[receiver/azureeventhub] Support app metrics

[olofmontin-truesec]

Component(s)

receiver/azureeventhub

Is your feature request related to a problem? Please describe.

I'm trying to gather metrics from various applications across multiple Azure tenants and subscriptions. These applications are emitting metrics into their own Application Insights, which are then forwarding a set of metrics into an Azure Event Hub in a remote Azure tenant. An OpenTelemetry Collector are receiving metrics from this Event Hub and ingesting them into a Azure Managed Prometheus data collection rule.

The endgoal is to have a centralized storage of metrics, available to query with PromQL, and used to monitor SLOs, without ingesting all telemetry from all applications.

Azure has different formats for their metrics, depending on their origin. The Azure Event Hub reciever supports metrics for Azure resources, and in my case, I also need it to support metrics emitted by OpenTelemetry through Application Insights.

Describe the solution you'd like

I've got this working actually and I've got a PR cooking. Although I'd just want to check if this is something you'd like to add or if there's any other concerns.

I've just extended the parsing of received json by adding another struct for these "AppMetrics". And some extendable code gluing this together, as there might be additional formats I haven't counted for.

Although, I'm not totally happy with the solution, as it converts all metrics into gauges and multiply them by four as the payload comes with a set of Sum, Min, Max and ItemCount values per metric.

I'm happy to let the code mature a bit for some days and then post a PR, if this seems interesting.

Describe alternatives you've considered

I've also tried to just forward metrics with Diagnostic Settings to a remote Log Analytics Workspace, but that will duplicate all metrics and logs in the origin. It'll also ingest metrics as logs and I won't be able to utilize PromQL.

Additional context

No response

Tip

_{React with 👍 to help prioritize this issue. Please use comments to provide useful context, avoiding +1 or me too, to help us triage it. Learn more here.}

[github-actions]

Pinging code owners:

• receiver/azureeventhub: @atoulme @cparkins

See Adding Labels via Comments if you do not have permissions to add labels yourself.

[olofmontin-truesec]

Ok, so, I did some more development for my usecase and drafted a PR, just to see if this is of any interest or along the direction you had in mind. If not, I'm happy to create an alternative component outside this repo for my usecase.

[axw]

IIUC this is about adding support for these formats? https://learn.microsoft.com/en-us/azure/azure-monitor/app/metrics-overview

[olofmontin-truesec]

IIUC this is about adding support for these formats? https://learn.microsoft.com/en-us/azure/azure-monitor/app/metrics-overview

Yes. I'm sorry I didn't provide any documentation.

I wrote a comment in the PR regarding this with some other links with more details

#41367 (comment)