Losing correlation in end-to-end transaction details in AppInsights when using latest version of Arcus

[ClementVaillantCodit]

Possible duplicate of Dependencies are not correctly linked to the incoming request #387

We are using Arcus in a project to setup the logging in an Azure function app and we are encountering issues.

We are using the following packages:

• Arcus.Observability.Telemetry.Serilog.Enrichers v2.5
• Arcus.Observability.Telemetry.Serilog.Sinks.ApplicationInsights v2.5

Just a note before to start, I tried installing Arcus.Observability.Telemetry.AzureFunctions v2.5 and use
.RemoveMicrosoftApplicationInsightsLoggerProvider()

But when I do the above, function app logs do not show up in AppInsights so I stopped using the above.

The root problem is that we are losing correlation for calls between components.
We have an APIM reaching out to a function app functions and I would expect to see something like this in AppInsights (calls are correlated so we can see the full end-to-end trace):

We can see which component called the function (using default behaviour when 2 components point to the same app insights instance).

Although when using the latest version of Arcus, we are losing correlation and the function Parent ID is set to the Operation ID of the call to the function.

Has this been fixed already? If yes when do you think this will be released?

[stijnmoreels]

Hi! Thx for the issue.

Just a note before to start, I tried installing Arcus.Observability.Telemetry.AzureFunctions v2.5 and use
.RemoveMicrosoftApplicationInsightsLoggerProvider()
But when I do the above, function app logs do not show up in AppInsights so I stopped using the above.

The line you mentioned here, is required when the Azure Function is run in-process. Please do add this as otherwise the Arcus and Microsoft logging will conflict in Application Insights. If the logs don't show, it could mean that there's a problem with minimum log level, setup of Serilog sink (connection string <> instrumentation key), and also the time it takes before telemetry shows up in Application Insights.

The root problem is that we are losing correlation for calls between components.
We have an APIM reaching out to a function app functions and I would expect to see something like this in AppInsights (calls are correlated so we can see the full end-to-end trace):

What I assume from your description, is that you expect the request-dependency correlation to be there from the start, automatically, in Azure Functions, but Azure Functions also does this when it is configured for Microsoft, not for Arcus. When using Arcus, in Azure Functions you have to (for now) make sure that you call the request tracking yourself. See the an example of our HTTP trigger template where we track the request.
Make sure to use WebApi v1.6 of Arcus also to get the correct correlation implementation.

Has this been fixed already? If yes when do you think this will be released?

All the issues mentioned here are already fixed and released, it is a matter of correctly setting Arcus up for Azure Functions. Currently, we don't cleanly support out-of-process Azure Functions as it still rather cumbersome to get it right, but we're working on a clean approach there, too.

[stijnmoreels]

Also, we're working on a dedicated user guide to help you with setting up service-to-service correlation in Web API and Messaging solution.

[stijnmoreels]

We have recently updated the Arcus Web API library to use the more recent W3C HTTP correlation: https://webapi.arcus-azure.net/Features/correlation-azure-functions. Feel free to have a look and update the example accordingly, as the older Hierarchical HTTP correlation is deprecated.

[fgheysels]

Is this still relevant ?
@spasal is running into a similar issue today

[stijnmoreels]

Is this still relevant ? @spasal is running into a similar issue today

I would think not 🤔
Assigned Clement to check their setup.