TelemetryClient FlushAsync will block for up to 100 seconds

[thegoodnerd]

• List of NuGet packages and version that you are using:

• Microsoft.ApplicationInsights 2.20.0

• Runtime version (e.g. net461, net48, netcoreapp2.1, netcoreapp3.1, etc. You can find this information from the *.csproj file):

• net8.0

• Hosting environment (e.g. Azure Web App, App Service on Linux, Windows, Ubuntu, etc.):

• k8s ubuntu based docker container

Describe the bug

The TelemetryClient.Flush method occasionally freezes for up to 100 seconds. We use the telemetry client configured with InMemoryTransmitter and InMemoryChannel. When we close our application due to pod termination, we observe that the FlushAsync method of the telemetry client—which accepts a cancellation token—blocks, and consequently, the pod is eventually killed by Kubernetes.

This occurs because the cancellation token isn't effectively observed. Specifically, inside FlushAsync, the Flush(timeout) method of the channels/transmitters is called using default(timeout), leading to a 100-second default timeout applied to the underlying HttpClient.SendAsync method. This freeze happens because, occasionally, actual requests to the Application Insights endpoint fail to complete and thus reach this timeout.

Additionally, the freeze can happen even if the Flush method itself isn't directly experiencing this timeout. A background task running DequeueAndSend with the default timeout can also become stuck. Since the logic for dequeuing and sending is guarded by synchronization mechanisms, FlushAsync can also freeze due to being unable to acquire this lock.

Ideally, the cancellation token provided to FlushAsync should be observed all the way down to the HttpClient.SendAsync method, ensuring that FlushAsync isn't blocked by the locking mechanism used in DequeueAndSend.

To Reproduce

Create a self-contained project using the template of your choice, apply the minimum required code to result in the issue you're observing.

We will close this issue if:

• The repro project you share with us is complex. We can't investigate custom projects, so don't point us to such, please.
• If we will not be able to repro the behavior you're reporting

For Immediate Support

For immediate support relating to the Application Insights .NET SDK we encourage you to file an Azure Support Request with Microsoft Azure instead of filing a GitHub Issue in this repository.
You can do so by going online to the Azure portal and submitting a support request. Access to subscription management and billing support is included with your Microsoft Azure subscription, and technical support is provided through one of the Azure Support Plans. For step-by-step guidance for the Azure portal, see How to create an Azure support request. Alternatively, you can create and manage your support tickets programmatically using the Azure Support ticket REST API