Frequent `ObjectDisposedException` thrown by `IServiceProvider`

[Ilia-Kosenkov]

Description

Hi. We recently updated several of our services to use the latest version of the DurableFunctions library, and after that we started observing random exceptions outside of user code, namely ObjectDisposedException thrown from IServicePobider. This effectively destroys our orchestrations and interrupts our workflows.

Exhibit A

Microsoft.Azure.WebJobs.Host.FunctionInvocationException: Exception while executing function: Functions.MoveJsonSnapshot\n ---> Microsoft.Azure.WebJobs.Script.Workers.Rpc.RpcException: Result: Failure\nException: Cannot access a disposed object.\nObject name: 'IServiceProvider'.\nStack: at Microsoft.Extensions.DependencyInjection.ServiceLookup.ThrowHelper.ThrowObjectDisposedException()\n at Microsoft.Extensions.DependencyInjection.ServiceProvider.CreateScope()\n at Microsoft.Azure.Functions.Worker.DefaultFunctionContext.get_InstanceServices() in //src/DotNetWorker.Core/Context/DefaultFunctionContext.cs:line 46\n at Microsoft.Extensions.Hosting.MiddlewareWorkerApplicationBuilderExtensions.<>c__DisplayClass3_01.<UseMiddleware>b__1(FunctionContext context) in /_/src/DotNetWorker.Core/Hosting/WorkerMiddlewareWorkerApplicationBuilderExtensions.cs:line 105\n at Microsoft.Azure.Functions.Worker.FunctionsApplication.InvokeFunctionAsync(FunctionContext context)\n at Microsoft.Azure.Functions.Worker.Handlers.InvocationHandler.InvokeAsync(InvocationRequest request) in /_/src/DotNetWorker.Grpc/Handlers/InvocationHandler.cs:line 89\n at Microsoft.Azure.WebJobs.Script.Description.WorkerFunctionInvoker.InvokeCore(Object[] parameters, FunctionInvocationContext context) in /_/src/WebJobs.Script/Description/Workers/WorkerFunctionInvoker.cs:line 102\n at Microsoft.Azure.WebJobs.Script.Description.FunctionInvokerBase.Invoke(Object[] parameters) in /_/src/WebJobs.Script/Description/FunctionInvokerBase.cs:line 82\n at Microsoft.Azure.WebJobs.Script.Description.FunctionGenerator.Coerce[T](Task1 src) in //src/WebJobs.Script/Description/FunctionGenerator.cs:line 225\n at Microsoft.Azure.WebJobs.Host.Executors.FunctionInvoker2.InvokeAsync(Object instance, Object[] arguments) in D:\\a\\_work\\1\\s\\src\\Microsoft.Azure.WebJobs.Host\\Executors\\FunctionInvoker.cs:line 53\n at Microsoft.Azure.WebJobs.Extensions.DurableTask.OutOfProcMiddleware.<>c__DisplayClass10_0.<<CallOrchestratorAsync>b__0>d.MoveNext() in /_/src/WebJobs.Extensions.DurableTask/OutOfProcMiddleware.cs:line 135\n--- End of stack trace from previous location ---\n at Microsoft.Azure.WebJobs.Host.Executors.TriggeredFunctionExecutor1.<>c__DisplayClass7_0.<b__0>d.MoveNext() in D:\a\_work\1\s\src\Microsoft.Azure.WebJobs.Host\Executors\TriggeredFunctionExecutor.cs:line 51\n--- End of stack trace from previous location ---\n at Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor.InvokeWithTimeoutAsync(IFunctionInvoker invoker, ParameterHelper parameterHelper, CancellationTokenSource timeoutTokenSource, CancellationTokenSource functionCancellationTokenSource, Boolean throwOnTimeout, TimeSpan timerInterval, IFunctionInstance instance) in D:\a\_work\1\s\src\Microsoft.Azure.WebJobs.Host\Executors\FunctionExecutor.cs:line 581\n at Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor.ExecuteWithWatchersAsync(IFunctionInstanceEx instance, ParameterHelper parameterHelper, ILogger logger, CancellationTokenSource functionCancellationTokenSource) in D:\a\_work\1\s\src\Microsoft.Azure.WebJobs.Host\Executors\FunctionExecutor.cs:line 527\n at Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor.ExecuteWithLoggingAsync(IFunctionInstanceEx instance, FunctionStartedMessage message, FunctionInstanceLogEntry instanceLogEntry, ParameterHelper parameterHelper, ILogger logger, CancellationToken cancellationToken) in D:\a\_work\1\s\src\Microsoft.Azure.WebJobs.Host\Executors\FunctionExecutor.cs:line 306\n --- End of inner exception stack trace ---\n at Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor.ExecuteWithLoggingAsync(IFunctionInstanceEx instance, FunctionStartedMessage message, FunctionInstanceLogEntry instanceLogEntry, ParameterHelper parameterHelper, ILogger logger, CancellationToken cancellationToken) in D:\a\_work\1\s\src\Microsoft.Azure.WebJobs.Host\Executors\FunctionExecutor.cs:line 352\n at Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor.TryExecuteAsync(IFunctionInstance functionInstance, CancellationToken cancellationToken) in D:\a\_work\1\s\src\Microsoft.Azure.WebJobs.Host\Executors\FunctionExecutor.cs:line 108

Exhibit B (different incident, same trace)

at Microsoft.Extensions.DependencyInjection.ServiceLookup.ThrowHelper.ThrowObjectDisposedException()
at Microsoft.Extensions.DependencyInjection.ServiceProvider.CreateScope()
at Microsoft.Azure.Functions.Worker.DefaultFunctionContext.get_InstanceServices() in //src/DotNetWorker.Core/Context/DefaultFunctionContext.cs:line 46
at Microsoft.Extensions.Hosting.MiddlewareWorkerApplicationBuilderExtensions.<>c__DisplayClass3_0`1.b__1(FunctionContext context) in //src/DotNetWorker.Core/Hosting/WorkerMiddlewareWorkerApplicationBuilderExtensions.cs:line 105
at Microsoft.Azure.Functions.Worker.FunctionsApplication.InvokeFunctionAsync(FunctionContext context) in //src/DotNetWorker.Core/FunctionsApplication.cs:line 96
at Microsoft.Azure.Functions.Worker.Handlers.InvocationHandler.InvokeAsync(InvocationRequest request) in //src/DotNetWorker.Grpc/Handlers/InvocationHandler.cs:line 89

App Details

• Durable Functions extension version (e.g. v1.8.3): Microsoft.Azure.Functions.Worker.Extensions.DurableTask 1.8.0
• Azure Functions runtime version (1.0 or 2.0): 2.0
• Programming language used: C#

Here is a screenshot of AppInsights timeline. Green line shows when we did a migration & updated a bunch of dependencies in one of our services (26.09.2025), after which we started to get occasional ObjectDisposedException.

I plan to roll back at least major version and see if it has any effect.

[Ilia-Kosenkov]

Likely related to #3216 , but happens without durable entity interaction.

[Ilia-Kosenkov]

Actually now it seems identical to #3216 and it breaks continuation. We have orchestration like this (pseudo-c#):

try 
{
     var result = await context.CallOrchestration(FirstStep);
     await context.CallOrchestration(SecondStep, result); // ObjectDisposedException surfaced here from two layers deep sub-orchestrator
}
finally 
{
     await context.CallOrchestration(EnsureFinalized); // ‼️Never got executed after ObjectDisposedException
}

[sophiatev]

This seems related to this issue in the Functions host Azure/azure-functions-host#11298, I would suggest tracking the issue there.