HTTP API: List Instances with a status filter returns empty data, but has a x-ms-continuation-token

[jaliyaudagedara]

Description

I was querying Instances of a Durable Function App with the HTTP API using the following.

GET /runtime/webhooks/durableTask/instances?
    taskHub={taskHub}
    &connection={connectionName}
    &code={systemKey}
    &createdTimeFrom={timestamp}
    &createdTimeTo={timestamp}
    &runtimeStatus={runtimeStatus1,runtimeStatus2,...}
    &instanceIdPrefix={prefix}
    &showInput=[true|false]
    &top={integer}

Function App uses a Netherite backend.

<PackageReference Include="Microsoft.Azure.Functions.Worker.Extensions.DurableTask" Version="1.8.0" />
<PackageReference Include="Microsoft.Azure.Functions.Worker.Extensions.DurableTask.Netherite" Version="3.1.0" />
<PackageReference Include="Microsoft.Azure.Functions.Worker" Version="2.1.0" />
<PackageReference Include="Microsoft.Azure.Functions.Worker.Sdk" Version="2.0.5" />

I am noticing this scenario where on the HTTP request, the response body has no data, but has response header: x-ms-continuation-token. In some cases, I am finding data when traversed through using the x-ms-continuation-token. In some cases, I am traversing like forever.

Expected behavior

When the response header contains x-ms-continuation-token, then the current response should contain the data.

Actual behavior

When the response header contains x-ms-continuation-token, current response body is empty.

Relevant source code snippets

N/A

Known workarounds

N/A

App Details

• Durable Functions extension version (e.g. v1.8.3): 1.8.0
• Azure Functions runtime version (1.0 or 2.0): 4.x
• Programming language used: C#

Screenshots

If deployed to Azure

Sharing an InstanceId, so you can look up the function app.

• Orchestration instance ID(s): 30e7b79d1163481bbc07d83b4c12dcf7

[cgillum]

I believe this may be a known issue with the Netherite backend (which is going out of support in March 2028). Netherite does not have an indexed state store for queries, so instance queries must traverse through multiple pages in some cases before it finds a result that matches your query.

If you haven't already, please consider upgrading to the Durable Task Scheduler storage provider, which is a more robust, higher performance alternative to Netherite. It will not only provide faster data queries, but will also come with an integrated dashboard where you can visualize and manage your orchestrations.

[jaliyaudagedara]

@cgillum,

Thanks for replying.

We wanted to increase the performance of our Durable Function App and switched to DTS back in September for testing. DTS Dashboard was wonderful, but performance wasn't that good.

Tested performance with starting new orchestrations in parallel.

DTS:

Netherite:

So we haven't explored DTS further due to project commitments.

In our case, we want to investigate the Instances/Entities in our Durable Function App.
https://app-durable-functions-console-prod-001.azurewebsites.net/

In some cases can't really query as the traverse takes like forever.