Feature Request: Option to Close Session After Each Message in Azure Functions Service Bus Trigger with Sessions #42187
Description
Is your feature request related to a problem? Please describe.
We’re using Azure Functions with a session-enabled Service Bus subscription to ensure message ordering per session, but we do not require session affinity (i.e., all messages from a session being processed by the same instance).
Currently, the sessionIdleTimeout starts counting after the last message is received, not after it is processed. This causes issues when a message takes longer to process than the idle timeout.
- The session is released prematurely -> The message is redelivered and processed again → duplicate processing on another instance in parallel with the first processing -> data integrity issues
We also autoscale our solution and it's ok to process messages from the same session on different instances, as long as ordering is preserved.
The obvious solution to these problems is to set the sessionIdleTimeout to a larger value. However, this creates a problem - the consumer is idle and is not processing any messages for (almost) the whole duration of the timeout (sessionIdleTimeout minus time to process the message). Then, the session is closed and another session processing is initiated. This means waste of resources, making our solution both slow and more costly than necessary.
Describe the solution you'd like
We would like an option to close the session immediately after processing a message.
Another option would be to have the sessionIdleTimeout measuring start after message processing completes, not before - then we could set the sessionIdleTimeout to a very short time, to ensure prompt session closing. I personally believe this is how the sessionIdleTimeout option should work - how long the consumer should wait for another message of the same session after processing the last message for the session before closing the session. However, this would be a breaking change so it's very unlikely to happen.
Either of the two options would allow us to:
- Avoid duplicate deliveries.
- Improve throughput.
- Continue using Azure Functions
Describe alternatives you've considered
- Closing sessions after every message in a custom session processor using the Azure SDK (adds both programming and deployment complexity - as far as we know this couldn't run as a trigger in azure functions).
- Increasing both sessionIdleTimeout and maxConcurrentSessions to make sure the CPU is utilized (can help with throughput, but makes the solution less stable - when processing of all sessions on an instance aligns, the CPU is overloaded when processing all sessions at the same time, and then idle for a long time until the sessionIdleTimeout passes, and then again...).
** Additional context**
Instance Idle - not processing any new sessions until the sessionIdleTimeout of 60 seconds runs out. Message processing takes around 300 ms in this test (otherwise there are outliers with processing time of several minutes)
Thanks for considering this! We’d be happy to provide more context or test early versions of this feature.