No obvious way to handle asyncio.CancelledError

[dybi]

Hi there,

In our project we had to deal with the following scenario:

1. There is an asynchronous web app.
2. During handling one (pretty long - say few seconds) request A, the service was order to reload - closing server and cancelling all the pending tasks (including task A).
3. There is an ugly traceback in the logs with asyncio.CancelledError and there is no obvious way to return i.e. 503 to A caller

What we did was adding custom middleware (the most outer one) catching all (concurent.future.CanccelledError, asyncio.CancelledError) exceptions.
The solution is not perfect (there are edge cases were it might not work), but is more less OK.

The much better one would be adding custom exception handler for asyncio.CancelledError, but ATM this is not possible, because starlette explicitly checks that every error handler is an Exception (https://github.com/encode/starlette/blob/d7558515020f4c0a96b199880c181a40678b18d5/starlette/middleware/exceptions.py#L42), whereas asyncio.CancelledError is a BaseException.

Where there any reasons for that? Not handling asyncio.CancllledError? Or what would be the recommended way of handling the situation described in the scenario?

(If I chose wrong category, please change it)

Thanks and keep on with your great work! 💪

[Kludex]

How do you have 6 upvotes? 🤔

What do you mean the "service was order to reload"? Are you using --reload in production? 🤔

Ideally, the user of starlette shouldn't need to worry about CancelledError.

[dybi]

hi @Kludex

Sorry for not being precise. We use gunciorn with custom workers based on UvicornWorker. In our service, we have multiple workers serving requests. When one worker gets deactivate command (our internal stuff) it propagates it to fellow workers, resulting in closing asyncio loop and getting unfinished requests (i.e. from different workers) to get cancelled, causing asyncio.CancelledError. We would like to have a way of handling such asyncio.CancelledErrors.

TL;DR - deactivating one worker will cause other workers to get deactivated (technically meaning uvicorn worker gets SIGTERM).

So, our case might be not 'ideal' ;)

[Kludex]

Ah... I see.

Is the middleware solution working fine? Cause as I said, the user shouldn't be concerned about cancellation error, and it sounds this is a very specific use case...

[dybi]

The solution with custom middleware is a bit hackish (oddball solution) taking into account, that starlette provides dedicated exception handling layer . Why not giving the user possibility to handle CancelledError if he so desires?

[Kludex]

Why not giving the user possibility to handle CancelledError if he so desires?

Because, as I said, the user shouldn't be worried about CancelledError, only Exception based exceptions are considered user exceptions, and the ones Starlette should care.

The solution with custom middleware is a bit hackish (oddball solution)

Well, you have quite an unusual setup, as you mentioned above...

If we want to discuss the idea, we need at least a good use-case. 🤔

[marcinbarczynski]

I think there are cases in which user has to deal with cancellation errors:

1. Custom error response when the server is shutting down. For instance, uvicorn always replies with 500 Internal Server Error.
2. Avoiding traceback in production-quality code. Consider the following code:

import asyncio

from starlette.applications import Starlette
from starlette.responses import JSONResponse
from starlette.routing import Route

app = Starlette()


async def long_request(request):
    await asyncio.sleep(123)
    return JSONResponse({"hello": "world"})


app = Starlette(
    routes=[
        Route("/long", long_request),
    ],
)

Now, stopping the server while long request is in progress generates a nasty traceback:

$ uvicorn demo-starlette:app
INFO:     Started server process [216821]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)
^CINFO:     Shutting down
INFO:     Waiting for connections to close. (CTRL+C to force quit)
^CINFO:     Finished server process [216821]
ERROR:    Traceback (most recent call last):
  File "PATH/python3.10/site-packages/starlette/routing.py", line 674, in lifespan
    await receive()
  File "PATH/python3.10/site-packages/uvicorn/lifespan/on.py", line 135, in receive
    return await self.receive_queue.get()
  File "PATH/python3.10/asyncio/queues.py", line 159, in get
    await getter
asyncio.exceptions.CancelledError

ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "PATH/python3.10/site-packages/uvicorn/protocols/http/h11_impl.py", line 366, in run_asgi
    result = await app(self.scope, self.receive, self.send)
  File "PATH/python3.10/site-packages/uvicorn/middleware/proxy_headers.py", line 75, in __call__
    return await self.app(scope, receive, send)
  File "PATH/python3.10/site-packages/starlette/applications.py", line 124, in __call__
    await self.middleware_stack(scope, receive, send)
  File "PATH/python3.10/site-packages/starlette/middleware/errors.py", line 162, in __call__
    await self.app(scope, receive, _send)
  File "PATH/python3.10/site-packages/starlette/middleware/exceptions.py", line 68, in __call__
    await self.app(scope, receive, sender)
  File "PATH/python3.10/site-packages/starlette/routing.py", line 706, in __call__
    await route.handle(scope, receive, send)
  File "PATH/python3.10/site-packages/starlette/routing.py", line 276, in handle
    await self.app(scope, receive, send)
  File "PATH/python3.10/site-packages/starlette/routing.py", line 66, in app
    response = await func(request)
  File "PATH/demo-starlette.py", line 11, in long_request
    await asyncio.sleep(123)
  File "PATH/python3.10/asyncio/tasks.py", line 605, in sleep
    return await future
asyncio.exceptions.CancelledError
INFO:     127.0.0.1:48166 - "GET /long HTTP/1.1" 500 Internal Server Error

IMO, making it possible to define a custom response for CancelledError in ExceptionMiddleware would be great!

[Kludex]

1 and 2 are the same above.

...and I don't think they are valid. When the server shuts down gracefully, it gives the task time to finish.