Raise custom exception instead of RuntimeError

[browniebroke]

We're using uvicorn with server side event in a Django application. We're sometimes getting some RuntimeError raised by uvicorn:

RuntimeError: Expected ASGI message 'websocket.send' or 'websocket.close', but got 'websocket.accept'.

Looking at the uvicorn code, it seems perfectly reasonable to raise something, so I was thinking to catch the appropriate exception and do something with it when that happens. However, we have a few framework layers between where the exception is raised by uvicorn and our code, so catching for plain RuntimeError seems a bit broad and I'm a bit concerned of hiding other issues.

Would it make sense for uvicorn to define its own set of exception classes and raise them instead of raising built-in exceptions from the standard library? It would certainly make it easier for us to target and handle the right error.

[Kludex]

I don't understand how changing the exception would make it easier for you. 🤔

You're sending the websocket.accept multiple times. You should send it once.

[browniebroke]

I don't understand how changing the exception would make it easier for you.

When it's raised, I catch it and move on with my life. Catching "RuntimeError" would be a bit too broad I think.

You're sending the websocket.accept multiple times. You should send it once.

I don't know why it's doing that, and probably I should, but in the context of our project, it's much harder than the alternative I'm suggesting 😄

From the outside, changing the exception type to seemed like a relatively simple change and seemed like a good idea anyway, so I thought I asked, but I understand if you say no, it's 100% up to you at the end.

[browniebroke]

Part of the reason I'm not understanding why it's happening is because I'm not sending it directly, my app is implementing a specialised version of AsyncWebsocketConsumer from Django channels, uvicorn calls channels which calls my code, which calls channels again, which ends up calling uvicorn's websocket implementation.

It seems that the error happens when my backend restarts (e.g. when a new version is deployed) and some operation gets interrupted...

[browniebroke]

FWIW, I manage to uncover more details about this. It seems the server is being overloaded with too many connections:

[2024-11-19 09:57:24 +0000] [61] [WARNING] Maximum request limit of 286 exceeded. Terminating process.
[2024-11-19 09:57:24 +0000] [61] [INFO] Shutting down
[2024-11-19 09:57:24 +0000] [61] [INFO] Error while closing socket [Errno 9] Bad file descriptor
[2024-11-19 09:57:24 +0000] [61] [INFO] connection rejected (503 Service Unavailable)
[2024-11-19 09:57:24 +0000] [61] [INFO] connection closed
[2024-11-19 09:57:24 +0000] [61] [INFO] connection closed
[2024-11-19 09:57:24 +0000] [61] [INFO] connection closed
[2024-11-19 09:57:24 +0000] [61] [INFO] connection closed
[2024-11-19 09:57:24 +0000] [61] [INFO] connection closed
[2024-11-19 09:57:24 +0000] [61] [INFO] connection closed
[2024-11-19 09:57:24 +0000] [61] [INFO] connection closed
[2024-11-19 09:57:24 +0000] [61] [INFO] connection closed
[2024-11-19 09:57:24 +0000] [61] [INFO] connection closed
[2024-11-19 09:57:24 +0000] [61] [INFO] connection closed
[2024-11-19 09:57:24 +0000] [61] [INFO] connection closed
[2024-11-19 09:57:24 +0000] [61] [INFO] connection closed
[2024-11-19 09:57:24 +0000] [61] [INFO] connection closed
[2024-11-19 09:57:24 +0000] [61] [INFO] connection closed
[2024-11-19 09:57:24 +0000] [61] [INFO] connection closed
[2024-11-19 09:57:24 +0000] [61] [INFO] connection closed
[2024-11-19 09:57:24 +0000] [61] [INFO] connection closed
[2024-11-19 09:57:24 +0000] [61] [INFO] connection closed
[2024-11-19 09:57:24 +0000] [61] [INFO] connection closed
[2024-11-19 09:57:24 +0000] [61] [INFO] connection closed
[2024-11-19 09:57:24 +0000] [61] [INFO] connection closed
[2024-11-19 09:57:24 +0000] [61] [INFO] connection closed
[2024-11-19 09:57:24 +0000] [61] [INFO] connection closed
[2024-11-19 09:57:24 +0000] [61] [INFO] connection closed
[2024-11-19 09:57:24 +0000] [61] [INFO] connection closed
[2024-11-19 09:57:24 +0000] [61] [INFO] connection closed
[2024-11-19 09:57:24 +0000] [61] [INFO] connection closed
[2024-11-19 09:57:24 +0000] [61] [INFO] connection closed
[2024-11-19 09:57:24 +0000] [61] [INFO] connection closed
[2024-11-19 09:57:24 +0000] [61] [INFO] connection closed
[2024-11-19 09:57:24 +0000] [61] [INFO] connection closed
[2024-11-19 09:57:24 +0000] [61] [INFO] connection closed
[2024-11-19 09:57:24 +0000] [61] [INFO] connection closed
[2024-11-19 09:57:24 +0000] [61] [INFO] connection closed
[2024-11-19 09:57:24 +0000] [61] [INFO] connection closed
[2024-11-19 09:57:24 +0000] [61] [INFO] connection closed
[2024-11-19 09:57:24 +0000] [61] [INFO] connection closed
[2024-11-19 09:57:24 +0000] [61] [INFO] connection closed
[2024-11-19 09:57:24 +0000] [61] [INFO] connection closed
[2024-11-19 09:57:24 +0000] [61] [INFO] connection closed
[2024-11-19 09:57:24 +0000] [61] [INFO] connection closed
[2024-11-19 09:57:24 +0000] [61] [INFO] connection closed
[2024-11-19 09:57:24 +0000] [61] [INFO] connection closed
[2024-11-19 09:57:24 +0000] [61] [INFO] connection closed
[2024-11-19 09:57:24 +0000] [61] [INFO] connection closed
[2024-11-19 09:57:24 +0000] [61] [INFO] connection closed
[2024-11-19 09:57:24 +0000] [61] [INFO] connection closed
[2024-11-19 09:57:24 +0000] [61] [INFO] connection closed
[2024-11-19 09:57:24 +0000] [61] [INFO] connection closed
[2024-11-19 09:57:24 +0000] [61] [INFO] connection closed
[2024-11-19 09:57:24 +0000] [61] [INFO] connection closed
[2024-11-19 09:57:24 +0000] [61] [INFO] connection closed
[2024-11-19 09:57:24 +0000] [61] [INFO] connection closed
[2024-11-19 09:57:24 +0000] [61] [INFO] connection closed
[2024-11-19 09:57:24 +0000] [61] [INFO] connection closed
[2024-11-19 09:57:24 +0000] [61] [INFO] connection closed
[2024-11-19 09:57:24 +0000] [61] [INFO] connection closed
[2024-11-19 09:57:24 +0000] [61] [INFO] connection closed
[2024-11-19 09:57:24 +0000] [61] [INFO] connection closed
[2024-11-19 09:57:24 +0000] [61] [INFO] connection closed
[2024-11-19 09:57:24 +0000] [61] [INFO] connection closed
[2024-11-19 09:57:24 +0000] [61] [INFO] connection closed
[2024-11-19 09:57:24 +0000] [61] [INFO] connection closed
[2024-11-19 09:57:24 +0000] [61] [INFO] connection closed
[2024-11-19 09:57:24 +0000] [61] [INFO] connection closed
[2024-11-19 09:57:24 +0000] [61] [INFO] connection closed
[2024-11-19 09:57:24 +0000] [61] [INFO] connection closed
[2024-11-19 09:57:24 +0000] [61] [INFO] connection closed
[2024-11-19 09:57:24 +0000] [61] [INFO] connection closed
[2024-11-19 09:57:24 +0000] [61] [INFO] connection closed
[2024-11-19 09:57:24 +0000] [61] [INFO] connection closed
[2024-11-19 09:57:24 +0000] [61] [INFO] connection closed
[2024-11-19 09:57:24 +0000] [61] [INFO] connection closed
[2024-11-19 09:57:24 +0000] [61] [INFO] connection closed
[2024-11-19 09:57:24 +0000] [61] [INFO] connection closed
[2024-11-19 09:57:24 +0000] [61] [INFO] connection closed
[2024-11-19 09:57:24 +0000] [61] [INFO] connection closed
[2024-11-19 09:57:24 +0000] [61] [INFO] connection closed
[2024-11-19 09:57:24 +0000] [61] [INFO] Waiting for background tasks to complete. (CTRL+C to force quit)
[2024-11-19 09:57:24 +0000] [61] [ERROR] Exception in ASGI application
Traceback (most recent call last):
  File "/opt/venv/lib/python3.12/site-packages/uvicorn/protocols/websockets/websockets_impl.py", line 242, in run_asgi
    result = await self.app(self.scope, self.asgi_receive, self.asgi_send)  # type: ignore[func-returns-value]
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/uvicorn/middleware/proxy_headers.py", line 60, in __call__
    return await self.app(scope, receive, send)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/channels/routing.py", line 62, in __call__
    return await application(scope, receive, send)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/channels/security/websocket.py", line 37, in __call__
    return await self.application(scope, receive, send)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/channels/sessions.py", line 47, in __call__
    return await self.inner(dict(scope, cookies=cookies), receive, send)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/channels/sessions.py", line 263, in __call__
    return await self.inner(wrapper.scope, receive, wrapper.send)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/channels/auth.py", line 185, in __call__
    return await super().__call__(scope, receive, send)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/channels/middleware.py", line 24, in __call__
    return await self.inner(scope, receive, send)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/channels/routing.py", line 132, in __call__
    return await application(
           ^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/channels/consumer.py", line 94, in app
    return await consumer(scope, receive, send)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/channels/consumer.py", line 58, in __call__
    await await_many_dispatch(
  File "/opt/venv/lib/python3.12/site-packages/channels/utils.py", line 50, in await_many_dispatch
    await dispatch(result)
  File "/opt/venv/lib/python3.12/site-packages/channels/consumer.py", line 73, in dispatch
    await handler(message)
  File "/opt/venv/lib/python3.12/site-packages/channels/generic/websocket.py", line 179, in websocket_connect
    await self.connect()
  File "/code/apps/server_side_events/consumers.py", line 50, in connect
    await self.accept()
  File "/opt/venv/lib/python3.12/site-packages/channels/generic/websocket.py", line 195, in accept
    await super().send(message)
  File "/opt/venv/lib/python3.12/site-packages/channels/consumer.py", line 81, in send
    await self.base_send(message)
  File "/opt/venv/lib/python3.12/site-packages/channels/sessions.py", line 226, in send
    return await self.real_send(message)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/venv/lib/python3.12/site-packages/uvicorn/protocols/websockets/websockets_impl.py", line 341, in asgi_send
    raise RuntimeError(msg % message_type)
RuntimeError: Expected ASGI message 'websocket.send' or 'websocket.close', but got 'websocket.accept'.
[2024-11-19 09:57:24 +0000] [61] [INFO] Finished server process [61]

The first line comes from here:

https://github.com/encode/uvicorn/blob/fe3910083e3990695bc19c2ef671dd447262ae18/uvicorn/server.py#L256

From I can see, the server is shutting down, but a request is still being processed? Still digging why we're getting so many connections, we're probably not cleaning up after ourselves...

Anyway, seems like an issue in our app I think, but in case someone comes across a similar issue.