Incompatibility with websockets 14

[MarcoGorelli]

Hey - we're temporarily pinning websockets in the Narwhals CI downstream shiny job, as the new websockets release (which came out today) seems to be causing some pyright failures narwhals-dev/narwhals#1343

/home/marcogorelli/py-shiny-dev/shiny/_autoreload.py
  /home/marcogorelli/py-shiny-dev/shiny/_autoreload.py:201:29 - error: Type of parameter "conn" is unknown (reportUnknownParameterType)
  /home/marcogorelli/py-shiny-dev/shiny/_autoreload.py:201:35 - error: Type of "WebSocketServerProtocol" is unknown (reportUnknownMemberType)
  /home/marcogorelli/py-shiny-dev/shiny/_autoreload.py:201:53 - error: "WebSocketServerProtocol" is not a known attribute of module ".server" (reportAttributeAccessIssue)
  /home/marcogorelli/py-shiny-dev/shiny/_autoreload.py:203:16 - error: Type of "path" is unknown (reportUnknownMemberType)
  /home/marcogorelli/py-shiny-dev/shiny/_autoreload.py:208:27 - error: Type of "send" is unknown (reportUnknownMemberType)
  /home/marcogorelli/py-shiny-dev/shiny/_autoreload.py:209:18 - error: Type of "path" is unknown (reportUnknownMemberType)
  /home/marcogorelli/py-shiny-dev/shiny/_autoreload.py:212:17 - error: Type of "req_secret" is unknown (reportUnknownVariableType)
  /home/marcogorelli/py-shiny-dev/shiny/_autoreload.py:212:30 - error: Type of "request_headers" is unknown (reportUnknownMemberType)
  /home/marcogorelli/py-shiny-dev/shiny/_autoreload.py:212:30 - error: Type of "get" is unknown (reportUnknownMemberType)
  /home/marcogorelli/py-shiny-dev/shiny/_autoreload.py:216:17 - error: Type of "data" is unknown (reportUnknownVariableType)
  /home/marcogorelli/py-shiny-dev/shiny/_autoreload.py:216:30 - error: Type of "recv" is unknown (reportUnknownMemberType)
  /home/marcogorelli/py-shiny-dev/shiny/_autoreload.py:240:9 - error: Argument type is partially unknown
    Argument corresponds to parameter "handler" in function "__init__"
    Argument type is "(conn: Unknown) -> Coroutine[Any, Any, None]" (reportUnknownArgumentType)
  /home/marcogorelli/py-shiny-dev/shiny/_autoreload.py:240:59 - error: Argument of type "(path: str, request_headers: Headers) -> Coroutine[Any, Any, tuple[HTTPStatus, HeadersLike, bytes] | None]" cannot be assigned to parameter "process_request" of type "((ServerConnection, Request) -> (Awaitable[Response | None] | Response | None)) | None" in function "__init__"
    Type "(path: str, request_headers: Headers) -> Coroutine[Any, Any, tuple[HTTPStatus, HeadersLike, bytes] | None]" is not assignable to type "((ServerConnection, Request) -> (Awaitable[Response | None] | Response | None)) | None"
      Type "(path: str, request_headers: Headers) -> Coroutine[Any, Any, tuple[HTTPStatus, HeadersLike, bytes] | None]" is not assignable to type "(ServerConnection, Request) -> (Awaitable[Response | None] | Response | None)"
        Parameter 1: type "ServerConnection" is incompatible with type "str"
          "ServerConnection" is not assignable to "str"
        Parameter 2: type "Request" is incompatible with type "Headers"
          "Request" is not assignable to "Headers"
        Function return type "Coroutine[Any, Any, tuple[HTTPStatus, HeadersLike, bytes] | None]" is incompatible with type "Awaitable[Response | None] | Response | None"
          Type "Coroutine[Any, Any, tuple[HTTPStatus, HeadersLike, bytes] | None]" is not assignable to type "Awaitable[Response | None] | Response | None"
    ... (reportArgumentType)
13 errors, 0 warnings, 0 informations

pinning websockets<14 it passes

[gadenbuie]

This also impacts running Shiny for Python apps with --reload, i.e. via the VSCode extension.

INFO:     Will watch for changes in these directories: ['/Users/garrick/$APP_DIR/posit-brand']
INFO:     Uvicorn running on http://127.0.0.1:50968 (Press CTRL+C to quit)
INFO:     Started reloader process [63264] using WatchFiles
opening handshake failed
Traceback (most recent call last):
  File "/Users/garrick/$APP_DIR/.venv/lib/python3.13/site-packages/websockets/http11.py", line 134, in parse
    request_line = yield from parse_line(read_line)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/garrick/$APP_DIR/.venv/lib/python3.13/site-packages/websockets/http11.py", line 380, in parse_line
    line = yield from read_line(MAX_LINE_LENGTH)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/garrick/$APP_DIR/.venv/lib/python3.13/site-packages/websockets/streams.py", line 46, in read_line
    raise EOFError(f"stream ends after {p} bytes, before end of line")
EOFError: stream ends after 0 bytes, before end of line

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/garrick/$APP_DIR/.venv/lib/python3.13/site-packages/websockets/asyncio/server.py", line 353, in conn_handler
    await connection.handshake(
    ...<3 lines>...
    )
  File "/Users/garrick/$APP_DIR/.venv/lib/python3.13/site-packages/websockets/asyncio/server.py", line 204, in handshake
    raise self.protocol.handshake_exc
  File "/Users/garrick/$APP_DIR/.venv/lib/python3.13/site-packages/websockets/server.py", line 551, in parse
    request = yield from Request.parse(
              ^^^^^^^^^^^^^^^^^^^^^^^^^
        self.reader.read_line,
        ^^^^^^^^^^^^^^^^^^^^^^
    )
    ^
  File "/Users/garrick/$APP_DIR/.venv/lib/python3.13/site-packages/websockets/http11.py", line 136, in parse
    raise EOFError("connection closed while reading HTTP request line") from exc
EOFError: connection closed while reading HTTP request line
INFO:     Started server process [63266]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
Task exception was never retrieved
future: <Task finished name='Task-3' coro=<reload_end.<locals>._() done, defined at /Users/garrick/$APP_DIR/.venv/lib/python3.13/site-packages/shiny/_autoreload.py:71> exception=TypeError("BaseEventLoop.create_connection() got an unexpected keyword argument 'extra_headers'")>
Traceback (most recent call last):
  File "/Users/garrick/$APP_DIR/.venv/lib/python3.13/site-packages/shiny/_autoreload.py", line 78, in _
    async with websockets.connect(
               ~~~~~~~~~~~~~~~~~~^
        url, **options  # pyright: ignore[reportArgumentType]
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    ) as websocket:
    ^
  File "/Users/garrick/$APP_DIR/.venv/lib/python3.13/site-packages/websockets/asyncio/client.py", line 484, in __aenter__
    return await self
           ^^^^^^^^^^
  File "/Users/garrick/$APP_DIR/.venv/lib/python3.13/site-packages/websockets/asyncio/client.py", line 441, in __await_impl__
    self.connection = await self.create_connection()
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/garrick/$APP_DIR/.venv/lib/python3.13/site-packages/websockets/asyncio/client.py", line 367, in create_connection
    _, connection = await loop.create_connection(factory, **kwargs)
                          ~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^
TypeError: BaseEventLoop.create_connection() got an unexpected keyword argument 'extra_headers'

[gadenbuie]

These changes seem very relevant:

process_request¶

The signature of process_request changed. This is easiest to illustrate with an example:

import http

# Original implementation

def process_request(path, request_headers):
    return http.HTTPStatus.OK, [], b"OK\n"

# New implementation

def process_request(connection, request):
    return connection.respond(http.HTTPStatus.OK, "OK\n")

serve(..., process_request=process_request, ...)

connection is always available in process_request. In the original implementation, if you wanted to make the connection object available in a process_request method, you had to write a subclass of WebSocketServerProtocol and pass it in the create_protocol argument. This pattern isn’t useful anymore; you can replace it with a process_request function or coroutine.

[gadenbuie]

@MarcoGorelli we've updated Shiny to work with websockets 14 and will require websockets >= 13.0. We're doing some internal QA testing now and hope to have a patch release on PyPI soon.

[karangattu]

@MarcoGorelli Please update the shiny package to 1.2.1 by running

pip install --upgrade shiny

to fix this issue.