Unclear documentation about limit_concurrency and backlog settings

[siminchengithub]

According to the doc, limit_concurrency is the maximum number of concurrent connections and any extra connection requests will get 503; backlog is the maximum number of connections in the backlog waiting to be handled.
My confusion about them:

1. when limit_concurrency is reached, do all extra requests get 503 straight without being put in the backlog first?
2. when there are requests in the backlog and limit_concurrency is reached, will those pending requests get 503 one by one? Gunicorn would just process them whenever possible, but the definition of limit_concurrency in Uvicorn seems to imply otherwise.
3. when backlog is full while limit_concurrency is not yet reached, what status code do the extra requests get?
4. when backlog is full and limit_concurrency is also reached, what status code do the extra requests get?

If someone can also describe more clearly how Uvicorn interacts with a socket and its backlog in relation to its limit_concurrency setting would be great!

[Kludex]

PR is always welcome to improve documentation...

[siminchengithub]

@Kludex those are my questions though, once I understand the answers I'm happy to improve the documentation

[chaselambda]

I'm trying to figure out what the backlog parameter does as well. It looks like it ultimately goes to the listen syscall.

The original PR doesn't seem to clarify what's going on: https://github.com/encode/uvicorn/pull/545/files

In playing around on Ubuntu 20.04:

• It does seem like backlog is indeed getting sent to the listen syscall:
  
• All my requests seem to get queued up with curl.. nothing gets prematurely dropped with a backlog of 1
• ss -a indeed shows Send-Q is updated (though I don't know precisely what this means):
  

I think ultimately, the backlog parameter doesn't actually drop TCP connections, because TCP will retry. Here are two sources that suggest this:

Given this, I don't have a good idea when anyone would ever use backlog. Maybe some TCP connections are really picky about getting SYN responses quickly, and will otherwise time out faster? Something detailed like that.

[chaselambda]

limit_concurrency docs seem to be off-by-one. limit_concurrency of 1 will drop all requests. And a value of 2 will allow at most 1 request. That's because this >= sign should probably be simply >.

That structure is added to before the check is made:

[chaselambda]

To answer this rephrased question: "when limit_concurrency is reached, do all extra requests get 503 immediately"

Answer: No. You first have to wait for one of the workers to finish its last request, it will then drop all requests. Per worker. That's because each worker is running on a single thread, so it can't go and tell some incoming request to drop. You would hope that there would be some OS-layer tool that would do that dropping for you (like backlog), but it seems like that's not the case. limit_concurrency is simply written in python, so that can't be run while there's a blocking request happening.

[euri10]

been a while but iirc there's 0 link between the limit_concurrency and the backlog flags, https://stackoverflow.com/a/72897345/3581357

[chaselambda]

@euri10 I'm curious what part of that answer is additionally helpful here?

That answer states "backlog is passed down the loop.create_server and ultimately will determine the number of sockets listening.", which I believe is incorrect. There's only one socket listening, with a backlog parameter passed to it, as I've mentioned above.

"limit-concurrency is just here to tell, after x responses issue a 503" I think is not very clear. My description above seems to be more detailed and specific.

Maybe you understand something I do not, but from what I can glean you're just adding that answer in case it's helpful, not because you have confidence it's helpful. Which is a fine thing, I just wanted to check.

[euri10]

I didnt read your answer, I was answering OP ;)

[siminchengithub]

@euri10 that stackoverflow link simply states there is no link with a wrong definition of limit_concurrency and no explanation to the dynamic between limit_concurrency and backlog. If you are confident that there is no link, then please explain more exactly why there is no relationship by elaborating the answers to my questions.

[siminchengithub]

@Kludex why is this issue converted to a discussion? There is a clear confusion to the behavior of Uvicorn's two settings. This is an issue for people troubleshooting timeouts if they can't find out what the settings are exactly doing.

[euri10]

Cant do better than quoting the docs :
socket.listen([backlog])¶
Enable a server to accept connections. If backlog is specified, it must be at least 0 (if it is lower, it is set to 0); it specifies the number of unaccepted connections that the system will allow before refusing new connections. If not specified, a default reasonable value is chosen.

[siminchengithub]

thanks for the reference. The Python socket.listen() calls the underlying listen syscall, whose backlog argument since Linux 2.2 "specifies the queue length for completely established sockets waiting to be accepted, instead of the number of incomplete connection requests." So this queue holds connections whose 3-way handshake is already completed

[siminchengithub]

@euri10 can you confirm whether this is correct?: when the number of concurrent connections or tasks equals to limit_concurrency, Uvicorn would pop and respond 503 to the next connection held the ACCEPT queue? And this operation will continue as long as ACCEPT queue is not empty and limit_concurrency is still reached.

[euri10]

Cant tell you really without looking back again at the source which right now is compromised for me

[siminchengithub]

limit_concurrency check is done inside on_headers_complete, I suppose it indeed happens as how I described.

[siminchengithub]

After reading the source code further, I think there is actually a bug.
The limit_concurrency check is done inside on_headers_complete after a connection is added and before a new task is added. This will work correctly to make sure any extra requests receive 503. However this won't make sure there are less than limit_concurrency connections. Connections can continue to pile up as long as they are not closed.

[ddelange]

--backlog

The maximum number of pending connections.

This refers to the number of clients that can be waiting to be served. Exceeding this number results in the client getting an error when attempting to connect. It should only affect servers under significant load.

Must be a positive integer. Generally set in the 64-2048 range.

combine that with @siminchengithub's answer, it looks like --limit-concurrency works on the task/thread level, and --backlog on the connection level.

[Cruuncher]

Going to hijack this discussion thread here because I'm still very confused about how these 2 flags work both after the answers to my stackoverflow question that was linked here, and with the responses in this thread.

Concrete example:
Setup is limit-concurrency of 5, backlog as some high number like 2024.
Request handling at the application is slow, let's say 1s for easy number.
10 requests come in at approximately the same time. What happens to those 10 requests, and what response codes do the callers get?

I can see at least 2 possible interpretations of how uvicorn behaves here from the documentation:

1. 5 requests are handled immediately, which end up with a total latency of 1s, while the other 5 requests wait in the backlog, are processed after, and have a total latency of 2s. 1s to wait for the backlog to clear, and 1s handling the request. Requests all 200
2. 5 requests are handled immediately, which end up with a total latency of 1s, while the other 5 requests return a 503 essentially immediately.

I suppose I can setup a test to get this result, but this really should be clear from the documentation.

[gregoiredx]

I did exactly this test, I tried with a sync endpoint and an async endpoint (so time.sleep(1) and anyio.sleep(1) as the endpoints' implementations).

I used apache bench to launch the requests ab -c 10 -n 10 http://0.0.0.0:8000/sleep

The result is that 9 requests are rejected with "Exceeded concurrency limit" and only one is accepted and correctly served 🙃

[Cruuncher]

It's also worth noting that when you ask ChatGPT, its interpretation of how uvicron+gunicorn works is that it behaves like option 1 in my post above, but this thread is making me thing that option 2 is actually correct. This is just further evidence that a natural language interpretation of the existing documentation and online material is misleading. Here's some output text:

The limit-concurrency setting in Uvicorn, as described, sets a threshold for the maximum number of concurrent connections or tasks the server will handle before starting to issue HTTP 503 responses. However, the behavior in response to reaching this limit is a bit more nuanced than simply moving excess requests to the backlog.

When you have limit-concurrency set to 5, and 10 requests come in simultaneously, here's what generally happens:

Handling Up to the Limit: The server will accept and start processing up to 5 of these requests concurrently, as that's the limit set by limit-concurrency.

Exceeding the Limit: For the requests that arrive after the limit is reached, their handling depends on the current state of the backlog queue and the server's overall capacity.

If the backlog queue is not full and can accommodate more pending connections, the additional requests (in your case, the remaining 5) will be queued in the backlog. They will be processed once one of the currently active requests is completed, and a slot opens up.

If the backlog is full (which would require a large number of pending connections, given the default is 2048), then new incoming connections will receive a 503 Service Unavailable response. This response indicates that the server is temporarily unable to handle the request due to overload.

In practice, for your scenario with a limit-concurrency of 5 and a default backlog of 2048, it's highly unlikely that a 503 error would be returned just because 10 requests come in simultaneously. The first 5 would be processed concurrently, and the next 5 would typically be queued in the backlog, waiting to be processed as soon as a slot becomes available.

503 errors become more likely in situations where the server is under significant, sustained load, and both the concurrent processing limit and the backlog capacity are fully utilized.

[siddhantoon]

Did an Experiment and found no use for backlog basically

Also that limit_concurrency has bug in implementation. when you set it to 10 it will take only 9 requests at a time.

[ddelange]

the PR that adds support for --backlog seems to just propagate the value to loop.create_server, with no additional implementation in uvicorn itself

[ddelange]

cc @Kludex does this warrant a new issue?

[ddelange]

gunicorn is clear:

--backlog

The maximum number of pending connections.

This refers to the number of clients that can be waiting to be served. Exceeding this number results in the client getting an error when attempting to connect. It should only affect servers under significant load.

Must be a positive integer. Generally set in the 64-2048 range.

[ddelange]

had another look:

looks like both gunivorn and uvicorn propagate the backlog value to socket.listen:
gunicorn directly to socket.listen
uvicorn as well but via loop.create_server

from python docs:

it specifies the number of unaccepted connections that the system will allow before refusing new connections

[ddelange]

Also that limit_concurrency has bug in implementation. when you set it to 10 it will take only 9 requests at a time.

@siddhantoon from tests it looks like this is intended.

[jearton]

Is there any conclusions?

[ddelange]

tl;dr

• --backlog is supposed to work on connection level (limit amount of unaccepted connections), but has no effect because the worker immediately accepts incoming connections.
• --limit-concurrency works on asyncio.Task level to reply with 503 upon overflow, and does not touch connections.

Findings

• It looks like --backlog is a no-op, it is supposed to work on connection level, but in practise I see no way to limit the amount of connections to a uvicorn worker.
  • In the MRE below, the worker accepts all incoming connections immediately, so no connection ever is in the backlog. I tested this with backlog=1 and backlog=0 (same results: no refused connections, all incoming connections are immediately accepted by the worker).
  • Both gunivorn and uvicorn propagate the backlog value to socket.listen:
    • gunicorn propagates directly to socket.listen
    • uvicorn as well but via loop.create_server
    • python docs for backlog:

      it specifies the number of unaccepted connections that the system will allow before refusing new connections

    • gunicorn docs for --backlog:

      The maximum number of pending connections.

      This refers to the number of clients that can be waiting to be served. Exceeding this number results in the client getting an error when attempting to connect. It should only affect servers under significant load.

      Must be a positive integer. Generally set in the 64-2048 range.

  • It looks to me like --backlog only has a function if uvicorn implements a gunicorn --worker-connections equivalent, assuming that this leaves incoming connections in the backlog when that limit is reached.
    • gunicorn docs for --worker-connections:

      The maximum number of simultaneous clients.

      This setting only affects the gthread, eventlet and gevent worker types.

• On asyncio.Task level, one can use the --limit-concurrency flag to limit the amount of concurrent requests.
  • This count includes the main thread: limit_concurrency=1 will block any incoming requests. With that flag, the worker will (after accepting the request) respond instantly with a 503 for any task that wants to acquire a slot when there is none.
    • uvicorn docs for --limit-concurrency:

      Maximum number of concurrent connections or tasks to allow, before issuing HTTP 503 responses. Useful for ensuring known memory usage patterns even under over-resourced loads.

  • This behaviour is also flaky: in the MRE below, sometimes 32-1 requests come through as expected. But many times, more than 1 request receives a 503 in both bomb_single_connection and bomb_separate_connections scenarios.
  • To quote @siminchengithub:

    The limit_concurrency check is done inside on_headers_complete after a connection is added and before a new task is added. This will work correctly to make sure any extra requests receive 503. However this won't make sure there are less than limit_concurrency connections. Connections can continue to pile up as long as they are not closed.

MRE

server.py

import fastapi, logging, asyncio, httpx

logging.basicConfig(level=logging.INFO, format="%(asctime)s:%(levelname)-7s %(filename)20s:%(lineno)-4d %(name)s:%(message)s")

app = fastapi.FastAPI()


@app.get("/")
async def read_root():
    await asyncio.sleep(2)
    return {"Hello": "World"}


async def bomb_single_connection(requests=32):
    """Send concurrent requests using HTTP connection pooling."""
    limits = httpx.Limits(max_keepalive_connections=0, max_connections=1)  # everything over 1 connection
    transport = httpx.AsyncHTTPTransport(retries=0)  # no retries on failed connections (to test --backlog)
    timeout = httpx.Timeout(None)  # no timeouts (also no timeout for waiting for a slot in the connection pool)
    async with httpx.AsyncClient(limits=limits, timeout=timeout, transport=transport) as client:
        # send all requests concurrently over the same connection
        await asyncio.gather(*[client.get('http://localhost:8001') for _ in range(requests)])

async def bomb_separate_connections(requests=32):
    """Send concurrent requests, each in their own connection."""
    await asyncio.gather(*[bomb_single_connection(requests=1) for _ in range(requests)])

terminal 1

$ uvicorn server:app --port 8001 --host 0.0.0.0 --backlog 0 --limit-concurrency 32
INFO:     Started server process [37996]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://0.0.0.0:8001 (Press CTRL+C to quit)


WARNING:  Exceeded concurrency limit.
INFO:     127.0.0.1:55433 - "GET / HTTP/1.1" 503 Service Unavailable
INFO:     127.0.0.1:55379 - "GET / HTTP/1.1" 200 OK
INFO:     127.0.0.1:55381 - "GET / HTTP/1.1" 200 OK
INFO:     127.0.0.1:55383 - "GET / HTTP/1.1" 200 OK
INFO:     127.0.0.1:55384 - "GET / HTTP/1.1" 200 OK
INFO:     127.0.0.1:55387 - "GET / HTTP/1.1" 200 OK
INFO:     127.0.0.1:55389 - "GET / HTTP/1.1" 200 OK
INFO:     127.0.0.1:55390 - "GET / HTTP/1.1" 200 OK
INFO:     127.0.0.1:55392 - "GET / HTTP/1.1" 200 OK
INFO:     127.0.0.1:55394 - "GET / HTTP/1.1" 200 OK
INFO:     127.0.0.1:55396 - "GET / HTTP/1.1" 200 OK
INFO:     127.0.0.1:55398 - "GET / HTTP/1.1" 200 OK
INFO:     127.0.0.1:55400 - "GET / HTTP/1.1" 200 OK
INFO:     127.0.0.1:55403 - "GET / HTTP/1.1" 200 OK
INFO:     127.0.0.1:55404 - "GET / HTTP/1.1" 200 OK
INFO:     127.0.0.1:55406 - "GET / HTTP/1.1" 200 OK
INFO:     127.0.0.1:55408 - "GET / HTTP/1.1" 200 OK
INFO:     127.0.0.1:55411 - "GET / HTTP/1.1" 200 OK
INFO:     127.0.0.1:55412 - "GET / HTTP/1.1" 200 OK
INFO:     127.0.0.1:55414 - "GET / HTTP/1.1" 200 OK
INFO:     127.0.0.1:55416 - "GET / HTTP/1.1" 200 OK
INFO:     127.0.0.1:55418 - "GET / HTTP/1.1" 200 OK
INFO:     127.0.0.1:55420 - "GET / HTTP/1.1" 200 OK
INFO:     127.0.0.1:55422 - "GET / HTTP/1.1" 200 OK
INFO:     127.0.0.1:55425 - "GET / HTTP/1.1" 200 OK
INFO:     127.0.0.1:55426 - "GET / HTTP/1.1" 200 OK
INFO:     127.0.0.1:55428 - "GET / HTTP/1.1" 200 OK
INFO:     127.0.0.1:55430 - "GET / HTTP/1.1" 200 OK
INFO:     127.0.0.1:55434 - "GET / HTTP/1.1" 200 OK
INFO:     127.0.0.1:55436 - "GET / HTTP/1.1" 200 OK
INFO:     127.0.0.1:55437 - "GET / HTTP/1.1" 200 OK
INFO:     127.0.0.1:55438 - "GET / HTTP/1.1" 200 OK


WARNING:  Exceeded concurrency limit.
INFO:     127.0.0.1:55523 - "GET / HTTP/1.1" 503 Service Unavailable
WARNING:  Exceeded concurrency limit.
WARNING:  Exceeded concurrency limit.
INFO:     127.0.0.1:55524 - "GET / HTTP/1.1" 503 Service Unavailable
INFO:     127.0.0.1:55525 - "GET / HTTP/1.1" 503 Service Unavailable
INFO:     127.0.0.1:55526 - "GET / HTTP/1.1" 200 OK
INFO:     127.0.0.1:55527 - "GET / HTTP/1.1" 200 OK
INFO:     127.0.0.1:55528 - "GET / HTTP/1.1" 200 OK
INFO:     127.0.0.1:55529 - "GET / HTTP/1.1" 200 OK
INFO:     127.0.0.1:55541 - "GET / HTTP/1.1" 200 OK
INFO:     127.0.0.1:55542 - "GET / HTTP/1.1" 200 OK
INFO:     127.0.0.1:55543 - "GET / HTTP/1.1" 200 OK
INFO:     127.0.0.1:55544 - "GET / HTTP/1.1" 200 OK
INFO:     127.0.0.1:55545 - "GET / HTTP/1.1" 200 OK
INFO:     127.0.0.1:55546 - "GET / HTTP/1.1" 200 OK
INFO:     127.0.0.1:55547 - "GET / HTTP/1.1" 200 OK
INFO:     127.0.0.1:55548 - "GET / HTTP/1.1" 200 OK
INFO:     127.0.0.1:55549 - "GET / HTTP/1.1" 200 OK
INFO:     127.0.0.1:55550 - "GET / HTTP/1.1" 200 OK
INFO:     127.0.0.1:55551 - "GET / HTTP/1.1" 200 OK
INFO:     127.0.0.1:55552 - "GET / HTTP/1.1" 200 OK
INFO:     127.0.0.1:55553 - "GET / HTTP/1.1" 200 OK
INFO:     127.0.0.1:55554 - "GET / HTTP/1.1" 200 OK
INFO:     127.0.0.1:55555 - "GET / HTTP/1.1" 200 OK
INFO:     127.0.0.1:55556 - "GET / HTTP/1.1" 200 OK
INFO:     127.0.0.1:55557 - "GET / HTTP/1.1" 200 OK
INFO:     127.0.0.1:55558 - "GET / HTTP/1.1" 200 OK
INFO:     127.0.0.1:55559 - "GET / HTTP/1.1" 200 OK
INFO:     127.0.0.1:55560 - "GET / HTTP/1.1" 200 OK
INFO:     127.0.0.1:55561 - "GET / HTTP/1.1" 200 OK
INFO:     127.0.0.1:55562 - "GET / HTTP/1.1" 200 OK
INFO:     127.0.0.1:55563 - "GET / HTTP/1.1" 200 OK
INFO:     127.0.0.1:55564 - "GET / HTTP/1.1" 200 OK
INFO:     127.0.0.1:55565 - "GET / HTTP/1.1" 200 OK

terminal 2

In [1]: %load_ext autotime  # https://pypi.org/project/ipython-autotime/
time: 359 µs (started: 2024-02-06 10:11:53 +01:00)
In [2]: from server import bomb_single_connection, bomb_separate_connections
time: 12.7 ms (started: 2024-02-06 10:12:02 +01:00)


In [3]: await bomb_single_connection(requests=32)
2024-02-06 10:13:21,781:INFO              _client.py:1729 httpx:HTTP Request: GET http://localhost:8001 "HTTP/1.1 503 Service Unavailable"
2024-02-06 10:13:23,723:INFO              _client.py:1729 httpx:HTTP Request: GET http://localhost:8001 "HTTP/1.1 200 OK"
2024-02-06 10:13:23,724:INFO              _client.py:1729 httpx:HTTP Request: GET http://localhost:8001 "HTTP/1.1 200 OK"
2024-02-06 10:13:23,725:INFO              _client.py:1729 httpx:HTTP Request: GET http://localhost:8001 "HTTP/1.1 200 OK"
2024-02-06 10:13:23,726:INFO              _client.py:1729 httpx:HTTP Request: GET http://localhost:8001 "HTTP/1.1 200 OK"
2024-02-06 10:13:23,727:INFO              _client.py:1729 httpx:HTTP Request: GET http://localhost:8001 "HTTP/1.1 200 OK"
2024-02-06 10:13:23,728:INFO              _client.py:1729 httpx:HTTP Request: GET http://localhost:8001 "HTTP/1.1 200 OK"
2024-02-06 10:13:23,732:INFO              _client.py:1729 httpx:HTTP Request: GET http://localhost:8001 "HTTP/1.1 200 OK"
2024-02-06 10:13:23,733:INFO              _client.py:1729 httpx:HTTP Request: GET http://localhost:8001 "HTTP/1.1 200 OK"
2024-02-06 10:13:23,734:INFO              _client.py:1729 httpx:HTTP Request: GET http://localhost:8001 "HTTP/1.1 200 OK"
2024-02-06 10:13:23,739:INFO              _client.py:1729 httpx:HTTP Request: GET http://localhost:8001 "HTTP/1.1 200 OK"
2024-02-06 10:13:23,740:INFO              _client.py:1729 httpx:HTTP Request: GET http://localhost:8001 "HTTP/1.1 200 OK"
2024-02-06 10:13:23,741:INFO              _client.py:1729 httpx:HTTP Request: GET http://localhost:8001 "HTTP/1.1 200 OK"
2024-02-06 10:13:23,745:INFO              _client.py:1729 httpx:HTTP Request: GET http://localhost:8001 "HTTP/1.1 200 OK"
2024-02-06 10:13:23,748:INFO              _client.py:1729 httpx:HTTP Request: GET http://localhost:8001 "HTTP/1.1 200 OK"
2024-02-06 10:13:23,750:INFO              _client.py:1729 httpx:HTTP Request: GET http://localhost:8001 "HTTP/1.1 200 OK"
2024-02-06 10:13:23,751:INFO              _client.py:1729 httpx:HTTP Request: GET http://localhost:8001 "HTTP/1.1 200 OK"
2024-02-06 10:13:23,755:INFO              _client.py:1729 httpx:HTTP Request: GET http://localhost:8001 "HTTP/1.1 200 OK"
2024-02-06 10:13:23,758:INFO              _client.py:1729 httpx:HTTP Request: GET http://localhost:8001 "HTTP/1.1 200 OK"
2024-02-06 10:13:23,760:INFO              _client.py:1729 httpx:HTTP Request: GET http://localhost:8001 "HTTP/1.1 200 OK"
2024-02-06 10:13:23,764:INFO              _client.py:1729 httpx:HTTP Request: GET http://localhost:8001 "HTTP/1.1 200 OK"
2024-02-06 10:13:23,770:INFO              _client.py:1729 httpx:HTTP Request: GET http://localhost:8001 "HTTP/1.1 200 OK"
2024-02-06 10:13:23,771:INFO              _client.py:1729 httpx:HTTP Request: GET http://localhost:8001 "HTTP/1.1 200 OK"
2024-02-06 10:13:23,776:INFO              _client.py:1729 httpx:HTTP Request: GET http://localhost:8001 "HTTP/1.1 200 OK"
2024-02-06 10:13:23,777:INFO              _client.py:1729 httpx:HTTP Request: GET http://localhost:8001 "HTTP/1.1 200 OK"
2024-02-06 10:13:23,779:INFO              _client.py:1729 httpx:HTTP Request: GET http://localhost:8001 "HTTP/1.1 200 OK"
2024-02-06 10:13:23,780:INFO              _client.py:1729 httpx:HTTP Request: GET http://localhost:8001 "HTTP/1.1 200 OK"
2024-02-06 10:13:23,781:INFO              _client.py:1729 httpx:HTTP Request: GET http://localhost:8001 "HTTP/1.1 200 OK"
2024-02-06 10:13:23,784:INFO              _client.py:1729 httpx:HTTP Request: GET http://localhost:8001 "HTTP/1.1 200 OK"
2024-02-06 10:13:23,785:INFO              _client.py:1729 httpx:HTTP Request: GET http://localhost:8001 "HTTP/1.1 200 OK"
2024-02-06 10:13:23,785:INFO              _client.py:1729 httpx:HTTP Request: GET http://localhost:8001 "HTTP/1.1 200 OK"
2024-02-06 10:13:23,786:INFO              _client.py:1729 httpx:HTTP Request: GET http://localhost:8001 "HTTP/1.1 200 OK"
time: 2.12 s (started: 2024-02-06 10:13:21 +01:00)


In [4]: await bomb_separate_connections(requests=32)
2024-02-06 10:16:26,863:INFO              _client.py:1729 httpx:HTTP Request: GET http://localhost:8001 "HTTP/1.1 503 Service Unavailable"
2024-02-06 10:16:26,865:INFO              _client.py:1729 httpx:HTTP Request: GET http://localhost:8001 "HTTP/1.1 503 Service Unavailable"
2024-02-06 10:16:26,865:INFO              _client.py:1729 httpx:HTTP Request: GET http://localhost:8001 "HTTP/1.1 503 Service Unavailable"
2024-02-06 10:16:28,864:INFO              _client.py:1729 httpx:HTTP Request: GET http://localhost:8001 "HTTP/1.1 200 OK"
2024-02-06 10:16:28,865:INFO              _client.py:1729 httpx:HTTP Request: GET http://localhost:8001 "HTTP/1.1 200 OK"
2024-02-06 10:16:28,866:INFO              _client.py:1729 httpx:HTTP Request: GET http://localhost:8001 "HTTP/1.1 200 OK"
2024-02-06 10:16:28,867:INFO              _client.py:1729 httpx:HTTP Request: GET http://localhost:8001 "HTTP/1.1 200 OK"
2024-02-06 10:16:28,868:INFO              _client.py:1729 httpx:HTTP Request: GET http://localhost:8001 "HTTP/1.1 200 OK"
2024-02-06 10:16:28,868:INFO              _client.py:1729 httpx:HTTP Request: GET http://localhost:8001 "HTTP/1.1 200 OK"
2024-02-06 10:16:28,869:INFO              _client.py:1729 httpx:HTTP Request: GET http://localhost:8001 "HTTP/1.1 200 OK"
2024-02-06 10:16:28,869:INFO              _client.py:1729 httpx:HTTP Request: GET http://localhost:8001 "HTTP/1.1 200 OK"
2024-02-06 10:16:28,872:INFO              _client.py:1729 httpx:HTTP Request: GET http://localhost:8001 "HTTP/1.1 200 OK"
2024-02-06 10:16:28,872:INFO              _client.py:1729 httpx:HTTP Request: GET http://localhost:8001 "HTTP/1.1 200 OK"
2024-02-06 10:16:28,873:INFO              _client.py:1729 httpx:HTTP Request: GET http://localhost:8001 "HTTP/1.1 200 OK"
2024-02-06 10:16:28,873:INFO              _client.py:1729 httpx:HTTP Request: GET http://localhost:8001 "HTTP/1.1 200 OK"
2024-02-06 10:16:28,874:INFO              _client.py:1729 httpx:HTTP Request: GET http://localhost:8001 "HTTP/1.1 200 OK"
2024-02-06 10:16:28,874:INFO              _client.py:1729 httpx:HTTP Request: GET http://localhost:8001 "HTTP/1.1 200 OK"
2024-02-06 10:16:28,875:INFO              _client.py:1729 httpx:HTTP Request: GET http://localhost:8001 "HTTP/1.1 200 OK"
2024-02-06 10:16:28,875:INFO              _client.py:1729 httpx:HTTP Request: GET http://localhost:8001 "HTTP/1.1 200 OK"
2024-02-06 10:16:28,876:INFO              _client.py:1729 httpx:HTTP Request: GET http://localhost:8001 "HTTP/1.1 200 OK"
2024-02-06 10:16:28,876:INFO              _client.py:1729 httpx:HTTP Request: GET http://localhost:8001 "HTTP/1.1 200 OK"
2024-02-06 10:16:28,879:INFO              _client.py:1729 httpx:HTTP Request: GET http://localhost:8001 "HTTP/1.1 200 OK"
2024-02-06 10:16:28,880:INFO              _client.py:1729 httpx:HTTP Request: GET http://localhost:8001 "HTTP/1.1 200 OK"
2024-02-06 10:16:28,880:INFO              _client.py:1729 httpx:HTTP Request: GET http://localhost:8001 "HTTP/1.1 200 OK"
2024-02-06 10:16:28,881:INFO              _client.py:1729 httpx:HTTP Request: GET http://localhost:8001 "HTTP/1.1 200 OK"
2024-02-06 10:16:28,881:INFO              _client.py:1729 httpx:HTTP Request: GET http://localhost:8001 "HTTP/1.1 200 OK"
2024-02-06 10:16:28,882:INFO              _client.py:1729 httpx:HTTP Request: GET http://localhost:8001 "HTTP/1.1 200 OK"
2024-02-06 10:16:28,882:INFO              _client.py:1729 httpx:HTTP Request: GET http://localhost:8001 "HTTP/1.1 200 OK"
2024-02-06 10:16:28,883:INFO              _client.py:1729 httpx:HTTP Request: GET http://localhost:8001 "HTTP/1.1 200 OK"
2024-02-06 10:16:28,883:INFO              _client.py:1729 httpx:HTTP Request: GET http://localhost:8001 "HTTP/1.1 200 OK"
2024-02-06 10:16:28,884:INFO              _client.py:1729 httpx:HTTP Request: GET http://localhost:8001 "HTTP/1.1 200 OK"
2024-02-06 10:16:28,884:INFO              _client.py:1729 httpx:HTTP Request: GET http://localhost:8001 "HTTP/1.1 200 OK"
time: 2.53 s (started: 2024-02-06 10:16:26 +01:00)

(empty lines added for readability)

[black-snow]

After two years this is still open, right? Just tested it.
And it's still unclear why this is now a discussion instead of an issue.

[ddelange]

correct cc @Kludex