Performance degrades under high load because of ConnectionNotAvailable exceptions

[kramar11]

Hi,

I'm using httpcore in a project and while doing some performance tests with locust, I noticed, that the performance degraded under very high load. After digging, I found, that the AsyncConnectionPool may (and under load will!) assign connections to more than one pool_requests, which then results in ConnectionNotAvailable Exceptions (lots of them) which lowers the performance as the pool_request needs again to wait for an available connection.

I then made a simple modification in _assign_requests_to_connections so that already assigned connections are not considered as available, which greatly increased performance and totally got rid of ConnectionNotAvailable Exceptions.

    def _assign_requests_to_connections(self) -> list[AsyncConnectionInterface]:
        ...
        # Assign queued requests to connections.
        queued_requests = [request for request in self._requests if request.is_queued()]
        assigned_connections = [ request.connection for request in self._requests if request.connection is not None ]
        for pool_request in queued_requests:
            origin = pool_request.request.url.origin
            available_connections = [
                connection
                for connection in self._connections
                if connection not in assigned_connections and connection.can_handle_request(origin) and connection.is_available()
            ]
        ...

Note: Only the 2 lines with the assigned_connections var were added/changed.

My test results:

-	without modifications	with above modifications
# Requests	51739	69643
Average (ms)	42.64	31.62
RPS	431	580

Don't know if this is the right fix to avoid the ConnectionNotAvailable Exceptions but it increases high load performance by more than 30 percent. I think it could be reasonable to keep two lists of connections, those which are available and those which are assigned - and move assigned connections accordingly.

What do you think?