Inconsistency observed between network_info stats and connection limits being hit

[sandreae]

I'm observing, what i feel to be, inconsistencies between the stats returned from nework_info and the behaviour around rejecting inbound connections because of connection limits being hit. Here is logging from within IncomingConnectionError on swarm:

[2023-06-09T09:27:06Z WARN  aquadoggo::network::service] IncomingConnectionError: /ip4/0.0.0.0/udp/2022/quic-v1 /ip4/192.168.1.87/udp/2024/quic-v1 Denied { cause: ConnectionDenied { inner: Exceeded { limit: 8, kind: PendingIncoming } } }
[2023-06-09T09:27:06Z DEBUG aquadoggo::network::service] NetworkInfo {
        num_peers: 1,
        connection_counters: ConnectionCounters {
            limits: ConnectionLimits {
                max_pending_incoming: None,
                max_pending_outgoing: None,
                max_established_incoming: None,
                max_established_outgoing: None,
                max_established_per_peer: None,
                max_established_total: None,
            },
            pending_incoming: 1,
            pending_outgoing: 0,
            established_incoming: 1,
            established_outgoing: 0,
        },
    }

Here's the default network configuration we're using:

impl Default for NetworkConfiguration {
    fn default() -> Self {
        Self {
            autonat: false,
            dial_concurrency_factor: 8,
            max_connections_in: 16,
            max_connections_out: 16,
            max_connections_pending_in: 8,
            max_connections_pending_out: 8,
            max_connections_per_peer: 8,
            mdns: false,
            notify_handler_buffer_size: 128,
            per_connection_event_buffer_size: 8,
            ping: false,
            quic_port: 2022,
            relay_address: None,
            relay_peer_id: None,
            relay_server_enabled: false,
            remote_peers: Vec::new(),
            rendezvous_address: None,
            rendezvous_peer_id: None,
            rendezvous_server_enabled: false,
            peer_id: None,
        }
    }
}

I wonder if there's any way connection_limits::Behaviour can get out of sync with the general network state?

If anyone has has ideas on where/how i can explore this further I'd very much appreciate any suggestions.

[sandreae]

We're using version 0.51.3 and I see ConnectionLimits is deprecated here in an upcoming version so maybe that's related.

[thomaseizinger]

Do you still experience the inconsistency if you don't use the deprecated functionality?

[sandreae]

I updated to use the current main branch and still experience the same behaviour.

It may be that we're not following correct patterns for closing connections, this is something we're experimenting with now, of course for this observing the current network state is very useful. Is there anyway to get a report on what current state of connection_limits::Behaviour is?

[thomaseizinger]

Is there anyway to get a report on what current state of connection_limits::Behaviour is?

Not at the moment. The state is very simple so I am not sure how useful it would be to observe. I'd accept a PR that adds trace logs.

[sandreae]

This behaviour turned out to be caused by some incorrect logic in our handler implementation. I'll close now, thanks again for your comments.

[mxinden]

Thanks for the update here @sandreae.