Connection management

[tthebst]

Hey!

I'm trying to better understand the concept of connections in libp2p. Few questions:

• Libp2p allows having many connections to the same peer, but what is the benefit of having multiple connections? Isn't one connection enough if your application runs on a single port and there only is one multiaddr for one peer?
• Does it makes sense to set with_max_established_per_peer to one?
• If you have multiple connections open to peer and you send a message (for example req/resp) how does libp2p decide which connection to use?
• Is there one connection handler task per connection per behavior or one connection handler per connection?
• With recent changes, it seems possible to write a "Peer Manager" network behaviour that does connection management (and not go through the swarm). Is this recommended?

[thomaseizinger]

Hey!

I'm trying to better understand the concept of connections in libp2p. Few questions:

All excellent questions!

* Libp2p allows having many connections to the same peer, but what is the benefit of having multiple connections? Isn't one connection enough if your application runs on a single port and there only is one multiaddr for one peer?

Technically, it is enough. Practically, you can't avoid it. How would you decide which connection to keep if two peers dial each other at the same time?

* Does it makes sense to set [with_max_established_per_peer](https://docs.rs/libp2p/latest/libp2p/swarm/struct.ConnectionLimits.html#method.with_max_established_per_peer) to one?

This function is deprecated in master already, see #3386. You can set it to 1 but may experience weird behaviour because both peers may close a different connection and you end up with none.

* If you have multiple connections open to peer and you send a message (for example req/resp) how does libp2p decide which connection to use?

A NetworkBehaviour can decide which connection to send a message to: https://docs.rs/libp2p/latest/libp2p/swarm/enum.NotifyHandler.html

The request-respones behaviour in particular uses a round-robin approach across all active connections:

rust-libp2p/protocols/request-response/src/lib.rs

Line 538 in 3f99c31

let ix = (request.request_id.0 as usize) % connections.len();

* Is there one connection handler task per connection per behavior or one connection handler per connection?

Behaviours are organised as "slices", i.e. they all share all connections. Each connection has a connection handler which is again composed out of all the handlers of all behaviours. It is one task per connection but the handlers are composed.

* With recent changes, it seems possible to write a "Peer Manager" network behaviour that does connection management (and not go through the swarm). Is this recommended?

Yes, see:

• feat: introduce libp2p-connection-limits connection management module #3386
• feat: introduce libp2p-allow-block-list connection management module #3590

[tthebst]

Thanks for the response!

So is there any benefit of having more than one or two open connections?

[mxinden]

In general, there is no benefit of having more than one connection between the same set of two peers.

On TCP one could argue that multiple connections prevent head-of-line blocking between two concurrent streams. Though I would consider that a potential optimization worth measuring first.

QUIC does not have head-of-line blocking on the connection level, thus I would not expect there to be any gains from having more than one connection between two peers.

Happy to go into detail @tthebst. Let me know if this is helpful.

[rkuhn]

There is a benefit to allowing a second connection: how else could the dialled peer validate that it can connect back to the dialling peer? Curating known-good addresses is a difficult and important part of p2p systems.

[mxinden]

True. My comment above only addressed "So is there any benefit of having more than one or two [long lived] open connections?"

Agreed that there is value in testing both directions. The second connection (in the reverse direction) can be short lived.