Questions regarding peer discovery in libp2p

[finlaydotb]

As can be seen from my recent posting here I have been tinkering with libp2p and especially trying to wrap my head around the discovery mechanism.

Only problem is, I have a couple of questions which I cannot find answers to into the documentations.

First observation is that the documentations all seems to be incomplete or outdated :( For example in the specification repository, only contains info for mdns as can be seen here

The section of the website that lists all the discovery mechanism supported by each implementation is also outdated, as can be seen here

I also found this README that outlined some of the discovery mechanisms. It does not look exhaustive.

So I am left with more questions than answers hence this post.

To the questions:

• Is there a place to see a list of the supported libp2p discovery mechanisms, per implementations?
• Will I be correct to say that discovery mechanisms like bootstrap or mdns are non-iterative. By non-iterative I mean, in the case of bootstraping, only connect to the node listed and stops, or in case of mdns only connect to nodes found on the same LAN. So effectively It does not offer any mechanism where new peers can be discovered from the initial peers first connected to?
• On a high level how does the webrtc-star discovery mechanism works?
• On a high level, what is the rendezvous discovery mechanism. I see only implementation in Rust here
• Is Gossip Sub also a discovery mechanism?
• can a node discover and connect another peer even though they don't support the same protocol? Or a node can discover another peer regardless of protocol, just that connection won't be established?
• What is the process that ensures a node only discover and connect to peers that are participating in a specific network? For example if a node only needs to connect to ethereum nodes? what is the mechanism that can be used to ensure this? Controlling the bootstrap nodes? Or something else?
• What is the general idea of Peer Routing? Unfortunately there is no article on it yet in the docs as can be seen here. Does Peer routing work with DHT? if so which ones? If not how does it work?

I continue tinkering and hopefully find answers to some of these questions myself...but will also appreciate if someone more familiar with the libp2p project can help provide answers/guidance

[mxinden]

Is there a place to see a list of the supported libp2p discovery mechanisms, per implementations?

You are right. We are not doing a good job. Ideally I would simply like to track on each specification which implementation implements the spec. This data can then e.g. fill https://libp2p.io/bundles/. Let me know in case you would like to help with that.

Will I be correct to say that discovery mechanisms like bootstrap or mdns are non-iterative. By non-iterative I mean, in the case of bootstraping, only connect to the node listed and stops, or in case of mdns only connect to nodes found on the same LAN. So effectively It does not offer any mechanism where new peers can be discovered from the initial peers first connected to?

Not sure what you are referring with bootstrap in rust-libp2p. mdns works on the local lan only. As long as all peers are within the local lan, all peers should discover all other peers after a small delay.

On a high level how does the webrtc-star discovery mechanism works?

To be honest, I am surprised it is listed as a discovery mechanism. Also note, that we are redesigning libp2p's way of doing webrtc libp2p/specs#412

On a high level, what is the rendezvous discovery mechanism. I see only implementation in Rust here

Is https://github.com/libp2p/specs/tree/master/rendezvous of some help?

Is Gossip Sub also a discovery mechanism?

No. It has the notion of peer exchange though that still depends on an external discovery mechanism.

can a node discover and connect another peer even though they don't support the same protocol? Or a node can discover another peer regardless of protocol, just that connection won't be established?

E.g. in the case of Kademlia, a node speaking only TCP can discovery a node only speaking QUIC. Though as you said, they won't be able to connect.

What is the process that ensures a node only discover and connect to peers that are participating in a specific network? For example if a node only needs to connect to ethereum nodes? what is the mechanism that can be used to ensure this? Controlling the bootstrap nodes? Or something else?

The identify protocol has a protocolVersion field. https://github.com/libp2p/specs/tree/master/identify

What is the general idea of Peer Routing? Unfortunately there is no article on it yet in the docs as can be seen here. Does Peer routing work with DHT? if so which ones? If not how does it work?

Peer routing would be using Kademlia https://github.com/libp2p/specs/blob/master/kad-dht/README.md

I continue tinkering and hopefully find answers to some of these questions myself...but will also appreciate if someone more familiar with the libp2p project can help provide answers/guidance

Let me know if the above is of some help.

[andrewdavidmackenzie]

Thanks for posting this. I had a very similar list of questions. Thanks for the response too. More info in the docs on those capabilities and when to use which for a given purpose would be of enormous help.

In some docs I read it mentioned protocol (a d I understood the programmer defined higher level protocol) and I'm not sure if the questions refers to that rather than TCP/QUIC.

I would like to discover peers who "opt-in" to a given network (related to my distributed app) and implement a given protocol (offer certain capabilities for peers), later they might need to share data between them on demand (content addressable?).

I guess mDNS is a good start as I won't be going outside my subnet (at least at the start), but I would like to know the options for finding peers on my network outside the subnet later on.

So, I'm looking for something similar to the author I think.

Andrew

[mxinden]

More info in the docs on those capabilities and when to use which for a given purpose would be of enormous help.

Unfortunately we don't have much capacity to write these. Contributions are very much appreciated.

I would like to discover peers who "opt-in" to a given network (related to my distributed app) and implement a given protocol (offer certain capabilities for peers)

Peers advertise what they provide via the identify protocol. Also you can use rendezvous protocol to search for peers supporting a specific protocol / capability.

Let me know if this is of some help @andrewdavidmackenzie.

[andrewdavidmackenzie]

Thanks for the hints, definitely helpful. I will go off and read those docs. Currently no rust examples of using them no?

[thomaseizinger]

The individual protocols also have examples in examples/.

[andrewdavidmackenzie]

I've used and explored the examples/. It looks like the ipfs-private example uses Identify a bit, but I don't see any using Rendezvous.

[thomaseizinger]

I was talking about the individual protocol crates :) See protocols/rendevouz/examples: https://github.com/libp2p/rust-libp2p/tree/master/protocols%2Frendezvous%2Fexamples

[andrewdavidmackenzie]

Aha! Thanks, I'd never have thought to look in there for samples. I see there is one in rendezvous. For Identify I will study the general example. Thanks!

[thomaseizinger]

All the protocols are individual crates and they follow the cargo conventions of src/, examples/ and tests/ :)

[andrewdavidmackenzie]

Obviously when examples and tests exist! :-) e.g. https://github.com/libp2p/rust-libp2p/tree/master/protocols/identify

[thomaseizinger]

Obviously when examples and tests exist! :-) e.g. https://github.com/libp2p/rust-libp2p/tree/master/protocols/identify

There is room for improvement 😁