PutValue context not utilized or propagated into bootstrapping

[aschmahmann]

PubsubValueStore.PutValue receives a context.Context but does not respect it. In particular, we have:

go-libp2p-pubsub-router/pubsub.go

Line 92 in 2c8ac20

bootstrapPubsub(p.ctx, p.cr, p.host, topic)

which uses the PubsubValueStore's context instead of the PutValue context when calling the very expensive bootstrapPubsub function.

Additionally, bootstrapPubsub could be much less expensive if:

go-libp2p-pubsub-router/pubsub.go

Line 345 in 2c8ac20

err := cr.Provide(ctx, rz, true)

was moved into the go func right below it.

After speaking with @Stebalien my current plan is to start by moving the Provide into the go func, and then look into moving bootstrapping into PubSub via the Discovery (https://github.com/libp2p/go-libp2p-discovery) interface, as indicated by:

go-libp2p-pubsub-router/pubsub.go

Line 338 in 2c8ac20

// Note: rendezvous/boostrap should really be handled by the pubsub implementation itself!

@vyzo @raulk Any thoughts or issues with this?

[vyzo]

Not so sure about moving bootstrap into pubsub, but I won't reject a good patch :)

[Stebalien]

Not so sure about moving bootstrap into pubsub, but I won't reject a good patch :)

My thinking is that there should be a pubsub option that passes in a discovery service. If provided, pubsub would:

1. Auto-discover peers interested in the pubsub topic iff we don't already have some.
2. Re-bootstrap pubsub topics if we run out of connected peers.
3. Announce the pubsub topic periodically while subscribed.

1 and 2 are difficult to do reliably outside pubsub.

[vyzo]

ok

[vyzo]

We might want to connect the bootstrap process with the routers as well, so that they have a say in whether more peers are needed.

[aschmahmann]

Any thoughts as to whether we should be maintaining/running a peer discovery service if we've Published to the topic but not Subscribed? Two of our options are:

1. Run a FindPeers query before sending each message
2. Keep an ongoing list of peers for a given topic as if we were subscribed to it

[Stebalien]

Ideally, we wouldn't block publishing to run publish. Should we provide a separate bootstrap method? Or something that returns a bootstrap handle (subset of subscribe functionality)?

@vyzo how does gossipsub handle publishing on topics we're not subscribed to?

[aschmahmann]

@Stebalien running a blocking FindPeers query before publishing feels pretty intentional, especially for a first time publish. For example, if you haven't yet found any peers for your topic you probably don't want to send out a message yet. This is especially true in the non-persistent forms of PubSub since once you send the message out it's game over and if the message didn't make it to a node connected to the bulk of the network your message will get swallowed. If we'd like to run Publish and have it perform this blocking in some external gofunc we can do that, but then we'd probably want some PublishAsync function we can call instead.

@vyzo please correct me if I'm wrong, but I think I can give a stab at this question. Regarding gossipsub and publishing topics we're not subscribed to, PubSub itself does nothing special it just calls Publish on the routers. The routers themselves use PubSub's topics map[string]map[peer.ID]struct{} field to discover relevant peers. This field is populated by handling incoming Subscribe RPCs, which occur either when a node newly subscribes to a topic or when a node learns about a new node for the first time. PubSub learns about new nodes by listening to Connected events from libp2p's Notifiee interface.

• FloodSub: Uses PubSub.topics to send to all of the peers subscribed to the topic
• RandomSub: Uses PubSub.topics to send to a random subset of the peers subscribed to the topic
• GossipSub: Uses PubSub.topics to select a random subset of the peers subscribed to the topic and stores them as fanout peers. These fanout peers are used for sending out messages in the future instead of selecting new random peers each time. Periodically the fanout peers are checked to see if we still have enough of them and if we don't then we randomly add more.

It's worth noting the GossipSub is the only one of these implementations in which a node cares at all about whether it is Subscribed or not, as seen by the fact that it is the only implementation where the Join function is non-empty.

[Stebalien]

running a blocking FindPeers query before publishing feels pretty intentional, especially for a first time publish. For example, if you haven't yet found any peers for your topic you probably don't want to send out a message yet.

I kind of agree but we need to make sure to not block forever if, e.g., we're offline.

[vyzo]

@aschmahmann this is correct, we don't do anything special other than using the existing peers on publish.

I kind of agree but we need to make sure to not block forever if, e.g., we're offline

We should have a timeout in the Provide.

[aschmahmann]

We should have a timeout in the Provide.

Given that operations like Provide and the Discovery interfaces all take context.Contexts we could just handle the timeouts at that level. For example, the PubSubValueRouter in this repo has all the mechanisms to perform timeouts, they're just not being properly utilized.

However, if we'd like to add some context passing into PubSub functions as well that seems reasonable. I'm just not convinced we should hard code any particular timeouts into PubSub itself.

[Stebalien]

@aschmahmann this has been fixed, right?

[aschmahmann]

@Stebalien not really. We followed through on the suggestion to make the Content Routing/DHT Provide in the bootstrapping process non-blocking (see #29) which makes this issue much less painful. However, the rest of the bootstrapping operation still does not respect the PutValue context.

My understanding was that instead of immediately tackling this by changing the issue in this repo that we would put together a PR to move the bootstrapping into the PubSub itself via the go-libp2p-discovery interfaces. Once that's done then propagating the context in the PubSubValueRouter should be fairly straightforward.