chore: bump the pubsub validation queue length to 256#13176
Conversation
There was a problem hiding this comment.
Pull Request Overview
This PR bumps the pubsub validation queue length to 4096 to better handle the increased rate of gossipsub message exchanges driven by F3. The changes include updating the pubsub instantiation in the code and adding an entry to the CHANGELOG.
- Updated the pubsub module to use a validation queue size of 4096.
- Updated the CHANGELOG to document the change.
Reviewed Changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| node/modules/lp2p/pubsub.go | Extended the pubsub options to include a higher validation queue size. |
| CHANGELOG.md | Added an entry documenting the change to the validation queue size. |
|
Many thanks to @dennis-tra at Probelab for pointing this out. |
|
Marked as draft to look into max message size (default 1MiB in go-libp2p-pubsub) unconfigured in Lotus. |
|
We know from Dennis that 2048 would be effective but not as much as we ideally want. Not sure what size would be a good compromise. |
|
Great @masih! I know from our Hermes deployment that 2048 still results in some discarded messages, so bumping to 4096 sounds reasonable to make sure that we don't miss any. |
|
@dennis-tra FYI, we optimised the validator (it could take a while to process the first message of an instance) in more recent go-f3 commits and lotus (go-f3 is not yet merged), so we expect maybe that to help. |
|
I'm at
I can't remember exactly but I think the validation functions weren't exposed back when I worked on that.
I can't see an open PR, so what's the latest state regarding the validation logic that I should look into? |
|
Here is Lotus with latest go-f3 #13144 |
0248ff0 to
f05d017
Compare
Increase the pubsub validation queue length to 256, to accommodate the increased rate of message exchange as a result of F3. Note that there's no topic-specific option for this setting, hence the bump at top-level pubsub instantiation. Relates to: libp2p/go-libp2p-pubsub#621
ebff113 to
089e886
Compare
|
@Kubuxu I ran a bunch of experiments on our test node and it seems like 256 is the smallest queue size with almost no impact on validation queue getting full. observed a dozen of such instances while running the test for ~30 minutes. I think this is an acceptable level of message loss and a massive improvement from the default of 32. |
|
For completeness if maintainers want the full context, this started out of an internal discussion in https://filecoinproject.slack.com/archives/C08715YFQH5/p1749815281170279 |
Increase the pubsub validation queue length to 256, to accommodate the increased rate of message exchange as a result of F3. Note that there's
no topic-specific option for this setting, hence the bump at top-level
pubsub instantiation.
Relates to: libp2p/go-libp2p-pubsub#621