refactor: generalize NetworkProcessor awaiting-block mechanism for all gossip types - #9059
Merged
Conversation
…ll gossip messages - Rename `searchUnknownSlotRoot` → `searchUnknownBlock` across network interface - Flatten `awaitingGossipsubMessagesByRootBySlot` (MapDef of MapDef) into `awaitingBlockByRoot` (single MapDef) for simpler lookups - Rename `unknownRootsBySlot` → `unknownBlocksBySlot` for clarity - Rename metrics `reprocessGossipAttestations` → `awaitingBlockGossipMessages` with added `topic` label - Add `BlockInputSource.network_processor` to label messages queued internally - Update dashboard panels to use new metric names and topic breakdowns Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Contributor
Summary of ChangesHello, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request refactors the Highlights
🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here. Footnotes
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Code Review
This pull request does a great job of generalizing the awaiting-block mechanism in the NetworkProcessor, which is a solid step towards supporting more gossip types. The refactoring simplifies the data structures by flattening the awaiting messages map and improves metrics by adding more granularity with the topic label. The code is cleaner and more maintainable as a result. I've found one potential issue where the generic message age check might be too strict for Deneb attestations, possibly leading to the rejection of valid messages. Please see my detailed comment.
Contributor
Performance Report✔️ no performance regression detected Full benchmark results
|
twoeths
marked this pull request as ready for review
March 18, 2026 07:36
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a142e115e7
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
nflaig
reviewed
Mar 20, 2026
| name: "lodestar_awaiting_block_gossip_messages_wait_time_reject_seconds", | ||
| help: "Time to wait for unknown block before being rejected", | ||
| labelNames: ["reason"], | ||
| labelNames: ["reason", "topic"], |
Member
There was a problem hiding this comment.
I suspect this is fine since it's a gauge, but flagging the 2 labels here
ensi321
reviewed
Mar 20, 2026
ensi321
left a comment
ensi321
left a comment
Member
There was a problem hiding this comment.
There are stale comments in createExtractBlockSlotRootFns. Need to update that too.
ensi321
reviewed
Mar 20, 2026
| for (const [slot, roots] of this.unknownBlocksBySlot) { | ||
| if (slot > minSlot) continue; | ||
| for (const rootHex of roots) { | ||
| const gossipMessagesByRoot = this.awaitingBlockByRoot.get(rootHex); |
Member
There was a problem hiding this comment.
gossipMessagesByRoot is a set of gossip messages. The naming suggests it is a map but it is not. Consider gossipMessages or something similar
| metrics.gossipValidationQueue.concurrency.set({topic}, this.gossipTopicConcurrency[topic]); | ||
| } | ||
| metrics.reprocessGossipAttestations.countPerSlot.set(this.unknownBlockGossipsubMessagesCount); | ||
| metrics.awaitingBlockGossipMessages.countPerSlot.set(this.unknownBlockGossipsubMessagesCount); |
Member
There was a problem hiding this comment.
Essentially this metric is just to track total number of messages sitting in awaitingBlockByRoot. I find having a variable unknownBlockGossipsubMessagesCount and we increment/decrement in several places is error prone.
Can we add a function and just track this instead?
private get unknownBlockGossipsubMessagesCount(): number {
let count = 0;
for (const messages of this.awaitingBlockByRoot.values()) {
count += messages.size;
}
return count;
}
ensi321
approved these changes
Mar 23, 2026
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## unstable #9059 +/- ##
============================================
+ Coverage 52.32% 52.36% +0.04%
============================================
Files 848 848
Lines 62326 62234 -92
Branches 4572 4572
============================================
- Hits 32612 32589 -23
+ Misses 29649 29580 -69
Partials 65 65 🚀 New features to boost your workflow:
|
lodekeeper
added a commit
to lodekeeper/lodestar
that referenced
this pull request
Mar 23, 2026
Resolve 1 conflict in network/processor/index.ts: - PR ChainSafe#9059 refactored awaiting-block mechanism for all gossip types - Keep unstable's refactored structure, add back ePBS execution_payload bypass
5 tasks
Member
|
🎉 This PR is included in v1.42.0 🎉 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
This PR is a step toward #9025 where we enhance
NetworkProcessorfor GLOAS.The network processor's "await unknown block" mechanism was originally built specifically for attestation messages. As we add support for more gossip types that depend on block availability (and eventually payload envelopes in GLOAS), the code needs to be generalized before those new types are introduced.
Description
Generalizes the network processor's unknown-block-waiting mechanism from attestation-specific to all gossip message types:
searchUnknownSlotRoot→searchUnknownBlockacrossINetwork,Network,NetworkProcessor, and all call sitesawaitingGossipsubMessagesByRootBySlot: MapDef<Slot, MapDef<RootHex, Set<...>>>into a singleawaitingBlockByRoot: MapDef<RootHex, Set<...>>, using a separateunknownBlocksBySlotmap for slot-based pruningreprocessGossipAttestations→awaitingBlockGossipMessages; add atopiclabel so per-topic breakdown is visible in dashboardsBlockInputSource.network_processorto distinguish messages queued by the processor itself{{topic}}legend formatAI Assistance Disclosure
Used Claude Code to assist with implementation.