Add `--semi-supernode` support #8254

jimmygchen · 2025-10-21T05:27:00Z

Issue Addressed

Addresses #8218

A simplified version of #8241 for the initial release.

I've tried to minimise the logic change in this PR, although introducing the NodeCustodyType enum still result in quite a bit a of diff, but the actual logic change in CustodyContext is quite small.

The main changes are in the CustdoyContext struct

combining validator_custody_count and current_is_supernode fields into a single custody_group_count_at_head field. We persist the cgc of the initial cli values into the custody_group_count_at_head field and only allow for increase (same behaviour as before).
I noticed the above approach caused a backward compatibility issue, I've made a fix and changed the approach slightly (which was actually what I had originally in mind):
- when initialising, only override the validator_custody_count value if either flag --supernode or --semi-supernode is used; otherwise leave it as the existing default 0. Most other logic remains unchanged.

All existing validator custody unit tests are still all passing, and I've added additional tests to cover semi-supernode, and restoring CustodyContext from disk.

Note: I've added a WARN if the user attempts to switch to a --semi-supernode or --supernode - this currently has no effect, but once @eserilev column backfill is merged, we should be able to support this quite easily.

Things to test

cgc in metadata / enr
cgc in metrics
subscribed subnets
getBlobs endpoint

Co-authored-by: pawanjay176 <[email protected]>

Squashed commit of the following: commit 7767a2e Author: Jimmy Chen <[email protected]> Date: Tue Oct 21 16:47:40 2025 +1100 More test fixes and update help text. commit 0932f36 Author: Jimmy Chen <[email protected]> Date: Tue Oct 21 16:14:46 2025 +1100 Add tests for restoring custody context from persisted. commit 5cc3186 Author: Jimmy Chen <[email protected]> Date: Tue Oct 21 16:07:47 2025 +1100 Implement semi-supernode. Co-authored-by: pawanjay176 <[email protected]>

… validator should have `validator_custody_at_head` set to 0.

jimmygchen · 2025-10-21T12:25:53Z

beacon_node/src/config.rs

+    let is_semi_supernode = parse_flag(cli_args, "semi-supernode");
+
+    client_config.chain.node_custody_type = if is_supernode {
+        client_config.network.subscribe_all_data_column_subnets = true;


I've just noticed subscribe_all_data_column_subnets is redundant in network config, because NetworkGlobals already has the sampling columns when initialised, but I can do it in a follow up PR.

Seems pretty straight forward to remove: jimmygchen@e623eb6

Squashed commit of the following: commit 45a4315 Author: Jimmy Chen <[email protected]> Date: Tue Oct 21 23:15:58 2025 +1100 Add tests for flags commit 15569bc Author: Jimmy Chen <[email protected]> Date: Tue Oct 21 22:40:49 2025 +1100 Revert some changes to preserve existing behaviour. Full node with no validator should have `validator_custody_at_head` set to 0. commit 7767a2e Author: Jimmy Chen <[email protected]> Date: Tue Oct 21 16:47:40 2025 +1100 More test fixes and update help text. commit 0932f36 Author: Jimmy Chen <[email protected]> Date: Tue Oct 21 16:14:46 2025 +1100 Add tests for restoring custody context from persisted. commit 5cc3186 Author: Jimmy Chen <[email protected]> Date: Tue Oct 21 16:07:47 2025 +1100 Implement semi-supernode. Co-authored-by: pawanjay176 <[email protected]>

jimmygchen · 2025-10-21T13:08:18Z

Some devnet-3 testing:

All three types of nodes able to sync to head
Sampling subnets correctly set
Gossip topics correctly subscribed
Able to fetch blobs from supernode and semi-supernode

mergify · 2025-10-21T13:52:42Z

Some required checks have failed. Could you please take a look @jimmygchen? 🙏

…mple

mergify · 2025-10-22T01:04:01Z

This pull request has been removed from the queue for the following reason: conflict with pull request ahead.

The pull request conflicts with at least one pull request ahead in queue.

There is nothing you can do for now. If the pull request ahead in the queue is merged, this pull request will become conflicting and you'll have to update it.
If the pull request ahead is not merged, you can requeue this pull request with a @mergifyio requeue comment.

michaelsproul · 2025-10-22T02:22:03Z

Are we waiting for custody backfill to merge before merging this? Don't we want to add the feature to start custody backfill when the node changes to a semi-supernode/supernode?

eserilev · 2025-10-22T03:40:09Z

if a node's cgc increases on start up, we just need two things for custody backfill to do its thing

data column custody info needs to be updated to reflect the cgc change
CustodyContext::validator_registrations::epoch_validator_custody_requirements needs to be updated to reflect the cgc change

…mple

jimmygchen · 2025-10-22T04:17:04Z

@mergify requeue

mergify · 2025-10-22T04:17:10Z

requeue

✅ The queue state of this pull request has been cleaned. It can be re-embarked automatically

jimmygchen · 2025-10-22T04:24:43Z

Are we waiting for custody backfill to merge before merging this? Don't we want to add the feature to start custody backfill when the node changes to a semi-supernode/supernode?

Oh yeah I've discussed with Pawan and going to make a follow up PR so it's bit easier to review
#8254 (comment)

Open PRs to include for the release - #7907 - #8247 - #8251 - #8253 - #8254 - #8265 - #8269 - #8266 Co-Authored-By: Jimmy Chen <[email protected]> Co-Authored-By: Jimmy Chen <[email protected]>

…rk (#8259) Addresses this comment: #8254 (comment) We're currently using `subscribe_all_data_column_subnets` here to subscribe to all subnets https://github.com/sigp/lighthouse/blob/522bd9e9c6ac167f2231525e937c9ebbcb86cf6e/beacon_node/lighthouse_network/src/types/topics.rs#L82-L92 But its unnecessary because the else path also works for supernode (uses `sampling_subnets` instead) The big diffs will disappear once #8254 is merged. Co-Authored-By: Jimmy Chen <[email protected]>

…rk (sigp#8259) Addresses this comment: sigp#8254 (comment) We're currently using `subscribe_all_data_column_subnets` here to subscribe to all subnets https://github.com/sigp/lighthouse/blob/522bd9e9c6ac167f2231525e937c9ebbcb86cf6e/beacon_node/lighthouse_network/src/types/topics.rs#L82-L92 But its unnecessary because the else path also works for supernode (uses `sampling_subnets` instead) The big diffs will disappear once sigp#8254 is merged. Co-Authored-By: Jimmy Chen <[email protected]>

jimmygchen requested a review from jxs as a code owner October 21, 2025 05:27

Implement semi-supernode.

5cc3186

Co-authored-by: pawanjay176 <[email protected]>

jimmygchen force-pushed the semi-supernode-simple branch 2 times, most recently from 0ffed11 to a8580d0 Compare October 21, 2025 05:36

Add tests for restoring custody context from persisted.

0932f36

jimmygchen force-pushed the semi-supernode-simple branch from a8580d0 to 0932f36 Compare October 21, 2025 05:39

More test fixes and update help text.

7767a2e

jimmygchen added ready-for-review The code is ready for review v8.0.0 Q4 2025 Fusaka Mainnet Release labels Oct 21, 2025

jimmygchen mentioned this pull request Oct 21, 2025

Release v8.0.0 rc.2 #8255

Merged

mergify bot added waiting-on-author The reviewer has suggested changes and awaits thier implementation. and removed ready-for-review The code is ready for review labels Oct 21, 2025

Revert some changes to preserve existing behaviour. Full node with no…

15569bc

… validator should have `validator_custody_at_head` set to 0.

jimmygchen added ready-for-review The code is ready for review and removed waiting-on-author The reviewer has suggested changes and awaits thier implementation. labels Oct 21, 2025

Add tests for flags

45a4315

jimmygchen commented Oct 21, 2025

View reviewed changes

sigp deleted a comment from mergify bot Oct 21, 2025

This comment was marked as outdated.

Sign in to view

mergify bot added waiting-on-author The reviewer has suggested changes and awaits thier implementation. and removed ready-for-review The code is ready for review labels Oct 21, 2025

jimmygchen added ready-for-review The code is ready for review and removed waiting-on-author The reviewer has suggested changes and awaits thier implementation. labels Oct 21, 2025

mergify bot added the waiting-on-author The reviewer has suggested changes and awaits thier implementation. label Oct 21, 2025

Merge remote-tracking branch 'origin/unstable' into semi-supernode-si…

e78f53e

…mple

jimmygchen added ready-for-merge This PR is ready to merge. and removed waiting-on-author The reviewer has suggested changes and awaits thier implementation. labels Oct 22, 2025

mergify bot added the queued label Oct 22, 2025

mergify bot added dequeued and removed queued labels Oct 22, 2025

jimmygchen mentioned this pull request Oct 22, 2025

Remove redundant subscribe_all_data_column_subnets field from network #8259

Merged

michaelsproul removed the ready-for-merge This PR is ready to merge. label Oct 22, 2025

Merge remote-tracking branch 'origin/unstable' into semi-supernode-si…

4c8c4ac

…mple

Fix build

a6c7aef

jimmygchen added the ready-for-merge This PR is ready to merge. label Oct 22, 2025

mergify bot added queued and removed dequeued labels Oct 22, 2025

mergify bot added a commit that referenced this pull request Oct 22, 2025

Merge of #8254

dc97575

mergify bot mentioned this pull request Oct 22, 2025

merge queue: embarking unstable (33e2163) and #8254 together #8261

Closed

6 tasks

mergify bot merged commit 43c5e92 into sigp:unstable Oct 22, 2025
37 checks passed

mergify bot removed the queued label Oct 22, 2025

This was referenced Oct 22, 2025

v8.0.0 release testing checklist #8233

Closed

CLI flag for "semi-supernode" #8218

Closed

jimmygchen mentioned this pull request Oct 29, 2025

Implement semi-supernode flag #8241

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Add `--semi-supernode` support #8254

Add `--semi-supernode` support #8254

Uh oh!

jimmygchen commented Oct 21, 2025 •

edited

Loading

Uh oh!

jimmygchen Oct 21, 2025 •

edited

Loading

Uh oh!

jimmygchen Oct 21, 2025

Uh oh!

jimmygchen commented Oct 21, 2025

Uh oh!

This comment was marked as outdated.

mergify bot commented Oct 21, 2025

Uh oh!

mergify bot commented Oct 22, 2025

Uh oh!

michaelsproul commented Oct 22, 2025

Uh oh!

eserilev commented Oct 22, 2025

Uh oh!

jimmygchen commented Oct 22, 2025

Uh oh!

mergify bot commented Oct 22, 2025

Uh oh!

jimmygchen commented Oct 22, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Add --semi-supernode support #8254

Add --semi-supernode support #8254

Uh oh!

Conversation

jimmygchen commented Oct 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Issue Addressed

Uh oh!

jimmygchen Oct 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jimmygchen Oct 21, 2025

Choose a reason for hiding this comment

Uh oh!

jimmygchen commented Oct 21, 2025

Uh oh!

This comment was marked as outdated.

mergify bot commented Oct 21, 2025

Uh oh!

mergify bot commented Oct 22, 2025

Uh oh!

michaelsproul commented Oct 22, 2025

Uh oh!

eserilev commented Oct 22, 2025

Uh oh!

jimmygchen commented Oct 22, 2025

Uh oh!

mergify bot commented Oct 22, 2025

✅ The queue state of this pull request has been cleaned. It can be re-embarked automatically

Uh oh!

jimmygchen commented Oct 22, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Add `--semi-supernode` support #8254

Add `--semi-supernode` support #8254

jimmygchen commented Oct 21, 2025 •

edited

Loading

jimmygchen Oct 21, 2025 •

edited

Loading