feat(shuffle): add PTC sampling by nazarhussain · Pull Request #563 · ChainSafe/lodestar-z

nazarhussain · 2026-08-14T08:08:04Z

Summary

Adds payload timeliness committee (PTC) sampling to the Zig swap_or_not_shuffle module and its JS binding, so ChainSafe/lodestar#9263 can stop computing it in JS.

Ports compute_ptc_indices and compute_ptc_indices_for_epoch, whose original implementation is feat: add computePtcIndices and computePtcIndicesForEpoch swap-or-not-shuffle#24 (453b639b).
Exposes a per-slot entry point and a whole-epoch one that derives each slot's seed and returns the committees concatenated.
Returns explicit errors for degenerate inputs that the reference package would crash or hang on.
Runs the epoch variant sequentially where the Rust uses rayon; still roughly 287x the naive JS path Lodestar uses today.

Testing

Added Zig vectors generated from the naive Lodestar sampler, plus epoch/per-slot consistency and error-path coverage.
Added JS tests comparing the binding against the naive sampler at full PTC size for one slot and across a full epoch.
Added per-slot and per-epoch benches alongside the existing shuffle ones.

Follow-ups

The epoch variant can be parallelised if its sequential cost ever matters.

AI assistance was used for drafting and implementation support.

## Summary Adds a Zig `swap_or_not_shuffle` module and JS bindings for the shuffle APIs Lodestar uses, so ChainSafe/lodestar can drop `@chainsafe/swap-or-not-shuffle`. - Shares one shuffle implementation between `state_transition` and the JS binding. - Matches the upstream Rust/reference behavior for implemented paths. - Returns explicit errors for degenerate inputs that the reference package would crash or hang on. - Leaves async binding support for a future zapi release, since Lodestar does not use it. ## Testing - Ported upstream shuffle reference tests and existing lodestar-z shuffle vectors. - Added Zig coverage for validation order, error paths, proposer index, and sync committee selection. - Checked beta metrics against unstable; no performance regression found. ## Follow-ups - #563 adds PTC sampling on top of this. - Async shuffle bindings can be added later if a consumer appears. AI assistance was used for drafting and implementation support. --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: bing <spiralladder@fastmail.com>

github-actions · 2026-08-20T08:29:36Z

Performance Report

✔️ no performance regression detected

Full benchmark results

Benchmark suite	Current: `c40bbff`	Previous: null	Ratio
aggregatePublicKeys 1 keys	2.1920 us/op
aggregatePublicKeys 8 keys	14.786 us/op
aggregatePublicKeys 32 keys	45.413 us/op
aggregatePublicKeys 128 keys	174.55 us/op
aggregatePublicKeys 256 keys	306.63 us/op
aggregateSignatures 1 sigs	2.1110 us/op
aggregateSignatures 8 sigs	24.238 us/op
aggregateSignatures 32 sigs	89.716 us/op
aggregateSignatures 128 sigs	341.47 us/op
aggregateSignatures 256 sigs	681.15 us/op
aggregateVerify 3 sets	2.2034 ms/op
aggregateVerify 8 sets	3.2621 ms/op
aggregateVerify 32 sets	9.9231 ms/op
aggregateVerify 64 sets	18.815 ms/op
aggregateVerify 128 sets	35.046 ms/op
verifyMultipleAggregateSignatures 3 sets	2.4874 ms/op
verifyMultipleAggregateSignatures 8 sets	3.9995 ms/op
verifyMultipleAggregateSignatures 32 sets	12.801 ms/op
verifyMultipleAggregateSignatures 64 sets	24.643 ms/op
verifyMultipleAggregateSignatures 128 sets	46.072 ms/op
native (internal serialize seed)	269.47 ms/op
TS (internal serialize seed)	1.2476 s/op
native (prebuilt seedValidatorsBytes)	56.974 ms/op
TS (prebuilt seedValidatorsBytes)	269.04 ms/op
JS - unshuffleList - 16384 indices	1.3740 ms/op
zig - unshuffleList - 16384 indices	545.88 us/op
JS - unshuffleList - 250000 indices	17.334 ms/op
zig - unshuffleList - 250000 indices	16.953 ms/op
JS - unshuffleList - 1000000 indices	72.134 ms/op
zig - unshuffleList - 1000000 indices	49.660 ms/op
JS - computeSyncCommitteeIndicesElectra - 16384 indices	949.57 ms/op
zig - computeSyncCommitteeIndices - 16384 indices	3.0554 ms/op
zig - computeProposerIndex - 16384 indices	28.186 us/op
JS - computeSyncCommitteeIndicesElectra - 250000 indices	918.20 ms/op
zig - computeSyncCommitteeIndices - 250000 indices	10.294 ms/op
zig - computeProposerIndex - 250000 indices	46.410 us/op
JS - computeSyncCommitteeIndicesElectra - 1000000 indices	909.28 ms/op
zig - computeSyncCommitteeIndices - 1000000 indices	13.606 ms/op
zig - computeProposerIndex - 1000000 indices	64.482 us/op
JS - computePtcIndices - 16384 indices	157.26 ms/op
zig - computePtcIndices - 16384 indices	200.47 us/op
JS - computePtcIndices - 250000 indices	140.59 ms/op
zig - computePtcIndices - 250000 indices	189.59 us/op
JS - computePtcIndices - 1000000 indices	147.69 ms/op
zig - computePtcIndices - 1000000 indices	193.87 us/op
JS - computePtcIndicesForEpoch - 250000 validators	4.9871 s/op
zig - computePtcIndicesForEpoch - 250000 validators	6.9711 ms/op
JS - computePtcIndicesForEpoch - 1000000 validators	4.7129 s/op
zig - computePtcIndicesForEpoch - 1000000 validators	6.8655 ms/op

by benchmarkbot/action

GrapeBaBa · 2026-08-20T13:16:30Z

+    ptc_size: js.Number,
+    max_effective_balance_electra: js.Number,
+    effective_balance_increment: js.Number,
+) !js.Uint32Array {


Is that possible to use a *Into API with out arg, seems a little bit effective for these kinds of invocation

The module already had an out-based inner function, so this was mostly deleting the allocation wrapper.

I think grapebaba is requesting an additional function(s) to the binding, *Into variants?

Ports compute_ptc_indices and compute_ptc_indices_for_epoch (head 453b639b) into the swap_or_not_shuffle module with the same graceful error policy as the rest of the port, exposes them on the shuffle binding, and adds the upstream naive lodestar PTC sampler to the TS oracle for differential tests and benches. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

spiral-ladder · 2026-08-24T14:23:00Z

+pub fn computePtcIndicesInto(
+    out: []u32,
+    seed: []const u8,
+    indices: []const u32,
+    effective_balance_increments: []const u16,
+    max_effective_balance_electra: i64,
+    effective_balance_increment: i64,


out should be the last param

Suggested change

pub fn computePtcIndicesInto(

out: []u32,

seed: []const u8,

indices: []const u32,

effective_balance_increments: []const u16,

max_effective_balance_electra: i64,

effective_balance_increment: i64,

pub fn computePtcIndicesInto(

seed: []const u8,

indices: []const u32,

effective_balance_increments: []const u16,

max_effective_balance_electra: i64,

effective_balance_increment: i64,

out: []u32,

spiral-ladder · 2026-08-24T14:28:30Z

+    indices: js.Uint32Array,
+    effective_balance_increments: js.Uint16Array,
+    ptc_size: js.Number,
+    max_effective_balance_electra: js.Number,


since this is known at comptime, can't we just not provide this as a param? Or are there reasons to provide this as a param

Really good point, and points to some pain we will have to figure out.

Current lodestar-ts handles both mainnet and minimal without recompilation, rather relies on load-time selection of the preset.

Lodestar-z handles the preset at compile time. So we will have to figure out how we can bundle code for both presets or else deploy separate versions for each preset.

spiral-ladder · 2026-08-24T14:32:30Z

+    );
+}
+
+test "computePtcIndicesForEpochInto rejects invalid slot offsets" {


i kinda prefer we put all invalid cases into the same test like in test "computePtcIndicesInto rejects invalid inputs gracefully"

spiral-ladder · 2026-08-24T14:50:23Z

+    var count: usize = 0;
+
+    var i: usize = 0;
+    outer: while (true) : (i += 16) {


i don't really like the while true here and think we should avoid this pattern as much as possible, can't we bound with count?

This makes an empty out a valid input without causing non-termination since an empty out would skip this entirely. with that said we should probably still check `out.len > 0

Suggested change

outer: while (true) : (i += 16) {

outer: while (count < out.len) : (i += 16) {

github-project-automation Bot added this to Lodestar Team Coordination Aug 14, 2026

github-project-automation Bot moved this to In Progress in Lodestar Team Coordination Aug 14, 2026

nazarhussain force-pushed the nh/swap-or-not-shuffle-ptc branch from 82ad017 to 929d3b8 Compare August 18, 2026 04:54

nazarhussain force-pushed the nh/swap-or-not-shuffle-ptc branch 2 times, most recently from 11d1021 to 0ba4381 Compare August 19, 2026 06:08

nazarhussain mentioned this pull request Aug 19, 2026

feat(shuffle): add swap-or-not shuffling module and binding #559

Merged

nazarhussain force-pushed the nh/swap-or-not-shuffle-ptc branch 2 times, most recently from b3b1992 to 186ebc1 Compare August 19, 2026 11:52

Base automatically changed from nh/swap-or-not-shuffle-parity to main August 19, 2026 14:22

wemeetagain force-pushed the nh/swap-or-not-shuffle-ptc branch from 186ebc1 to 44852ad Compare August 19, 2026 14:22

nazarhussain force-pushed the nh/swap-or-not-shuffle-ptc branch from 44852ad to 2ca9b06 Compare August 20, 2026 08:21

nazarhussain changed the title ~~feat: add PTC sampling from swap-or-not-shuffle#24~~ feat(shuffle): add PTC sampling Aug 20, 2026

nazarhussain force-pushed the nh/swap-or-not-shuffle-ptc branch 2 times, most recently from 19ad4b2 to 8511dfc Compare August 20, 2026 09:20

nazarhussain marked this pull request as ready for review August 20, 2026 10:21

nazarhussain requested a review from a team as a code owner August 20, 2026 10:21

nazarhussain requested review from GrapeBaBa, spiral-ladder and wemeetagain August 20, 2026 10:24

GrapeBaBa reviewed Aug 20, 2026

View reviewed changes

nazarhussain force-pushed the nh/swap-or-not-shuffle-ptc branch from 8511dfc to 848d561 Compare August 20, 2026 18:23

nazarhussain requested a review from GrapeBaBa August 20, 2026 18:25

GrapeBaBa mentioned this pull request Aug 24, 2026

(WIP)perf(state-transition): use native PTC sampling ChainSafe/lodestar#9903

Draft

spiral-ladder reviewed Aug 24, 2026

View reviewed changes

Navigation Menu

Uh oh!

feat(shuffle): add PTC sampling - #563

feat(shuffle): add PTC sampling#563
nazarhussain wants to merge 1 commit into
mainfrom
nh/swap-or-not-shuffle-ptc

nazarhussain commented Aug 14, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Aug 20, 2026 •

edited

Loading

Uh oh!

GrapeBaBa Aug 20, 2026

Uh oh!

nazarhussain Aug 20, 2026

Uh oh!

wemeetagain Aug 24, 2026

Uh oh!

spiral-ladder Aug 24, 2026

Uh oh!

spiral-ladder Aug 24, 2026

Uh oh!

wemeetagain Aug 24, 2026

Uh oh!

spiral-ladder Aug 24, 2026

Uh oh!

spiral-ladder Aug 24, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

	outer: while (true) : (i += 16) {
	outer: while (count < out.len) : (i += 16) {

Uh oh!

Conversation

nazarhussain commented Aug 14, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Testing

Follow-ups

Uh oh!

github-actions Bot commented Aug 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Performance Report

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

nazarhussain commented Aug 14, 2026 •

edited

Loading

github-actions Bot commented Aug 20, 2026 •

edited

Loading