perf(bls): add cache-aware signature verifier by wemeetagain · Pull Request #562 · ChainSafe/lodestar-z

wemeetagain · 2026-08-13T14:19:15Z

Summary

Add native, cache-aware verification for Lodestar signature sets.

Implementation details

src/bls
- Add Signature.sigValidate, matching PublicKey.keyValidate.
- Add verifier.zig and VerifySignatureSetsOptions.
- Borrow BatchVerifyItem.message as *const SigningRoot.
- Return false for cryptographic failures.
- Propagate thread-pool lifecycle errors.
src/state_transition
- Add fixed-capacity SignatureSetBatch and SameMessageSignatureSetBatch.
- Fall back to ordered individual verification.
- Add PubkeyCache.aggregateIndices to support []u32 input.
bindings
- Add the blsVerifier N-API module and @chainsafe/lodestar-z/bls-verifier export.
- Support indexed, aggregate-index, and raw-key sets.
- Throw on malformed inputs, cache misses, or unavailable state.
- Reuse the verifier from the existing blst bindings.
Bounds
- General batches: 256 sets.
- Same-message batches: 128 sets.
- Aggregate indices: consensus committee limits.

Why

Keeping validator indices through the native boundary reduces JavaScript/native serialization and reuses the native public-key cache. Native same-message fallback also avoids another worker-queue pass when identifying invalid signatures.

Paired with ChainSafe/lodestar#9820.

AI assistance

Written with Codex assistance.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 0f53b807f4

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

lodekeeper-z

Reviewed the native verifier and companion contract. The core design is coherent and the targeted build/tests pass. I found one additional error-classification issue below. I also confirmed the existing cumulative aggregate-index concern in the open thread: the maximum 256 x 131,072-index call took about 16.8 seconds synchronously in a local probe, so I consider a cumulative bound blocking before merge.

lodekeeper-z

Approved. Correction to my earlier review: after applying the repository threat model in #557, I withdraw the cumulative aggregate-work bound as a blocking finding. My probe exercised an arbitrary same-process Cartesian maximum; it did not establish a least-privileged hostile production path, account for protocol and Lodestar rate bounds, or show amplification beyond expected cryptographic work. The measured timing remains useful performance data, not a demonstrated merge blocker. The remaining cache-error ordering note is non-blocking API semantics. Native implementation, companion contract, and targeted verification look good.

GrapeBaBa · 2026-08-14T15:22:43Z

+    return typed_ptr[0..info.length];
+}
+
+fn uint32(value: napi.Value) !u32 {


may bump the zapi to use PR

yes that would be nice. will leave for a separate PR since next release of zapi will be breaking

GrapeBaBa · 2026-08-15T08:05:13Z

+    const results = js.Array.createWithLength(count);
+    if (count == 0) return results;
+
+    const message_slice = try message.toSlice();


The refactor making message as a pointer seems violate the NAPI v8 memory rule, because there are some js function calls may triggered GC before get the value later. And store the pointer in the item seems also not safety.

Confirmed. The shared message slice was retained across per-set property reads that can execute JavaScript. I added a regression where a signature getter mutates the source message, watched it fail as [false], then fixed the boundary by copying the 32-byte message before further NAPI calls. Since this account cannot push the original branch, the verified fix is in #564. The focused binding suite passes all 9 tests.

Can you check all such kind of code including the existing code? I suspected there are some similar issues.

I audited the BLS NAPI typed-array lifetimes and found three additional affected paths: the mixed-set verifier, fastAggregateVerify, and the existing verifyMultipleAggregateSignatures. Each retained a message backing-store pointer across a property or array read that can execute JavaScript. I added red/green getter-mutation regressions and native snapshots for all three in #564. The combined focused suites pass all 173 tests, and the ReleaseSafe mainnet binding build passes.

I don't think issue is GC, by itself, here. Its a synchronous function with a handle scope around the Uint8Array. The handle scope ensures that the Uint8Array doesn't get GC'd. Any synchronous function has the same guarantees.

But there is a real (a contrived, but real nonetheless) re-entrency risk, where an overridden getter of a later object access can modify the Uint8Array, or detach the underlying ArrayBuffer.

Imo these kinds of getters are deliberately contrived and malicious, and obviously not what Lodestar does or what these bindings need to handle (This is all trusted input, lodestar just constructs vanilla objects).

Happy to change it but its worth noting that this is a very low-risk correctness hardening, not a real production bug.

I don't think issue is GC, by itself, here. Its a synchronous function with a handle scope around the Uint8Array. The handle scope ensures that the Uint8Array doesn't get GC'd. Any synchronous function has the same guarantees.

But there is a real (a contrived, but real nonetheless) re-entrency risk, where an overridden getter of a later object access can modify the Uint8Array, or detach the underlying ArrayBuffer.

Imo these kinds of getters are deliberately contrived and malicious, and obviously not what Lodestar does or what these bindings need to handle (This is all trusted input, lodestar just constructs vanilla objects).

Happy to change it but its worth noting that this is a very low-risk correctness hardening, not a real production bug.

Yes, I noticed most are get accessors called here, I am not quite sure if it is possible to trigger GC or not. But current keeping the message pointer in item, seems there are a lot JS function be called throughout the lifecycle, is it same safety?

Yes—the distinction is liveness versus stability. The handle scope keeps the Uint8Array reachable, so GC alone should not invalidate it, but it does not snapshot or freeze the backing bytes. Once later sets.get, object-property reads, or getters execute JavaScript, that code can mutate (or potentially detach) a previously captured backing store. In verifyMultipleAggregateSignatures, an earlier item’s pointer survives all subsequent set/property reads before native verification, so it has that same re-entrancy exposure. #564 removes the dependency by copying each 32-byte message into native-owned SigningRoot storage as it is extracted. I would therefore describe this as backing-store stability/re-entrancy hardening, not a demonstrated GC lifetime bug; the regressions in #564 specifically prove mutation across later JS calls.

OK, seems correct, but the zapi comment for typed array seems a little bit inaccurate, so that we don't need #564.

spiral-ladder

bunch of nits

spiral-ladder · 2026-08-17T06:24:32Z

+/// cache misses encountered before a result is known.
+pub fn verifySignatureSets(sets: js.Array) !js.Boolean {
+    const count = try sets.length();
+    if (count == 0) return js.Boolean.from(false);


semantics nit: would passing an empty set count as cryptographic failure? I guess in some ways it can be interpreted as such (passing in sets of length 0 = nothing to verify = not verified?)

spiral-ladder

LGTM

GrapeBaBa

Looks good 👍

edit: integrated the comments section of ChainSafe/lodestar#9833 here in addition to the note below This is more of a personal taste thing, but i always prefer seeing the possible result(s) of a function first, then justification. Hope this enforces a more straightforward documentation written by agents. Happy to hear thoughts or if people have different opinions Opened this after reviewing #562 , i found that agents tend to like to write a lot before stating exactly what the expected result(s) are, which reads to me like beating around the bush

🤖 I have created a release *beep* *boop* --- ## [1.0.0](v0.1.2...v1.0.0) (2026-08-19) ### Features * add `state.getBuildersLength()` binding ([#472](#472)) ([be2b5ab](be2b5ab)) * **beacon-node:** add block state cache and checkpoint datastore ([#452](#452)) ([2145faa](2145faa)) * bindings to `getExpectedWithdrawals` and native tweaks ([#350](#350)) ([f47bc66](f47bc66)) * **bindings:** add pubkey cache syncPubkeys ([#537](#537)) ([542779f](542779f)) * **bindings:** aggregate cached public keys by validator index ([#397](#397)) ([2f90603](2f90603)) * **bindings:** align `BeaconStateView` with `IBeaconStateView` ([#347](#347)) ([b8ec273](b8ec273)) * **bindings:** configurable pubkey cache growth step ([#481](#481)) ([133ef24](133ef24)) * **bindings:** expose more APIs for STF ([#444](#444)) ([7fe2609](7fe2609)) * **bls:** add small MSM for npoints < 32 ([#393](#393)) ([b430638](b430638)) * **blst:** use external buffers for blst operations ([#358](#358)) ([78e4678](78e4678)) * **ci:** conditionally publish bindings with tag ([#355](#355)) ([ea77919](ea77919)) * **clock:** add clock module for slot/epoch timing ([#354](#354)) ([385b077](385b077)) * **fork_choice:** add Prometheus metrics module ([#309](#309)) ([cbc9d8d](cbc9d8d)) * **forkchoice:** implement the forkchoice module ([#246](#246)) ([7c62a9b](7c62a9b)) * getSyncCommitteesWitness ([#367](#367)) ([ef77649](ef77649)) * implement `loadState` API and binding ([#165](#165)) ([f903519](f903519)), closes [#159](#159) * **metrics:** metrics bindings ([#455](#455)) ([dd41999](dd41999)) * migrate blst,pubkeys to use zapi js dsl ([#331](#331)) ([fcd26ca](fcd26ca)) * **pubkeys:** add getPubkeyBytes binding ([#555](#555)) ([4ca51cf](4ca51cf)) * publish ARM64 musl bindings ([#482](#482)) ([ac764c9](ac764c9)) * **shuffle:** add swap-or-not shuffling module and binding ([#559](#559)) ([c2db37c](c2db37c)) * split nextValue fn ([#464](#464)) ([b47faeb](b47faeb)) * support getLatestWeakSubjectivityCheckpointEpoch ([#366](#366)) ([dcf3883](dcf3883)) * update fulu deposit processing ([#442](#442)) ([064335c](064335c)) ### Bug Fixes * avoid set ([#484](#484)) ([2e25d97](2e25d97)) * better generation of rand scalar ([#388](#388)) ([74dce77](74dce77)) * **bindings:** accept `dontTransferCache` in processSlots for backward compatibility ([#460](#460)) ([65df5af](65df5af)) * **bindings:** check signature infinity by default ([#509](#509)) ([2f5f281](2f5f281)) * **bindings:** clean up failed async BLS work ([#527](#527)) ([1111b00](1111b00)) * **bindings:** free metrics writer on scrape failure ([#529](#529)) ([4c8d94a](4c8d94a)) * **bindings:** harden random aggregate scalars ([#528](#528)) ([8e89a63](8e89a63)) * **bindings:** log level for missing fields ([#435](#435)) ([08faf41](08faf41)) * **bindings:** misordering of print for cpu count ([#381](#381)) ([752a972](752a972)) * **bindings:** populate epoch participation for test fixtures ([#436](#436)) ([8dbdd2e](8dbdd2e)) * **bindings:** refcount Pool to fix teardown panic ([#352](#352)) ([23b2f68](23b2f68)) * **bindings:** roll back partial N-API initialization ([#491](#491)) ([31c5ebb](31c5ebb)) * **bindings:** size BLS thread pool by cgroup-aware CPU count ([#386](#386)) ([3ae9522](3ae9522)) * **bindings:** validate class types before unwrap ([#514](#514)) ([2fd2ad5](2fd2ad5)) * **bindings:** validate secret key hex length ([#517](#517)) ([136e415](136e415)) * **bls:** align PublicKey.uncompress validation with Signature.uncompress ([#508](#508)) ([5a8dbe9](5a8dbe9)) * **bls:** bound randomized aggregation inputs ([#548](#548)) ([779d0bf](779d0bf)), closes [#542](#542) * **bls:** clean up partial thread pool initialization ([#490](#490)) ([d55e598](d55e598)) * **bls:** convert pippenger scratch bytes to element counts ([#513](#513)) ([a12ca92](a12ca92)) * **bls:** enforce 32-byte signing roots ([#545](#545)) ([72fd308](72fd308)) * **bls:** make batch cardinality structural ([#547](#547)) ([a06d8b2](a06d8b2)) * **bls:** preserve aggregate outputs on failure ([#521](#521)) ([e0b6dd1](e0b6dd1)) * **bls:** reject empty keygen salts ([#524](#524)) ([d2a9c86](d2a9c86)) * **bls:** reject unknown BLST error codes ([#525](#525)) ([9e4a6ad](9e4a6ad)) * **bls:** size pairing buffers for 32-bit targets ([#531](#531)) ([dc64a27](dc64a27)) * **blst:** default signature infinity check to true if not provided ([#387](#387)) ([021cdcb](021cdcb)) * **build:** remove `zig-out` from `files` ([#360](#360)) ([c52af09](c52af09)) * **ci:** fix caching spec test version ([#439](#439)) ([96885a1](96885a1)) * dangling state pointer in loadOtherState ([#450](#450)) ([81cbd5f](81cbd5f)) * **epoch_cache:** compute missing `next_proposers` ([#447](#447)) ([0088a29](0088a29)) * **epoch_cache:** populate decision roots in afterProcessEpoch ([#453](#453)) ([4b70a5e](4b70a5e)) * export asyncAggregateWithRandomness through napi binding ([#371](#371)) ([1d04c2b](1d04c2b)) * harden memory safety across PMT, SSZ tree views, and state transition ([#377](#377)) ([d6f5897](d6f5897)) * improve atomic ordering in ThreadPool and NAPI init ([#310](#310)) ([4b0a1cc](4b0a1cc)) * interface compatbility with NativeBeaconStateView ([#445](#445)) ([89e13d1](89e13d1)) * missing deinits in loadOtherState ([#459](#459)) ([094d278](094d278)) * missing state commits ([#454](#454)) ([a432b55](a432b55)) * no-op when syncPubkeys run on a pk cache with shrinking validator set ([#432](#432)) ([ed05a99](ed05a99)) * param order in BeaconBlockBody ([#348](#348)) ([d8b9c06](d8b9c06)) * pendingConsolidations bindings ([#449](#449)) ([b9c497e](b9c497e)) * **pmt,ssz:** harden chunked-leaf and zero-copy tree-view memory safety ([#400](#400)) ([de50c53](de50c53)) * populate cache balances during rewards/penalties processing ([#474](#474)) ([5bf23dc](5bf23dc)) * re-expose sizes ([#369](#369)) ([64b81f3](64b81f3)) * remove `slashValidator` gating on active status ([#448](#448)) ([d319a0d](d319a0d)) * **ssz:** drop redundant default-init pass in fixed-list decode ([#468](#468)) ([0c757be](0c757be)) * **ssz:** publish child cache entries after lookup ([#565](#565)) ([21e78c9](21e78c9)) * state transition binding exports ([#456](#456)) ([895982c](895982c)) * **state-transition:** group-check signature sets ([#515](#515)) ([42774e9](42774e9)), closes [#502](#502) * **state-transition:** isolate epoch step cache mutations ([#535](#535)) ([a83741a](a83741a)) * **state-transition:** repair Pool.init call broken by [#346](https://github.com/ChainSafe/lodestar-z/issues/346)×[#367](https://github.com/ChainSafe/lodestar-z/issues/367) merge skew ([#394](#394)) ([b42944f](b42944f)) * various fixes around config ([#433](#433)) ([c4f082c](c4f082c)) ### Performance Improvements * **bindings:** drop TS BLS comparison benches and report benchmarks on PRs ([#552](#552)) ([c909c6f](c909c6f)) * **bls:** add cache-aware signature verifier ([#562](#562)) ([063857e](063857e)) * **bls:** bypass worker queue for small batches ([#553](#553)) ([3f8a6df](3f8a6df)) * **epoch:** replace AutoHashMap with array lookup in reward/penalty caches ([#286](#286)) ([e4e181b](e4e181b)), closes [#243](#243) * **pmt:** chunked-leaf packing for basic lists and container_struct ([#346](#346)) ([ba156c4](ba156c4)) ### Code Refactoring * allocate `AsyncAggRandData` in one obj ([#384](#384)) ([459750f](459750f)) * **bindings/pubkeys:** simplify allocation strategy for aggregate ([#518](#518)) ([b82750f](b82750f)) * **bindings:** rename blst Lifecycle to State ([#516](#516)) ([0a9c179](0a9c179)) * **bindings:** use zapi js.io() instead of local io module ([#469](#469)) ([2b34cc0](2b34cc0)) * **bindings:** wake only required number of workers ([#383](#383)) ([1db57f1](1db57f1)) * **bls:** allocations around VMAS ([#395](#395)) ([dfda58c](dfda58c)) * **bls:** clean up bls ([#398](#398)) ([e0f3b9b](e0f3b9b)) * **bls:** remove need for tracking results for verifyMultipleAggregateSignatures ([#389](#389)) ([6fe5c3f](6fe5c3f)) * **bls:** remove single-threaded fallback ([#390](#390)) ([e057713](e057713)) * **clock:** single public Clock; internalize SlotClock ([#463](#463)) ([fbab1fa](fbab1fa)) * make XXXDecisionRoot fns return `js.String` ([#342](#342)) ([aef4420](aef4420)) * move shuffle into swap_or_not_shuffle module ([#558](#558)) ([e56efb2](e56efb2)) * **pubkeys:** centralize the process-wide cache ([#522](#522)) ([dc9669d](dc9669d)) ### Miscellaneous Chores * avoid slow tests in AGENTS.md ([#546](#546)) ([c60f2a9](c60f2a9)) * bump zapi to include musl build ([#485](#485)) ([0b488cc](0b488cc)) * **ci:** pin github actions with sha hashes ([#507](#507)) ([167b8f5](167b8f5)) * deprecate unused blst APIs ([#575](#575)) ([7b547fa](7b547fa)) * **deps:** bump zapi v2.1.0 -> v2.2.0 ([#376](#376)) ([0c240d8](0c240d8)) * **deps:** bump zbuild ([#403](#403)) ([e2545de](e2545de)) * **deps:** compile blst with ReleaseFast ([#391](#391)) ([753a896](753a896)) * **deps:** update zapi to 3.1.0 ([#483](#483)) ([f3e5827](f3e5827)) * **deps:** use zapi v2.1.0 ([#372](#372)) ([88f403a](88f403a)) * disable gemini auto code review ([#382](#382)) ([63e42a4](63e42a4)), closes [#380](#380) * **docs:** add comments section in AGENTS.md ([#566](#566)) ([0c09750](0c09750)) * move state clones out of benchmark run functions ([#324](#324)) ([e4035de](e4035de)) * prepare 1.0.0 release ([#576](#576)) ([20b657b](20b657b)) * release v0.1.2-rc.3 ([#370](#370)) ([e4fc551](e4fc551)) * **release:** 0.1.2-rc.2 ([#365](#365)) ([7046128](7046128)) * **release:** v0.1.2-rc.10 ([#477](#477)) ([9a4fad5](9a4fad5)) * **release:** v0.1.2-rc.4 ([#373](#373)) ([09468f1](09468f1)) * **release:** v0.1.2-rc.5 ([#374](#374)) ([f344efa](f344efa)) * **release:** v0.1.2-rc.6 ([#375](#375)) ([bdf5b67](bdf5b67)) * **release:** v0.1.2-rc.8 ([#401](#401)) ([06f91c2](06f91c2)) * **release:** v0.1.2-rc.9 ([#404](#404)) ([6024800](6024800)) * remove merge transition code ([#359](#359)) ([09b175d](09b175d)) * remove stale epoch cache TODOs ([#534](#534)) ([27a547a](27a547a)) * rename era shortHistoricalRoot to shortEraRoot ([#473](#473)) ([c75a4d3](c75a4d3)) * **scripts:** build bindings with preset ([#434](#434)) ([a1b5ef7](a1b5ef7)) * silence debug log when used in release builds ([#486](#486)) ([c5377d7](c5377d7)) * support dev workflow ([#364](#364)) ([fcb9a78](fcb9a78)) * update gloas types to align with the latest specs ([#431](#431)) ([1f065b5](1f065b5)) * update spec test version to v1.7.0-alpha.11 ([#451](#451)) ([5875660](5875660)) * update spec-test-version: v1.6.0-beta.2 -> v1.7.0-alpha.10 ([#441](#441)) ([f932b1c](f932b1c)) * update zapi to 4.0.0 ([#571](#571)) ([de8e3fd](de8e3fd)) ### Documentation * document security threat model ([#557](#557)) ([e678b87](e678b87)) * more comprehensive AGENTS.md ([#520](#520)) ([c74b386](c74b386)) * **pkix:** document load provenance requirement ([#556](#556)) ([37e0aa2](37e0aa2)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

## Summary - Move indexed, aggregate, and raw-public-key BLS verification behind the cache-aware lodestar-z verifier. - Return final per-signature results for same-message jobs and remove the worker retry fanout. - Count same-message signatures at their actual queue cost, simplify worker transport, and remove stale BLS metrics. - Pin the required lodestar-z commit through an HTTPS codeload tarball. ## Why The previous flow serialized or aggregated public keys in TypeScript and treated a same-message job containing many signatures as one queued signature. Failed same-message aggregate checks were also expanded into new worker jobs, repeating parsing and cache work. The new interface keeps validator indices through the worker boundary and resolves cached public keys inside lodestar-z. ## Impact This reduces JavaScript/native boundary work, removes duplicate fallback verification, and prevents oversized worker batches caused by undercounted same-message jobs. Depends on ChainSafe/lodestar-z#562. Written with codex assistance --------- Co-authored-by: bing <spiralladder@fastmail.com> Co-authored-by: lodekeeper-z <258924193+lodekeeper-z@users.noreply.github.com> Co-authored-by: matthewkeil <me@matthewkeil.com>

perf(bls): add cache-aware signature verifier

0f53b80

github-project-automation Bot added this to Lodestar Team Coordination Aug 13, 2026

github-project-automation Bot moved this to In Progress in Lodestar Team Coordination Aug 13, 2026

wemeetagain marked this pull request as ready for review August 13, 2026 14:32

wemeetagain requested a review from a team as a code owner August 13, 2026 14:32

chatgpt-codex-connector Bot reviewed Aug 13, 2026

View reviewed changes