feat: add SHA256 chunk hash verification for CRS downloads#21087
Closed
feat: add SHA256 chunk hash verification for CRS downloads#21087
Conversation
Adds integrity verification for BN254 CRS data downloaded over HTTP. Instead of requiring SSL/TLS, the downloaded data is verified against SHA256 hashes of 1MB chunks embedded in the binary. - Embeds 2049 SHA256 hashes covering the full 2^25+1 point CRS (2GB) - Verifies all complete 1MB chunks on network download (not cached loads) - Retains first-element check as fast sanity check for all download sizes - Replaces the second-element-only check with comprehensive hash coverage - Adds test that validates hash checking and corruption detection - Includes script to regenerate hashes from CRS source data Addresses AztecProtocol/barretenberg#1616
…nloads - Use 8MB chunks instead of 1MB (257 hashes vs 2049, ~53KB vs ~400KB header) - Round up downloads to 8MB boundaries so every chunk is fully verified - Update test to verify alignment: requesting 200k points downloads 262144 (2×8MB) - Update generation script to match
Use parallel_for with ThreadChunk to verify SHA256 chunk hashes across all available cores. Includes early-exit on first mismatch.
AztecBot
force-pushed
the
claudebox/crs-hash-verification
branch
from
0c6572a to
5125aad
Compare
verify_bn254_crs_integrity now throws if data size is not a multiple of the 8MB chunk size, since this function is designed for data downloaded in aligned 8MB chunks.
- Cap aligned download to 256 full chunks to avoid requesting past EOF
(full CRS has 256 full 8MB chunks + 64-byte remainder)
- verify_bn254_crs_integrity now skips trailing bytes < chunk size
instead of erroring, since the CRS tail is inherently non-aligned
- Fix misleading CAS comment ("smallest failing index" -> "first CAS wins")
- Sync generation script with header changes
ludamad
pushed a commit
that referenced
this pull request
Mar 4, 2026
…#21113) Adds SHA-256 chunk hash verification for BN254 CRS downloads, combining the best of #21087 and #20864. - **8MB chunks** (131K points each, 257 hashes embedded at compile time) - **Parallel verification** with atomic early-exit on first mismatch - **`std::span`-based** hashing (zero per-chunk allocation) - Retains fast first-element sanity check before full hash verification - Covers partial last chunk (64-byte tail) so every downloaded byte is verified - Primary/fallback CDN URLs with HTTP Range requests --------- Co-authored-by: Jonathan Hao <jonathan@aztec-labs.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds integrity verification for BN254 CRS data downloaded over HTTP, addressing the concern raised in AztecProtocol/barretenberg#1616 without the complexity of adding SSL/TLS support.
Instead of HTTPS, we embed SHA256 hashes of 8MB chunks of the CRS file directly in the binary. Downloads are rounded up to 8MB boundaries so every downloaded chunk can be fully verified.
Changes
bn254_crs_hashes.hpp(new): 257 SHA256 hashes covering the full 2^25+1 point CRS file (~53KB header), plus an inlineverify_bn254_crs_integrity()functionget_bn254_crs.cpp: Rounds up downloads to 8MB chunk boundaries; calls hash verification after download; retains first-element check as fast sanity check for small downloadssha256.cpp: Addsstd::span<const uint8_t>template instantiation needed by the verification functioncrs_factory.test.cpp: NewBn254HashVerificationtest that requests 200k points (non-aligned), verifies the download rounds up to 2×8MB (262144 points), validates hash checking passes, and confirms corruption detection worksgenerate_crs_hashes.sh(new): Script to regenerate the hash header from a CRS file or by downloading from the CDNDesign decisions
Test plan
CrsFactory.bn254— existing test passesCrsFactory.Bn254Fallback— existing fallback test passesCrsFactory.Bn254HashVerification— new test: downloads 16MB (aligned from 200k points), verifies hash integrity, validates corruption detection