feat(ci/bb): enable BB_BENCH benchmarking in wasm#17569
Merged
ludamad merged 27 commits intomerge-train/barretenbergfrom Oct 28, 2025
Merged
feat(ci/bb): enable BB_BENCH benchmarking in wasm#17569ludamad merged 27 commits intomerge-train/barretenbergfrom
ludamad merged 27 commits intomerge-train/barretenbergfrom
Conversation
ludamad
changed the title
Adds ENABLE_WASM_BENCH compile-time definition to conditionally enable BB_BENCH benchmarking in WASM builds. By default, WASM builds exclude benchmarking overhead, but the new wasm-threads-bench preset enables it for performance measurements. Changes: - Add wasm-threads-bench preset with ENABLE_WASM_BENCH flag - Update bb_bench.hpp/cpp to conditionally compile for WASM when flag set - Update bootstrap.sh to use wasm-threads-bench for benchmark builds - wasmtime.sh already supports BB_BENCH environment variable passthrough
…d/wasmtime-experiments
- Update CMakePresets.json to use CMAKE_CXX_FLAGS for ENABLE_WASM_BENCH - Update cli.cpp to enable bench printing when ENABLE_WASM_BENCH is defined - Fix type consistency in bb_bench.hpp (use uint64_t throughout) - wasm-threads-bench preset now properly defines ENABLE_WASM_BENCH flag
Remove unnecessary static_cast<size_t> that caused compilation errors on macOS with Zig where size_t (unsigned long) and uint64_t (unsigned long long) are incompatible types. Both arguments are now uint64_t. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
Add closing #endif directive to match the #if on line 4. This fixes the "unterminated #if" compilation error on GCC builds. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
Change SerializableEntry fields from std::size_t to uint64_t to match the types from AggregateEntry. This fixes narrowing conversion errors on wasm32 where size_t is 32-bit but the source fields are 64-bit. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
…ENCH Update preprocessor condition in backing_memory.cpp from `#ifndef __wasm__` to `#if !defined(__wasm__) || defined(ENABLE_WASM_BENCH)` to match the condition used in cli.cpp. This ensures parse_size_string, storage_budget, and current_storage_usage are defined in wasm-threads-bench builds. Fixes linker errors: - undefined symbol: parse_size_string - undefined symbol: storage_budget 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
Collaborator
Flakey Tests🤖 says: This CI run detected 1 tests that failed, but were tolerated due to a .test_patterns.yml entry. Flaked tests |
Add a simplified stub implementation of parse_size_string for WASM builds with BB_NO_EXCEPTIONS defined. The stub returns unlimited storage budget, which is appropriate for WASM benchmarking where storage budget enforcement is not needed. This fixes the compilation error: cannot use 'try' with exceptions disabled The full exception-handling version is used for native builds, while WASM bench builds get a lightweight stub that satisfies the linker requirements from cli.cpp without requiring exception support. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
…Make option This change converts the dedicated wasm-threads-bench CMake preset into a CMake option that can be enabled during development but is excluded from semver/release builds. Changes: - Added ENABLE_WASM_BENCH CMake option in CMakeLists.txt - Removed wasm-threads-bench preset from CMakePresets.json - Updated bootstrap.sh to use ENABLE_WASM_BENCH=1 with wasm-threads preset - Updated ci_benchmark_ivc_flows.sh to use build-wasm-threads directory - Modified build_preset function to support ENABLE_WASM_BENCH environment variable This provides more flexibility for developers while ensuring release builds don't inadvertently enable benchmarking overhead. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
Modified the build_preset function to automatically enable ENABLE_WASM_BENCH for WASM preset builds unless it's a semver release build. This ensures: - Development/CI builds: WASM benchmarks are automatically enabled - Release builds (semver tags): WASM benchmarks are disabled by default - Manual override: ENABLE_WASM_BENCH=1 still works for all cases Changes: - Updated build_preset logic to check preset name and semver status - Removed explicit ENABLE_WASM_BENCH=1 from build_bench and bench_ivc functions - Added comments explaining the auto-enable behavior This simplifies the build process while ensuring release builds remain optimized without benchmarking overhead. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
…ions Refactored to remove ENABLE_WASM_BENCH as an environment variable option in build_preset. Instead, auto-enable logic is now specific to WASM build functions only: - build_wasm_threads: Auto-enables for non-semver builds - build_wasm_bench: New function for benchmark builds with auto-enable - build_wasm_bb: Inline function in bench_ivc for bb target This approach ensures: - No environment variable dependency (removed ENABLE_WASM_BENCH env var) - Auto-enable only applies to WASM thread builds, not all WASM presets - Each function has clear, isolated logic for when to enable benchmarking - Semver/release builds never enable WASM benchmarks 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
Simplified WASM benchmark configuration by exporting wasm_cmake_args early in the script based on semver check. This eliminates duplicate logic across multiple functions. Changes: - Export wasm_cmake_args at top: empty for semver, -DENABLE_WASM_BENCH=ON for non-semver - Removed build_wasm_bench and build_wasm_bb helper functions - Updated build_wasm_threads, build_bench, and bench_ivc to use exported variable - All WASM builds now consistently use the same logic This ensures: - DRY principle: single source of truth for WASM cmake args - Consistent behavior across all WASM builds - Cleaner, more maintainable code 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
Simplified by centralizing ENABLE_WASM_BENCH logic inside build_preset. The function now checks if the preset is "wasm-threads" and automatically adds -DENABLE_WASM_BENCH=ON for non-semver builds. Changes: - Removed wasm_cmake_args export from top of script - Added check in build_preset: if preset is wasm-threads and not semver, enable bench - Updated build_wasm_threads to use build_preset - Updated build_bench to use build_preset wasm-threads - Updated bench_ivc to use build_preset wasm-threads This ensures: - Single source of truth: logic is in build_preset only - Consistent behavior: all wasm-threads builds get same treatment - Simpler code: no exported variables or inline cmake commands 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
johnathan79717
approved these changes
Oct 27, 2025
barretenberg/cpp/bootstrap.sh
Outdated
| set -eu | ||
| if ! cache_download barretenberg-benchmarks-$hash.zst; then | ||
| # Run builds in parallel with different targets per preset | ||
| # WASM benchmarks auto-enable ENABLE_WASM_BENCH for non-semver builds |
Contributor
There was a problem hiding this comment.
Nit: Duplicated comment from L71?
barretenberg/cpp/bootstrap.sh
Outdated
| commit_hash="${2:-origin/next~3}" # commit from which to download flow inputs | ||
|
|
||
| # Build both native and wasm benchmark binaries | ||
| # WASM benchmarks auto-enable ENABLE_WASM_BENCH for non-semver builds |
Contributor
There was a problem hiding this comment.
Nit: Duplicated comment from L71?
ludamad
changed the title
ludamad
added a commit
that referenced
this pull request
Dec 16, 2025
Co-authored-by: Claude <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.