feat(greybox_fuzzer): Parallel fuzz tests by Rumata888 · Pull Request #8432 · noir-lang/noir

Rumata888 · 2025-05-09T11:13:22Z

Description

Problem*

Resolves

Summary*

Fuzzing tests are now run after the main tests and each fuzzing test uses parallelisation internally instead of just being assigned 1 thread. The limits have also been changed to 100k executions and 1 second.

Additional Context

Documentation*

Check one:

No documentation needed.
Documentation included in this PR.
[For Experimental Features] Documentation to be submitted in a separate PR.

PR Checklist*

I have tested the changes locally.
I have formatted the changes with Prettier and/or cargo fmt on default settings.

github-actions · 2025-05-09T11:25:29Z

🚀 Deployed on https://681e2654426f2e211b624ab7--noir-docs.netlify.app

github-actions

⚠️ Performance Alert ⚠️

Possible performance regression was detected for benchmark 'Test Suite Duration'.
Benchmark result of this commit is worse than the previous benchmark result exceeding threshold 1.20.

Benchmark suite	Current: `cf18fb7`	Previous: `94e839d`	Ratio
`test_report_AztecProtocol_aztec-packages_noir-projects_aztec-nr`	`141` s	`54` s	`2.61`

This comment was automatically generated by workflow using github-action-benchmark.

CC: @TomAFrench

tooling/nargo_cli/src/cli/test_cmd.rs

asterite

Looks great!

docs/docs/tooling/testing.md

asterite

I think there's a mismatch between the 1 second by default and the documented 10 seconds.

Rumata888 · 2025-05-09T15:46:51Z

I think there's a mismatch between the 1 second by default and the documented 10 seconds.

Yes, thanks for catching that

github-actions · 2025-05-09T15:57:58Z

FYI @noir-lang/developerrelations on Noir doc changes.

Automated pull of nightly from the [noir](https://github.com/noir-lang/noir) programming language, a dependency of Aztec. BEGIN_COMMIT_OVERRIDE fix: always type-check turbofish, and error when it's not allowed (noir-lang/noir#8437) chore: Release Noir(1.0.0-beta.5) (noir-lang/noir#7955) feat(greybox_fuzzer): Parallel fuzz tests (noir-lang/noir#8432) fix(ssa): Mislabeled instructions with side effects in EnableSideEffectsIf removal pass (noir-lang/noir#8355) feat: SSA pass impact report (noir-lang/noir#8393) chore: bump external pinned commits (noir-lang/noir#8433) chore: separate benchmarking from github actions more (noir-lang/noir#7943) chore(fuzz): Break up the AST fuzzer `compare` module (noir-lang/noir#8431) chore(fuzz): Rename `init_vs_final` to `min_vs_full` (noir-lang/noir#8430) fix!: error on tuple mismatch (noir-lang/noir#8424) chore: bump external pinned commits (noir-lang/noir#8429) chore(acir): Test whether the predicate has an effect on slice intrinsics (noir-lang/noir#8421) feat(ssa): Mark transitively dead parameters during DIE (noir-lang/noir#8254) fix(ssa_gen): Do not code gen fetching of empty arrays when initializing the data bus (noir-lang/noir#8426) chore: remove `.aztec-sync-commit` (noir-lang/noir#8415) chore(test): Add more unit tests for `inline_functions_with_at_most_one_instruction` (noir-lang/noir#8418) chore: add minor docs for interpreter (noir-lang/noir#8397) fix: print slice composite types surrounded by parentheses (noir-lang/noir#8412) feat: Skip SSA passes that contain any of the given messages (noir-lang/noir#8416) fix: disable range constraints using the predicate (noir-lang/noir#8396) chore: bumping external libraries (noir-lang/noir#8406) chore: redo typo PR by shystrui1199 (noir-lang/noir#8405) feat(test): add `nargo_fuzz_target` (noir-lang/noir#8308) fix: allow names to collide in the values/types namespaces (noir-lang/noir#8286) fix: Fix sequencing of side-effects in lvalue (noir-lang/noir#8384) feat(greybox_fuzzer): Maximum executions parameter added (noir-lang/noir#8390) fix: warn on and discard unreachable statements after break and continue (noir-lang/noir#8382) fix: add handling for u128 infix ops in interpreter (noir-lang/noir#8392) chore: move acirgen tests into separate file (noir-lang/noir#8376) feat(fuzz): initial version of comptime vs brillig target for AST fuzzer (noir-lang/noir#8335) chore: apply lints to `ast_fuzzer` (noir-lang/noir#8386) chore: add note on AI generated PRs in `CONTRIBUTING.md` (noir-lang/noir#8385) chore: document flattening pass (noir-lang/noir#8312) fix: comptime shift-right overflow is zero (noir-lang/noir#8380) feat: let static_assert accept any type for its message (noir-lang/noir#8322) fix(expand): output safety comment before statements (noir-lang/noir#8378) chore: avoid need to rebuild after running tests (noir-lang/noir#8379) chore: bump dependencies (noir-lang/noir#8372) chore: Add GITHUB_TOKEN to cross build (noir-lang/noir#8370) chore: redo typo PR by GarmashAlex (noir-lang/noir#8364) chore: remove unsafe code from greybox fuzzer (noir-lang/noir#8315) feat: add `--fuzz-timeout` to `nargo test` options (noir-lang/noir#8326) chore: bump external pinned commits (noir-lang/noir#8334) fix(expand): try to use "Self" in function calls (noir-lang/noir#8353) fix: Fix evaluation order of assignments with side-effects in their rhs (noir-lang/noir#8342) fix: let comptime Field value carry the field's sign (noir-lang/noir#8343) fix: Ordering of items in callstacks (noir-lang/noir#8338) chore: add snapshosts for nargo expand tests (noir-lang/noir#8318) fix(ownership): Clone global arrays (noir-lang/noir#8328) chore: Replace all SSA interpreter panics with error variants (noir-lang/noir#8311) feat: Metamorphic AST fuzzing (noir-lang/noir#8299) fix: fix some Display implementations for AST nodes (noir-lang/noir#8316) chore: remove leftover file (noir-lang/noir#8313) fix: uses non-zero points with ec-add-unsafe (noir-lang/noir#8248) END_COMMIT_OVERRIDE --------- Co-authored-by: AztecBot <tech@aztecprotocol.com> Co-authored-by: Tom French <15848336+TomAFrench@users.noreply.github.com>

github-actions bot added the documentation Improvements or additions to documentation label May 9, 2025

Rumata888 added 3 commits May 9, 2025 11:16

parallelisation for fuzz tests

ce68aea

implemented

5e79f97

docs

b4294be

Rumata888 force-pushed the is/parallel-fuzz-tests branch from 425da12 to b4294be Compare May 9, 2025 11:17

github-actions bot reviewed May 9, 2025

View reviewed changes

Rumata888 added 4 commits May 9, 2025 14:16

Fix

87d9545

Clippy

cf18fb7

smaller times

912812f

update aztec.nr timeout

20ab890

Rumata888 self-assigned this May 9, 2025

Rumata888 requested a review from asterite May 9, 2025 15:25

asterite reviewed May 9, 2025

View reviewed changes

tooling/nargo_cli/src/cli/test_cmd.rs Show resolved Hide resolved

asterite approved these changes May 9, 2025

View reviewed changes

Rumata888 added this pull request to the merge queue May 9, 2025

asterite reviewed May 9, 2025

View reviewed changes

docs/docs/tooling/testing.md Outdated Show resolved Hide resolved

asterite requested changes May 9, 2025

View reviewed changes

asterite removed this pull request from the merge queue due to a manual request May 9, 2025

change docs

a1c9f08

Rumata888 requested a review from asterite May 9, 2025 15:47

typo

2601706

asterite approved these changes May 9, 2025

View reviewed changes

asterite added this pull request to the merge queue May 9, 2025

Merged via the queue into master with commit c7ff3e7 May 9, 2025
116 checks passed

asterite deleted the is/parallel-fuzz-tests branch May 9, 2025 17:07

noirwhal mentioned this pull request May 9, 2025

chore: Release Noir(1.0.0-beta.5) #7955

Merged

AztecBot mentioned this pull request May 10, 2025

chore: Bump Noir reference AztecProtocol/aztec-packages#14055

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(greybox_fuzzer): Parallel fuzz tests#8432

feat(greybox_fuzzer): Parallel fuzz tests#8432
asterite merged 9 commits intomasterfrom
is/parallel-fuzz-tests

Rumata888 commented May 9, 2025 •

edited

Loading

Uh oh!

github-actions bot commented May 9, 2025 •

edited

Loading

Uh oh!

github-actions bot left a comment •

edited

Loading

Uh oh!

Uh oh!

asterite left a comment

Uh oh!

Uh oh!

asterite left a comment

Uh oh!

Uh oh!

Rumata888 commented May 9, 2025

Uh oh!

github-actions bot commented May 9, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

Rumata888 commented May 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Problem*

Summary*

Additional Context

Documentation*

PR Checklist*

Uh oh!

github-actions bot commented May 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

⚠️ Performance Alert ⚠️

Uh oh!

Uh oh!

asterite left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

asterite left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Rumata888 commented May 9, 2025

Uh oh!

github-actions bot commented May 9, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Rumata888 commented May 9, 2025 •

edited

Loading

github-actions bot commented May 9, 2025 •

edited

Loading

github-actions bot left a comment •

edited

Loading