Skip to content

chore(experimental): use larger stack size for parsing#8347

Merged
asterite merged 8 commits intomasterfrom
ab/parse-all-larger-stack-size
May 19, 2025
Merged

chore(experimental): use larger stack size for parsing#8347
asterite merged 8 commits intomasterfrom
ab/parse-all-larger-stack-size

Conversation

@asterite
Copy link
Collaborator

@asterite asterite commented May 5, 2025

Description

Problem

Related to #8354

Summary

Except for WASM, in which setting a custom stack size doesn't work, we now parse all files still in parallel, but in custom threads that have a larger stack size.

Also changes the infix parsing to avoid recursively calling functions, then using #[inline(always)] in a few places so that when the compiler is built in release mode it allows for even more deepness.

Additional Context

This doesn't solve the stack overflow issue, which can still happen, but at least makes it harder to trigger.

Documentation

Check one:

  • No documentation needed.
  • Documentation included in this PR.
  • [For Experimental Features] Documentation to be submitted in a separate PR.

PR Checklist

  • I have tested the changes locally.
  • I have formatted the changes with Prettier and/or cargo fmt on default settings.

@asterite asterite closed this May 7, 2025
@asterite
Copy link
Collaborator Author

asterite commented May 7, 2025

It seems this isn't possible in all platforms.

@asterite asterite deleted the ab/parse-all-larger-stack-size branch May 7, 2025 13:18
@asterite asterite restored the ab/parse-all-larger-stack-size branch May 7, 2025 13:57
@asterite asterite reopened this May 7, 2025
@asterite asterite added the bench-show Display benchmark results on PR label May 7, 2025
Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ACVM Benchmarks

Details
Benchmark suite Current: 87f4fdd Previous: 96aa763 Ratio
purely_sequential_opcodes 255416 ns/iter (± 360) 254353 ns/iter (± 1012) 1.00
perfectly_parallel_opcodes 224820 ns/iter (± 6578) 223438 ns/iter (± 4598) 1.01
perfectly_parallel_batch_inversion_opcodes 3209856 ns/iter (± 6268) 3208426 ns/iter (± 10325) 1.00

This comment was automatically generated by workflow using github-action-benchmark.

Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Execution Time

Details
Benchmark suite Current: 87f4fdd Previous: 96aa763 Ratio
private-kernel-inner 0.028 s 0.029 s 0.97
private-kernel-reset 0.166 s 0.161 s 1.03
private-kernel-tail 0.011 s 0.011 s 1
rollup-base-private 0.304 s 0.308 s 0.99
rollup-base-public 0.194 s 0.191 s 1.02
rollup-block-root 11.2 s 11.4 s 0.98
rollup-merge 0.004 s 0.004 s 1
rollup-root 0.013 s 0.012 s 1.08
semaphore-depth-10 0.02 s 0.021 s 0.95

This comment was automatically generated by workflow using github-action-benchmark.

Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Compilation Time

Details
Benchmark suite Current: 87f4fdd Previous: 96aa763 Ratio
private-kernel-inner 2.296 s 2.506 s 0.92
private-kernel-reset 7.03 s 6.984 s 1.01
private-kernel-tail 1.164 s 1.082 s 1.08
rollup-base-private 16.06 s 16.36 s 0.98
rollup-base-public 12.48 s 12.52 s 1.00
rollup-block-root-empty 1.286 s 1.36 s 0.95
rollup-block-root-single-tx 126 s 133 s 0.95
rollup-block-root 130 s 128 s 1.02
rollup-merge 1.088 s 1.178 s 0.92
rollup-root 1.636 s 1.624 s 1.01
semaphore-depth-10 0.826 s 0.885 s 0.93

This comment was automatically generated by workflow using github-action-benchmark.

Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Performance Alert ⚠️

Possible performance regression was detected for benchmark 'Execution Time'.
Benchmark result of this commit is worse than the previous benchmark result exceeding threshold 1.20.

Benchmark suite Current: 7ef794c Previous: a686856 Ratio
rollup-merge 0.004 s 0.003 s 1.33

This comment was automatically generated by workflow using github-action-benchmark.

CC: @TomAFrench

Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Test Suite Duration

Details
Benchmark suite Current: 87f4fdd Previous: 96aa763 Ratio
test_report_AztecProtocol_aztec-packages_noir-projects_aztec-nr 64 s 65 s 0.98
test_report_AztecProtocol_aztec-packages_noir-projects_noir-contracts 99 s 103 s 0.96
test_report_AztecProtocol_aztec-packages_noir-projects_noir-protocol-circuits_crates_blob 49 s 45 s 1.09
test_report_AztecProtocol_aztec-packages_noir-projects_noir-protocol-circuits_crates_private-kernel-lib 190 s 197 s 0.96
test_report_AztecProtocol_aztec-packages_noir-projects_noir-protocol-circuits_crates_rollup-lib 200 s 193 s 1.04
test_report_AztecProtocol_aztec-packages_noir-projects_noir-protocol-circuits_crates_types 60 s 61 s 0.98
test_report_noir-lang_noir-bignum_ 370 s 372 s 0.99
test_report_noir-lang_noir_bigcurve_ 217 s 226 s 0.96
test_report_noir-lang_sha512_ 31 s 30 s 1.03
test_report_zkpassport_noir_rsa_ 23 s 23 s 1

This comment was automatically generated by workflow using github-action-benchmark.

Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Execution Memory

Details
Benchmark suite Current: 87f4fdd Previous: 96aa763 Ratio
private-kernel-inner 241.29 MB 241.29 MB 1
private-kernel-reset 263.55 MB 263.55 MB 1
private-kernel-tail 216.78 MB 216.78 MB 1
rollup-base-private 549.27 MB 549.27 MB 1
rollup-base-public 541.72 MB 541.72 MB 1
rollup-block-root 1450 MB 1450 MB 1
rollup-merge 370.98 MB 370.98 MB 1
rollup-root 376.99 MB 376.99 MB 1
semaphore_depth_10 93.04 MB 93.04 MB 1

This comment was automatically generated by workflow using github-action-benchmark.

Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Compilation Memory

Details
Benchmark suite Current: 87f4fdd Previous: 96aa763 Ratio
private-kernel-inner 316.51 MB 316.51 MB 1
private-kernel-reset 572.22 MB 572.22 MB 1
private-kernel-tail 231.45 MB 231.43 MB 1.00
rollup-base-private 1440 MB 1440 MB 1
rollup-base-public 1470 MB 1470 MB 1
rollup-block-root-empty 400.75 MB 400.77 MB 1.00
rollup-block-root-single-tx 7220 MB 7220 MB 1
rollup-block-root 7220 MB 7220 MB 1
rollup-merge 385.03 MB 385.02 MB 1.00
rollup-root 446.35 MB 446.35 MB 1
semaphore_depth_10 128.47 MB 128.47 MB 1

This comment was automatically generated by workflow using github-action-benchmark.

Co-authored-by: Maxim Vezenov <mvezenov@gmail.com>
@asterite asterite enabled auto-merge May 19, 2025 18:17
Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Artifact Size

Details
Benchmark suite Current: 87f4fdd Previous: 96aa763 Ratio
private-kernel-inner 1127.1 KB 1127.1 KB 1
private-kernel-reset 2051.1 KB 2051.1 KB 1
private-kernel-tail 583.1 KB 583.1 KB 1
rollup-base-private 5123.1 KB 5123.1 KB 1
rollup-base-public 3945.1 KB 3945.1 KB 1
rollup-block-root-empty 256.8 KB 256.8 KB 1
rollup-block-root-single-tx 25674.8 KB 25674.8 KB 1
rollup-block-root 25714.3 KB 25714.3 KB 1
rollup-merge 181.7 KB 181.7 KB 1
rollup-root 477.8 KB 477.8 KB 1
semaphore-depth-10 636.3 KB 636.3 KB 1

This comment was automatically generated by workflow using github-action-benchmark.

@asterite asterite added this pull request to the merge queue May 19, 2025
Merged via the queue into master with commit 4972733 May 19, 2025
115 of 116 checks passed
@asterite asterite deleted the ab/parse-all-larger-stack-size branch May 19, 2025 18:59
github-merge-queue bot pushed a commit to AztecProtocol/aztec-packages that referenced this pull request May 22, 2025
Automated pull of nightly from the
[noir](https://github.com/noir-lang/noir) programming language, a
dependency of Aztec.
BEGIN_COMMIT_OVERRIDE
fix(licm): Account for nested loops being control dependent when
analyzing outer loops (noir-lang/noir#8593)
chore(refactor): Switch unreachable function removal to use centralized
call graph (noir-lang/noir#8578)
chore(test): Allow lambdas in fuzzing
(noir-lang/noir#8584)
chore: use insta for execution_success stdout
(noir-lang/noir#8576)
chore: use generator instead of zero for ec-add predicate
(noir-lang/noir#8552)
fix: use predicate expression as binary result
(noir-lang/noir#8583)
fix(ssa): Do not generate apply functions when no lambda variants exist
(noir-lang/noir#8573)
chore: put `nargo expand` snapshosts in the same directory
(noir-lang/noir#8577)
chore: Use FxHashMap for TypeBindings
(noir-lang/noir#8574)
chore(experimental): use larger stack size for parsing
(noir-lang/noir#8347)
chore: use insta snapshots for compile_failure stderr
(noir-lang/noir#8569)
chore(inlining): Mark functions with <= 10 instructions and no control
flow as inline always (noir-lang/noir#8533)
chore(ssa): Add weighted edges to call graph, move callers and callees
methods to call graph (noir-lang/noir#8513)
fix(frontend): Override to allow empty array input
(noir-lang/noir#8568)
fix: avoid logging all unused params in DIE pass
(noir-lang/noir#8566)
chore: bump external pinned commits
(noir-lang/noir#8562)
chore(deps): bump base-x from 3.0.9 to 3.0.11
(noir-lang/noir#8555)
chore(fuzz): Call function pointers
(noir-lang/noir#8531)
feat: C++ codegen for msgpack
(noir-lang/noir#7716)
feat(performance): brillig array set optimization
(noir-lang/noir#8550)
chore(fuzz): AST fuzzer to use function valued arguments (Part 1)
(noir-lang/noir#8514)
fix(licm): Check whether the loop is executed when hoisting with a
predicate (noir-lang/noir#8546)
feat: Implement $crate (noir-lang/noir#8537)
fix: add offset to ArrayGet
(noir-lang/noir#8536)
chore: remove some unused enum variants and functions
(noir-lang/noir#8538)
fix: disallow `()` in entry points
(noir-lang/noir#8529)
chore: Remove println in ssa interpreter
(noir-lang/noir#8528)
fix: don't overflow when casting signed value to u128
(noir-lang/noir#8526)
chore(performance): Enable hoisting pure with predicate calls
(noir-lang/noir#8522)
feat(fuzz): AST fuzzing with SSA interpreter
(noir-lang/noir#8436)
chore: Add u1 ops to interpreter, convert Value panics to errors
(noir-lang/noir#8469)
chore: Release Noir(1.0.0-beta.6)
(noir-lang/noir#8438)
chore(fuzz): AST generator to add `ctx_limit` to all functions
(noir-lang/noir#8507)
fix(inlining): Use centralized CallGraph structure for inline info
computation (noir-lang/noir#8489)
fix: remove private builtins from `Field` impl
(noir-lang/noir#8496)
feat: primitive types are no longer keywords
(noir-lang/noir#8470)
fix: parenthesized pattern, and correct 1-element tuple printing
(noir-lang/noir#8482)
fix: fix visibility of methods in `std::meta`
(noir-lang/noir#8497)
fix: Change `can_be_main` to be recursive
(noir-lang/noir#8501)
chore: add SSA interpreter test for higher order functions
(noir-lang/noir#8486)
fix(frontend)!: Ban zero sized arrays and strings as program input
(noir-lang/noir#8491)
fix!: remove `to_be_radix` and `to_le_radix` from stdlib interface
(noir-lang/noir#8495)
END_COMMIT_OVERRIDE

---------

Co-authored-by: AztecBot <tech@aztecprotocol.com>
Co-authored-by: Tom French <15848336+TomAFrench@users.noreply.github.com>
AztecBot added a commit to AztecProtocol/aztec-nr that referenced this pull request May 23, 2025
Automated pull of nightly from the
[noir](https://github.com/noir-lang/noir) programming language, a
dependency of Aztec.
BEGIN_COMMIT_OVERRIDE
fix(licm): Account for nested loops being control dependent when
analyzing outer loops (noir-lang/noir#8593)
chore(refactor): Switch unreachable function removal to use centralized
call graph (noir-lang/noir#8578)
chore(test): Allow lambdas in fuzzing
(noir-lang/noir#8584)
chore: use insta for execution_success stdout
(noir-lang/noir#8576)
chore: use generator instead of zero for ec-add predicate
(noir-lang/noir#8552)
fix: use predicate expression as binary result
(noir-lang/noir#8583)
fix(ssa): Do not generate apply functions when no lambda variants exist
(noir-lang/noir#8573)
chore: put `nargo expand` snapshosts in the same directory
(noir-lang/noir#8577)
chore: Use FxHashMap for TypeBindings
(noir-lang/noir#8574)
chore(experimental): use larger stack size for parsing
(noir-lang/noir#8347)
chore: use insta snapshots for compile_failure stderr
(noir-lang/noir#8569)
chore(inlining): Mark functions with <= 10 instructions and no control
flow as inline always (noir-lang/noir#8533)
chore(ssa): Add weighted edges to call graph, move callers and callees
methods to call graph (noir-lang/noir#8513)
fix(frontend): Override to allow empty array input
(noir-lang/noir#8568)
fix: avoid logging all unused params in DIE pass
(noir-lang/noir#8566)
chore: bump external pinned commits
(noir-lang/noir#8562)
chore(deps): bump base-x from 3.0.9 to 3.0.11
(noir-lang/noir#8555)
chore(fuzz): Call function pointers
(noir-lang/noir#8531)
feat: C++ codegen for msgpack
(noir-lang/noir#7716)
feat(performance): brillig array set optimization
(noir-lang/noir#8550)
chore(fuzz): AST fuzzer to use function valued arguments (Part 1)
(noir-lang/noir#8514)
fix(licm): Check whether the loop is executed when hoisting with a
predicate (noir-lang/noir#8546)
feat: Implement $crate (noir-lang/noir#8537)
fix: add offset to ArrayGet
(noir-lang/noir#8536)
chore: remove some unused enum variants and functions
(noir-lang/noir#8538)
fix: disallow `()` in entry points
(noir-lang/noir#8529)
chore: Remove println in ssa interpreter
(noir-lang/noir#8528)
fix: don't overflow when casting signed value to u128
(noir-lang/noir#8526)
chore(performance): Enable hoisting pure with predicate calls
(noir-lang/noir#8522)
feat(fuzz): AST fuzzing with SSA interpreter
(noir-lang/noir#8436)
chore: Add u1 ops to interpreter, convert Value panics to errors
(noir-lang/noir#8469)
chore: Release Noir(1.0.0-beta.6)
(noir-lang/noir#8438)
chore(fuzz): AST generator to add `ctx_limit` to all functions
(noir-lang/noir#8507)
fix(inlining): Use centralized CallGraph structure for inline info
computation (noir-lang/noir#8489)
fix: remove private builtins from `Field` impl
(noir-lang/noir#8496)
feat: primitive types are no longer keywords
(noir-lang/noir#8470)
fix: parenthesized pattern, and correct 1-element tuple printing
(noir-lang/noir#8482)
fix: fix visibility of methods in `std::meta`
(noir-lang/noir#8497)
fix: Change `can_be_main` to be recursive
(noir-lang/noir#8501)
chore: add SSA interpreter test for higher order functions
(noir-lang/noir#8486)
fix(frontend)!: Ban zero sized arrays and strings as program input
(noir-lang/noir#8491)
fix!: remove `to_be_radix` and `to_le_radix` from stdlib interface
(noir-lang/noir#8495)
END_COMMIT_OVERRIDE

---------

Co-authored-by: AztecBot <tech@aztecprotocol.com>
Co-authored-by: Tom French <15848336+TomAFrench@users.noreply.github.com>
Thunkar pushed a commit to AztecProtocol/aztec-packages that referenced this pull request May 23, 2025
Automated pull of nightly from the
[noir](https://github.com/noir-lang/noir) programming language, a
dependency of Aztec.
BEGIN_COMMIT_OVERRIDE
fix(licm): Account for nested loops being control dependent when
analyzing outer loops (noir-lang/noir#8593)
chore(refactor): Switch unreachable function removal to use centralized
call graph (noir-lang/noir#8578)
chore(test): Allow lambdas in fuzzing
(noir-lang/noir#8584)
chore: use insta for execution_success stdout
(noir-lang/noir#8576)
chore: use generator instead of zero for ec-add predicate
(noir-lang/noir#8552)
fix: use predicate expression as binary result
(noir-lang/noir#8583)
fix(ssa): Do not generate apply functions when no lambda variants exist
(noir-lang/noir#8573)
chore: put `nargo expand` snapshosts in the same directory
(noir-lang/noir#8577)
chore: Use FxHashMap for TypeBindings
(noir-lang/noir#8574)
chore(experimental): use larger stack size for parsing
(noir-lang/noir#8347)
chore: use insta snapshots for compile_failure stderr
(noir-lang/noir#8569)
chore(inlining): Mark functions with <= 10 instructions and no control
flow as inline always (noir-lang/noir#8533)
chore(ssa): Add weighted edges to call graph, move callers and callees
methods to call graph (noir-lang/noir#8513)
fix(frontend): Override to allow empty array input
(noir-lang/noir#8568)
fix: avoid logging all unused params in DIE pass
(noir-lang/noir#8566)
chore: bump external pinned commits
(noir-lang/noir#8562)
chore(deps): bump base-x from 3.0.9 to 3.0.11
(noir-lang/noir#8555)
chore(fuzz): Call function pointers
(noir-lang/noir#8531)
feat: C++ codegen for msgpack
(noir-lang/noir#7716)
feat(performance): brillig array set optimization
(noir-lang/noir#8550)
chore(fuzz): AST fuzzer to use function valued arguments (Part 1)
(noir-lang/noir#8514)
fix(licm): Check whether the loop is executed when hoisting with a
predicate (noir-lang/noir#8546)
feat: Implement $crate (noir-lang/noir#8537)
fix: add offset to ArrayGet
(noir-lang/noir#8536)
chore: remove some unused enum variants and functions
(noir-lang/noir#8538)
fix: disallow `()` in entry points
(noir-lang/noir#8529)
chore: Remove println in ssa interpreter
(noir-lang/noir#8528)
fix: don't overflow when casting signed value to u128
(noir-lang/noir#8526)
chore(performance): Enable hoisting pure with predicate calls
(noir-lang/noir#8522)
feat(fuzz): AST fuzzing with SSA interpreter
(noir-lang/noir#8436)
chore: Add u1 ops to interpreter, convert Value panics to errors
(noir-lang/noir#8469)
chore: Release Noir(1.0.0-beta.6)
(noir-lang/noir#8438)
chore(fuzz): AST generator to add `ctx_limit` to all functions
(noir-lang/noir#8507)
fix(inlining): Use centralized CallGraph structure for inline info
computation (noir-lang/noir#8489)
fix: remove private builtins from `Field` impl
(noir-lang/noir#8496)
feat: primitive types are no longer keywords
(noir-lang/noir#8470)
fix: parenthesized pattern, and correct 1-element tuple printing
(noir-lang/noir#8482)
fix: fix visibility of methods in `std::meta`
(noir-lang/noir#8497)
fix: Change `can_be_main` to be recursive
(noir-lang/noir#8501)
chore: add SSA interpreter test for higher order functions
(noir-lang/noir#8486)
fix(frontend)!: Ban zero sized arrays and strings as program input
(noir-lang/noir#8491)
fix!: remove `to_be_radix` and `to_le_radix` from stdlib interface
(noir-lang/noir#8495)
END_COMMIT_OVERRIDE

---------

Co-authored-by: AztecBot <tech@aztecprotocol.com>
Co-authored-by: Tom French <15848336+TomAFrench@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bench-show Display benchmark results on PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants