Skip to content

fix(ownership): Clone global arrays #8328

Merged
jfecher merged 7 commits intomasterfrom
mv/rc-on-global-arrays
May 2, 2025
Merged

fix(ownership): Clone global arrays #8328
jfecher merged 7 commits intomasterfrom
mv/rc-on-global-arrays

Conversation

@vezenovm
Copy link
Contributor

@vezenovm vezenovm commented May 2, 2025

Description

Problem*

Resolves #8259

Taking the snippet from the issue:

global G_C: [bool; 3] = [true, false, true];
fn main(a: bool) -> pub [bool; 3] {
    let b = func_1(a, G_C);
    if a {
        G_C
    } else {
        b
    }
}
fn func_1(a: bool, mut b: [bool; 3]) -> [bool; 3] {
    b[1] = a;
    b
}

We have the following SSA:

g0 = u1 1
g1 = u1 0
g2 = make_array [u1 1, u1 0, u1 1] : [u1; 3]

brillig(inline) predicate_pure fn main f0 {
  b0(v3: u1):
    v5 = call f1(v3, g2) -> [u1; 3]
    v6 = not v3
    v7 = if v3 then g2 else (if v6) v5
    return v7
}
brillig(inline) predicate_pure fn func_1 f1 {
  b0(v3: u1, v4: [u1; 3]):
    v6 = array_set v4, index u32 1, value v3
    return v6
}

The crux of the issue is a missing inc_rc on g2. This indicates a bug in the ownership pass and that we should clone the global.

Summary*

We now clone any globals which contain arrays.

The snippet above produces the following SSA:

g0 = u1 1
g1 = u1 0
g2 = make_array [u1 1, u1 0, u1 1] : [u1; 3]

brillig(inline) predicate_pure fn main f0 {
  b0(v3: u1):
    inc_rc g2
    v5 = call f1(v3, g2) -> [u1; 3]
    inc_rc g2
    v6 = not v3
    v7 = if v3 then g2 else (if v6) v5
    return v7
}
brillig(inline) predicate_pure fn func_1 f1 {
  b0(v3: u1, v4: [u1; 3]):
    v6 = array_set v4, index u32 1, value v3
    return v6
}

Additional Context

This PR initially considering optimizing this process by initializing global arrays to have an RC > 1. This was chosen as unideal as it complicates array initialization in Brillig gen. It also essentially means our monomorphization output is false and being correct by Brillig gen. You can follow the discussion on this PR for why we moved away from this optimization to simplfy always cloning global arrays.

Documentation*

Check one:

  • No documentation needed.
  • Documentation included in this PR.
  • [For Experimental Features] Documentation to be submitted in a separate PR.

PR Checklist*

  • I have tested the changes locally.
  • I have formatted the changes with Prettier and/or cargo fmt on default settings.

@github-actions
Copy link
Contributor

github-actions bot commented May 2, 2025

Changes to number of Brillig opcodes executed

Generated at commit: 7d54ae8b49f8484830ebeab788c8d4d1f09a84c1, compared to commit: b2d3a42c638536ad982ebdea0dbe2ac63e5f6e8b

🧾 Summary (10% most significant diffs)

Program Brillig opcodes (+/-) %
regression_8174_inliner_max +33 ❌ +12.79%
regression_8174_inliner_min +33 ❌ +12.79%
regression_8174_inliner_zero +33 ❌ +12.79%

Full diff report 👇
Program Brillig opcodes (+/-) %
regression_8174_inliner_max 291 (+33) +12.79%
regression_8174_inliner_min 291 (+33) +12.79%
regression_8174_inliner_zero 291 (+33) +12.79%
strings_inliner_max 1,524 (+3) +0.20%
strings_inliner_zero 1,816 (+3) +0.17%
strings_inliner_min 2,463 (+3) +0.12%

@vezenovm vezenovm requested a review from jfecher May 2, 2025 19:12
@vezenovm
Copy link
Contributor Author

vezenovm commented May 2, 2025

I think it may be best to simply update the ownership pass. It would simplify Brillig gen and how we initialize arrays.

@jfecher
Copy link
Contributor

jfecher commented May 2, 2025

I think it may be best to simply update the ownership pass. It would simplify Brillig gen and how we initialize arrays.

I think so too. It is an easy change in should_clone_ident to add the global case:

    fn should_clone_ident(&self, ident: &Ident) -> bool {
        match &ident.definition {
            Definition::Local(local_id) => {
                contains_array_or_str_type(&ident.typ) && !self.should_move(*local_id, ident.id)
            },
            // Globals are always cloned if they contain arrays
            Definition::Global(_) => contains_array_or_str_type(&ident.typ),
            _ => false,
        }
    }

This PRs version with starting RC of globals at 2 could be more efficient but I don't like that it means our monomorphized output is essentially incorrect and is relying on brillig to later correct it. For debugging in the future it can also complicate the mental model if we have to think about global arrays in SSA differently from normal arrays.

@jfecher
Copy link
Contributor

jfecher commented May 2, 2025

Aside: all the snap changes make this PR look much larger than it actually is

@vezenovm
Copy link
Contributor Author

vezenovm commented May 2, 2025

I think so too. It is an easy change in should_clone_ident to add the global case:

Yeah that is what I did initially. I would like to track globals in last uses as well, although that should not be too bad to implement as it will copy what exists for local variables. For example in the code posted in the original issue we would get this SSA:

g0 = u1 1
g1 = u1 0
g2 = make_array [u1 1, u1 0, u1 1] : [u1; 3]

brillig(inline) predicate_pure fn main f0 {
  b0(v3: u1):
    inc_rc g2
    v5 = call f1(v3, g2) -> [u1; 3]
    inc_rc g2
    v6 = not v3
    v7 = if v3 then g2 else (if v6) v5
    return v7
}
brillig(inline) predicate_pure fn func_1 f1 {
  b0(v3: u1, v4: [u1; 3]):
    v6 = array_set v4, index u32 1, value v3
    return v6
}

The second inc_rc g2 is unneeded.

Aside: all the snap changes make this PR look much larger than it actually is

Yes, it is a bit unfortunate.

@vezenovm
Copy link
Contributor Author

vezenovm commented May 2, 2025

I would like to track globals in last uses as well

For now, I will make this a follow-up as to fix the issue first.

@jfecher
Copy link
Contributor

jfecher commented May 2, 2025

I would like to track globals in last uses as well

We shouldn't track globals in last_uses since we should never move globals values. We don't know the ordering globals are used in in-between function calls which prevents us from determining where they're last used generally. If we limit the analysis to only entry point functions, these can be called recursively. I think it'd be a lot of effort to remove just 1 inc_rc from each global in the best case.

@vezenovm
Copy link
Contributor Author

vezenovm commented May 2, 2025

We shouldn't track globals in last_uses since we should never move globals values. We don't know the ordering globals are used in in-between function calls which prevents us from determining where they're last used generally. If we limit the analysis to only entry point functions, these can be called recursively. I think it'd be a lot of effort to remove just 1 inc_rc from each global in the best case.

Ah yes! 🤦 Good point that would be way too much effort for the benefits. Makes things easier!

@vezenovm vezenovm changed the title fix(brillig): Reference counter for global arrays and slices fix(ownership): Clone global arrays May 2, 2025
@vezenovm
Copy link
Contributor Author

vezenovm commented May 2, 2025

@jfecher this is ready for review again. As you said the actual changes are very small and almost all of the diff are snapshots.

@github-actions
Copy link
Contributor

github-actions bot commented May 2, 2025

Changes to Brillig bytecode sizes

Generated at commit: 7d54ae8b49f8484830ebeab788c8d4d1f09a84c1, compared to commit: b2d3a42c638536ad982ebdea0dbe2ac63e5f6e8b

🧾 Summary (10% most significant diffs)

Program Brillig opcodes (+/-) %
regression_8174_inliner_max +3 ❌ +1.68%
regression_8174_inliner_min +3 ❌ +1.68%
regression_8174_inliner_zero +3 ❌ +1.68%

Full diff report 👇
Program Brillig opcodes (+/-) %
regression_8174_inliner_max 182 (+3) +1.68%
regression_8174_inliner_min 182 (+3) +1.68%
regression_8174_inliner_zero 182 (+3) +1.68%
strings_inliner_zero 912 (+3) +0.33%
strings_inliner_max 917 (+3) +0.33%
strings_inliner_min 1,045 (+3) +0.29%

Copy link
Contributor

@jfecher jfecher left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@jfecher jfecher enabled auto-merge May 2, 2025 20:20
@jfecher jfecher added this pull request to the merge queue May 2, 2025
Merged via the queue into master with commit b8d04d6 May 2, 2025
119 checks passed
@jfecher jfecher deleted the mv/rc-on-global-arrays branch May 2, 2025 20:54
github-merge-queue bot pushed a commit to AztecProtocol/aztec-packages that referenced this pull request May 12, 2025
Automated pull of nightly from the
[noir](https://github.com/noir-lang/noir) programming language, a
dependency of Aztec.
BEGIN_COMMIT_OVERRIDE
fix: always type-check turbofish, and error when it's not allowed
(noir-lang/noir#8437)
chore: Release Noir(1.0.0-beta.5)
(noir-lang/noir#7955)
feat(greybox_fuzzer): Parallel fuzz tests
(noir-lang/noir#8432)
fix(ssa): Mislabeled instructions with side effects in
EnableSideEffectsIf removal pass
(noir-lang/noir#8355)
feat: SSA pass impact report
(noir-lang/noir#8393)
chore: bump external pinned commits
(noir-lang/noir#8433)
chore: separate benchmarking from github actions more
(noir-lang/noir#7943)
chore(fuzz): Break up the AST fuzzer `compare` module
(noir-lang/noir#8431)
chore(fuzz): Rename `init_vs_final` to `min_vs_full`
(noir-lang/noir#8430)
fix!: error on tuple mismatch
(noir-lang/noir#8424)
chore: bump external pinned commits
(noir-lang/noir#8429)
chore(acir): Test whether the predicate has an effect on slice
intrinsics (noir-lang/noir#8421)
feat(ssa): Mark transitively dead parameters during DIE
(noir-lang/noir#8254)
fix(ssa_gen): Do not code gen fetching of empty arrays when initializing
the data bus (noir-lang/noir#8426)
chore: remove `.aztec-sync-commit`
(noir-lang/noir#8415)
chore(test): Add more unit tests for
`inline_functions_with_at_most_one_instruction`
(noir-lang/noir#8418)
chore: add minor docs for interpreter
(noir-lang/noir#8397)
fix: print slice composite types surrounded by parentheses
(noir-lang/noir#8412)
feat: Skip SSA passes that contain any of the given messages
(noir-lang/noir#8416)
fix: disable range constraints using the predicate
(noir-lang/noir#8396)
chore: bumping external libraries
(noir-lang/noir#8406)
chore: redo typo PR by shystrui1199
(noir-lang/noir#8405)
feat(test): add `nargo_fuzz_target`
(noir-lang/noir#8308)
fix: allow names to collide in the values/types namespaces
(noir-lang/noir#8286)
fix: Fix sequencing of side-effects in lvalue
(noir-lang/noir#8384)
feat(greybox_fuzzer): Maximum executions parameter added
(noir-lang/noir#8390)
fix: warn on and discard unreachable statements after break and continue
(noir-lang/noir#8382)
fix: add handling for u128 infix ops in interpreter
(noir-lang/noir#8392)
chore: move acirgen tests into separate file
(noir-lang/noir#8376)
feat(fuzz): initial version of comptime vs brillig target for AST fuzzer
(noir-lang/noir#8335)
chore: apply lints to `ast_fuzzer`
(noir-lang/noir#8386)
chore: add note on AI generated PRs in `CONTRIBUTING.md`
(noir-lang/noir#8385)
chore: document flattening pass
(noir-lang/noir#8312)
fix: comptime shift-right overflow is zero
(noir-lang/noir#8380)
feat: let static_assert accept any type for its message
(noir-lang/noir#8322)
fix(expand): output safety comment before statements
(noir-lang/noir#8378)
chore: avoid need to rebuild after running tests
(noir-lang/noir#8379)
chore: bump dependencies (noir-lang/noir#8372)
chore: Add GITHUB_TOKEN to cross build
(noir-lang/noir#8370)
chore: redo typo PR by GarmashAlex
(noir-lang/noir#8364)
chore: remove unsafe code from greybox fuzzer
(noir-lang/noir#8315)
feat: add `--fuzz-timeout` to `nargo test` options
(noir-lang/noir#8326)
chore: bump external pinned commits
(noir-lang/noir#8334)
fix(expand): try to use "Self" in function calls
(noir-lang/noir#8353)
fix: Fix evaluation order of assignments with side-effects in their rhs
(noir-lang/noir#8342)
fix: let comptime Field value carry the field's sign
(noir-lang/noir#8343)
fix: Ordering of items in callstacks
(noir-lang/noir#8338)
chore: add snapshosts for nargo expand tests
(noir-lang/noir#8318)
fix(ownership): Clone global arrays
(noir-lang/noir#8328)
chore: Replace all SSA interpreter panics with error variants
(noir-lang/noir#8311)
feat: Metamorphic AST fuzzing
(noir-lang/noir#8299)
fix: fix some Display implementations for AST nodes
(noir-lang/noir#8316)
chore: remove leftover file
(noir-lang/noir#8313)
fix: uses non-zero points with ec-add-unsafe
(noir-lang/noir#8248)
END_COMMIT_OVERRIDE

---------

Co-authored-by: AztecBot <tech@aztecprotocol.com>
Co-authored-by: Tom French <15848336+TomAFrench@users.noreply.github.com>
github-merge-queue bot pushed a commit to AztecProtocol/aztec-packages that referenced this pull request May 12, 2025
Automated pull of nightly from the
[noir](https://github.com/noir-lang/noir) programming language, a
dependency of Aztec.
BEGIN_COMMIT_OVERRIDE
fix: always type-check turbofish, and error when it's not allowed
(noir-lang/noir#8437)
chore: Release Noir(1.0.0-beta.5)
(noir-lang/noir#7955)
feat(greybox_fuzzer): Parallel fuzz tests
(noir-lang/noir#8432)
fix(ssa): Mislabeled instructions with side effects in
EnableSideEffectsIf removal pass
(noir-lang/noir#8355)
feat: SSA pass impact report
(noir-lang/noir#8393)
chore: bump external pinned commits
(noir-lang/noir#8433)
chore: separate benchmarking from github actions more
(noir-lang/noir#7943)
chore(fuzz): Break up the AST fuzzer `compare` module
(noir-lang/noir#8431)
chore(fuzz): Rename `init_vs_final` to `min_vs_full`
(noir-lang/noir#8430)
fix!: error on tuple mismatch
(noir-lang/noir#8424)
chore: bump external pinned commits
(noir-lang/noir#8429)
chore(acir): Test whether the predicate has an effect on slice
intrinsics (noir-lang/noir#8421)
feat(ssa): Mark transitively dead parameters during DIE
(noir-lang/noir#8254)
fix(ssa_gen): Do not code gen fetching of empty arrays when initializing
the data bus (noir-lang/noir#8426)
chore: remove `.aztec-sync-commit`
(noir-lang/noir#8415)
chore(test): Add more unit tests for
`inline_functions_with_at_most_one_instruction`
(noir-lang/noir#8418)
chore: add minor docs for interpreter
(noir-lang/noir#8397)
fix: print slice composite types surrounded by parentheses
(noir-lang/noir#8412)
feat: Skip SSA passes that contain any of the given messages
(noir-lang/noir#8416)
fix: disable range constraints using the predicate
(noir-lang/noir#8396)
chore: bumping external libraries
(noir-lang/noir#8406)
chore: redo typo PR by shystrui1199
(noir-lang/noir#8405)
feat(test): add `nargo_fuzz_target`
(noir-lang/noir#8308)
fix: allow names to collide in the values/types namespaces
(noir-lang/noir#8286)
fix: Fix sequencing of side-effects in lvalue
(noir-lang/noir#8384)
feat(greybox_fuzzer): Maximum executions parameter added
(noir-lang/noir#8390)
fix: warn on and discard unreachable statements after break and continue
(noir-lang/noir#8382)
fix: add handling for u128 infix ops in interpreter
(noir-lang/noir#8392)
chore: move acirgen tests into separate file
(noir-lang/noir#8376)
feat(fuzz): initial version of comptime vs brillig target for AST fuzzer
(noir-lang/noir#8335)
chore: apply lints to `ast_fuzzer`
(noir-lang/noir#8386)
chore: add note on AI generated PRs in `CONTRIBUTING.md`
(noir-lang/noir#8385)
chore: document flattening pass
(noir-lang/noir#8312)
fix: comptime shift-right overflow is zero
(noir-lang/noir#8380)
feat: let static_assert accept any type for its message
(noir-lang/noir#8322)
fix(expand): output safety comment before statements
(noir-lang/noir#8378)
chore: avoid need to rebuild after running tests
(noir-lang/noir#8379)
chore: bump dependencies (noir-lang/noir#8372)
chore: Add GITHUB_TOKEN to cross build
(noir-lang/noir#8370)
chore: redo typo PR by GarmashAlex
(noir-lang/noir#8364)
chore: remove unsafe code from greybox fuzzer
(noir-lang/noir#8315)
feat: add `--fuzz-timeout` to `nargo test` options
(noir-lang/noir#8326)
chore: bump external pinned commits
(noir-lang/noir#8334)
fix(expand): try to use "Self" in function calls
(noir-lang/noir#8353)
fix: Fix evaluation order of assignments with side-effects in their rhs
(noir-lang/noir#8342)
fix: let comptime Field value carry the field's sign
(noir-lang/noir#8343)
fix: Ordering of items in callstacks
(noir-lang/noir#8338)
chore: add snapshosts for nargo expand tests
(noir-lang/noir#8318)
fix(ownership): Clone global arrays
(noir-lang/noir#8328)
chore: Replace all SSA interpreter panics with error variants
(noir-lang/noir#8311)
feat: Metamorphic AST fuzzing
(noir-lang/noir#8299)
fix: fix some Display implementations for AST nodes
(noir-lang/noir#8316)
chore: remove leftover file
(noir-lang/noir#8313)
fix: uses non-zero points with ec-add-unsafe
(noir-lang/noir#8248)
END_COMMIT_OVERRIDE

---------

Co-authored-by: AztecBot <tech@aztecprotocol.com>
Co-authored-by: Tom French <15848336+TomAFrench@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ACIR != Brillig: mutating global arrays with minimum inliner aggressiveness

2 participants