Merged
Conversation
# Check out Noir during build Changes the way we sync the code from Noir away from having a periodically synced copy of the codebase, to checking out the code during the build based on a tag/branch/commit indicated by a file or env var. The following new files were added: * `noir/noir-repo-ref` contains the reference in Noir that should be checked out * [noir/scripts/sync.sh](https://github.com/AztecProtocol/aztec-packages/blob/noir-sync-rework/noir/scripts/sync.sh) performs the checkouts; the `init` command does the initial checkout and the `update` command switches to new branches * `noir/bootstrap.sh` performs an `init` and `update` before every command except `clean` TODO: - [x] Sync during the build - [x] Detect the need to switch branches - [x] Create a patch file from local commits - [x] Apply the patch file during checkouts - [x] Fix patch creation to only include non-empty commits because `git am` stops at them, and with `git am --allow-empty` it just fails - [x] Add to the git hooks something to prevent the user from switching branches without creating and committing a patch file, if there are commits in `noir-repo`. (Currently an error is returned by `sync.sh` but if we created a patch file, it should be committed on the aztec-packages branch we switched away from).\ - [x] Update the daily workflow to update `noir-repo-ref` to the next release. - [x] Test the `pull-noir` workflow: * Action that create the PR: https://github.com/AztecProtocol/aztec-packages/actions/runs/13768222542 * PR that bumps the ref: #12608 ## Rationale Currently there is a two-way sync between Noir and aztec-packages, with automated workflows running daily to push changes between the `master` branches by opening (or appending to already open) sync PRs. The two-way sync was required because for example frequent changes in the `bb` interface were easier to update in this repo and pushed back to Noir, than to have to wait for a release cycle. The problem we frequently encounter with the sync from Noir is that some of the integration tests in aztec-packages break due to subtle changes in the compiler, which we don't know about until we open the sync PR. At that point we're looking at a potentially long list of changes (every commit on `master` since the last sync), and it's not clear which one is the culprit. We bisect the build to find a failure, fix it in the PR, but this often just reveals the next bug to be fixed. This process can take days, and during that time the sync process has to be paused - if it runs it just makes the PR bigger, with potentially more bugs. Regardless of whether we pause it or let it append to the PR, no new features or fixes get merged into aztec-packages until all the bugs are squashed. Instead what we wish to achieve here is that we can: 1. Point aztec-packages at an integration/release branch, or nightly tags to be synced from, and if there is a problem then we can cherry pick changes from `master` onto the release branch so that the good stuff can go in. 2. Initiate CI runs against arbitrary branches on Noir, so that we can test smaller change sets before they are synced to aztec-packages, to see their effect in isolation. ## Example The following example shows: 1. Initialising `noir-repo` to a tag 2. Creating a file and commit 3. Trying to switch branch is rejected because the commit is not in the patch 4. Creating a patch 5. Switching to a branch 6. Creating another commit and adding it to the patch 7. Switching to a commit 8. See both commits are re-applied ```console % rm -rf noir-repo % cat noir-repo-ref nightly-2025-03-07 % scripts/sync.sh init Cloning into 'noir-repo'... ... Note: switching to '37be49fd081f33dc7256d23cee8ccc0719c50a82'. ... HEAD is now at 37be49fd08 chore!: convert `TraitMethodNotInScope` to error (#7427) ➤ YN0000: ┌ Resolution step ➤ YN0002: │ @algolia/autocomplete-core@npm:1.9.3 doesn't provide @algolia/client-search (pbc410), requested by @algolia/autocomplete-shared ... ➤ YN0000: Done with warnings in 20s 227ms [detached HEAD 6cee7cd4d6] Noir local fixup commit. 4 files changed, 8 insertions(+), 9 deletions(-) [detached HEAD 94cffa4cb1] Noir local patch commit. % ./scripts/sync.sh info Fixup commit: 30d08b921c71edb1c75491e459df6d54917c5e51 Checkout commit: 37be49fd081f33dc7256d23cee8ccc0719c50a82 Wanted: nightly-2025-03-07 Needs switch: no Needs patch: no Detached: yes On branch: no Branch name: n/a Has wanted tag: yes Has tag commit: yes Has patch commit: yes Last commit is patch: yes Has fixup and patch: yes % cd noir-repo % git status Not currently on any branch. nothing to commit, working tree clean % echo "Hello" > hello.txt && git add . && git commit -m "Hello" [detached HEAD 6053b5e04c] Hello 1 file changed, 1 insertion(+) create mode 100644 hello.txt % cd .. % NOIR_REPO_REF=master scripts/sync.sh update noir-repo is on a detached HEAD and the last commit is not the patch marker commit; switching from nightly-2025-03-07 to master could meand losing those commits. Please use the 'make-patch' command to create a noir-repo.patch file and commit it in aztec-packages, so that it is applied after each checkout; make sure to commit the patch on the branch where it should be. % scripts/sync.sh make-patch ../patches/0001-Noir-local-patch-commit.patch ../patches/0002-Hello.patch [detached HEAD 28f53e097b] Noir local patch commit. % NOIR_REPO_REF=master scripts/sync.sh update ... From https://github.com/noir-lang/noir * branch master -> FETCH_HEAD Warning: you are leaving 4 commits behind, not connected to any of your branches: 28f53e097b Noir local patch commit. 6053b5e04c Hello 94cffa4cb1 Noir local patch commit. 6cee7cd4d6 Noir local fixup commit. If you want to keep them by creating a new branch, this may be a good time to do so with: git branch <new-branch-name> 28f53e097b branch 'master' set up to track 'origin/master'. Switched to a new branch 'master' Already up to date. ➤ YN0000: ┌ Resolution step ➤ YN0002: │ @algolia/autocomplete-core@npm:1.9.3 doesn't provide @algolia/client-search (pbc410), requested by @algolia/autocomplete-shared ... ➤ YN0000: Done with warnings in 2s 686ms [master 7cee0c05bb] Noir local fixup commit. 4 files changed, 8 insertions(+), 9 deletions(-) Applying: Hello [master ad6812fae7] Noir local patch commit. % cd noir-repo % echo "World" > world.txt && git add . && git commit -m "World" [master 16f0054dc9] World 1 file changed, 1 insertion(+) create mode 100644 world.txt % cd .. % cat .noir-repo-last-ref master % scripts/sync.sh make-patch ../patches/0001-Hello.patch ../patches/0002-Noir-local-patch-commit.patch ../patches/0003-World.patch [master 2cc18fd434] Noir local patch commit. % cat noir-repo.patch From 6a8c19cd6fd70e4b982ec8ca10b4b70363636f97 Mon Sep 17 00:00:00 2001 From: aakoshh <akosh@aztecprotocol.com> Date: Fri, 7 Mar 2025 23:46:36 +0000 Subject: [PATCH 1/3] Hello --- hello.txt | 1 + 1 file changed, 1 insertion(+) create mode 100644 hello.txt diff --git a/hello.txt b/hello.txt new file mode 100644 index 0000000000..e965047ad7 --- /dev/null +++ b/hello.txt @@ -0,0 +1 @@ +Hello -- 9.43.0 From 16f0054dc961105cdb264d6ace10ff9061b7b96d Mon Sep 17 00:00:00 2001 From: aakoshh <akosh@aztecprotocol.com> Date: Fri, 7 Mar 2025 23:49:19 +0000 Subject: [PATCH 3/3] World --- world.txt | 1 + 1 file changed, 1 insertion(+) create mode 100644 world.txt diff --git a/world.txt b/world.txt new file mode 100644 index 0000000000..216e97ce08 --- /dev/null +++ b/world.txt @@ -0,0 +1 @@ +World -- 2.43.0 % export NOIR_REPO_REF=a9de769f3743733ac754a5d6c4e147ba8283a336 % scripts/sync.sh update ... From https://github.com/noir-lang/noir * branch a9de769f3743733ac754a5d6c4e147ba8283a336 -> FETCH_HEAD Note: switching to 'a9de769f3743733ac754a5d6c4e147ba8283a336'. ... HEAD is now at a9de769f37 feat: add optional oracle resolver url in `acvm_cli` (#7630) ➤ YN0000: ┌ Resolution step ... ➤ YN0000: └ Completed in 1s 219ms ➤ YN0000: Done with warnings in 2s 643ms [detached HEAD 53cbe7bae4] Noir local fixup commit. 4 files changed, 8 insertions(+), 9 deletions(-) Applying: Hello Applying: World [detached HEAD c4921bc4a8] Noir local patch commit. % ls noir-repo/*.txt 16s noir-repo/hello.txt noir-repo/world.txt % ``` --------- Co-authored-by: ludamad <adam.domurad@gmail.com> Co-authored-by: Tom French <15848336+TomAFrench@users.noreply.github.com>
5 tasks
TomAFrench
approved these changes
Mar 14, 2025
TomAFrench
added a commit
that referenced
this pull request
Mar 14, 2025
* master: fix: filter for empty attestations when pulling from block (#12740) feat: one-way noir sync (#12592) feat(avm): Address derivation gadget (#12721) chore(ci): add workflow dispatch to masternet (#12739) feat: add default accounts (#12734) feat: gas reports and snapshots (#12724) fix(avm): fix vm1 fake proof size (#12733) feat(bb): extend_edges optimization for zero values past end_index (#12703) fix: remove hard coding of constructor for account manager (#12678) git subrepo push --branch=master noir-projects/aztec-nr git_subrepo.sh: Fix parent in .gitrepo file. [skip ci] chore: replace relative paths to noir-protocol-circuits git subrepo push --branch=master barretenberg fix: verify_honk_proof inputs generation in bootstrap (#12457) fix: Patches to cycle_group and cycle_group fuzzer (#12385)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Contains the changes from #12574 to replace the copy of the noir-repo with a
noir-repo-reffile to point at the nightly version which we pull during the build.Updated
noir-repo-refin the PR to match the last merge sync PR #12624Alternatively it could be set to include the commits from the open #12669 by setting it to nightly-2025-03-14
Added a patch to replicate the changes to honk test programs in #12266
Opened noir-lang/noir#7706 to migrate the patch back to Noir