Skip to content

fix(tag_attr): keep whitespace tokens when parsing#9981

Merged
asterite merged 6 commits intonoir-lang:masterfrom
blocksense-network:fix/keep-whitespace-in-attributes
Oct 6, 2025
Merged

fix(tag_attr): keep whitespace tokens when parsing#9981
asterite merged 6 commits intonoir-lang:masterfrom
blocksense-network:fix/keep-whitespace-in-attributes

Conversation

@Aristotelis2002
Copy link
Copy Markdown
Contributor

Description

Problem*

Resolves #9980

The parser for #[tag] attributes currently ignores all whitespace. This prevents the use of expressions that contain whitespace, such as x as u32, which is incorrectly parsed as xasu32.

This parser behavior blocks the use of verification attributes. Specifically, it has two major impacts:

  1. We cannot produce accurate error spans if the whitespaces are missing.
  2. We cannot use common expressions like x as u32 in verification attributes.

Summary*

This PR fixes a bug in the #[tag] attribute parser. The fix modifies the lexer and parser to ensure that whitespace within #[tag] attributes is handled correctly. This is important for our formal verification tool, as we use #[tag] attributes for our annotations.

The proposed solution involves temporarily disabling whitespace skipping in the lexer while parsing the attribute's content. This ensures that all characters, including whitespace, are preserved.

This change will make the parsing of #['tag] attributes more robust and predictable, enabling the correct use of verification attributes and improving error reporting.

Additional Context

This is one of the two PRs that will allow us to move our formal verification tool off of a Noir fork.

Documentation*

Check one:

  • No documentation needed.
  • Documentation included in this PR.
  • [For Experimental Features] Documentation to be submitted in a separate PR.

PR Checklist*

  • I have tested the changes locally.
  • I have formatted the changes with Prettier and/or cargo fmt on default settings.

@github-actions
Copy link
Copy Markdown
Contributor

Thank you for your contribution to the Noir language.

Please do not force push to this branch after the Noir team have started review of this PR. Doing so will only delay us merging your PR as we will need to start the review process from scratch.

Thanks for your understanding.

@aakoshh aakoshh requested a review from asterite September 24, 2025 19:54
@asterite
Copy link
Copy Markdown
Collaborator

Thank you! I think this is probably fine though it would be nice if there were tests for this.

We found a bug with the tests and we resolved it. The function
`eat_inner_attribute` would consume the whitespaces when
parsing the first token after `#[`.
This has now been resolved.
@Aristotelis2002
Copy link
Copy Markdown
Contributor Author

Thank you for the advice. We added some tests.

Fixed an issue similar to the one in 95c53b3 but for tag attributes
instead of inner tag attributes.
Added tests.
Added comments.
Copy link
Copy Markdown
Collaborator

@asterite asterite left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great, thank you!

@asterite
Copy link
Copy Markdown
Collaborator

Hm, it seems the formatter now needs to be updated to account for these extra whitespaces...

When the start of an attribute was being consumed we would keep the
starting whitespaces even for non-tag attributes which resulted in some
tests failing. This now has been addressed.
Now the formatter removes the trailing whitespaces from tag attributes.
A new test has been added.
@Aristotelis2002
Copy link
Copy Markdown
Contributor Author

I have now updated the formatter. I also fixed a bug which made some formatter tests fail. More details can be found in the commits descriptions.

Copy link
Copy Markdown
Collaborator

@asterite asterite left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great, thank you!

@asterite asterite closed this Oct 6, 2025
@asterite asterite reopened this Oct 6, 2025
@asterite asterite added this pull request to the merge queue Oct 6, 2025
Merged via the queue into noir-lang:master with commit 1d260df Oct 6, 2025
387 of 395 checks passed
AztecBot added a commit to AztecProtocol/aztec-packages that referenced this pull request Oct 8, 2025
Automated pull of nightly from the [noir](https://github.com/noir-lang/noir) programming language, a dependency of Aztec.
BEGIN_COMMIT_OVERRIDE
chore: Remove unnecessary allocation in `expr_with` (noir-lang/noir#10103)
chore(ACIR): inline `maybe_eq_predicate` (noir-lang/noir#10095)
chore: use `u64` over `Field` in template program (noir-lang/noir#10096)
chore: disallow slice arguments to blackbox functions (noir-lang/noir#10090)
chore(brillig_vm): Separate fuzzing module (noir-lang/noir#10091)
chore: ConstrainNotEqual requires acir predicate (noir-lang/noir#10062)
chore: bump version of bb used in tests (noir-lang/noir#10093)
chore: wrapping arithmetic tests (noir-lang/noir#9714)
chore(brillig_vm): Foreign call module and test re-org (noir-lang/noir#10089)
chore: add extra constraint folding pass (noir-lang/noir#9766)
chore: refactor brillig_blocks (noir-lang/noir#10088)
chore: add regression test for #9986 (noir-lang/noir#10087)
chore: Release Noir(1.0.0-beta.14) (noir-lang/noir#9942)
fix(tag_attr): keep whitespace tokens when parsing (noir-lang/noir#9981)
fix: hoist and then deduplicate (noir-lang/noir#10047)
chore: typos and some refactors in `acvm/src/pwg` (noir-lang/noir#10086)
chore: add in hack for `public_dispatch` (noir-lang/noir#10084)
fix(ssa): Avoid going through `i128` when casting signed to `u128` (noir-lang/noir#10045)
chore: avoid zero bits range-checks (noir-lang/noir#10083)
chore: bump external pinned commits (noir-lang/noir#10082)
fix(stdlib): Only compute the garbage `embedded_curve_result` result if we know we will need it (noir-lang/noir#10077)
fix(ssa): Keep defaults for values returned in the databus (noir-lang/noir#10042)
chore: remove unused predicate from mem-op solver (noir-lang/noir#10079)
chore(ACIR): snapshot tests for each instruction (noir-lang/noir#10071)
fix: remove generic length from ECDSA message hash in stdlib (noir-lang/noir#10043)
chore: validate that no jumps to function entry block exist (noir-lang/noir#10076)
feat(brillig): Centralize memory layout policy and reorganize memory regions (noir-lang/noir#9985)
chore(ci): fix permissions about publishing rustdoc (noir-lang/noir#10075)
chore(ACVM): use Vec instead of Hash for memory blocks (noir-lang/noir#10072)
feat(ssa): `constant_folding` with loop (noir-lang/noir#10019)
chore: take truncate into account for bit size (noir-lang/noir#10059)
chore: update check for `u128` overflow in `check_u128_mul_overflow` (noir-lang/noir#9998)
chore: update check for field overflow in `check_u128_mul_overflow` (noir-lang/noir#9968)
chore(ACIR): binary instructions snapshots (noir-lang/noir#10054)
chore(acir): SliceRemove refactor (noir-lang/noir#10058)
fix(fuzzer): Mark DivisionByZero with different types as equivalent (noir-lang/noir#10066)
chore(fuzz): Remove `is_frontend_friendly` from the AST fuzzer (noir-lang/noir#10046)
chore: use new ACIR syntax in docs, and some tests (noir-lang/noir#10057)
fix(ssa): SSA interpreter to use the 2nd arg in `slice_refcount` (noir-lang/noir#10034)
fix(ssa): SSA interpreter to return 0 for `Intrinsic::*RefCount` when constrained (noir-lang/noir#10033)
chore(ssa_fuzzer): fix array get/set  (noir-lang/noir#10031)
fix(acir): Extend slice on dynamic insertion and compilation panic when flattening (noir-lang/noir#10051)
chore(ACIR): extract convert_constrain_error helper (noir-lang/noir#10050)
chore(ACIR): expand signed lt, div and mod in SSA (noir-lang/noir#10036)
chore(ACIR): more consistent syntax and with less noise (noir-lang/noir#10014)
chore(acir): Code gen tests for slice intrinsics (noir-lang/noir#10017)
feat: parse and display SSA databus (noir-lang/noir#9991)
fix(ssa): Handle partially removed `ArrayGet` groups of complex type during OOB checks (noir-lang/noir#10027)
chore(acir): binary operations always have the same operand types (noir-lang/noir#10028)
feat: Add Module::parent and Module::child_modules (noir-lang/noir#10005)
chore: green light for ACVM optimisation (noir-lang/noir#10002)
chore: unit test for brillig solver (greenlight ACVM execution) (noir-lang/noir#9967)
chore(acir): avoid duplication when invoking brillig stdlib call (noir-lang/noir#10025)
chore: Use 8 partitions for rust tests (noir-lang/noir#10026)
chore: green light for ACVM execution audit (noir-lang/noir#9982)
chore: greenlight for ACVM execution (PWG) (noir-lang/noir#9961)
feat: optimize out noop casts on constants (noir-lang/noir#10024)
fix(mem2reg): consider call return aliases (noir-lang/noir#10016)
chore: bump external pinned commits (noir-lang/noir#10022)
fix(parser): enforce left brace after match expression (noir-lang/noir#10018)
chore(acir): Intrinsics and slice_ops modules as well as slice_ops doc comments (noir-lang/noir#10012)
fix: signed division by -1 can overflow (noir-lang/noir#9976)
chore(ci): fix external checks (noir-lang/noir#10009)
chore(ci): add provenance attestations to npm packages (noir-lang/noir#10011)
chore(ACIR): show all expressions as polynomials (noir-lang/noir#10007)
chore: remove unused feature flag (noir-lang/noir#9993)
chore(ci): fix docs breaking JS releases (noir-lang/noir#10010)
chore(ssa_fuzzer): add external coverage registration  (noir-lang/noir#9974)
END_COMMIT_OVERRIDE

Co-authored-by: Tom French <15848336+TomAFrench@users.noreply.github.com>
github-merge-queue bot pushed a commit to AztecProtocol/aztec-packages that referenced this pull request Oct 8, 2025
Automated pull of nightly from the
[noir](https://github.com/noir-lang/noir) programming language, a
dependency of Aztec.
BEGIN_COMMIT_OVERRIDE
chore: Remove unnecessary allocation in `expr_with`
(noir-lang/noir#10103)
chore(ACIR): inline `maybe_eq_predicate`
(noir-lang/noir#10095)
chore: use `u64` over `Field` in template program
(noir-lang/noir#10096)
chore: disallow slice arguments to blackbox functions
(noir-lang/noir#10090)
chore(brillig_vm): Separate fuzzing module
(noir-lang/noir#10091)
chore: ConstrainNotEqual requires acir predicate
(noir-lang/noir#10062)
chore: bump version of bb used in tests
(noir-lang/noir#10093)
chore: wrapping arithmetic tests
(noir-lang/noir#9714)
chore(brillig_vm): Foreign call module and test re-org
(noir-lang/noir#10089)
chore: add extra constraint folding pass
(noir-lang/noir#9766)
chore: refactor brillig_blocks
(noir-lang/noir#10088)
chore: add regression test for #9986
(noir-lang/noir#10087)
chore: Release Noir(1.0.0-beta.14)
(noir-lang/noir#9942)
fix(tag_attr): keep whitespace tokens when parsing
(noir-lang/noir#9981)
fix: hoist and then deduplicate
(noir-lang/noir#10047)
chore: typos and some refactors in `acvm/src/pwg`
(noir-lang/noir#10086)
chore: add in hack for `public_dispatch`
(noir-lang/noir#10084)
fix(ssa): Avoid going through `i128` when casting signed to `u128`
(noir-lang/noir#10045)
chore: avoid zero bits range-checks
(noir-lang/noir#10083)
chore: bump external pinned commits
(noir-lang/noir#10082)
fix(stdlib): Only compute the garbage `embedded_curve_result` result if
we know we will need it (noir-lang/noir#10077)
fix(ssa): Keep defaults for values returned in the databus
(noir-lang/noir#10042)
chore: remove unused predicate from mem-op solver
(noir-lang/noir#10079)
chore(ACIR): snapshot tests for each instruction
(noir-lang/noir#10071)
fix: remove generic length from ECDSA message hash in stdlib
(noir-lang/noir#10043)
chore: validate that no jumps to function entry block exist
(noir-lang/noir#10076)
feat(brillig): Centralize memory layout policy and reorganize memory
regions (noir-lang/noir#9985)
chore(ci): fix permissions about publishing rustdoc
(noir-lang/noir#10075)
chore(ACVM): use Vec instead of Hash for memory blocks
(noir-lang/noir#10072)
feat(ssa): `constant_folding` with loop
(noir-lang/noir#10019)
chore: take truncate into account for bit size
(noir-lang/noir#10059)
chore: update check for `u128` overflow in `check_u128_mul_overflow`
(noir-lang/noir#9998)
chore: update check for field overflow in `check_u128_mul_overflow`
(noir-lang/noir#9968)
chore(ACIR): binary instructions snapshots
(noir-lang/noir#10054)
chore(acir): SliceRemove refactor
(noir-lang/noir#10058)
fix(fuzzer): Mark DivisionByZero with different types as equivalent
(noir-lang/noir#10066)
chore(fuzz): Remove `is_frontend_friendly` from the AST fuzzer
(noir-lang/noir#10046)
chore: use new ACIR syntax in docs, and some tests
(noir-lang/noir#10057)
fix(ssa): SSA interpreter to use the 2nd arg in `slice_refcount`
(noir-lang/noir#10034)
fix(ssa): SSA interpreter to return 0 for `Intrinsic::*RefCount` when
constrained (noir-lang/noir#10033)
chore(ssa_fuzzer): fix array get/set
(noir-lang/noir#10031)
fix(acir): Extend slice on dynamic insertion and compilation panic when
flattening (noir-lang/noir#10051)
chore(ACIR): extract convert_constrain_error helper
(noir-lang/noir#10050)
chore(ACIR): expand signed lt, div and mod in SSA
(noir-lang/noir#10036)
chore(ACIR): more consistent syntax and with less noise
(noir-lang/noir#10014)
chore(acir): Code gen tests for slice intrinsics
(noir-lang/noir#10017)
feat: parse and display SSA databus
(noir-lang/noir#9991)
fix(ssa): Handle partially removed `ArrayGet` groups of complex type
during OOB checks (noir-lang/noir#10027)
chore(acir): binary operations always have the same operand types
(noir-lang/noir#10028)
feat: Add Module::parent and Module::child_modules
(noir-lang/noir#10005)
chore: green light for ACVM optimisation
(noir-lang/noir#10002)
chore: unit test for brillig solver (greenlight ACVM execution)
(noir-lang/noir#9967)
chore(acir): avoid duplication when invoking brillig stdlib call
(noir-lang/noir#10025)
chore: Use 8 partitions for rust tests
(noir-lang/noir#10026)
chore: green light for ACVM execution audit
(noir-lang/noir#9982)
chore: greenlight for ACVM execution (PWG)
(noir-lang/noir#9961)
feat: optimize out noop casts on constants
(noir-lang/noir#10024)
fix(mem2reg): consider call return aliases
(noir-lang/noir#10016)
chore: bump external pinned commits
(noir-lang/noir#10022)
fix(parser): enforce left brace after match expression
(noir-lang/noir#10018)
chore(acir): Intrinsics and slice_ops modules as well as slice_ops doc
comments (noir-lang/noir#10012)
fix: signed division by -1 can overflow
(noir-lang/noir#9976)
chore(ci): fix external checks
(noir-lang/noir#10009)
chore(ci): add provenance attestations to npm packages
(noir-lang/noir#10011)
chore(ACIR): show all expressions as polynomials
(noir-lang/noir#10007)
chore: remove unused feature flag
(noir-lang/noir#9993)
chore(ci): fix docs breaking JS releases
(noir-lang/noir#10010)
chore(ssa_fuzzer): add external coverage registration
(noir-lang/noir#9974)
END_COMMIT_OVERRIDE
mralj pushed a commit to AztecProtocol/aztec-packages that referenced this pull request Oct 13, 2025
Automated pull of nightly from the [noir](https://github.com/noir-lang/noir) programming language, a dependency of Aztec.
BEGIN_COMMIT_OVERRIDE
chore: Remove unnecessary allocation in `expr_with` (noir-lang/noir#10103)
chore(ACIR): inline `maybe_eq_predicate` (noir-lang/noir#10095)
chore: use `u64` over `Field` in template program (noir-lang/noir#10096)
chore: disallow slice arguments to blackbox functions (noir-lang/noir#10090)
chore(brillig_vm): Separate fuzzing module (noir-lang/noir#10091)
chore: ConstrainNotEqual requires acir predicate (noir-lang/noir#10062)
chore: bump version of bb used in tests (noir-lang/noir#10093)
chore: wrapping arithmetic tests (noir-lang/noir#9714)
chore(brillig_vm): Foreign call module and test re-org (noir-lang/noir#10089)
chore: add extra constraint folding pass (noir-lang/noir#9766)
chore: refactor brillig_blocks (noir-lang/noir#10088)
chore: add regression test for #9986 (noir-lang/noir#10087)
chore: Release Noir(1.0.0-beta.14) (noir-lang/noir#9942)
fix(tag_attr): keep whitespace tokens when parsing (noir-lang/noir#9981)
fix: hoist and then deduplicate (noir-lang/noir#10047)
chore: typos and some refactors in `acvm/src/pwg` (noir-lang/noir#10086)
chore: add in hack for `public_dispatch` (noir-lang/noir#10084)
fix(ssa): Avoid going through `i128` when casting signed to `u128` (noir-lang/noir#10045)
chore: avoid zero bits range-checks (noir-lang/noir#10083)
chore: bump external pinned commits (noir-lang/noir#10082)
fix(stdlib): Only compute the garbage `embedded_curve_result` result if we know we will need it (noir-lang/noir#10077)
fix(ssa): Keep defaults for values returned in the databus (noir-lang/noir#10042)
chore: remove unused predicate from mem-op solver (noir-lang/noir#10079)
chore(ACIR): snapshot tests for each instruction (noir-lang/noir#10071)
fix: remove generic length from ECDSA message hash in stdlib (noir-lang/noir#10043)
chore: validate that no jumps to function entry block exist (noir-lang/noir#10076)
feat(brillig): Centralize memory layout policy and reorganize memory regions (noir-lang/noir#9985)
chore(ci): fix permissions about publishing rustdoc (noir-lang/noir#10075)
chore(ACVM): use Vec instead of Hash for memory blocks (noir-lang/noir#10072)
feat(ssa): `constant_folding` with loop (noir-lang/noir#10019)
chore: take truncate into account for bit size (noir-lang/noir#10059)
chore: update check for `u128` overflow in `check_u128_mul_overflow` (noir-lang/noir#9998)
chore: update check for field overflow in `check_u128_mul_overflow` (noir-lang/noir#9968)
chore(ACIR): binary instructions snapshots (noir-lang/noir#10054)
chore(acir): SliceRemove refactor (noir-lang/noir#10058)
fix(fuzzer): Mark DivisionByZero with different types as equivalent (noir-lang/noir#10066)
chore(fuzz): Remove `is_frontend_friendly` from the AST fuzzer (noir-lang/noir#10046)
chore: use new ACIR syntax in docs, and some tests (noir-lang/noir#10057)
fix(ssa): SSA interpreter to use the 2nd arg in `slice_refcount` (noir-lang/noir#10034)
fix(ssa): SSA interpreter to return 0 for `Intrinsic::*RefCount` when constrained (noir-lang/noir#10033)
chore(ssa_fuzzer): fix array get/set  (noir-lang/noir#10031)
fix(acir): Extend slice on dynamic insertion and compilation panic when flattening (noir-lang/noir#10051)
chore(ACIR): extract convert_constrain_error helper (noir-lang/noir#10050)
chore(ACIR): expand signed lt, div and mod in SSA (noir-lang/noir#10036)
chore(ACIR): more consistent syntax and with less noise (noir-lang/noir#10014)
chore(acir): Code gen tests for slice intrinsics (noir-lang/noir#10017)
feat: parse and display SSA databus (noir-lang/noir#9991)
fix(ssa): Handle partially removed `ArrayGet` groups of complex type during OOB checks (noir-lang/noir#10027)
chore(acir): binary operations always have the same operand types (noir-lang/noir#10028)
feat: Add Module::parent and Module::child_modules (noir-lang/noir#10005)
chore: green light for ACVM optimisation (noir-lang/noir#10002)
chore: unit test for brillig solver (greenlight ACVM execution) (noir-lang/noir#9967)
chore(acir): avoid duplication when invoking brillig stdlib call (noir-lang/noir#10025)
chore: Use 8 partitions for rust tests (noir-lang/noir#10026)
chore: green light for ACVM execution audit (noir-lang/noir#9982)
chore: greenlight for ACVM execution (PWG) (noir-lang/noir#9961)
feat: optimize out noop casts on constants (noir-lang/noir#10024)
fix(mem2reg): consider call return aliases (noir-lang/noir#10016)
chore: bump external pinned commits (noir-lang/noir#10022)
fix(parser): enforce left brace after match expression (noir-lang/noir#10018)
chore(acir): Intrinsics and slice_ops modules as well as slice_ops doc comments (noir-lang/noir#10012)
fix: signed division by -1 can overflow (noir-lang/noir#9976)
chore(ci): fix external checks (noir-lang/noir#10009)
chore(ci): add provenance attestations to npm packages (noir-lang/noir#10011)
chore(ACIR): show all expressions as polynomials (noir-lang/noir#10007)
chore: remove unused feature flag (noir-lang/noir#9993)
chore(ci): fix docs breaking JS releases (noir-lang/noir#10010)
chore(ssa_fuzzer): add external coverage registration  (noir-lang/noir#9974)
END_COMMIT_OVERRIDE

Co-authored-by: Tom French <15848336+TomAFrench@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

tag_attr parser incorrectly handles whitespace

2 participants