fix(tag_attr): keep whitespace tokens when parsing#9981
fix(tag_attr): keep whitespace tokens when parsing#9981asterite merged 6 commits intonoir-lang:masterfrom
Conversation
|
Thank you for your contribution to the Noir language. Please do not force push to this branch after the Noir team have started review of this PR. Doing so will only delay us merging your PR as we will need to start the review process from scratch. Thanks for your understanding. |
|
Thank you! I think this is probably fine though it would be nice if there were tests for this. |
We found a bug with the tests and we resolved it. The function `eat_inner_attribute` would consume the whitespaces when parsing the first token after `#[`. This has now been resolved.
|
Thank you for the advice. We added some tests. |
Fixed an issue similar to the one in 95c53b3 but for tag attributes instead of inner tag attributes. Added tests. Added comments.
asterite
left a comment
There was a problem hiding this comment.
Looks great, thank you!
|
Hm, it seems the formatter now needs to be updated to account for these extra whitespaces... |
When the start of an attribute was being consumed we would keep the starting whitespaces even for non-tag attributes which resulted in some tests failing. This now has been addressed.
Now the formatter removes the trailing whitespaces from tag attributes. A new test has been added.
|
I have now updated the formatter. I also fixed a bug which made some formatter tests fail. More details can be found in the commits descriptions. |
asterite
left a comment
There was a problem hiding this comment.
Looks great, thank you!
Automated pull of nightly from the [noir](https://github.com/noir-lang/noir) programming language, a dependency of Aztec. BEGIN_COMMIT_OVERRIDE chore: Remove unnecessary allocation in `expr_with` (noir-lang/noir#10103) chore(ACIR): inline `maybe_eq_predicate` (noir-lang/noir#10095) chore: use `u64` over `Field` in template program (noir-lang/noir#10096) chore: disallow slice arguments to blackbox functions (noir-lang/noir#10090) chore(brillig_vm): Separate fuzzing module (noir-lang/noir#10091) chore: ConstrainNotEqual requires acir predicate (noir-lang/noir#10062) chore: bump version of bb used in tests (noir-lang/noir#10093) chore: wrapping arithmetic tests (noir-lang/noir#9714) chore(brillig_vm): Foreign call module and test re-org (noir-lang/noir#10089) chore: add extra constraint folding pass (noir-lang/noir#9766) chore: refactor brillig_blocks (noir-lang/noir#10088) chore: add regression test for #9986 (noir-lang/noir#10087) chore: Release Noir(1.0.0-beta.14) (noir-lang/noir#9942) fix(tag_attr): keep whitespace tokens when parsing (noir-lang/noir#9981) fix: hoist and then deduplicate (noir-lang/noir#10047) chore: typos and some refactors in `acvm/src/pwg` (noir-lang/noir#10086) chore: add in hack for `public_dispatch` (noir-lang/noir#10084) fix(ssa): Avoid going through `i128` when casting signed to `u128` (noir-lang/noir#10045) chore: avoid zero bits range-checks (noir-lang/noir#10083) chore: bump external pinned commits (noir-lang/noir#10082) fix(stdlib): Only compute the garbage `embedded_curve_result` result if we know we will need it (noir-lang/noir#10077) fix(ssa): Keep defaults for values returned in the databus (noir-lang/noir#10042) chore: remove unused predicate from mem-op solver (noir-lang/noir#10079) chore(ACIR): snapshot tests for each instruction (noir-lang/noir#10071) fix: remove generic length from ECDSA message hash in stdlib (noir-lang/noir#10043) chore: validate that no jumps to function entry block exist (noir-lang/noir#10076) feat(brillig): Centralize memory layout policy and reorganize memory regions (noir-lang/noir#9985) chore(ci): fix permissions about publishing rustdoc (noir-lang/noir#10075) chore(ACVM): use Vec instead of Hash for memory blocks (noir-lang/noir#10072) feat(ssa): `constant_folding` with loop (noir-lang/noir#10019) chore: take truncate into account for bit size (noir-lang/noir#10059) chore: update check for `u128` overflow in `check_u128_mul_overflow` (noir-lang/noir#9998) chore: update check for field overflow in `check_u128_mul_overflow` (noir-lang/noir#9968) chore(ACIR): binary instructions snapshots (noir-lang/noir#10054) chore(acir): SliceRemove refactor (noir-lang/noir#10058) fix(fuzzer): Mark DivisionByZero with different types as equivalent (noir-lang/noir#10066) chore(fuzz): Remove `is_frontend_friendly` from the AST fuzzer (noir-lang/noir#10046) chore: use new ACIR syntax in docs, and some tests (noir-lang/noir#10057) fix(ssa): SSA interpreter to use the 2nd arg in `slice_refcount` (noir-lang/noir#10034) fix(ssa): SSA interpreter to return 0 for `Intrinsic::*RefCount` when constrained (noir-lang/noir#10033) chore(ssa_fuzzer): fix array get/set (noir-lang/noir#10031) fix(acir): Extend slice on dynamic insertion and compilation panic when flattening (noir-lang/noir#10051) chore(ACIR): extract convert_constrain_error helper (noir-lang/noir#10050) chore(ACIR): expand signed lt, div and mod in SSA (noir-lang/noir#10036) chore(ACIR): more consistent syntax and with less noise (noir-lang/noir#10014) chore(acir): Code gen tests for slice intrinsics (noir-lang/noir#10017) feat: parse and display SSA databus (noir-lang/noir#9991) fix(ssa): Handle partially removed `ArrayGet` groups of complex type during OOB checks (noir-lang/noir#10027) chore(acir): binary operations always have the same operand types (noir-lang/noir#10028) feat: Add Module::parent and Module::child_modules (noir-lang/noir#10005) chore: green light for ACVM optimisation (noir-lang/noir#10002) chore: unit test for brillig solver (greenlight ACVM execution) (noir-lang/noir#9967) chore(acir): avoid duplication when invoking brillig stdlib call (noir-lang/noir#10025) chore: Use 8 partitions for rust tests (noir-lang/noir#10026) chore: green light for ACVM execution audit (noir-lang/noir#9982) chore: greenlight for ACVM execution (PWG) (noir-lang/noir#9961) feat: optimize out noop casts on constants (noir-lang/noir#10024) fix(mem2reg): consider call return aliases (noir-lang/noir#10016) chore: bump external pinned commits (noir-lang/noir#10022) fix(parser): enforce left brace after match expression (noir-lang/noir#10018) chore(acir): Intrinsics and slice_ops modules as well as slice_ops doc comments (noir-lang/noir#10012) fix: signed division by -1 can overflow (noir-lang/noir#9976) chore(ci): fix external checks (noir-lang/noir#10009) chore(ci): add provenance attestations to npm packages (noir-lang/noir#10011) chore(ACIR): show all expressions as polynomials (noir-lang/noir#10007) chore: remove unused feature flag (noir-lang/noir#9993) chore(ci): fix docs breaking JS releases (noir-lang/noir#10010) chore(ssa_fuzzer): add external coverage registration (noir-lang/noir#9974) END_COMMIT_OVERRIDE Co-authored-by: Tom French <15848336+TomAFrench@users.noreply.github.com>
Automated pull of nightly from the [noir](https://github.com/noir-lang/noir) programming language, a dependency of Aztec. BEGIN_COMMIT_OVERRIDE chore: Remove unnecessary allocation in `expr_with` (noir-lang/noir#10103) chore(ACIR): inline `maybe_eq_predicate` (noir-lang/noir#10095) chore: use `u64` over `Field` in template program (noir-lang/noir#10096) chore: disallow slice arguments to blackbox functions (noir-lang/noir#10090) chore(brillig_vm): Separate fuzzing module (noir-lang/noir#10091) chore: ConstrainNotEqual requires acir predicate (noir-lang/noir#10062) chore: bump version of bb used in tests (noir-lang/noir#10093) chore: wrapping arithmetic tests (noir-lang/noir#9714) chore(brillig_vm): Foreign call module and test re-org (noir-lang/noir#10089) chore: add extra constraint folding pass (noir-lang/noir#9766) chore: refactor brillig_blocks (noir-lang/noir#10088) chore: add regression test for #9986 (noir-lang/noir#10087) chore: Release Noir(1.0.0-beta.14) (noir-lang/noir#9942) fix(tag_attr): keep whitespace tokens when parsing (noir-lang/noir#9981) fix: hoist and then deduplicate (noir-lang/noir#10047) chore: typos and some refactors in `acvm/src/pwg` (noir-lang/noir#10086) chore: add in hack for `public_dispatch` (noir-lang/noir#10084) fix(ssa): Avoid going through `i128` when casting signed to `u128` (noir-lang/noir#10045) chore: avoid zero bits range-checks (noir-lang/noir#10083) chore: bump external pinned commits (noir-lang/noir#10082) fix(stdlib): Only compute the garbage `embedded_curve_result` result if we know we will need it (noir-lang/noir#10077) fix(ssa): Keep defaults for values returned in the databus (noir-lang/noir#10042) chore: remove unused predicate from mem-op solver (noir-lang/noir#10079) chore(ACIR): snapshot tests for each instruction (noir-lang/noir#10071) fix: remove generic length from ECDSA message hash in stdlib (noir-lang/noir#10043) chore: validate that no jumps to function entry block exist (noir-lang/noir#10076) feat(brillig): Centralize memory layout policy and reorganize memory regions (noir-lang/noir#9985) chore(ci): fix permissions about publishing rustdoc (noir-lang/noir#10075) chore(ACVM): use Vec instead of Hash for memory blocks (noir-lang/noir#10072) feat(ssa): `constant_folding` with loop (noir-lang/noir#10019) chore: take truncate into account for bit size (noir-lang/noir#10059) chore: update check for `u128` overflow in `check_u128_mul_overflow` (noir-lang/noir#9998) chore: update check for field overflow in `check_u128_mul_overflow` (noir-lang/noir#9968) chore(ACIR): binary instructions snapshots (noir-lang/noir#10054) chore(acir): SliceRemove refactor (noir-lang/noir#10058) fix(fuzzer): Mark DivisionByZero with different types as equivalent (noir-lang/noir#10066) chore(fuzz): Remove `is_frontend_friendly` from the AST fuzzer (noir-lang/noir#10046) chore: use new ACIR syntax in docs, and some tests (noir-lang/noir#10057) fix(ssa): SSA interpreter to use the 2nd arg in `slice_refcount` (noir-lang/noir#10034) fix(ssa): SSA interpreter to return 0 for `Intrinsic::*RefCount` when constrained (noir-lang/noir#10033) chore(ssa_fuzzer): fix array get/set (noir-lang/noir#10031) fix(acir): Extend slice on dynamic insertion and compilation panic when flattening (noir-lang/noir#10051) chore(ACIR): extract convert_constrain_error helper (noir-lang/noir#10050) chore(ACIR): expand signed lt, div and mod in SSA (noir-lang/noir#10036) chore(ACIR): more consistent syntax and with less noise (noir-lang/noir#10014) chore(acir): Code gen tests for slice intrinsics (noir-lang/noir#10017) feat: parse and display SSA databus (noir-lang/noir#9991) fix(ssa): Handle partially removed `ArrayGet` groups of complex type during OOB checks (noir-lang/noir#10027) chore(acir): binary operations always have the same operand types (noir-lang/noir#10028) feat: Add Module::parent and Module::child_modules (noir-lang/noir#10005) chore: green light for ACVM optimisation (noir-lang/noir#10002) chore: unit test for brillig solver (greenlight ACVM execution) (noir-lang/noir#9967) chore(acir): avoid duplication when invoking brillig stdlib call (noir-lang/noir#10025) chore: Use 8 partitions for rust tests (noir-lang/noir#10026) chore: green light for ACVM execution audit (noir-lang/noir#9982) chore: greenlight for ACVM execution (PWG) (noir-lang/noir#9961) feat: optimize out noop casts on constants (noir-lang/noir#10024) fix(mem2reg): consider call return aliases (noir-lang/noir#10016) chore: bump external pinned commits (noir-lang/noir#10022) fix(parser): enforce left brace after match expression (noir-lang/noir#10018) chore(acir): Intrinsics and slice_ops modules as well as slice_ops doc comments (noir-lang/noir#10012) fix: signed division by -1 can overflow (noir-lang/noir#9976) chore(ci): fix external checks (noir-lang/noir#10009) chore(ci): add provenance attestations to npm packages (noir-lang/noir#10011) chore(ACIR): show all expressions as polynomials (noir-lang/noir#10007) chore: remove unused feature flag (noir-lang/noir#9993) chore(ci): fix docs breaking JS releases (noir-lang/noir#10010) chore(ssa_fuzzer): add external coverage registration (noir-lang/noir#9974) END_COMMIT_OVERRIDE
Automated pull of nightly from the [noir](https://github.com/noir-lang/noir) programming language, a dependency of Aztec. BEGIN_COMMIT_OVERRIDE chore: Remove unnecessary allocation in `expr_with` (noir-lang/noir#10103) chore(ACIR): inline `maybe_eq_predicate` (noir-lang/noir#10095) chore: use `u64` over `Field` in template program (noir-lang/noir#10096) chore: disallow slice arguments to blackbox functions (noir-lang/noir#10090) chore(brillig_vm): Separate fuzzing module (noir-lang/noir#10091) chore: ConstrainNotEqual requires acir predicate (noir-lang/noir#10062) chore: bump version of bb used in tests (noir-lang/noir#10093) chore: wrapping arithmetic tests (noir-lang/noir#9714) chore(brillig_vm): Foreign call module and test re-org (noir-lang/noir#10089) chore: add extra constraint folding pass (noir-lang/noir#9766) chore: refactor brillig_blocks (noir-lang/noir#10088) chore: add regression test for #9986 (noir-lang/noir#10087) chore: Release Noir(1.0.0-beta.14) (noir-lang/noir#9942) fix(tag_attr): keep whitespace tokens when parsing (noir-lang/noir#9981) fix: hoist and then deduplicate (noir-lang/noir#10047) chore: typos and some refactors in `acvm/src/pwg` (noir-lang/noir#10086) chore: add in hack for `public_dispatch` (noir-lang/noir#10084) fix(ssa): Avoid going through `i128` when casting signed to `u128` (noir-lang/noir#10045) chore: avoid zero bits range-checks (noir-lang/noir#10083) chore: bump external pinned commits (noir-lang/noir#10082) fix(stdlib): Only compute the garbage `embedded_curve_result` result if we know we will need it (noir-lang/noir#10077) fix(ssa): Keep defaults for values returned in the databus (noir-lang/noir#10042) chore: remove unused predicate from mem-op solver (noir-lang/noir#10079) chore(ACIR): snapshot tests for each instruction (noir-lang/noir#10071) fix: remove generic length from ECDSA message hash in stdlib (noir-lang/noir#10043) chore: validate that no jumps to function entry block exist (noir-lang/noir#10076) feat(brillig): Centralize memory layout policy and reorganize memory regions (noir-lang/noir#9985) chore(ci): fix permissions about publishing rustdoc (noir-lang/noir#10075) chore(ACVM): use Vec instead of Hash for memory blocks (noir-lang/noir#10072) feat(ssa): `constant_folding` with loop (noir-lang/noir#10019) chore: take truncate into account for bit size (noir-lang/noir#10059) chore: update check for `u128` overflow in `check_u128_mul_overflow` (noir-lang/noir#9998) chore: update check for field overflow in `check_u128_mul_overflow` (noir-lang/noir#9968) chore(ACIR): binary instructions snapshots (noir-lang/noir#10054) chore(acir): SliceRemove refactor (noir-lang/noir#10058) fix(fuzzer): Mark DivisionByZero with different types as equivalent (noir-lang/noir#10066) chore(fuzz): Remove `is_frontend_friendly` from the AST fuzzer (noir-lang/noir#10046) chore: use new ACIR syntax in docs, and some tests (noir-lang/noir#10057) fix(ssa): SSA interpreter to use the 2nd arg in `slice_refcount` (noir-lang/noir#10034) fix(ssa): SSA interpreter to return 0 for `Intrinsic::*RefCount` when constrained (noir-lang/noir#10033) chore(ssa_fuzzer): fix array get/set (noir-lang/noir#10031) fix(acir): Extend slice on dynamic insertion and compilation panic when flattening (noir-lang/noir#10051) chore(ACIR): extract convert_constrain_error helper (noir-lang/noir#10050) chore(ACIR): expand signed lt, div and mod in SSA (noir-lang/noir#10036) chore(ACIR): more consistent syntax and with less noise (noir-lang/noir#10014) chore(acir): Code gen tests for slice intrinsics (noir-lang/noir#10017) feat: parse and display SSA databus (noir-lang/noir#9991) fix(ssa): Handle partially removed `ArrayGet` groups of complex type during OOB checks (noir-lang/noir#10027) chore(acir): binary operations always have the same operand types (noir-lang/noir#10028) feat: Add Module::parent and Module::child_modules (noir-lang/noir#10005) chore: green light for ACVM optimisation (noir-lang/noir#10002) chore: unit test for brillig solver (greenlight ACVM execution) (noir-lang/noir#9967) chore(acir): avoid duplication when invoking brillig stdlib call (noir-lang/noir#10025) chore: Use 8 partitions for rust tests (noir-lang/noir#10026) chore: green light for ACVM execution audit (noir-lang/noir#9982) chore: greenlight for ACVM execution (PWG) (noir-lang/noir#9961) feat: optimize out noop casts on constants (noir-lang/noir#10024) fix(mem2reg): consider call return aliases (noir-lang/noir#10016) chore: bump external pinned commits (noir-lang/noir#10022) fix(parser): enforce left brace after match expression (noir-lang/noir#10018) chore(acir): Intrinsics and slice_ops modules as well as slice_ops doc comments (noir-lang/noir#10012) fix: signed division by -1 can overflow (noir-lang/noir#9976) chore(ci): fix external checks (noir-lang/noir#10009) chore(ci): add provenance attestations to npm packages (noir-lang/noir#10011) chore(ACIR): show all expressions as polynomials (noir-lang/noir#10007) chore: remove unused feature flag (noir-lang/noir#9993) chore(ci): fix docs breaking JS releases (noir-lang/noir#10010) chore(ssa_fuzzer): add external coverage registration (noir-lang/noir#9974) END_COMMIT_OVERRIDE Co-authored-by: Tom French <15848336+TomAFrench@users.noreply.github.com>
Description
Problem*
Resolves #9980
The parser for
#[tag]attributes currently ignores all whitespace. This prevents the use of expressions that contain whitespace, such asx as u32, which is incorrectly parsed asxasu32.This parser behavior blocks the use of verification attributes. Specifically, it has two major impacts:
x as u32in verification attributes.Summary*
This PR fixes a bug in the
#[tag]attribute parser. The fix modifies the lexer and parser to ensure that whitespace within#[tag]attributes is handled correctly. This is important for our formal verification tool, as we use#[tag]attributes for our annotations.The proposed solution involves temporarily disabling whitespace skipping in the lexer while parsing the attribute's content. This ensures that all characters, including whitespace, are preserved.
This change will make the parsing of
#['tag]attributes more robust and predictable, enabling the correct use of verification attributes and improving error reporting.Additional Context
This is one of the two PRs that will allow us to move our formal verification tool off of a Noir fork.
Documentation*
Check one:
PR Checklist*
cargo fmton default settings.