chore: transcript+codec+poseidon2 fixes by iakovenkos · Pull Request #19419 · AztecProtocol/aztec-packages

iakovenkos · 2026-01-08T12:03:51Z

Summary

This PR addresses findings from the external audit of the Transcript and Poseidon2 components. The changes ensure consistent behavior between native and in-circuit verification, particularly around field element serialization/deserialization.

1. Field Element Serialization/Deserialization (Issues 5, 6, 9, 10, 11)

Problem: Inconsistent handling of field elements during serialization, particularly around edge cases involving the field modulus and aliased values.

validate_split_in_field_unsafe now properly rejects hi||lo == field_modulus (Issue 5)
Replaced hardcoded $254$ with generically calculated max_bits (Issue 6)
Added proper borrow boolean check even when lo is constant (Issue 10)
Unified deserialization behavior: both FrCodec (native) and StdlibCodec (in-circuit) now use assert_is_in_field() to reject unreduced values (Issues 9 & 11)
For Mega arithmetization, validation is deferred to Translator+ECCVM; for Ultra, strict validation happens immediately via bigfield

2. Transcript Round Tracking (Issues 3, 4)

Removed redundant round_number field, now using only round_index (Issue 4)
Consecutive challenges with no data between them now stay in the same round
prover_init_empty and verifier_init_empty are now properly marked as testing methods (Issue 3)

3. Poseidon2 Cleanup (Issues 1, 2, 7, 8)

Strategy:

Not marking the first element of the sponge state as used.
Added documentation for $-1$ offset in internal_matrix_diagonal (Issue 2)
Removed unnecessary normalization assert in matrix_multiplication_external (Issue 7)
Fixed transcript initialization in circuit failure tests (Issue 8)

Commits:

Issue 1 (68011cc): First element of of FieldSponge::hash_internal is not marked as used
Issue 2 (2dd4197): Missing annotation about $-1$ offset in poseidon2 internal_matrix_diagonal
Issue 3 (1dddf54): prover_init_empty and verifier_init_empty are testing methods
Issue 4 (ced1a88): round_number is redundant with round_index
Issues 5 & 6 (e4fa3c5): validate_split_in_field_unsafe accepts hi||lo == modulus + hardcoded $254$
Issue 7 (dd1f644): Unnecessary assert for matrix_multiplication_external output normalization
Issue 8 (d064598): Verification in poseidon2.circuit.failure.test.cpp fails on valid input
Issues 9 & 11 (7f1cb65): deserialize_from_fields rejects unreduced fqs + (fq_modulus, fq_modulus) handling
Issue 10 (98178c5): validate_split_in_field_unsafe skips borrow check when lo is constant

Documentation

Added barretenberg/cpp/src/barretenberg/stdlib/primitives/field/CODEC_README.md documenting:

Codec architecture (FrCodec, StdlibCodec, U256Codec)
Field element encoding (2-limb representation for fq)
Canonical representation requirements and point at infinity handling
Ultra vs Mega arithmetization differences
Threat model: why native/recursive consistency matters
Mega/Goblin deferred validation flow and ECCVM↔Translator translation check

New Tests

field_utils.test.cpp (new file):

ValidateSplitRejectsModulus — Issue 5: split rejects hi||lo == modulus
ValidateSplitAcceptsModulusMinusOne — edge case: modulus - 1 is valid
SplitUniqueRejectsModulus — Issue 5: unique split rejects modulus
SplitUniqueMaxValue — edge case handling
ValidateSplitConstantLoWitnessHiRejectsModulus — Issue 10: constant lo with witness hi
ValidateSplitWitnessLoConstantHiRejectsModulus — Issue 10: witness lo with constant hi

field_conversion.test.cpp:

BigfieldDeserializationFailsOnLimbOverflow — limb bounds validation
BothCodecsRejectPointAtInfinityAlias — Issue 11: aliased infinity rejected
BothCodecsAcceptCanonicalRejectAlias — Issues 9, 11: native/circuit consistency
AcceptCanonicalPointAtInfinity — canonical infinity accepted
RejectPointNotOnCurve — off-curve points rejected

poseidon2.test.cpp:

PointCoordinatesVsAliasProduceDifferentHashes — aliased coords produce different hashes

poseidon2.circuit.failure.test.cpp:

ValidCircuitVerifies — Issue 8: baseline validity check

notnotraju · 2026-01-12T15:23:40Z

barretenberg/cpp/src/barretenberg/stdlib/primitives/field/field_utils.test.cpp

+    EXPECT_TRUE(CircuitChecker::check(builder));
+}
+
+/**


barretenberg/cpp/src/barretenberg/stdlib/hash/poseidon2/poseidon2.test.cpp

suyash67

Looks good. Thanks for adding the codec readme, its very useful!

notnotraju · 2026-01-13T13:52:49Z

barretenberg/cpp/src/barretenberg/stdlib/primitives/field/field_conversion.hpp

-            // Cases 2 and 3: a bigfield/goblin_field element is reconstructed from low and high limbs.
+        } else if constexpr (IsAnyOf<T, bigfield_ct>) {
+            // Case 2: bigfield is reconstructed from low and high limbs with in-field validation.
+            // This ensures aliased values (>= Fq::modulus) are rejected.


…recise

iakovenkos · 2026-01-15T11:52:19Z

barretenberg/cpp/src/barretenberg/stdlib/primitives/field/CODEC_README.md

+|---------|-------|--------|----------------|
+| **Native** | All | `for each limb: if (limb != 0) return false` | Direct limb-by-limb zero check. |
+| **Circuit** | BN254 | `sum(limbs) == 0` | Sum of 4 valid limbs (2×136-bit + 2×118-bit) ≤ 2^138, cannot wrap to 0 mod Fr (254 bits). Only all-zero limbs satisfy this. |
+| **Circuit** | Grumpkin | `x² + 5y² == 0` | Equation `x² = -5y²` requires -5 to be a quadratic residue. Since -5 is not a square mod p, only `(0,0)` satisfies this. |


@arielgabizon I updated the comment

iakovenkos · 2026-01-15T11:53:18Z

barretenberg/cpp/src/barretenberg/stdlib/primitives/field/field_conversion.hpp

            return (fr::accumulate(std::vector<fr>(fr_vec.begin(), fr_vec.end())).is_zero());
        } else {
+            // For Grumpkin infinity check: verify that Fr modulus p ≡ 2 mod 5
+            static_assert(bb::fr::modulus % 5 == 2, "Grumpkin infinity check requires Fr modulus p ≡ 2 mod 5");


@arielgabizon + added compile-time check

umm is this the right check? I thought it was p-1 =1 mod 4. Is this equivalent?

i updated the condition to check $x^2 - 5y^2 == 0$. This way we only need to check that $5$ is a non-residue <=> p mod 5 == 2,3 by quadratic reciprocity (previously i was implicitly assuming that we're checking somewhere that -1 is a residue and skipping the check mod 4)

Looks good!

But it looks like you're checking is 2 mod 5 - not 2 or 3.
Besides qr says (p/5)(5/p)= -1^((p-1)/2).
So I'm not sure it's equivalent to p being non-residue mod 5
(which seems to be what you're implying)

This points to an outdated code snippet, you'd need to go to the diff to see the current state
You're missing (5-1)/2 in the exponent -- we have $(5/p) (p/5) = (-1)^{\frac{(p-1)(5-1)}{4}} = (-1)^{p-1}$, which is $1$ since $p$ is odd (I guess it's safe to assume at this point)

I think it's (p/5)(5/p)= -1^((p-1)/2) * -1^((5-1)/2) = -1^((p-1)/2).

nvm - I got the qr formula wrong, this looks good!

notnotraju · 2026-01-15T15:03:21Z

barretenberg/cpp/src/barretenberg/stdlib/primitives/field/CODEC_README.md

+
+| Codec | Enforcement | Method |
+|-------|-------------|--------|
+| `FrCodec` | Native assertion | `BB_ASSERT_LT(value, fq::modulus)` |


looks good.

notnotraju

Great, thanks!

…i/transcript-post-audit-fixes

federicobarbacovi · 2026-01-23T17:26:29Z

barretenberg/cpp/src/barretenberg/vm2/constraining/recursion/recursive_flavor.hpp

-                transcript->add_element_frs_to_hash_buffer(derived_label, proof_span.subspan(proof_idx, num_frs_comm));
-                proof_idx += num_frs_comm;
+                [[maybe_unused]] auto _ =
+                    transcript->template receive_from_prover<GoblinStdlibProofCommitment>(derived_label);


Why is this Goblin while the previous isn't?

AztecBot · 2026-01-23T18:09:07Z

Flakey Tests

🤖 says: This CI run detected 2 tests that failed, but were tolerated due to a .test_patterns.yml entry.

\033FLAKED\033 (8;;http://ci.aztec-labs.com/2a1ca98cadc2b65f�2a1ca98cadc2b65f8;;�):  yarn-project/end-to-end/scripts/run_test.sh simple src/e2e_epochs/epochs_l1_reorgs.test.ts (410s) (code: 1) group:e2e-p2p-epoch-flakes (\033iakovenkos\033: oops)
\033FLAKED\033 (8;;http://ci.aztec-labs.com/4f4f5dd38a31368a�4f4f5dd38a31368a8;;�):  yarn-project/end-to-end/scripts/run_test.sh simple src/e2e_p2p/valid_epoch_pruned_slash.test.ts (395s) (code: 1) group:e2e-p2p-epoch-flakes (\033iakovenkos\033: oops)

BEGIN_COMMIT_OVERRIDE feat: support JSON input files for bb verify command (#19800) fix: update bootstrap.sh to use new JSON field names chore: Update `index.js` so that `HAS_ZK` and `PUBLIC_INPUTS` variables must always be set in tests (#19884) chore: pippenger int audit (#19302) chore: deduplicate batch affine addition trick (#19788) chore: transcript+codec+poseidon2 fixes (#19419) chore!: explicitly constrain inputs and intermediate witnesses (#19826) fix: exclude nlohmann/json from WASM builds in json_output.hpp chore: translator circuit builder and flavor audit (#19798) Revert "fix: exclude nlohmann/json from WASM builds in json_output.hpp" Revert "feat: support JSON input files for bb verify command (#19800)" Revert "fix: update bootstrap.sh to use new JSON field names" END_COMMIT_OVERRIDE

### Summary This PR addresses findings from the external audit of the Transcript and Poseidon2 components. The changes ensure consistent behavior between native and in-circuit verification, particularly around field element serialization/deserialization. #### 1. Field Element Serialization/Deserialization (Issues 5, 6, 9, 10, 11) Problem: Inconsistent handling of field elements during serialization, particularly around edge cases involving the field modulus and aliased values. - `validate_split_in_field_unsafe` now properly rejects `hi||lo == field_modulus` (Issue 5) - Replaced hardcoded $254$ with generically calculated `max_bits` (Issue 6) - Added proper borrow boolean check even when `lo` is constant (Issue 10) - Unified deserialization behavior: both `FrCodec` (native) and `StdlibCodec` (in-circuit) now use `assert_is_in_field()` to reject unreduced values (Issues 9 & 11) - For `Mega` arithmetization, validation is deferred to `Translator+ECCVM`; for `Ultra`, strict validation happens immediately via `bigfield` #### 2. Transcript Round Tracking (Issues 3, 4) - Removed redundant `round_number` field, now using only `round_index` (Issue 4) - Consecutive challenges with no data between them now stay in the same round - `prover_init_empty` and `verifier_init_empty` are now properly marked as testing methods (Issue 3) #### 3. Poseidon2 Cleanup (Issues 1, 2, 7, 8) Strategy: - Not marking the first element of the sponge state as `used`. - Added documentation for $-1$ offset in `internal_matrix_diagonal` (Issue 2) - Removed unnecessary normalization assert in `matrix_multiplication_external` (Issue 7) - Fixed transcript initialization in circuit failure tests (Issue 8) --- #### Commits: - Issue 1 (68011cc): First element of of `FieldSponge::hash_internal` is **not** marked as used - Issue 2 (2dd4197): Missing annotation about $-1$ offset in poseidon2 `internal_matrix_diagonal` - Issue 3 (1dddf54): `prover_init_empty` and `verifier_init_empty` are testing methods - Issue 4 (ced1a88): `round_number` is redundant with `round_index` - Issues 5 & 6 (e4fa3c5): `validate_split_in_field_unsafe` accepts `hi||lo == modulus` + hardcoded $254$ - Issue 7 (dd1f644): Unnecessary assert for matrix_multiplication_external output normalization - Issue 8 (d064598): Verification in poseidon2.circuit.failure.test.cpp fails on valid input - Issues 9 & 11 (7f1cb65): `deserialize_from_fields` rejects unreduced `fq`s + (fq_modulus, fq_modulus) handling - Issue 10 (98178c5): `validate_split_in_field_unsafe` skips borrow check when `lo` is constant --- #### Documentation Added barretenberg/cpp/src/barretenberg/stdlib/primitives/field/CODEC_README.md documenting: - Codec architecture (FrCodec, StdlibCodec, U256Codec) - Field element encoding (2-limb representation for fq) - Canonical representation requirements and point at infinity handling - Ultra vs Mega arithmetization differences - Threat model: why native/recursive consistency matters - Mega/Goblin deferred validation flow and ECCVM↔Translator translation check --- #### New Tests field_utils.test.cpp (new file): - ValidateSplitRejectsModulus — Issue 5: split rejects hi||lo == modulus - ValidateSplitAcceptsModulusMinusOne — edge case: modulus - 1 is valid - SplitUniqueRejectsModulus — Issue 5: unique split rejects modulus - SplitUniqueMaxValue — edge case handling - ValidateSplitConstantLoWitnessHiRejectsModulus — Issue 10: constant lo with witness hi - ValidateSplitWitnessLoConstantHiRejectsModulus — Issue 10: witness lo with constant hi field_conversion.test.cpp: - BigfieldDeserializationFailsOnLimbOverflow — limb bounds validation - BothCodecsRejectPointAtInfinityAlias — Issue 11: aliased infinity rejected - BothCodecsAcceptCanonicalRejectAlias — Issues 9, 11: native/circuit consistency - AcceptCanonicalPointAtInfinity — canonical infinity accepted - RejectPointNotOnCurve — off-curve points rejected poseidon2.test.cpp: - PointCoordinatesVsAliasProduceDifferentHashes — aliased coords produce different hashes poseidon2.circuit.failure.test.cpp: - ValidCircuitVerifies — Issue 8: baseline validity check

iakovenkos added 12 commits January 5, 2026 13:17

Issue 1

68011cc

Issue 2

2dd4197

Issue 3

1dddf54

Issue 4

ced1a88

Issues 5 and 6

e4fa3c5

Issue 7

dd1f644

Issue 8

d064598

Issues 9 and 11

17f3c74

Issue 10

98178c5

Redo Issues 9 and 11

7f1cb65

clean up tests + fix comment

59f601f

fix tests

aeab64c

iakovenkos self-assigned this Jan 8, 2026

move readme

11f4391

iakovenkos requested a review from notnotraju January 8, 2026 16:32

iakovenkos marked this pull request as ready for review January 8, 2026 16:33

iakovenkos requested review from IlyasRidhuan, fcarreiro and jeanmon as code owners January 8, 2026 16:33

notnotraju reviewed Jan 12, 2026

View reviewed changes

suyash67 reviewed Jan 12, 2026

View reviewed changes

barretenberg/cpp/src/barretenberg/stdlib/hash/poseidon2/poseidon2.test.cpp Show resolved Hide resolved

suyash67 approved these changes Jan 12, 2026

View reviewed changes

notnotraju self-requested a review January 13, 2026 13:20

notnotraju reviewed Jan 13, 2026

View reviewed changes

update codec readme, make the comment about grumpkin inf check more p…

7be41f1

…recise

iakovenkos commented Jan 15, 2026

View reviewed changes

upd infinity check and docs

8926462

notnotraju self-requested a review January 15, 2026 14:17

notnotraju reviewed Jan 15, 2026

View reviewed changes

notnotraju self-requested a review January 15, 2026 15:21

notnotraju approved these changes Jan 15, 2026

View reviewed changes

iakovenkos added 7 commits January 16, 2026 10:23

Merge remote-tracking branch 'origin/merge-train/barretenberg' into s…

1744b32

…i/transcript-post-audit-fixes

fix tests

14dc90b

fix constraint + add test

b12f81f

use fr(1)

8d4f789

Merge remote-tracking branch 'origin/merge-train/barretenberg' into s…

95f7378

…i/transcript-post-audit-fixes

upd gate count after merge

fce6dc5

vks hash upd

0f73dca

iakovenkos added the ci-full Run all master checks. label Jan 23, 2026

iakovenkos added 3 commits January 23, 2026 14:30

upd vk hash + throw or abort in failure test for wasm

a7d9f59

make wasm threads happy

ee9d62d

fix avm transcript

d6241c9

federicobarbacovi reviewed Jan 23, 2026

View reviewed changes

oops

b634269

iakovenkos merged commit cd3d1a3 into merge-train/barretenberg Jan 23, 2026
8 checks passed

iakovenkos deleted the si/transcript-post-audit-fixes branch January 23, 2026 18:22

AztecBot mentioned this pull request Jan 23, 2026

feat: merge-train/barretenberg #19887

Merged

Conversation

iakovenkos commented Jan 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

1. Field Element Serialization/Deserialization (Issues 5, 6, 9, 10, 11)

2. Transcript Round Tracking (Issues 3, 4)

3. Poseidon2 Cleanup (Issues 1, 2, 7, 8)

Commits:

Documentation

New Tests

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

suyash67 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

arielgabizon Jan 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

notnotraju left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

AztecBot commented Jan 23, 2026

Flakey Tests

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

iakovenkos commented Jan 8, 2026 •

edited

Loading

arielgabizon Jan 15, 2026 •

edited

Loading