feat!: blob batching

[MirandaWood]

The blobs are back in town.

This PR reworks blobs so that instead of calling the point evaluation precompile for each blob (currently up to 3 per block => up to 96 (?) calls per epoch), we call it once per epoch by batching blobs to a single kzg commitment, opening, challenge, and proof.

How we can be sure that this one pairing check is equivalent to a check per blob is covered in the maths by @iAmMichaelConnor here 🎉

Overview

Instead of pushing to a long array of BlobPublicInputs, which are then individually checked on L1, we batch each blob together to a single set of BlobAccumulatorPublicInputs. The start accumulator state is fed into each block root circuit, where the block's blobs are accumulated and the end state is set. Each block merge circuit checks that the state follows on correctly and, finally, the root circuit checks that the very start state was empty and finalises the last end state.

This last end state makes up the set of inputs for the point evaluation precompile. If the pairing check in that precompile passes, we know that all blobs for all blocks in the epoch are valid and contain only the tx effects validated by the rollup.

Circuits

Key changes:

• Integrate BLS12-381 curve operations with bignum and bigcurve libraries, plus tests.
• Rework the blob package to batch blobs and store in reworked structs, plus tests.
• Rework the rollup circuits from block_root above to handle blob accumulation state rather than a list of individual blob inputs, plus (you guessed it) tests.

Contracts

The contracts:

• No longer call the point evaluation precompile on propose, instead inject the blob commitments, check they correspond to the broadcast blobs, and stores them in the blobCommitmentsHash.
• Do not store any blob public inputs apart from the blobCommitmentsHash.
• Call the point evaluation precompile once on submitEpochRootProof for ALL blobs in the epoch.
• Use the same precompile inputs as pubic inputs to the root proof verification along with the blobCommitmentsHash to link the circuit batched blob, real L1 blobs, and the batched blob verified on L1.

TypeScript

Key changes:

• Edit all the structs and methods reliant on the circuits/contracts to match the above changes.
• Inject the final blob challenges used on each blob into all block building methods in orchestrator.
• Accumulate blobs in ts when building blocks and use as inputs to each rollup circuit, plus tests.
• Return the blob inputs required for submitEpochRootProof on finaliseEpoch().

TODOs/Related Issues

• Choose field for hashing challenge: feat(blobs): Finalise hash method for challenge gamma (used in blob batching) #13608
• Instead of exponentiating gamma (expensive!), hash it for each iteration: feat(blobs): explore hashing gamma instead of using powers to form linear combination of y, C, Q #13740
• Number of public inputs: BLS points in noir take up 9 fields (4 for each coordinate as a limbed bignum, 1 for the is_inf flag) but can be compressed to only 2. For recursive verification in block root and above, would it be worth the gates to compress these? It depends whether the gate cost of compression is more/less than gate cost of recursively verifying 7 more public inputs.
• Remove the large trusted setup file from yarn-project/blob-lib/src/trusted_setup_bit_reversed.json? Used in testing, but may not be worth keeping (see code comments).
• Cleanup old, unused blob stuff in chore: post blob batching cleanup #14637.

PR Stack

• mw/blob-batching <- main feature
• ^ mw/blob-batching-bls-utils <- BLS12-381 bigcurve and bignum utils (noir) (feat: blob batching methods #13583)
• ^ mw/blob-batching-bls-utils-ts <- BLS12-381 bigcurve and bignum utils (ts) (feat: blob batching methods (ts) #13606)
• ^ mw/blob-batching-integration <- Integrate batching into noir protocol circuits (feat: WIP batch blobs and validate in rollup #13817)
• ^ mw/blob-batching-integration-ts-sol <- Integrate batching into ts and solidity (feat: WIP integrate batched blobs into l1 contracts + ts #14329)
• ^ mw/blob-batching-cleanup <- Remove old blob code

[LHerskind]

Did a pretty high level look here. Seems pretty sane, but have reservations about some of the todo as I think they don't account for how blocks are produced and at the same time would provide negligable saving compared to some of the other changes we might have to make anyway.

[LHerskind]

Are we using this to have a slightly easier way to fetch it? Not a big thing atm, but it takes up extra space in the contract, and put us with more code paths which can be kinda annoying. We have a good bunch of these flowing around.

Created #14854, no need to address here. Guess there will be plenty at once as part of #14854.

[MirandaWood]

Yeah, just an easier way to fetch - thanks for adding an issue!

[LHerskind]

I don't think you would really be able to do it per epoch, for the reason that it is not guaranteed that the whole epoch lands. This is because we have partial epochs.

Ignoring that, I think a more likely outcome is that the BlockLog struct won't be saved in its entirety, but that we instead store just a hash, and whenever reading need to provide data. It is less neat and a bit of a pain to handle, but it takes at least a good chunk of the cost out of the question. And at that point it is not much more expensive to read more values in the BlockLog than just one.

I think I would lean towards, we get the things you have in. And then more or less leave the contracts alone so we can apply optimisations and things for gas without running into big changes etc.

TL;DR: I would probably close the part of per epoch, because it is not sound given we have partial epochs.

[MirandaWood]

Makes sense - considering partial epochs, how would the running hash work? No need to reply now, was just my assumption that the reason we could overwrite one value was that we could 'start fresh' with each epoch (whether it goes through or is pruned).

[LHerskind]

Why not just input[0]?

[MirandaWood]

Insanity caused by original blob PR 5 months ago. Will change

[LHerskind]

Oh, my comment related to this one (on the previous pr) was because people have been talking about same actor sending a tx that propose blocks for multiple rollups at the same time, so there could be a blob before the aztec specific, or after.

I'm ok with us checking like this, but would expect that we could get away with just checking blobhash(numBlobs) == 0 🤔.

[MirandaWood]

Ah ok I misunderstood! I was thinking that it might be possible to submit blobs like blob_0, blob_1, blob_2 where blob_1 is empty and wanted to cover that case. I wasn't sure whether blob submission would count blob_1 as 'blank' (with blobhash 0, covered by new check) or assign a blobhash which would just be the hash of the (0, 0) commitment (covered by protocol).

so there could be a blob before the aztec specific, or after.

We can't have one before the aztec specific since any commitment before numBlobs would be 'stored' and fail the epoch verification. I don't think there is a way to 'attack' this by injecting a commitment that matches the non-aztec blob both to this fn and in the rollup circuits (@iAmMichaelConnor?)

[MirandaWood]

Note: This is an unrelated change - contants.in.ts was unable to read this constant because the ; was on a new line