feat(sequencer): set block building limits from checkpoint limits#20974
feat(sequencer): set block building limits from checkpoint limits#20974spalladino merged 12 commits intomerge-train/spartanfrom
Conversation
| // Remaining L2 gas (mana) | ||
| // IMPORTANT: This assumes mana is computed solely based on L2 gas used in transactions. | ||
| // This may change in the future. | ||
| const usedMana = sum(existingBlocks.map(b => b.header.totalManaUsed.toNumber())); | ||
| const remainingMana = this.config.rollupManaLimit - usedMana; | ||
|
|
||
| // Remaining DA gas: DA gas = tx blob fields * DA_GAS_PER_FIELD | ||
| // IMPORTANT: This assumes DA gas is computed solely based on the number of blob fields in transactions | ||
| // This may change in the future, but we cannot access the actual DA gas used in a block since it's not exposed | ||
| // in the L2BlockHeader, so we have to rely on recomputing it here. | ||
| const usedDAGas = | ||
| sum(existingBlocks.map(b => sum(b.body.txEffects.map(tx => tx.getNumBlobFields())))) * DA_GAS_PER_FIELD; | ||
| const remainingDAGas = MAX_PROCESSABLE_DA_GAS_PER_CHECKPOINT - usedDAGas; | ||
|
|
||
| // Remaining blob fields (block blob fields include both tx data and block-end overhead) | ||
| const usedBlobFields = sum(existingBlocks.map(b => b.toBlobFields().length)); | ||
| const totalBlobCapacity = BLOBS_PER_CHECKPOINT * FIELDS_PER_BLOB - NUM_CHECKPOINT_END_MARKER_FIELDS; | ||
| const isFirstBlock = existingBlocks.length === 0; | ||
| const blockEndOverhead = getNumBlockEndBlobFields(isFirstBlock); | ||
| const maxBlobFieldsForTxs = totalBlobCapacity - usedBlobFields - blockEndOverhead; | ||
|
|
||
| // Cap L2 gas by remaining checkpoint mana | ||
| const cappedL2Gas = Math.min(opts.maxBlockGas?.l2Gas ?? remainingMana, remainingMana); | ||
|
|
||
| // Cap DA gas by remaining checkpoint DA gas budget | ||
| const cappedDAGas = Math.min(opts.maxBlockGas?.daGas ?? remainingDAGas, remainingDAGas); | ||
|
|
||
| // Cap blob fields by remaining checkpoint blob capacity | ||
| const cappedBlobFields = | ||
| opts.maxBlobFields !== undefined ? Math.min(opts.maxBlobFields, maxBlobFieldsForTxs) : maxBlobFieldsForTxs; | ||
|
|
||
| return { | ||
| maxBlockGas: new Gas(cappedDAGas, cappedL2Gas), | ||
| maxBlobFields: cappedBlobFields, | ||
| }; |
There was a problem hiding this comment.
@sirasistant can I ask you to review this part of the code?
There was a problem hiding this comment.
It looks good, however the cappedDAGas is a bit pointless since we'd always run out of maxBlobFields first (since DA gas doesn't account block builder blob fields overheads)
There was a problem hiding this comment.
since DA gas doesn't account block builder blob fields overheads
Should it? I'd feel more comfortable if we could remove the max-blob-fields check eventually and just rely on DA gas.
There was a problem hiding this comment.
Problem is that some fields don't come from txs: come from builder decided things like how many blocks you create in one checkpoint. We'd need blocks to increase da gas used and have da gas used in the header. Also it'd remove the ability for us (which we don't exercise now) of overcharging DA gas for long term storage for example (making the DA gas of nullifiers higher, for example) So i'm not sure it should
There was a problem hiding this comment.
@sirasistant also the changes on this file
|
|
||
| // Default gas limits. Users should use gas estimation, or they will overpay gas fees. | ||
| // TODO: consider moving to typescript | ||
| // TODO: These are overridden in typescript-land. Remove them from here. |
There was a problem hiding this comment.
I didn't delete them in this PR because I was scared of triggering a change in a vk
There was a problem hiding this comment.
If they are not used in any circuit / protocol contract, it shouldn't trigger a vk change
… building During re-execution (validation/proving), the public processor must process the exact txs from the proposal. Pre-processing skip checks for estimated blob fields and gas limits are now gated behind a new `isBuildingProposal` flag on PublicProcessorLimits, which is only set by the sequencer. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
0cdd6ac to
0e27a6c
Compare
0e27a6c to
b0c9e08
Compare
b0c9e08 to
f3b702f
Compare
BEGIN_COMMIT_OVERRIDE fix: track last seen nonce in case of stale fallback L1 RPC node (#20855) feat: Validate num txs in block proposals (#20850) fix(archiver): enforce checkpoint boundary on rollbackTo (#20908) fix: tps zero metrics (#20656) fix: handle scientific notation in bigintConfigHelper (#20929) feat(aztec): node enters standby mode on genesis root mismatch (#20938) fix: logging of class instances (#20807) feat(slasher): make slash grace period relative to rollup upgrade time (#20942) chore: add script to find PRs to backport (#20956) chore: remove unused prover-node dep (#20955) fix: increase minFeePadding in e2e_bot bridge resume tests and harden GasFees.mul() (#20962) feat(sequencer): (A-526) rotate publishers when send fails (#20888) chore: (A-554) bump reth version 1.6.0 -> 1.11.1 for eth devnet (#20889) chore: metric on how many epochs validator has been on committee (#20967) fix: set wallet minFeePadding in BotFactory constructor (#20992) chore: deflake epoch invalidate block test (#21001) chore(sequencer): e2e tests for invalid signature recovery in checkpoint attestations (#20971) chore: deflake duplicate proposals and attestations (#20990) chore: deflake epochs mbps test (#21003) feat: reenable function selectors in txPublicSetupAllowList (#20909) fix: limit offenses when voting in tally slashing mode by slashMaxPayloadSize (#20683) fix(spartan): wire SEQ_L1_PUBLISHING_TIME_ALLOWANCE_IN_SLOT env var (#21017) END_COMMIT_OVERRIDE
| const l1PublishingTimeBasedOnChain = isAnvilTestChain(config.l1ChainId) ? 1 : ethereumSlotDuration; | ||
| const l1PublishingTime = config.l1PublishingTime ?? l1PublishingTimeBasedOnChain; | ||
|
|
||
| const { maxL2BlockGas, maxDABlockGas } = this.computeBlockGasLimits(config, rollupManaLimit, l1PublishingTime, log); |
There was a problem hiding this comment.
So we take the combination of rollup defined limits and user provided config to produce specific configuration to be passed to the sequencer.
There was a problem hiding this comment.
Not just that. We take user-defined block-level limits, along with checkpoint-level limits (which are sourced from either the Rollup contract, constants, or more user config), and derive the final block-level limits to be used, and feed them into the sequencer.
| const maxBlobFieldsForTxs = remainingBlobFields - blockEndOverhead; | ||
|
|
||
| // Note that gas and blob field limits are further capped by checkpoint-level budgets inside CheckpointBuilder | ||
| const blockBuilderOptions: PublicProcessorLimits = { |
There was a problem hiding this comment.
And that configuration is what we are looking at here.
| ...numberConfigHelper(DefaultSequencerConfig.maxDABlockGas), | ||
| parseEnv: (val: string) => (val ? parseInt(val, 10) : undefined), | ||
| }, | ||
| gasPerBlockAllocationMultiplier: { |
There was a problem hiding this comment.
Is this to allow sequencers to overfill earlier blocks if they want to, but then they would need to ease off in later blocks to adhere to the checkpoint limit defined on the rollup?
There was a problem hiding this comment.
Correct. It's also there because different blocks may hit different limits (l2 gas, da gas, blob fields, tx count). So if we limited each block by exactly checkpoint_limit / num_blocks on all dimensions, we'd be wasting a lot of checkpoint capacity.
| ); | ||
| getPrivateTxEffectsSizeInFields(): number { | ||
| // 3 fields overhead: tx_start_marker, tx_hash, tx_fee | ||
| const overheadFields = TX_DA_GAS_OVERHEAD / DA_GAS_PER_FIELD; |
There was a problem hiding this comment.
Does this need a Math.ceil?
There was a problem hiding this comment.
Not really, since TX_DA_GAS_OVERHEAD is defined as:
pub global TX_DA_GAS_OVERHEAD: u32 = 3 * DA_GAS_PER_FIELD| // set if the operator explicitly configures maxL2BlockGas/maxDABlockGas. | ||
| // Checkpoint-level caps are still enforced inside CheckpointBuilder.buildBlock via | ||
| // capLimitsByCheckpointBudgets, so L1 rejection is prevented even without per-block limits. | ||
| maxBlockGas: |
There was a problem hiding this comment.
Not sure I follow why we are setting any limits here?
There was a problem hiding this comment.
I was on the fence on whether to add it or not.
Rationale was that if we want to set a hard limit to (let's say) L2 gas per block (independent of the accumulated value across the checkpoint), then if a sequencer sent a block with more than that, it wouldn't be considered valid. But it's an easy way to brick things.
I'll remove it for now, and we can discuss later how we want to handle it. Options are:
- Validators only check checkpoint-level limits, not block-level ones. Simplest option, but it means the network cannot enforce any block-level limit.
- Validators check block-level limits as well. This means that if a single operator wants to produce slightly larger blocks than what's configured per-block, they won't get attested to. This also means that gradually increasing block size (while staying within checkpoint limits) is more difficult.
- We have separate config for block-level limits for proposal and validation. Most complex option, but allows the greatest deal of flexibility.
There was a problem hiding this comment.
Created A-613 to track
|
❌ Failed to cherry-pick to |
…ckport #20974) (#21055) ## Summary Backport of #20974 to v4. The checkpoint builder now tracks remaining L2 gas, DA gas, and blob fields in a checkpoint while building each block, and forwards them to the public processor. This means that a proposer will not propose blocks that overall exceed checkpoint limits, and validators will properly reject them. **Cherry-picked commits:** - feat(sequencer): set block building limits from checkpoint limits - feat(sequencer): only skip txs due to gas/blob limits during proposal building - fix(gas): saner defaults for da gas limit per tx - fix: fix spread when computing processor limits - fix(processor): check gas limits using used gas on reexecution - test: check that validators reject exceeded gas limits - chore: re-validate checkpoint limits before proposal and attestation - test: fix tests related to avm gas limits - fix(validator): do not check block-level limits on validation - chore: add inline comments **Conflict resolution:** - `timetable.ts`: Resolved log level conflict — v4 uses `createLogger` (always defined) so kept `this.log.info(` without optional chaining (matching PR's intent to upgrade from `verbose` to `info`). ClaudeBox log: http://ci.aztec-labs.com/2433a99083562757-1 --------- Co-authored-by: Santiago Palladino <santiago@aztec-labs.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> Co-authored-by: Santiago Palladino <santiago@aztecprotocol.com>
BEGIN_COMMIT_OVERRIDE test: update proving-real test to mbps (#20991) chore: epoch proving log analyzer (#21033) chore: update pause script to allow resume (#21032) feat: price bump for RPC transaction replacement (#20806) refactor: remove update checker, retain version checks (#20898) fix: (A-592) p2p client proposal tx collector test (#20998) refactor: use publishers-per-pod in deployments (#21039) chore: web3signer refreshes keystore (#21045) feat(sequencer): set block building limits from checkpoint limits (#20974) chore(e2e): fix e2e bot L1 tx nonce reuse (#21052) feat: Update L1 to L2 message APIs (#20913) fix: (A-589) epochs l1 reorgs test (#20999) feat(sequencer): add SEQ_MAX_TX_PER_CHECKPOINT config (#21016) fix: drop --pid=host from docker_isolate (#21081) feat: standby mode for prover broker (#21098) fix(p2p): remove default block handler in favor of block handler (#21105) feat(validator): add VALIDATOR_ env vars for independent block limits (#21060) refactor(p2p): decouple proposal validators from base class via composition (#21075) feat: additional validation in public setup allowlist (onlySelf + null msg sender) (#21122) fix: (A-591) aztecProofSubmissionEpochs incorrectly named as aztecProofSubmissionWindow (#21108) refactor(sequencer): rename SEQ_GAS_PER_BLOCK_ALLOCATION_MULTIPLIER to SEQ_PER_BLOCK_ALLOCATION_MULTIPLIER (#21125) fix: unbound variable in check_doc_references.sh with set -u (#21126) feat: calldata length validation of public setup function allowlist (#21139) fix: include mismatched values in tx metadata validation errors (#21147) feat: single-node implementation of slash-protection signer (#20894) feat: Remove non-protocol contracts from public setup allowlist (#21154) chore: More updated Alpha configuration (#21155) chore: tally slashing pruning improvements (#21161) fix: update dependencies (#20997) fix: omit bigint priceBumpPercentage from IPC config in testbench worker (#21169) refactor(p2p): (A-588) maintain sorted array in tx pool instead of sorting on read (#21079) fix(p2p): report most severe failure in runValidations (#21185) fix: use dedicated L1 account for bot bridge resume tests to avoid nonce race (#21148) fix: parse error.message in formatViemError (#21163) fix: bump lighthouse consensus client v7.1.0 -> v8.0.1 (#21170) chore: code decuplication + refactor (public setup allowlist) (#21200) END_COMMIT_OVERRIDE
The checkpoint builder now tracks remaining L2 gas, DA gas, and blob fields in a checkpoint while building each block, and forwards them to the public processor. This means that a proposer will not propose blocks that overall exceed checkpoint limits, and validators will properly reject them.
In addition, the proposer defaults the L2 and DA gas limits per block to the checkpoint limits divided by expected number of blocks, times two. This value is still capped by the remaining gas in the checkpoint builder, but means that a proposer will not waste the entire checkpoint gas allocation on the first block.
Fixes A-528
Changes
As described by Claude
rollupManaLimit,MAX_PROCESSABLE_DA_GAS_PER_CHECKPOINT) using the timetable'smaxNumberOfBlocksand a configurable multiplier (default: 2) for creating proposals only (not for validation)CheckpointBuilder.capLimitsByCheckpointBudgets(), which caps per-block limits by remaining checkpoint budgets for both proposer and validator pathsmaxTxsPerBlock,maxL2BlockGas,maxDABlockGas, androllupManaLimitthrough to validator re-execution so limits are enforced during block validationmaxBlockSizeInByteswith field-based blob limits and a pre-processing blob field estimation (getPrivateTxEffectsSizeInFields)gasPerBlockAllocationMultiplierconfig (SEQ_GAS_PER_BLOCK_ALLOCATION_MULTIPLIER, default: 2)maxL2BlockGasandmaxDABlockGasoptional (auto-computed from checkpoint limits when not set)🤖 Generated with Claude Code