feat(pipeline): allow syncing blocks ontop of the proposed chain by Maddiaa0 · Pull Request #21025 · AztecProtocol/aztec-packages

Maddiaa0 · 2026-03-03T02:17:30Z

Overview

Key contributions:

In the pr above feat(pipeline): introduce pipeline views for building #21026 publishing was a blocking action, in this pr we move publishing to be a non blocking option, there a publisher can schedule when it should start trying to publish a block.
This keeps track of valid checkpoints that are pending and not settled to L1 - and allows building ontop of them.

Adds a second p2p callback that separates what runs for all nodes / validator nodes

Testing

epochs_mbps.pipeline now expects 3 blocks per checkpoint, just like the original epochs_mbps test, now it is fully pipelined.

Upcoming

updating the timetable to allow for longer time building in the slot - this pr does not extend the time allocated to block building.
handing rollbacks when the pendingCheckpoint needs to be rolled back / cleared.

Maddiaa0 · 2026-03-03T20:51:20Z

spalladino · 2026-03-04T15:39:52Z

yarn-project/archiver/src/modules/validation.ts

  }

-  const requiredAttestationCount = Math.floor((committee.length * 2) / 3) + 1;
+  const requiredAttestationCount = computeQuorum(committee.length);


spalladino · 2026-03-04T15:46:39Z

yarn-project/archiver/src/modules/data_store_updater.ts

+        // Atomically set the pending checkpoint number alongside the block if provided
+        pendingCheckpointNumber !== undefined &&
+          this.store.blockStore.setPendingCheckpointNumber(pendingCheckpointNumber),


This doesn't match the comment that this Sets the pending checkpoint number (quorum-attested but not yet L1-confirmed), right?

That said, let's discuss the model. Weirdly, I like the concept of having an uncheckpointed checkpoint. But it seems like we have two different things:

A checkpoint-being-built, which is the checkpoint being built via proposed blocks. This is just a checkpoint number, since a Checkpoint object needs all its blocks to be ready. Today we already have this, but don't need to explicitly store it.

A checkpoint-being-proposed, which is the Checkpoint object for which the current proposer has sent a checkpoint proposal, and would ultimately make it onto L1.

We need to define which ones we expose to clients of the archiver, and also to users via APIs like getL2Tips.

spalladino · 2026-03-04T15:50:08Z

yarn-project/sequencer-client/src/sequencer/sequencer.ts

-    });
+    const additionalSlotOffset = slot.now !== slot.pipeline ? getEthSlotsPerAztecSlot(this.l1Constants) : undefined;
+
+    // When pipelining, if the previous checkpoint has quorum attestations from P2P, we can skip the


Do we want to wait for the attestations, or just for our own reexecution?

Updated to just be own re-execution

spalladino · 2026-03-04T15:51:20Z

yarn-project/sequencer-client/src/sequencer/checkpoint_proposal_job.ts

+  /** Tracks the fire-and-forget L1 submission promise so it can be awaited during shutdown. */
+  private pendingL1Submission: Promise<void> | undefined;


Do we want to fire-and-forget the L1 submission from the checkpoint proposal job, or the entire checkpoint proposal job from the sequencer?

As discussed, its really the same thing since the checkpoint proposal job owns this.

We keep track of this item just so that we can cancel a pending submission job in the event of a rollback (cancelled in next pr)

yarn-project/validator-client/src/block_proposal_handler.ts

Maddiaa0 · 2026-03-19T00:45:08Z

yarn-project/archiver/src/store/kv_archiver_store.ts

  }

+  /** Sets the pipelining tree-in-progress boundary for building ahead of L1 confirmation. */
+  public setPipeliningTreeInProgress(value: bigint): Promise<void> {


this was added to deal with #21494 - i did not want to directly set the other tree in progress, I think my current solution is nasty and would appreciate a discussion around it

Maddiaa0 · 2026-03-19T00:45:34Z

yarn-project/archiver/src/store/message_store.ts

  /** Stores the checkpoint number whose message tree is currently being filled on L1. */
  #inboxTreeInProgress: AztecAsyncSingleton<bigint>;
+  /** Stores the pipelining tree-in-progress boundary, set by the sequencer when building ahead.
+   *  Separate from the synced value so normal callers still use the strict L1-synced guard. */


nasty solution in question

How is this nasty? Isn't this just tracking the inbox tree that is at the tip of the proposed chain?

I mean more that I'm not confident it is correct

Maddiaa0 · 2026-03-19T00:46:52Z

yarn-project/aztec-node/src/aztec-node/server.ts

      debugLogStore,
    );

+    // Register a callback for all nodes to set the pending checkpoint number when a checkpoint proposal is received.


Adds a lovely callback, the existing one was only relevant to nodes performing validation duties - allNodesCheckpointProposalHandler is triggered by everyone, not just validators

Maddiaa0 · 2026-03-19T00:48:58Z

yarn-project/ethereum/src/contracts/rollup.ts

+  }
+
+  /** Merges multiple StateOverride arrays, combining stateDiff entries for the same address. */
+  public static mergeStateOverrides(...overrides: StateOverride[]): StateOverride {


This duplicates packing the fee slot

Whenever checkpoint blocks become very full, the fee will drift from block to block, this prevents simulating into the future failing

Maddiaa0 · 2026-03-19T00:49:25Z

yarn-project/p2p/src/client/p2p_client.ts

-    // Knowledege of pending checkpoints is in the PR above
-    const { targetSlot } = this.epochCache.getTargetAndNextSlot();
-    if (targetSlot <= this.lastSlotProcessed) {
+    let slot;


@PhilWindle

PhilWindle · 2026-03-19T11:26:59Z

yarn-project/archiver/src/store/kv_archiver_store.test.ts

+      await store.addCheckpoints([checkpoint1]);
+
+      // Set pending checkpoint to 3 (far ahead)
+      await store.blockStore.setPendingCheckpoint({


Should it allow this?

You have a good point, will restrict

PhilWindle · 2026-03-19T11:28:12Z

yarn-project/archiver/src/store/kv_archiver_store.test.ts

+      await store.addCheckpoints([checkpoint1, checkpoint2, checkpoint3]);
+
+      // Set pending to 2 (already confirmed, but that's fine for the test)
+      await store.blockStore.setPendingCheckpoint({


Should this be allowed either?

PhilWindle · 2026-03-19T11:30:59Z

yarn-project/archiver/src/store/kv_archiver_store.test.ts

+        lastArchive: block2.archive,
+      });
+
+      // Block for checkpoint 2 should work (previous confirmed = 1)


Maybe I misunderstand the flow. But shoudl we allow adding proposed blocks for multiple different checkpoints?

Unless we are already potentially supporting build ahead of multiple checkpoints?

It has been restricted to +1

PhilWindle · 2026-03-19T11:36:41Z

yarn-project/archiver/src/store/l2_tips_cache.ts

        block: { number: provenBlockNumber, hash: provenBlockData.blockHash.toString() },
        checkpoint: provenCheckpointId,
      },
+      pendingCheckpoint: pendingCheckpointBlockData


Shall we just call this pending?

PhilWindle · 2026-03-19T11:46:16Z

yarn-project/aztec-node/src/aztec-node/server.ts

+    // The checkpoint proposal often arrives before the last block finishes re-execution.
+    // Trigger a sync to flush any queued blocks, then retry.
+    if (!blockData) {
+      await archiver.syncImmediate();


Will probably need a retry loop here won't we?

PhilWindle · 2026-03-19T12:00:08Z

yarn-project/p2p/src/services/libp2p/libp2p_service.ts

+    this.allNodesCheckpointReceivedCallback = (
+      _checkpoint: CheckpointProposalCore,
+    ): Promise<CheckpointAttestation[] | undefined> => {
+      return Promise.resolve(undefined);


Should probably log an error here shouldn't we? Everyone should register a handler here.

PhilWindle · 2026-03-19T12:00:44Z

yarn-project/p2p/src/services/libp2p/libp2p_service.ts

+    this.validatorCheckpointReceivedCallback = (
      checkpoint: CheckpointProposalCore,
    ): Promise<CheckpointAttestation[] | undefined> => {
      this.logger.debug(


And this log line could maybe go if this is validator only. Otherwise full nodes will print this routinely.

Maddiaa0 · 2026-03-19T14:01:12Z

yarn-project/sequencer-client/src/sequencer/sequencer.ts

-    const canProposeCheck = await publisher.canProposeAt(syncedTo.archive, proposer ?? EthAddress.ZERO, {
-      ...invalidateCheckpoint,
-    });
+    // Determine the correct archive and L1 state overrides for the canProposeAt check.


This block has been edited since last review @PhilWindle

PhilWindle · 2026-03-19T14:11:07Z

yarn-project/archiver/src/modules/l1_synchronizer.ts

    }

-    // What's the slot of the first uncheckpointed block?
+    // Don't prune blocks that are covered by a pending checkpoint (awaiting L1 submission from pipelining)


Slightly confused by this. What happens if a checkpoint fails to land on L1? Surely the blocks covered by that (pending) checkpoint are removed? As well as all blocks built afterwards?

ah, md/pipeline-recovery-2 im dealing with this currently in this branch

Maddiaa0 force-pushed the md/pipelining-syncing branch from 8de6157 to 31f941d Compare March 3, 2026 20:51

Maddiaa0 force-pushed the md/update-epoch-cache-for-buildahead branch from 3a75fdb to 1ca98d8 Compare March 3, 2026 20:51

This was referenced Mar 3, 2026

feat(sequencer): add build-ahead config and metrics #20779

Merged

feat(pipeline): introduce pipeline views for building #21026

Merged

chore(pipeline): timetable adjustments under pipelining #21076

Open

spalladino reviewed Mar 4, 2026

View reviewed changes

Maddiaa0 force-pushed the md/pipelining-syncing branch 2 times, most recently from 6a16e98 to 28a0520 Compare March 6, 2026 17:19

Maddiaa0 commented Mar 9, 2026

View reviewed changes

yarn-project/validator-client/src/block_proposal_handler.ts Outdated Show resolved Hide resolved

Maddiaa0 force-pushed the md/pipelining-syncing branch 2 times, most recently from 27f8a4b to f5c6308 Compare March 9, 2026 12:12

Maddiaa0 force-pushed the md/update-epoch-cache-for-buildahead branch from b1b1b14 to ce0e422 Compare March 9, 2026 12:12

Maddiaa0 force-pushed the md/pipelining-syncing branch from f5c6308 to ae6b651 Compare March 9, 2026 12:26

Maddiaa0 marked this pull request as ready for review March 9, 2026 12:29

Maddiaa0 mentioned this pull request Mar 9, 2026

feat(pipeline): handle pipeline prunes #21250

Open

Maddiaa0 force-pushed the md/update-epoch-cache-for-buildahead branch from ce0e422 to d8b8d31 Compare March 9, 2026 16:01

Maddiaa0 force-pushed the md/pipelining-syncing branch 2 times, most recently from 3a47fbf to 4bb6845 Compare March 9, 2026 16:07

Maddiaa0 force-pushed the md/update-epoch-cache-for-buildahead branch from d8b8d31 to 6973d24 Compare March 9, 2026 16:41

Maddiaa0 force-pushed the md/pipelining-syncing branch from 4bb6845 to 0e9b52e Compare March 9, 2026 16:41

Maddiaa0 mentioned this pull request Mar 10, 2026

chore(pipeline): spartan config #21285

Open

Maddiaa0 force-pushed the md/pipelining-syncing branch from 0e9b52e to dee426a Compare March 12, 2026 15:52

Maddiaa0 force-pushed the md/update-epoch-cache-for-buildahead branch from 6973d24 to 21cdcd5 Compare March 12, 2026 15:52

This was referenced Mar 12, 2026

feat(pipeline): minimize deadzone w cross slot attesting #21435

Open

chore: Update latency explorer #21590

Open

chore(pipeline): shorten last block in checkpoint build duration to allow checkpoint construction #21591

Draft

Maddiaa0 force-pushed the md/update-epoch-cache-for-buildahead branch from 21cdcd5 to 9543a61 Compare March 16, 2026 18:08

Maddiaa0 force-pushed the md/pipelining-syncing branch 2 times, most recently from 7c2c34a to 2548aed Compare March 18, 2026 14:37

Maddiaa0 force-pushed the md/update-epoch-cache-for-buildahead branch from c60ca91 to 9bba2e5 Compare March 18, 2026 14:55

Maddiaa0 changed the base branch from md/update-epoch-cache-for-buildahead to graphite-base/21025 March 18, 2026 15:27

Maddiaa0 force-pushed the graphite-base/21025 branch from 9bba2e5 to 4902133 Compare March 19, 2026 00:41

Maddiaa0 force-pushed the md/pipelining-syncing branch from 2548aed to 4c620ca Compare March 19, 2026 00:41

Maddiaa0 changed the base branch from graphite-base/21025 to merge-train/spartan March 19, 2026 00:41

Maddiaa0 commented Mar 19, 2026

View reviewed changes

Maddiaa0 force-pushed the md/pipelining-syncing branch from 4c620ca to c2cc23c Compare March 19, 2026 00:55

Maddiaa0 requested review from PhilWindle and spalladino March 19, 2026 00:56

Maddiaa0 force-pushed the md/pipelining-syncing branch from c2cc23c to 366f809 Compare March 19, 2026 02:07

PhilWindle reviewed Mar 19, 2026

View reviewed changes

Maddiaa0 force-pushed the md/pipelining-syncing branch 2 times, most recently from dcd68a6 to 2d31671 Compare March 19, 2026 13:58

Maddiaa0 commented Mar 19, 2026

View reviewed changes

PhilWindle reviewed Mar 19, 2026

View reviewed changes

feat(pipeline): allow syncing blocks ontop of the proposed chain

2591475

Maddiaa0 force-pushed the md/pipelining-syncing branch from 2d31671 to 2591475 Compare March 19, 2026 21:03

chore: dont build too far ahead

7ee46e9

		/** Tracks the fire-and-forget L1 submission promise so it can be awaited during shutdown. */
		private pendingL1Submission: Promise<void> \| undefined;

Conversation

Maddiaa0 commented Mar 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Overview

Testing

Upcoming

Uh oh!

Maddiaa0 commented Mar 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Maddiaa0 Mar 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Maddiaa0 commented Mar 3, 2026 •

edited

Loading

Maddiaa0 commented Mar 3, 2026 •

edited

Loading

Maddiaa0 Mar 6, 2026 •

edited

Loading