diff --git a/proposals/0337-parent-ready-update-marker.md b/proposals/0337-parent-ready-update-marker.md new file mode 100644 index 000000000..a418a3f55 --- /dev/null +++ b/proposals/0337-parent-ready-update-marker.md @@ -0,0 +1,307 @@ +--- +simd: '0337' +title: Markers for Alpenglow Fast Leader Handover +authors: + - ksn6 (Anza) + - Ashwin Sekar (Anza) + - Kobi Sliwinski (Anza) +category: Standard +type: Core +status: Review +created: 2025-08-13 +feature: TBD +--- + +## Summary + +We propose augmenting `BlockMarkerV1`, introduced in SIMD-0307, to include +`BlockHeader` and `UpdateParent` variants. `BlockHeader`, placed at +the beginning of a block, indicates the parent of the block. `UpdateParent` +signals that the intended parent is different from what was initially indicated, +paving the way for Fast Leader Handover support in Alpenglow (section 2.7 of +https://www.anza.xyz/alpenglow-1-1). + +Fast Leader Handover will minimize the synchronization delay between consecutive +leaders, increasing throughput. + +## Motivation + +SIMD-0326 proposes Alpenglow, a consensus protocol specified in +https://www.anza.xyz/alpenglow-1-1. Section 2.7 in the paper describes Fast +Leader Handover. This mechanism allows leaders to begin constructing blocks on +tentative parent blocks before the parent is ensured to be correct. In the +common case, this optimization increases throughput, by giving more time to the +leader to stream the block. To ensure correctness, the leader may need to change +the indicated parent while streaming the block. + +## New Terminology + +- **BlockHeader**: Block marker variant indicating the block's parent. Placed in + the beginning of block data. +- **UpdateParent**: Block marker variant indicating a changed parent. + +## Detailed Design + +### Specification + +`BlockHeader` contains info about the parent block. `UpdateParent` marker +contains info about the parent block that the leader switched to as a result +of Fast Leader Handover. Each variant includes its own version field: + +``` +VersionedBlockHeader Layout: ++---------------------------------------+ +| Version (1 byte) | ++---------------------------------------+ +| Payload (version-specific) (variable) | ++---------------------------------------+ +``` + +``` +VersionedUpdateParent Layout: ++---------------------------------------+ +| Version (1 byte) | ++---------------------------------------+ +| Payload (version-specific) (variable) | ++---------------------------------------+ +``` + +For Version 1: + +``` +BlockHeaderV1 Payload: ++---------------------------------------+ +| Parent Slot (8 bytes) | ++---------------------------------------+ +| Parent Block ID (32 bytes) | ++---------------------------------------+ + +Total payload size: 40 bytes +Total size with version: 41 bytes +``` + +``` +UpdateParentV1 Payload: ++---------------------------------------+ +| Parent Slot (8 bytes) | ++---------------------------------------+ +| Parent Block ID (32 bytes) | ++---------------------------------------+ + +Total payload size: 40 bytes +Total size with version: 41 bytes +``` + +Fields: + +- **Version**: Versioning byte for forward compatibility (u8) +- **Parent Slot**: The slot number of the (new) parent block (u64, + little-endian) +- **Parent Block ID**: The block ID identifying the parent block (32-byte + hash). The meaning of this field is left to specification in future + Alpenglow-related SIMDs. + +### `UpdateParent` Code Sample + +NOTE: `BlockHeader` has a nearly identical implementation. The markers are +separate to allow future header extensions. + +```rust +/// Always deserializes to the `Current` variant for forward compatibility. +/// ┌─────────────────────────────────────────┐ +/// │ Version (1 byte) │ +/// ├─────────────────────────────────────────┤ +/// │ Update Data (variable) │ +/// └─────────────────────────────────────────┘ +pub enum VersionedUpdateParent { + V1(UpdateParentV1), + Current(UpdateParentV1), +} + +/// Version 1 parent update data. +/// ┌─────────────────────────────────────────┐ +/// │ Parent Slot (8 bytes) │ +/// ├─────────────────────────────────────────┤ +/// │ Parent Block ID (32 bytes) │ +/// └─────────────────────────────────────────┘ +pub struct UpdateParentV1 { + pub new_parent_slot: Slot, + pub new_parent_block_id: Hash, +} +``` + +### Integration with BlockComponent + +The `UpdateParent` marker integrates with the existing `BlockComponent` system: + +```rust +/// Updated BlockMarkerV1. +/// Supports V1 features plus BlockHeader and UpdateParent. +/// ┌─────────────────────────────────────────┐ +/// │ Variant ID (1 byte) │ +/// ├─────────────────────────────────────────┤ +/// │ Byte Length (2 bytes) │ +/// ├─────────────────────────────────────────┤ +/// │ Variant Data (variable) │ +/// └─────────────────────────────────────────┘ +/// +/// The byte length field indicates the size of the variant data that follows, +/// allowing for proper parsing even if unknown variants are encountered. +pub enum BlockMarkerV1 { + BlockFooter(VersionedBlockFooter), + BlockHeader(VersionedBlockHeader), + UpdateParent(VersionedUpdateParent), +} +``` + +### Serialization Format + +Both `BlockHeader` and `UpdateParent` (variant 1 and 2) have the following +structure while serialized within a `BlockComponent` in the first version: + +``` ++---------------------------------------+ +| Entry Count = 0 (8 bytes) | ++---------------------------------------+ +| Marker Version = 1 (2 bytes) | ++---------------------------------------+ +| Variant ID = {1, 2} (1 byte) | ++---------------------------------------+ +| Length = 41 (2 bytes) | ++---------------------------------------+ +| Version = 1 (1 byte) | ++---------------------------------------+ +| Parent Slot (8 bytes) | ++---------------------------------------+ +| Parent Block ID (32 bytes) | ++---------------------------------------+ + +Total size: 54 bytes per marker +``` + +### Marker Placement Rules + +`BlockHeader` must be placed at the beginning of shred data of the first FEC +set of every block. + +At most one `UpdateParent` marker might be placed in a block, at the beginning +of the shred data of one FEC set. Only the first block of a leader window can +include the `UpdateParent` marker. + +Otherwise, the given block is invalid. + +### `DATA_COMPLETE_SHRED` Placement Rules + +Due to the logic of Alpenglow, validators need to be able to process the +`UpdateParent` marker outside of replay, even without ever replaying the initial +parent block. To make this possible, we introduce additional rules for the +`DATA_COMPLETE_SHRED` flag in data shreds. Most implementations already follow +this rule, but it's not yet enforced. + +FEC sets in Alpenglow will follow a fixed structure: + +- Each FEC set contains exactly 32 data shreds (insufficient data is + zero-padded) +- Each FEC set contains exactly 32 coding shreds for error correction. + +All implementations MUST enforce that the `DATA_COMPLETE_SHRED` flag can only +appear as `true` on the final data shred within an FEC set (index position 31 +within the set). Otherwise, the block is invalid. + +All implementations MUST enforce that `UpdateParent` marker can only start at +the beginning of an FEC set, AND the previous FEC set's last data shred shows +`DATA_COMPLETE_SHRED` as `true`. Otherwise, the block is invalid. + +With these rules, `UpdateParent` marker can be detected at shred ingestion, +without replaying block contents up to this point, like so: + +- Check the 0th shred of the next FEC set when `DATA_COMPLETE_SHRED` is `true` + in the current shred +- Check the current shred when the `DATA_COMPLETE_SHRED` is `true` in the + previous shred. + +See https://github.com/anza-xyz/alpenglow/pull/459 for an example implementation. + +**State Transition Rules**: + +After Fast Leader Handover is implemented, state transition will work as follows: + +- State changes from transactions before the `UpdateParent` marker are discarded +- The `UpdateParent` marker itself does not modify state +- Post-`UpdateParent` marker transactions are built on the new parent's state +- Notably, despite pre-marker transaction state changes being discarded, all + pre-marker shred data WILL be included in the constructed block. + +## Alternatives Considered + +- **Special Transaction Type**. + Rejected, because of overhead, and difficult and inconvenient implementation. + +- **Extended Block Footer**. + Rejected, because: + - Difficult to represent the switch point withing the block. + - The footer appears at the end of the block, delaying the parent switch + detection. + +- **Shred Modification**. + Rejected, due to significant protocol changes required, backward compatibility + challenges, complexity and impact on existing shred processing. + +## Impact + +- **Positive**: + - Paves the way for Fast Leader Handover that will increase throughput. + +- **Negative**: + - Slight block size overhead. + +### Examples of Edge Cases + +**Unexpected `UpdateParent`**: A malicious leader attempting to +include an `UpdateParent` in slots 2-4 of their leader window. + +`UpdateParent` markers are only allowed in the first +slot of a leader window. Upon witnessing an `UpdateParent` marker on a +different block, a receiving validator MUST deem the block invalid and +invoke `TrySkipWindow(slot)` according to the protocol logic. + +**Multiple `UpdateParent`s in a Single Block**: A malicious leader attempting +to create blocks on multiple parents simultaneously: + +At most a single `UpdateParent` marker per block is allowed. +Upon witnessing two or more markers within a block, a receiving +validator MUST deem the block invalid and invoke `TrySkipWindow(slot)` +according to the protocol logic. + +**Invalid Parent Reference**: Leader references a non-existent or +invalid parent: + +If the receiving validator cannot validate the parent block in time, the +receiving validator will time out and invoke `TrySkipWindow(slot)`. + +NOTE: in each of the above cases, `TrySkipWindow(slot)` is NOT the same as +unconditionally issuing a skip vote. + +**Feature Flag Protection**: + +- `BlockHeader` and `UpdateParent` markers will be processed if and only if the + Alpenglow feature is activated +- Validators must run with a client containing Alpenglow code to process blocks + with `UpdateParent` and/or `BlockHeader` markers +- Missing or malformed version fields will fail deserialization gracefully + +## Security Considerations + +Upon launching Alpenglow, validators must validate blocks according to the +Alpenglow protocol. + +## Backwards Compatibility + +Blocks containing `UpdateParent` markers are **not** backward compatible with +validators that do not support Alpenglow. Blocks will not include `UpdateParent` +markers until the Alpenglow feature flag has been activated. + +## Reference Implementation + +A reference implementation is available here: +https://github.com/anza-xyz/alpenglow/pull/364