From 5b719836f5db878ddc5e3b39bad205ec87f73fc0 Mon Sep 17 00:00:00 2001 From: "lightclient@protonmail.com" Date: Wed, 3 May 2023 16:00:25 -0600 Subject: [PATCH 1/7] 4844: de-sszify spec --- EIPS/eip-4844.md | 163 ++++++++++++----------------------------------- 1 file changed, 42 insertions(+), 121 deletions(-) diff --git a/EIPS/eip-4844.md b/EIPS/eip-4844.md index 54c5f641789514..86da01d52341c0 100644 --- a/EIPS/eip-4844.md +++ b/EIPS/eip-4844.md @@ -101,93 +101,28 @@ def fake_exponential(factor: int, numerator: int, denominator: int) -> int: ### New transaction type -We introduce a new [EIP-2718](./eip-2718.md) transaction type, -with the format being the single byte `BLOB_TX_TYPE` followed by an SSZ encoding of the -`SignedBlobTransaction` container comprising the transaction contents: +We introduce a new [EIP-2718](./eip-2718.md) transaction, "blob transaction", where the `TransactionType` is `BLOB_TX_TYPE` and the `TransactionPayload` is the following RLP value: -```python -class SignedBlobTransaction(Container): - message: BlobTransaction - signature: ECDSASignature - -class BlobTransaction(Container): - chain_id: uint256 - nonce: uint64 - max_priority_fee_per_gas: uint256 - max_fee_per_gas: uint256 - gas: uint64 - to: Union[None, Address] # Address = Bytes20 - value: uint256 - data: ByteList[MAX_CALLDATA_SIZE] - access_list: List[AccessTuple, MAX_ACCESS_LIST_SIZE] - max_fee_per_data_gas: uint256 - versioned_hashes: List[VersionedHash, MAX_VERSIONED_HASHES_LIST_SIZE] - -class AccessTuple(Container): - address: Address # Bytes20 - storage_keys: List[Hash, MAX_ACCESS_LIST_STORAGE_KEYS] - -class ECDSASignature(Container): - y_parity: boolean - r: uint256 - s: uint256 +``` +rlp([chain_id, nonce, max_priority_fee_per_gas, max_fee_per_gas, gas_limit, destination, amount, data, blob_versioned_hashes, access_list, y_parity, r, s])`. ``` -The `max_priority_fee_per_gas` and `max_fee_per_gas` fields follow [EIP-1559](./eip-1559.md) semantics, -and `access_list` as in [`EIP-2930`](./eip-2930.md). - -[`EIP-2718`](./eip-2718.md) is extended with a "wrapper data", the typed transaction can be encoded in two forms, dependent on the context: - -- Network (default): `TransactionType || TransactionNetworkPayload`, or `LegacyTransaction` -- Minimal (as in execution payload): `TransactionType || TransactionPayload`, or `LegacyTransaction` - -Execution-payloads / blocks use the minimal encoding of transactions. -In the transaction-pool and local transaction-journal the network encoding is used. - -For previous types of transactions the network encoding is no different, i.e. `TransactionNetworkPayload == TransactionPayload`. - -The `TransactionNetworkPayload` wraps a `TransactionPayload` with additional data: -this wrapping data SHOULD be verified directly before or after signature verification. - -When a blob transaction is passed through the network (see the [Networking](#networking) section below), -the `TransactionNetworkPayload` version of the transaction also includes `blobs`, `commitments` and `proofs`. -The execution layer verifies the wrapper validity against the inner `TransactionPayload` after signature verification as: +The `max_priority_fee_per_gas` and `max_fee_per_gas` fields follow [EIP-1559](./eip-1559.md) semantics +and `access_list` follows [EIP-2930](./eip-2930.md). -- `versioned_hashes` must not be empty -- All hashes in `versioned_hashes` must start with the byte `BLOB_COMMITMENT_VERSION_KZG` -- There may be at most `MAX_DATA_GAS_PER_BLOCK // DATA_GAS_PER_BLOB` total blob commitments in a valid block. -- There is an equal amount of versioned hashes, commitments, blobs and proofs. -- The commitments hash to the versioned hashes, i.e. `kzg_to_versioned_hash(commitment[i]) == versioned_hash[i]` -- The commitments match the blob contents. (Note: this can be optimized with additional data, using a proof for a - random evaluation at two points derived from the commitment and blob data) +The `blob_versioned_hashes` field represents a list hash outputs from `kzg_to_versioned_hash`. +The [EIP-2718](./eip-2718.md) `ReceiptPayload` for this transaction is `rlp([status, cumulative_transaction_gas_used, logs_bloom, logs])`. -The signature is verified and `tx.origin` is calculated as follows: +#### Signature -```python -def unsigned_tx_hash(tx: SignedBlobTransaction) -> Bytes32: - # The pre-image is prefixed with the transaction-type to avoid hash collisions with other tx hashers and types - return keccak256(BLOB_TX_TYPE + ssz.serialize(tx.message)) - -def get_origin(tx: SignedBlobTransaction) -> Address: - sig = tx.signature - # v = int(y_parity) + 27, same as EIP-1559 - return ecrecover(unsigned_tx_hash(tx), int(sig.y_parity)+27, sig.r, sig.s) -``` +The signature values `y_parity`, `r`, and `s` are calculated by constructing a secp256k1 signature over the following digest: -The hash of a signed blob transaction should be computed as: - -```python -def signed_tx_hash(tx: SignedBlobTransaction) -> Bytes32: - return keccak256(BLOB_TX_TYPE + ssz.serialize(tx)) -``` - -Blob transactions with empty `versioned_hashes` are also considered invalid during execution block verification, and must not be included in a valid block. +`keccak256(BLOB_TX_TYPE || rlp([chain_id, nonce, max_priority_fee_per_gas, max_fee_per_gas, gas_limit, destination, amount, data, blob_versioned_hashes, access_list]))`. ### Header extension -The current header encoding is extended with a new 256-bit unsigned integer field `excess_data_gas`. This is the running -total of excess data gas consumed on chain since this EIP was activated. If the total amount of data gas is below the +The current header encoding is extended with a new 256-bit unsigned integer field `excess_data_gas`. This is the running total of excess data gas consumed on chain since this EIP was activated. If the total amount of data gas is below the target, `excess_data_gas` is capped at zero. The resulting RLP encoding of the header is therefore: @@ -247,8 +182,8 @@ The `ethereum/consensus-specs` repository defines the following beacon-node chan ### Opcode to get versioned hashes We add an opcode `DATAHASH` (with byte value `HASH_OPCODE_BYTE`) which reads `index` from the top of the stack -as big-endian `uint256`, and replaces it on the stack with `tx.message.versioned_hashes[index]` -if `index < len(tx.message.versioned_hashes)`, and otherwise with a zeroed `bytes32` value. +as big-endian `uint256`, and replaces it on the stack with `tx.blob_versioned_hashes[index]` +if `index < len(tx.blob_versioned_hashes)`, and otherwise with a zeroed `bytes32` value. The opcode has a gas cost of `HASH_OPCODE_GAS`. ### Point evaluation precompile @@ -294,7 +229,7 @@ def calc_data_fee(tx: SignedBlobTransaction, parent: Header) -> int: return get_total_data_gas(tx) * get_data_gasprice(parent) def get_total_data_gas(tx: SignedBlobTransaction) -> int: - return DATA_GAS_PER_BLOB * len(tx.message.versioned_hashes) + return DATA_GAS_PER_BLOB * len(tx.blob_versioned_hashes) def get_data_gasprice(header: Header) -> int: return fake_exponential( @@ -315,12 +250,12 @@ def validate_block(block: Block) -> None: ... # the signer must be able to afford the transaction - assert signer(tx).balance >= tx.message.gas * tx.message.max_fee_per_gas + get_total_data_gas(tx) * tx.message.max_fee_per_data_gas + assert signer(tx).balance >= tx.gas * tx.max_fee_per_gas + get_total_data_gas(tx) * tx.max_fee_per_data_gas # ensure that the user was willing to at least pay the current data gasprice - assert tx.message.max_fee_per_data_gas >= get_data_gasprice(parent(block).header) + assert tx.max_fee_per_data_gas >= get_data_gasprice(parent(block).header) - num_blobs += len(tx.message.versioned_hashes) + num_blobs += len(tx.blob_versioned_hashes) # check that the excess data gas is correct expected_edg = calc_excess_data_gas(parent(block).header, num_blobs) @@ -331,42 +266,32 @@ The actual `data_fee` as calculated via `calc_data_fee` is deducted from the sen ### Networking -Nodes must not automatically broadcast blob transactions to their peers. -Instead, those transactions are only announced using `NewPooledTransactionHashes` messages, and can then be manually requested via `GetPooledTransactions`. - -Transactions are presented as `TransactionType || TransactionNetworkPayload` on the execution layer network, -the payload is a SSZ encoded container: +Blob transactions have two network representations. During transaction gossip responses (`PooledTransactions`), the EIP-2718 `TransactionPayload` of the blob transaction is wrapped to become: -```python -class BlobTransactionNetworkWrapper(Container): - tx: SignedBlobTransaction - # KZGCommitment = Bytes48 - commitments: List[KZGCommitment, MAX_TX_WRAP_COMMITMENTS] - # BLSFieldElement = uint256 - blobs: List[Vector[BLSFieldElement, FIELD_ELEMENTS_PER_BLOB], LIMIT_BLOBS_PER_TX] - # KZGProof = Bytes48 - proofs: List[KZGProof, MAX_TX_WRAP_COMMITMENTS] +``` +rlp([blob_tx_payload, blob_kzgs, blobs, kzg_aggregated_proof]) ``` -We do network-level validation of `BlobTransactionNetworkWrapper` objects as follows: +Each of these elements are defined as follows: -```python -def validate_blob_transaction_wrapper(wrapper: BlobTransactionNetworkWrapper): - versioned_hashes = wrapper.tx.message.versioned_hashes - commitments = wrapper.commitments - blobs = wrapper.blobs - proofs = wrapper.proofs - # note: assert blobs are not malformatted - assert len(versioned_hashes) == len(commitments) == len(blobs) == len(proofs) - assert len(versioned_hashes) > 0 - - # Verify that commitments match the blobs by checking the KZG proofs - assert verify_blob_kzg_proof_batch(blobs, commitments, proofs) - - # Now that all commitments have been verified, check that versioned_hashes matches the commitments - for versioned_hash, commitment in zip(versioned_hashes, commitments): - assert versioned_hash == kzg_to_versioned_hash(commitment) -``` +* `blob_tx_payload` - standard EIP-2718 blob transaction `TransactionPayload` +* `blob_kzgs` - list of `KZGCommitment` +* `blobs` - list of `blob` where `blob` is a list of `BLSFieldElement` +* `kzg_aggregated_proof` - `KZGProof` + +The node MUST validate `blob_tx_payload` and verify the wrapped data against it. To do so, ensure that: +- `blob_tx_payload.blob_versioned_hashes` must not be empty +- All hashes in `blob_tx_payload.blob_versioned_hashes` must start with the byte `BLOB_COMMITMENT_VERSION_KZG` +- There must be at most `MAX_DATA_GAS_PER_BLOCK // DATA_GAS_PER_BLOB` total blob commitments in the transaction. +- There are an equal number of versioned hashes, kzg commitments, and blobs. +- The KZG commitments hash to the versioned hashes, i.e. `kzg_to_versioned_hash(kzg[i]) == versioned_hash[i]` +- The KZG commitments match the blob contents. (Note: this can be optimized with additional data, using a proof for a + random evaluation at two points derived from the commitment and blob data) + +For body retreival responses (`BlockBodies`), the standard EIP-2718 blob transaction `TransactionPayload` is used. + +Nodes MUST NOT automatically broadcast blob transactions to their peers. +Instead, those transactions are only announced using `NewPooledTransactionHashes` messages, and can then be manually requested via `GetPooledTransactions`. ## Rationale @@ -405,11 +330,7 @@ The work that remains to be done to get to full sharding includes: - PBS (proposer/builder separation), to avoid requiring individual validators to process 32 MB of data in one slot - Proof of custody or similar in-protocol requirement for each validator to verify a particular part of the sharded data in each block -This EIP also sets the stage for longer-term protocol cleanups: - -- It adds an SSZ transaction type, and paves the precedent that all new transaction types should be SSZ -- It defines `TransactionNetworkPayload` to separate network and block encodings of a transaction type -- Its (cleaner) gas price update rule could be applied to the primary basefee +This EIP also sets the stage for longer-term protocol cleanups. For example, its (cleaner) gas price update rule could be applied to the primary basefee calculation. ### How rollups would function @@ -460,8 +381,8 @@ The values for `TARGET_DATA_GAS_PER_BLOCK` and `MAX_DATA_GAS_PER_BLOCK` are chos ### Blob non-accessibility -This EIP introduces a transaction type that has a distinct mempool version (`BlobTransactionNetworkWrapper`) and execution-payload version (`SignedBlobTransaction`), -with only one-way convertibility between the two. The blobs are in the `BlobTransactionNetworkWrapper` and not in the `SignedBlobTransaction`; +This EIP introduces a transaction type that has a distinct mempool version and execution-payload version, +with only one-way convertibility between the two. The blobs are in the network representation and not in the consensus representation; instead, they go into the `BeaconBlockBody`. This means that there is now a part of a transaction that will not be accessible from the web3 API. ### Mempool issues From b5a30dcce424e9f4760cadd63b326cac5dd81007 Mon Sep 17 00:00:00 2001 From: "lightclient@protonmail.com" Date: Wed, 3 May 2023 17:21:59 -0600 Subject: [PATCH 2/7] 4844: fix the fucking linter --- EIPS/eip-4844.md | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/EIPS/eip-4844.md b/EIPS/eip-4844.md index 86da01d52341c0..fb5c5556c6b06e 100644 --- a/EIPS/eip-4844.md +++ b/EIPS/eip-4844.md @@ -274,12 +274,13 @@ rlp([blob_tx_payload, blob_kzgs, blobs, kzg_aggregated_proof]) Each of these elements are defined as follows: -* `blob_tx_payload` - standard EIP-2718 blob transaction `TransactionPayload` -* `blob_kzgs` - list of `KZGCommitment` -* `blobs` - list of `blob` where `blob` is a list of `BLSFieldElement` -* `kzg_aggregated_proof` - `KZGProof` +- `blob_tx_payload` - standard EIP-2718 blob transaction `TransactionPayload` +- `blob_kzgs` - list of `KZGCommitment` +- `blobs` - list of `blob` where `blob` is a list of `BLSFieldElement` +- `kzg_aggregated_proof` - `KZGProof` The node MUST validate `blob_tx_payload` and verify the wrapped data against it. To do so, ensure that: + - `blob_tx_payload.blob_versioned_hashes` must not be empty - All hashes in `blob_tx_payload.blob_versioned_hashes` must start with the byte `BLOB_COMMITMENT_VERSION_KZG` - There must be at most `MAX_DATA_GAS_PER_BLOCK // DATA_GAS_PER_BLOB` total blob commitments in the transaction. From 5ffb5c6f28dbb8bfd95b642bfceed2e031aabf44 Mon Sep 17 00:00:00 2001 From: "lightclient@protonmail.com" Date: Thu, 4 May 2023 13:34:51 -0600 Subject: [PATCH 3/7] 4844: reorder fields in rlp add missing fee field for data --- EIPS/eip-4844.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/EIPS/eip-4844.md b/EIPS/eip-4844.md index fb5c5556c6b06e..c661824ce4ebf0 100644 --- a/EIPS/eip-4844.md +++ b/EIPS/eip-4844.md @@ -104,13 +104,13 @@ def fake_exponential(factor: int, numerator: int, denominator: int) -> int: We introduce a new [EIP-2718](./eip-2718.md) transaction, "blob transaction", where the `TransactionType` is `BLOB_TX_TYPE` and the `TransactionPayload` is the following RLP value: ``` -rlp([chain_id, nonce, max_priority_fee_per_gas, max_fee_per_gas, gas_limit, destination, amount, data, blob_versioned_hashes, access_list, y_parity, r, s])`. +rlp([chain_id, nonce, max_priority_fee_per_gas, max_fee_per_gas, gas_limit, destination, amount, data, access_list, max_fee_per_data_gas, blob_versioned_hashes, y_parity, r, s])`. ``` The `max_priority_fee_per_gas` and `max_fee_per_gas` fields follow [EIP-1559](./eip-1559.md) semantics and `access_list` follows [EIP-2930](./eip-2930.md). -The `blob_versioned_hashes` field represents a list hash outputs from `kzg_to_versioned_hash`. +The `max_fee_per_data_gas` is `uint256` and the `blob_versioned_hashes` field represents a list hash outputs from `kzg_to_versioned_hash`. The [EIP-2718](./eip-2718.md) `ReceiptPayload` for this transaction is `rlp([status, cumulative_transaction_gas_used, logs_bloom, logs])`. @@ -118,7 +118,7 @@ The [EIP-2718](./eip-2718.md) `ReceiptPayload` for this transaction is `rlp([sta The signature values `y_parity`, `r`, and `s` are calculated by constructing a secp256k1 signature over the following digest: -`keccak256(BLOB_TX_TYPE || rlp([chain_id, nonce, max_priority_fee_per_gas, max_fee_per_gas, gas_limit, destination, amount, data, blob_versioned_hashes, access_list]))`. +`keccak256(BLOB_TX_TYPE || rlp([chain_id, nonce, max_priority_fee_per_gas, max_fee_per_gas, gas_limit, destination, amount, data, access_list, max_fee_per_data_gas, blob_versioned_hashes]))`. ### Header extension From f66370f3c97556c7e8bcbc98e7c8fff6669771fd Mon Sep 17 00:00:00 2001 From: "lightclient@protonmail.com" Date: Tue, 16 May 2023 11:27:36 +0200 Subject: [PATCH 4/7] better names --- EIPS/eip-4844.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/EIPS/eip-4844.md b/EIPS/eip-4844.md index c661824ce4ebf0..0c41685640b5c1 100644 --- a/EIPS/eip-4844.md +++ b/EIPS/eip-4844.md @@ -104,7 +104,7 @@ def fake_exponential(factor: int, numerator: int, denominator: int) -> int: We introduce a new [EIP-2718](./eip-2718.md) transaction, "blob transaction", where the `TransactionType` is `BLOB_TX_TYPE` and the `TransactionPayload` is the following RLP value: ``` -rlp([chain_id, nonce, max_priority_fee_per_gas, max_fee_per_gas, gas_limit, destination, amount, data, access_list, max_fee_per_data_gas, blob_versioned_hashes, y_parity, r, s])`. +rlp([chain_id, nonce, max_priority_fee_per_gas, max_fee_per_gas, gas_limit, to, value, data, access_list, max_fee_per_data_gas, blob_versioned_hashes, y_parity, r, s])`. ``` The `max_priority_fee_per_gas` and `max_fee_per_gas` fields follow [EIP-1559](./eip-1559.md) semantics @@ -118,7 +118,7 @@ The [EIP-2718](./eip-2718.md) `ReceiptPayload` for this transaction is `rlp([sta The signature values `y_parity`, `r`, and `s` are calculated by constructing a secp256k1 signature over the following digest: -`keccak256(BLOB_TX_TYPE || rlp([chain_id, nonce, max_priority_fee_per_gas, max_fee_per_gas, gas_limit, destination, amount, data, access_list, max_fee_per_data_gas, blob_versioned_hashes]))`. +`keccak256(BLOB_TX_TYPE || rlp([chain_id, nonce, max_priority_fee_per_gas, max_fee_per_gas, gas_limit, to, value, data, access_list, max_fee_per_data_gas, blob_versioned_hashes]))`. ### Header extension From 7782d6cd6af7b6d9034a6374be1c5a7c87d9560d Mon Sep 17 00:00:00 2001 From: lightclient <14004106+lightclient@users.noreply.github.com> Date: Tue, 16 May 2023 12:32:06 +0200 Subject: [PATCH 5/7] 4844: one kzg proof per blob Co-authored-by: dankrad --- EIPS/eip-4844.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/EIPS/eip-4844.md b/EIPS/eip-4844.md index 0c41685640b5c1..3ddff8747fcacb 100644 --- a/EIPS/eip-4844.md +++ b/EIPS/eip-4844.md @@ -269,7 +269,7 @@ The actual `data_fee` as calculated via `calc_data_fee` is deducted from the sen Blob transactions have two network representations. During transaction gossip responses (`PooledTransactions`), the EIP-2718 `TransactionPayload` of the blob transaction is wrapped to become: ``` -rlp([blob_tx_payload, blob_kzgs, blobs, kzg_aggregated_proof]) +rlp([blob_tx_payload, blob_kzgs, blobs, blob_kzg_proofs]) ``` Each of these elements are defined as follows: From 39a4857e175a32a0449079c3433288148e9b6a76 Mon Sep 17 00:00:00 2001 From: lightclient <14004106+lightclient@users.noreply.github.com> Date: Tue, 16 May 2023 12:33:16 +0200 Subject: [PATCH 6/7] 4844: fix info about optimization Co-authored-by: dankrad --- EIPS/eip-4844.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/EIPS/eip-4844.md b/EIPS/eip-4844.md index 3ddff8747fcacb..f3934f4a56abeb 100644 --- a/EIPS/eip-4844.md +++ b/EIPS/eip-4844.md @@ -286,8 +286,8 @@ The node MUST validate `blob_tx_payload` and verify the wrapped data against it. - There must be at most `MAX_DATA_GAS_PER_BLOCK // DATA_GAS_PER_BLOB` total blob commitments in the transaction. - There are an equal number of versioned hashes, kzg commitments, and blobs. - The KZG commitments hash to the versioned hashes, i.e. `kzg_to_versioned_hash(kzg[i]) == versioned_hash[i]` -- The KZG commitments match the blob contents. (Note: this can be optimized with additional data, using a proof for a - random evaluation at two points derived from the commitment and blob data) +- The KZG commitments match the blob contents. (Note: this can be optimized using `blob_kzg_proofs`, with a proof for a + random evaluation at a point derived from the commitment and blob data for each blob) For body retreival responses (`BlockBodies`), the standard EIP-2718 blob transaction `TransactionPayload` is used. From 6d1e8403a4ced6bd151b9a680652e81751a19dc9 Mon Sep 17 00:00:00 2001 From: lightclient <14004106+lightclient@users.noreply.github.com> Date: Tue, 16 May 2023 12:33:27 +0200 Subject: [PATCH 7/7] 4844: fix typo Co-authored-by: dankrad --- EIPS/eip-4844.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/EIPS/eip-4844.md b/EIPS/eip-4844.md index f3934f4a56abeb..78e4c291b1b666 100644 --- a/EIPS/eip-4844.md +++ b/EIPS/eip-4844.md @@ -289,7 +289,7 @@ The node MUST validate `blob_tx_payload` and verify the wrapped data against it. - The KZG commitments match the blob contents. (Note: this can be optimized using `blob_kzg_proofs`, with a proof for a random evaluation at a point derived from the commitment and blob data for each blob) -For body retreival responses (`BlockBodies`), the standard EIP-2718 blob transaction `TransactionPayload` is used. +For body retrieval responses (`BlockBodies`), the standard EIP-2718 blob transaction `TransactionPayload` is used. Nodes MUST NOT automatically broadcast blob transactions to their peers. Instead, those transactions are only announced using `NewPooledTransactionHashes` messages, and can then be manually requested via `GetPooledTransactions`.