diff --git a/docs/running_tests/execute/hive.md b/docs/running_tests/execute/hive.md index c4d89e48c0..82527386e5 100644 --- a/docs/running_tests/execute/hive.md +++ b/docs/running_tests/execute/hive.md @@ -51,3 +51,27 @@ The genesis file is passed to the client with the appropriate configuration for All tests will be executed in the same network, in the same client, and serially, but when the `-n auto` parameter is passed to the command, the tests can also be executed in parallel. One important feature of the `execute hive` command is that, since there is no consensus client running in the network, the command drives the chain by the use of the Engine API to prompt the execution client to generate new blocks and include the transactions in them. + +## Using `testing_buildBlockV1` + +Clients that implement the `testing_buildBlockV1` endpoint can use it as an alternative to the standard Engine API block building flow. Instead of sending transactions to the mempool and building blocks through `engine_forkchoiceUpdatedVX` / `engine_getPayloadVX`, the plugin sends transactions directly inside the `testing_buildBlockV1` call, which builds a block containing exactly those transactions. + +To enable this route, pass the `--use-testing-build-block` flag: + +```bash +uv run execute hive --fork=Prague --use-testing-build-block +``` + +Or in dev mode: + +```bash +./hive --dev --client go-ethereum +uv run execute hive --fork=Prague --use-testing-build-block +``` + +This is useful when: + +- The client supports the endpoint and you want faster block building (the `--get-payload-wait-time` delay is skipped). +- You want deterministic transaction ordering in each block (transactions are included in the exact order provided). + +See [Block Building with `testing_buildBlockV1`](./index.md#block-building-with-testing_buildblockv1) for architectural details. diff --git a/docs/running_tests/execute/index.md b/docs/running_tests/execute/index.md index 8f61c2c81d..3aba6d9f7f 100644 --- a/docs/running_tests/execute/index.md +++ b/docs/running_tests/execute/index.md @@ -80,3 +80,24 @@ A warning is logged when `max_transactions_per_batch` exceeds 1000, as this may - **Benchmark tests**: Tests that measure gas consumption often generate many transactions - **Stress testing**: When intentionally testing RPC endpoint limits - **Slow RPC endpoints**: Reduce batch size to avoid timeouts on slower endpoints + +### Block Building with `testing_buildBlockV1` + +By default, the `execute` plugin drives block production through the Engine API: transactions are sent to the client's mempool via `eth_sendRawTransaction`, and blocks are built using the `engine_forkchoiceUpdatedVX` / `engine_getPayloadVX` / `engine_newPayloadVX` sequence. + +Clients that implement the [`testing_buildBlockV1`](https://github.com/ethereum/execution-apis/blob/main/src/testing/testing_buildBlockV1.yaml) endpoint offer an alternative route that collapses transaction submission and block building into a single RPC call. When enabled, the plugin: + +1. Collects the raw RLP-encoded transactions for each batch. +2. Calls `testing_buildBlockV1` with the parent block hash, payload attributes, and the transaction list. +3. Finalizes the returned payload with `engine_newPayloadVX` and `engine_forkchoiceUpdatedVX`. + +Because transactions are included directly in the built block (rather than pulled from the mempool), the standard Engine API `engine_getPayloadVX` call and the `--get-payload-wait-time` delay are both skipped. + +**CLI Configuration:** + +```bash +# Enable the testing_buildBlockV1 route +execute hive --fork=Prague --use-testing-build-block +``` + +This flag is available for both `execute hive` and `execute remote` (when an engine endpoint is configured). See [Execute Hive](./hive.md) and [Execute Remote](./remote.md) for mode-specific details. diff --git a/docs/running_tests/execute/remote.md b/docs/running_tests/execute/remote.md index 9f395ad553..674503cba8 100644 --- a/docs/running_tests/execute/remote.md +++ b/docs/running_tests/execute/remote.md @@ -87,6 +87,23 @@ The JWT secret file must contain only the JWT secret as a hex string. When an engine endpoint is provided, the test execution will use the Engine API to create new blocks and include transactions, giving you full control over the chain progression. +### Using `testing_buildBlockV1` with a Remote Engine + +If the execution client supports the `testing_buildBlockV1` endpoint, you can enable it alongside the engine endpoint: + +```bash +uv run execute remote --fork=Prague \ + --rpc-endpoint=https://rpc.endpoint.io \ + --rpc-seed-key 0x... --chain-id 12345 \ + --engine-endpoint=https://engine.endpoint.io \ + --engine-jwt-secret-file /path/to/jwt-secret.txt \ + --use-testing-build-block +``` + +This flag requires `--engine-endpoint` to be set, because `engine_newPayload` and `engine_forkchoiceUpdated` are still needed to finalize blocks built by `testing_buildBlockV1`. Note that `testing_buildBlockV1` itself is served on the unauthenticated ETH RPC port. + +See [Block Building with `testing_buildBlockV1`](./index.md#block-building-with-testing_buildblockv1) for architectural details. + The `execute remote` command will connect to the client via the RPC endpoint and will start executing every test in the `./tests` folder in the same way as the `execute hive` command, but instead of using the Engine API to generate blocks, it will send the transactions to the client via the RPC endpoint. It is recommended to only run a subset of the tests when executing on a live network. To do so, a path to a specific test can be provided to the command: diff --git a/packages/testing/src/execution_testing/cli/pytest_commands/plugins/execute/execute.py b/packages/testing/src/execution_testing/cli/pytest_commands/plugins/execute/execute.py index 79cb2f8f6b..d2e6efed5e 100644 --- a/packages/testing/src/execution_testing/cli/pytest_commands/plugins/execute/execute.py +++ b/packages/testing/src/execution_testing/cli/pytest_commands/plugins/execute/execute.py @@ -157,6 +157,17 @@ def pytest_addoption(parser: pytest.Parser) -> None: "RPC. Default=750. Higher values may cause RPC instability." ), ) + execute_group.addoption( + "--use-testing-build-block", + action="store_true", + dest="use_testing_build_block", + default=False, + help=( + "Use testing_buildBlockV1 to build blocks with transactions " + "directly, instead of the standard Engine API flow. " + "Only for clients that implement this endpoint." + ), + ) report_group = parser.getgroup( "tests", "Arguments defining html report behavior" @@ -338,6 +349,14 @@ def max_transactions_per_batch(request: pytest.FixtureRequest) -> int | None: return request.config.getoption("max_tx_per_batch") +@pytest.fixture(scope="session") +def use_testing_build_block( + request: pytest.FixtureRequest, +) -> bool: + """Return whether to use testing_buildBlockV1 for block building.""" + return request.config.getoption("use_testing_build_block") + + @pytest.fixture(scope="session") def default_max_fee_per_gas( request: pytest.FixtureRequest, diff --git a/packages/testing/src/execution_testing/cli/pytest_commands/plugins/execute/rpc/chain_builder_eth_rpc.py b/packages/testing/src/execution_testing/cli/pytest_commands/plugins/execute/rpc/chain_builder_eth_rpc.py index c23276b064..a0df484a25 100644 --- a/packages/testing/src/execution_testing/cli/pytest_commands/plugins/execute/rpc/chain_builder_eth_rpc.py +++ b/packages/testing/src/execution_testing/cli/pytest_commands/plugins/execute/rpc/chain_builder_eth_rpc.py @@ -6,18 +6,25 @@ import time from contextlib import AbstractContextManager from pathlib import Path -from typing import Any, List +from typing import Any, List, Sequence from filelock import FileLock -from execution_testing.base_types import Address, Hash, HexNumber +from execution_testing.base_types import ( + Address, + Bytes, + Hash, + HexNumber, +) from execution_testing.forks import Fork -from execution_testing.rpc import EngineRPC +from execution_testing.rpc import EngineRPC, TestingRPC from execution_testing.rpc import EthRPC as BaseEthRPC from execution_testing.rpc.rpc_types import ( ForkchoiceState, + GetPayloadResponse, PayloadAttributes, PayloadStatusEnum, + TransactionProtocol, ) @@ -32,6 +39,7 @@ class ChainBuilderEthRPC(BaseEthRPC, namespace="eth"): engine_rpc: EngineRPC get_payload_wait_time: float block_building_lock: FileLock + testing_rpc: TestingRPC | None def __init__( self, @@ -44,6 +52,7 @@ def __init__( initial_forkchoice_update_retries: int = 5, transaction_wait_timeout: int = 60, max_transactions_per_batch: int | None = None, + testing_rpc: TestingRPC | None = None, ): """Initialize the Ethereum RPC client for the hive simulator.""" super().__init__( @@ -57,6 +66,7 @@ def __init__( session_temp_folder / "chain_builder_fcu.lock" ) self.get_payload_wait_time = get_payload_wait_time + self.testing_rpc = testing_rpc # Send initial forkchoice updated only if we are the first worker base_name = "eth_rpc_forkchoice_updated" @@ -111,15 +121,8 @@ def transaction_polling_context(self) -> AbstractContextManager: """ return self.block_building_lock - def generate_block(self: "ChainBuilderEthRPC") -> None: - """Generate a block using the Engine API.""" - # Get the head block hash - head_block = self.get_block_by_number("latest") - assert head_block is not None - - forkchoice_state = ForkchoiceState( - head_block_hash=head_block["hash"], - ) + def _payload_attributes(self, head_block: dict) -> PayloadAttributes: + """Build payload attributes from the current head block.""" parent_beacon_block_root = ( Hash(0) if self.fork.header_beacon_root_required( @@ -127,7 +130,7 @@ def generate_block(self: "ChainBuilderEthRPC") -> None: ) else None ) - payload_attributes = PayloadAttributes( + return PayloadAttributes( timestamp=HexNumber(head_block["timestamp"]) + 1, prev_randao=Hash(0), suggested_fee_recipient=Address(0), @@ -150,6 +153,63 @@ def generate_block(self: "ChainBuilderEthRPC") -> None: else None ), ) + + def _finalize_payload( + self, + payload: GetPayloadResponse, + parent_beacon_block_root: Hash | None, + ) -> None: + """ + Execute *payload* via ``engine_newPayload`` and set it as + the canonical head via ``engine_forkchoiceUpdated``. + """ + new_payload_args: List[Any] = [ + payload.execution_payload, + ] + if payload.blobs_bundle is not None: + new_payload_args.append( + payload.blobs_bundle.blob_versioned_hashes() + ) + if parent_beacon_block_root is not None: + new_payload_args.append(parent_beacon_block_root) + if payload.execution_requests is not None: + new_payload_args.append(payload.execution_requests) + new_payload_version = self.fork.engine_new_payload_version() + assert new_payload_version is not None, ( + "Fork does not support engine new_payload" + ) + new_payload_response = self.engine_rpc.new_payload( + *new_payload_args, version=new_payload_version + ) + assert new_payload_response.status == PayloadStatusEnum.VALID, ( + "Payload was invalid" + ) + + fcu_version = self.fork.engine_forkchoice_updated_version() + assert fcu_version is not None, ( + "Fork does not support engine forkchoice_updated" + ) + new_forkchoice_state = ForkchoiceState( + head_block_hash=(payload.execution_payload.block_hash), + ) + response = self.engine_rpc.forkchoice_updated( + new_forkchoice_state, + None, + version=fcu_version, + ) + assert response.payload_status.status == PayloadStatusEnum.VALID, ( + "Payload was invalid" + ) + + def generate_block(self: "ChainBuilderEthRPC") -> None: + """Generate a block using the Engine API.""" + head_block = self.get_block_by_number("latest") + assert head_block is not None + + forkchoice_state = ForkchoiceState( + head_block_hash=head_block["hash"], + ) + payload_attributes = self._payload_attributes(head_block) forkchoice_updated_version = ( self.fork.engine_forkchoice_updated_version() ) @@ -176,43 +236,51 @@ def generate_block(self: "ChainBuilderEthRPC") -> None: response.payload_id, version=get_payload_version, ) - new_payload_args: List[Any] = [new_payload.execution_payload] - if new_payload.blobs_bundle is not None: - new_payload_args.append( - new_payload.blobs_bundle.blob_versioned_hashes() - ) - if parent_beacon_block_root is not None: - new_payload_args.append(parent_beacon_block_root) - if new_payload.execution_requests is not None: - new_payload_args.append(new_payload.execution_requests) - new_payload_version = self.fork.engine_new_payload_version() - assert new_payload_version is not None, ( - "Fork does not support engine new_payload" - ) - new_payload_response = self.engine_rpc.new_payload( - *new_payload_args, version=new_payload_version - ) - assert new_payload_response.status == PayloadStatusEnum.VALID, ( - "Payload was invalid" - ) - - new_forkchoice_state = ForkchoiceState( - head_block_hash=new_payload.execution_payload.block_hash, - ) - response = self.engine_rpc.forkchoice_updated( - new_forkchoice_state, - None, - version=forkchoice_updated_version, - ) - assert response.payload_status.status == PayloadStatusEnum.VALID, ( - "Payload was invalid" + self._finalize_payload( + new_payload, + payload_attributes.parent_beacon_block_root, ) def pending_transactions_handler(self) -> None: """ Called inside the transaction inclusion wait-loop. - This class triggers the block building process if it's still waiting - for transactions to be included. + This class triggers the block building process if it's still + waiting for transactions to be included. """ self.generate_block() + + def send_transactions( + self, + transactions: Sequence[TransactionProtocol], + ) -> List[Hash]: + """ + Send transactions to the execution client. + + When ``testing_rpc`` is configured, build and finalize a + block containing *transactions* via + ``testing_buildBlockV1`` instead of sending them to the + mempool with ``eth_sendRawTransaction``. + """ + if self.testing_rpc is None: + return super().send_transactions(transactions) + if not transactions: + return [] + + with self.block_building_lock: + head_block = self.get_block_by_number("latest") + assert head_block is not None + + payload_attributes = self._payload_attributes(head_block) + new_payload = self.testing_rpc.build_block( + parent_block_hash=Hash(head_block["hash"]), + payload_attributes=payload_attributes, + transactions=transactions, + extra_data=Bytes(b""), # TODO: This is marked as optional + ) + self._finalize_payload( + new_payload, + payload_attributes.parent_beacon_block_root, + ) + + return [tx.hash for tx in transactions] diff --git a/packages/testing/src/execution_testing/cli/pytest_commands/plugins/execute/rpc/hive.py b/packages/testing/src/execution_testing/cli/pytest_commands/plugins/execute/rpc/hive.py index 218511dd50..592f612ca4 100644 --- a/packages/testing/src/execution_testing/cli/pytest_commands/plugins/execute/rpc/hive.py +++ b/packages/testing/src/execution_testing/cli/pytest_commands/plugins/execute/rpc/hive.py @@ -35,7 +35,7 @@ ) from ...consume.simulators.helpers.ruleset import ruleset -from .chain_builder_eth_rpc import ChainBuilderEthRPC +from .chain_builder_eth_rpc import ChainBuilderEthRPC, TestingRPC def pytest_addoption(parser: pytest.Parser) -> None: @@ -398,10 +398,14 @@ def eth_rpc( session_fork: Fork, session_temp_folder: Path, max_transactions_per_batch: int | None, + use_testing_build_block: bool, ) -> EthRPC: """Initialize ethereum RPC client for the execution client under test.""" get_payload_wait_time = request.config.getoption("get_payload_wait_time") tx_wait_timeout = request.config.getoption("tx_wait_timeout") + testing_rpc = None + if use_testing_build_block: + testing_rpc = TestingRPC(f"http://{client.ip}:8545") return ChainBuilderEthRPC( rpc_endpoint=f"http://{client.ip}:8545", fork=session_fork, @@ -410,4 +414,5 @@ def eth_rpc( get_payload_wait_time=get_payload_wait_time, transaction_wait_timeout=tx_wait_timeout, max_transactions_per_batch=max_transactions_per_batch, + testing_rpc=testing_rpc, ) diff --git a/packages/testing/src/execution_testing/cli/pytest_commands/plugins/execute/rpc/remote.py b/packages/testing/src/execution_testing/cli/pytest_commands/plugins/execute/rpc/remote.py index b2045e8978..7b69093e4e 100644 --- a/packages/testing/src/execution_testing/cli/pytest_commands/plugins/execute/rpc/remote.py +++ b/packages/testing/src/execution_testing/cli/pytest_commands/plugins/execute/rpc/remote.py @@ -12,7 +12,7 @@ ) from ..pre_alloc import AddressStubs -from .chain_builder_eth_rpc import ChainBuilderEthRPC +from .chain_builder_eth_rpc import ChainBuilderEthRPC, TestingRPC def pytest_addoption(parser: pytest.Parser) -> None: @@ -179,16 +179,25 @@ def eth_rpc( session_fork: Fork, session_temp_folder: Path, max_transactions_per_batch: int | None, + use_testing_build_block: bool, ) -> EthRPC: """Initialize ethereum RPC client for the execution client under test.""" tx_wait_timeout = request.config.getoption("tx_wait_timeout") if engine_rpc is None: + if use_testing_build_block: + raise pytest.UsageError( + "--use-testing-build-block requires " + "--engine-endpoint to be set" + ) return EthRPC( rpc_endpoint, transaction_wait_timeout=tx_wait_timeout, max_transactions_per_batch=max_transactions_per_batch, ) get_payload_wait_time = request.config.getoption("get_payload_wait_time") + testing_rpc = None + if use_testing_build_block: + testing_rpc = TestingRPC(rpc_endpoint) return ChainBuilderEthRPC( rpc_endpoint=rpc_endpoint, fork=session_fork, @@ -197,4 +206,5 @@ def eth_rpc( get_payload_wait_time=get_payload_wait_time, transaction_wait_timeout=tx_wait_timeout, max_transactions_per_batch=max_transactions_per_batch, + testing_rpc=testing_rpc, ) diff --git a/packages/testing/src/execution_testing/rpc/__init__.py b/packages/testing/src/execution_testing/rpc/__init__.py index 6e0eef7680..87c62607f1 100644 --- a/packages/testing/src/execution_testing/rpc/__init__.py +++ b/packages/testing/src/execution_testing/rpc/__init__.py @@ -13,6 +13,7 @@ NetRPC, PeerConnectionTimeoutError, SendTransactionExceptionError, + TestingRPC, ) from .rpc_types import ( BlobAndProofV1, @@ -45,5 +46,6 @@ "RPCCall", "PeerConnectionTimeoutError", "SendTransactionExceptionError", + "TestingRPC", "TransactionProtocol", ] diff --git a/packages/testing/src/execution_testing/rpc/rpc.py b/packages/testing/src/execution_testing/rpc/rpc.py index 7f1477f1af..677f2905d2 100644 --- a/packages/testing/src/execution_testing/rpc/rpc.py +++ b/packages/testing/src/execution_testing/rpc/rpc.py @@ -240,6 +240,14 @@ def _build_json_rpc_request( id=request_id, ) + def namespace_extra_headers(self) -> Dict[str, str]: + """ + Extra headers that are included by default in this namespace. + + For non-jwt namespaces, this method returns an empty dictionary. + """ + return {} + def post_request( self, *, @@ -258,7 +266,7 @@ def post_request( base_header = { "Content-Type": "application/json", } - headers = base_header | extra_headers + headers = base_header | extra_headers | self.namespace_extra_headers() logger.debug( f"Sending RPC request to {self.url}, " @@ -293,7 +301,7 @@ def post_batch_request( base_header = { "Content-Type": "application/json", } - headers = base_header | extra_headers + headers = base_header | extra_headers | self.namespace_extra_headers() logger.debug( f"Sending batch RPC request to {self.url}, " @@ -326,6 +334,40 @@ def post_batch_request( return results +class BaseJwtRPC(BaseRPC): + """ + Represents an RPC namespace class that uses JWT authentication. + """ + + jwt_secret: bytes + + # Default secret used in hive + DEFAULT_JWT_SECRET: bytes = b"secretsecretsecretsecretsecretse" + + def __init__( + self, + *args: Any, + jwt_secret: bytes = DEFAULT_JWT_SECRET, + **kwargs: Any, + ) -> None: + """Initialize Engine RPC class with the given JWT secret.""" + super().__init__(*args, **kwargs) + self.jwt_secret = jwt_secret + + def namespace_extra_headers(self) -> Dict[str, str]: + """ + Overload to include JWT authentication header field. + """ + jwt_token = encode( + {"iat": int(time.time())}, + self.jwt_secret, + algorithm="HS256", + ) + return { + "Authorization": f"Bearer {jwt_token}", + } + + class EthRPC(BaseRPC): """ Represents an `eth_X` RPC class for every default ethereum RPC method used @@ -355,10 +397,7 @@ def __init__( max_transactions_per_batch: int | None = None, **kwargs: Any, ) -> None: - """ - Initialize EthRPC class with the given url and transaction wait - timeout. - """ + """Initialize JWT-authenticated RPC class with the given JWT secret.""" super().__init__(*args, **kwargs) self.transaction_wait_timeout = transaction_wait_timeout @@ -1073,74 +1112,12 @@ def trace_call(self, tr: dict[str, str], block_number: str) -> Any | None: ).result_or_raise() -class EngineRPC(BaseRPC): +class EngineRPC(BaseJwtRPC): """ Represents an Engine API RPC class for every Engine API method used within EEST based hive simulators. """ - jwt_secret: bytes - - # Default secret used in hive - DEFAULT_JWT_SECRET: bytes = b"secretsecretsecretsecretsecretse" - - def __init__( - self, - *args: Any, - jwt_secret: bytes = DEFAULT_JWT_SECRET, - **kwargs: Any, - ) -> None: - """Initialize Engine RPC class with the given JWT secret.""" - super().__init__(*args, **kwargs) - self.jwt_secret = jwt_secret - - def _jwt_extra_headers( - self, extra_headers: Dict[str, str] | None = None - ) -> Dict[str, str]: - """Build extra headers with JWT authentication.""" - if extra_headers is None: - extra_headers = {} - jwt_token = encode( - {"iat": int(time.time())}, - self.jwt_secret, - algorithm="HS256", - ) - return { - "Authorization": f"Bearer {jwt_token}", - } | extra_headers - - def post_request( - self, - *, - request: RPCCall, - extra_headers: Dict[str, str] | None = None, - timeout: int | None = None, - ) -> JSONRPCResponse: - """ - Send JSON-RPC POST request with Engine API JWT authentication. - """ - return super().post_request( - request=request, - extra_headers=self._jwt_extra_headers(extra_headers), - timeout=timeout, - ) - - def post_batch_request( - self, - *, - calls: Sequence[RPCCall], - extra_headers: Dict[str, str] | None = None, - timeout: int | None = None, - ) -> List[JSONRPCResponse]: - """ - Send JSON-RPC batch POST request with Engine API JWT authentication. - """ - return super().post_batch_request( - calls=calls, - extra_headers=self._jwt_extra_headers(extra_headers), - timeout=timeout, - ) - def new_payload(self, *params: Any, version: int) -> PayloadStatus: """ `engine_newPayloadVX`: Attempts to execute the given payload on an @@ -1375,6 +1352,47 @@ def _wait_for_peers() -> int: return _wait_for_peers() +class TestingRPC(BaseRPC): + """ + RPC class for the testing namespace, providing access to + testing-only methods like ``testing_buildBlockV1``. + """ + + def build_block( + self, + parent_block_hash: Hash, + payload_attributes: PayloadAttributes, + transactions: Sequence[TransactionProtocol] | None, + extra_data: Bytes | None = None, + *, + version: int = 1, + ) -> GetPayloadResponse: + """ + Build a block on top of *parent_block_hash* using the + provided *payload_attributes* and *transactions*. + + Calls ``testing_buildBlockVX``. + """ + method = f"buildBlockV{version}" + params: List[Any] = [ + str(parent_block_hash), + to_json(payload_attributes), + ] + if transactions is not None: + params.append([tx.rlp().hex() for tx in transactions]) + else: + params.append(None) + if extra_data is not None: + params.append(str(extra_data)) + + return GetPayloadResponse.model_validate( + self.post_request( + request=RPCCall(method=method, params=params) + ).result_or_raise(), + context=self.response_validation_context, + ) + + class AdminRPC(BaseRPC): """Represents an admin RPC class for administrative RPC calls."""