feat(consume): initial implementation of enginex

[danceratopz]

🗒️ Description

This PR adds an initial implementation of a consume enginex simulator that reuses client instances across tests sharing the same pre-allocation state to optimize test execution.

This PR does have basic pytest-xdist support which will be optimized in subsequent PRs.

Pending optimization: A huge performance gain is made if this PR is rebased on @marioevz's "optimize fill" PR ethereum#1804. For example, this command of 20 tests over two groups runs ~3x faster:

uv run consume enginex --input=https://github.com/danceratopz/execution-specs/releases/download/v0.1.0a1/fixtures.tar.gz -v -s -k "/shanghai/ and (fork_shanghai or fork_cancun) and push0"

This is because as-is, the genesis state root is (unnecessarily) calculated for every access of the pre-alloc group.

Note on fixtures:

1. Extended testing of this PR doesn't make a lot of sense, as some largish changes will come when xdist gets optimized.
2. The custom release above:
   • Only contains EEST tests --until Prague (no static tests).
   • Removes separate groups for all EEST tests marked as slow, which reduces the number of pre-alloc groups from 23938 to 312. This change will be PR'd independently. You can get this group information from the release, for example, via:

     uv run groupstats ~/.cache/ethereum-execution-spec-tests/cached_downloads/ethereum/execution-spec-tests/v5.3.0/fixtures_stable/fixtures/blockchain_tests_engine_x/pre_alloc/
     

Overview

The enginex simulator groups tests by pre_hash (hash of genesis + pre-state). Tests in the same group run against a single client instance instead of starting/stopping per test.

Key Components

New Files

• simulators/enginex/conftest.py - Client fixture with reuse logic, test sorting by group
• simulators/multi_test_client.py - MultiTestClientManager for client lifecycle, parallel fixtures to single_test_client.py
• simulators/helpers/test_tracker.py - Tracks test completion per group for cleanup
• fixtures/pre_alloc_groups.py - PreAllocGroup model for shared genesis/pre-state

Modified

• consume.py - Adds xdist_group marker based on pre_hash for enginex fixtures
• test_via_engine.py - Shared test logic between engine and enginex
• pytest_hive.py - shared_hive_test fixture for cross-test client sharing

Flow

1. Collection: pytest_collection_modifyitems counts tests per group, sorts by pre_hash
2. First test in group: Starts client, registers with manager
3. Subsequent tests: Reuses existing client
4. Last test in group: mark_test_completed triggers client shutdown
5. Session end: Cleanup any remaining clients

Fixture Chain

pre_alloc_group (cached) → client_genesis (cached) → buffered_genesis → client_files
                        → environment (cached)     ↘
                                                    → client (reused per group)

Usage

To test, you can use the recommended bootleg release here: https://github.com/danceratopz/execution-specs/releases/tag/v0.1.0a1

uv run consume enginex --input=https://github.com/danceratopz/execution-specs/releases/download/v0.1.0a1/fixtures.tar.gz -v -s -k "/shanghai/ and (fork_shanghai or fork_cancun) and push0"

🔗 Related Issues or PRs

N/A.

✅ Checklist

• All: Ran fast tox checks to avoid unnecessary CI fails, see also Code Standards and Enabling Pre-commit Checks:

  uvx tox -e static

• All: PR title adheres to the repo standard - it will be used as the squash commit message and should start type(scope):.
• All: Considered adding an entry to CHANGELOG.md. skipped
• All: Considered updating the online docs in the ./docs/ directory.

[danceratopz]

More xdist code landed in this PR than intended. Indeed, this should run correctly with xdist if loadgroup is provided to xdist's distribution flag:

-n 8 --dist=loadgroup

The xdist_group test markers, which set the xdist group to be the test's pre-alloc group hash, provide an elegant way of keeping track of test execution, so I decided to leave them in this initial PR. The purpose of these markers in xdist is to enforce that one group (which is equivalent to execution on one client, in this PR) is distributed to the same worker, not amongst multiple workers. It could even work without (if no fcu is applied), but this approach feels safest/more optimal with regards to loading resources (e.g., genesis, env).

The caching of genesis, env,.. is not necessary until we perform further optimizations, but I don't think it's too confusing if the reviewer bears in mind what's coming: We will enforce a max-group size on the pre-alloc groups and further divide them based on that parameter. This allows for better distribution of (our very skewed group distribution; some very large, many very small) groups across workers, which helps avoid worker starvation.

[danceratopz]

We add this xdist-specific marker now, even in sequential mode, as it's an elegant solution to help track the count of tests pending/executed in each pre-alloc group (in order to terminate the client upon completion of all collected tests in the group).

[danceratopz]

Update: This PR now contains basic, unoptimized xdist support.

[danceratopz]

Ah, these snuck in here, these caches are not necessary in sequential mode, where each pre-alloc group only gets executed once.

This caching is actually only useful when splitting large pre-alloc groups into smaller ones (we do this in xdist mode to allow better distribution of groups amongst workers and avoid worker starvation).