feat(docs,ci): enable docs build and publishing for the mainnet branch#2775
Merged
danceratopz merged 20 commits intoMay 8, 2026
Merged
Conversation
Bring the `Justfile`, `mkdocs.yml`, `whitelist.txt`, dependency pins, and `docc.py` plugin updates needed to build the docs on the `mainnet` branch. Includes the `docc>=0.5`, `pillow>=12`, and `rich<15` bumps plus the `docc.listing` handler additions in `src/ethereum_spec_tools/docc.py`.
Sync the `docs/` tree to match `forks/amsterdam` so the documentation published from `mainnet` reflects the current spec/EEST narrative. Drops the `--include-benchmark` flag from `gen_test_case_reference.py` since `mainnet`'s `fill` predates that flag and already collects `tests/benchmark/` by default.
Replace the `mainnet` `test-docs.yaml` with the unified `docs-build.yaml` and update `gh-pages.yaml` to match `forks/amsterdam`. Adds the `setup-uv` composite action that the new docs workflows depend on. With `mainnet` already in `.github/configs/docs-branches.yaml`, this enables the docs-aggregator to publish `mainnet` alongside `forks/amsterdam`.
Refresh `README.md` and `CONTRIBUTING.md`, add `CLAUDE.md`, and drop `EIP_AUTHORS_MANUAL.md` and `RELEASING.md` (replaced by `docs/specs/spec_releases.md`).
danceratopz
added
A-doc
`mkdocstrings` needs each test subpackage referenced by the generated test-case reference pages (e.g. `tests.benchmark.compute`) to be an importable Python package. Port the missing markers from `forks/amsterdam` so the docs build can resolve them on `mainnet`.
* chore(docs): add `docs-spec-fast` target `just docs-spec` re-renders every fork-pair diff and takes ~13–18 minutes, which makes iterating on the docc plugin painful. The plugin already honors a `DOCC_SKIP_DIFFS` environment variable that skips the diff discovery step entirely. Add a `docs-spec-fast` recipe that sets that variable and writes to a separate `docs-spec-fast` output directory so it does not stomp the full build output. * feat(doc): sort `src/ethereum/forks/` listing in fork order The diff directory listing now renders in fork chronological order, but the source listing at \`src/ethereum/forks/index.html\` still rendered forks alphabetically (\`amsterdam\`, \`arrow_glacier\`, ..., \`byzantium\`, \`cancun\`, ...). The same chronological ordering is more useful here. Compute a \`fork_order\` map in \`EthereumListingDiscover.__init__\` from \`Hardfork.discover()\` and add a third branch in \`_listing_source\` that wraps any parent path matching \`src/ethereum/forks/<fork>\` with an \`_EthereumSort\` keyed by that fork's chronological index. The existing \`_EthereumListingSource\` propagation branch carries the key upward through the parent chain, exactly as it already does for diff sources. Verified by building with \`just docs-spec-fast\` and inspecting the rendered listing: forks now render in order \`frontier\`, \`homestead\`, \`dao_fork\`, \`tangerine_whistle\`, \`spurious_dragon\`, ..., \`amsterdam\`. * refactor(doc): replace `_EthereumSort` with a path-driven plain tuple The `_EthereumListingSource` wrapping previously propagated upward through every parent of every wrapped descendant, so an `_EthereumSort` key could end up sibling to a plain `ListingSource` whose key contains a raw `PurePath`. Mixed-type tuple comparison forced `_EthereumSort` to ship a custom `__lt__(PurePath)` branch, `@total_ordering`, and the `_typeshed` `SupportsDunderGT`/`SupportsDunderLT` typing dance. Drive the wrapping decision purely from the parent path instead. Wrap exactly when `parent` is `diffs/<fork>` or `src/ethereum/forks/<fork>`, i.e. precisely the listings whose siblings we want to render in fork order. Every other listing falls through to the default plain `ListingSource`, so siblings at every level are now homogeneous in key shape and the mixed-type comparison never occurs. The key collapses to a plain `(int, PurePath)` tuple sorted by Python's built-in tuple comparison. Drop `_EthereumSort`, the `@total_ordering` decorator, and the `if TYPE_CHECKING: from _typeshed import ...` block. Also drop the `_sort` field threaded through `DiffSource`: the fork order now comes from a single `fork_order` map computed once in `EthereumListingDiscover.__init__` and looked up by path component. Trim the matching vulture whitelist entries. Verified with `just docs-spec`: both `diffs/index.html` and `src/ethereum/forks/index.html` render in chronological fork order (`frontier`, `homestead`, `dao_fork`, ..., `bpo5`, `amsterdam`). * factor out some paths, use is_relative_to * refactor justfile --------- Co-authored-by: Sam Wilson <sam@binarycake.ca>
`mainnet`'s latest fork is `BPO2`, but `DocsConfig.TARGET_FORK` / `GENERATE_UNTIL_FORK` were left at `Amsterdam` from the `forks/amsterdam` port. Setting them to `BPO2` so the docs build stops at `mainnet`'s actual fork range.
danceratopz
force-pushed
the
experiments/mainnet-docs
branch
from
b39242d to
d02bf8a
Compare
`MD060` is a cosmetic table-pipe alignment rule that doesn't affect rendering. Several ported docs (`eip_testing_checklist_template.md`, `reference_specification.md`, `writing_a_new_test.md`) trip it without harming the built site.
`docs-build.yaml` reads the publish allowlist from `.github/configs/docs-branches.yaml` via `yq`. The file lives on `forks/amsterdam` but wasn't on `mainnet`; without it the workflow fails with "Could not read allowlist". Port the file so `mainnet` and `forks/amsterdam` both publish, and `devnets/bal/4` keeps its existing entry.
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## mainnet #2775 +/- ##
========================================
Coverage 87.49% 87.49%
========================================
Files 523 523
Lines 32748 32748
Branches 3016 3016
========================================
Hits 28654 28654
Misses 3521 3521
Partials 573 573
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
`mainnet`'s EELS only goes up to `BPO2`, but `packages/testing/`'s `test_log_mismatch_during_generation` (in `src/execution_testing/specs/tests/test_expect.py`) hard-codes `fork=Amsterdam` and exits with `Unsupported state fork: Amsterdam`. The xdist worker crash from that `SystemExit` also takes down `test_fork_covariant_markers[with_all_call_opcodes_and_create_opcodes]` as collateral. Deselect the offending parametrizations so `tests_pytest_py3` and `tests_pytest_pypy3` pass on `mainnet`.
SamWilsn
reviewed
May 4, 2026
…thereum#2666 On `mainnet` the test case reference sidebar renders `Tangerine Whistle`, `Benchmark`, and `Amsterdam` at the bottom instead of in chronological order with benchmarks pinned on top. Port the ordering and counting fixes from ethereum#2666: - Snake-case the fork name when building `fork_order` so `TangerineWhistle` matches the on-disk `tests/tangerine_whistle/` directory. - Pin `benchmark` above all forks via `fork_order["benchmark"] = -1`. - Resolve transition forks (e.g. `OsakaToBPO1AtTime15k`) to their base fork when picking the displayed fork and when counting cases. - Compute `is_benchmark` for directory pages from path location rather than requiring a `@pytest.mark.benchmark` marker on a collected module.
The EIP checklist plugin attaches `checklist.md` pages under whichever test directory it first sees an EIP in. On `mainnet` (target_fork=BPO2), EIP-7928 lives in `tests/amsterdam/eip7928_block_level_access_lists/`, so a checklist page leaks in under that path and `mkdocs-gen-files` infers an empty `Amsterdam` parent in the sidebar. Filter `page_props` after collection: drop any page whose top-level test fork directory belongs to a fork newer than `target_fork`. This also lifts `_dir_name` to a module-level `_fork_dir_name` helper since it is now used in two places.
…urces The amsterdam merge (`6efdfe35e89`) replaced this branch's `vulture_whitelist.py` with amsterdam's version, which omits entries for several files that exist on `mainnet` but not on `forks/amsterdam`. Restore the entries that match files still present here: - `BaseLoad.json_to_header/state/block` and `json_data` parameter (`src/ethereum_spec_tools/evm_tools/loaders/fixture_loader.py`). - `PatchHygiene` (discovered dynamically by `ethereum_spec_tools.lint`). - `SetConstantCommand.visit_AnnAssign_target`, `leave_AnnAssign_target`, `leave_AnnAssign` (libcst visitor hooks resolved by name). - `RemoveDocstringCommand`. Entries that mainnet had but no longer apply here are not restored: `UintLenHygiene` (file removed) and the enginex fixtures (`_configure_client_manager`, `test_suite_name`, `genesis_header`).
`self._forks = Hardfork.discover()` was set in `__init__` but never read anywhere in the codebase. Drop the attribute and its annotation.
…-tests` `tox.ini` already deselects `test_log_mismatch_during_generation` on `mainnet` because the test pins `state_fork=Amsterdam`, which `Hardfork.discover()` does not return on this branch. Mirror the same deselect in the `just test-tests` and `test-tests-pypy` recipes so the local commands match CI behavior.
Member
Author
|
Hey @SamWilsn, I deleted the duplicated docs section. I added some fixes for CI, some fails remain. The state of I think it's ok to get a set of docs up. If we want to release from this branch, we will have undo (respectively redo) this work. In that case, I would favor making Wdyt about merge strategy? I would lean towards a squash-merge. |
Member
Author
|
Btw, we don't need a clean CI (in this PR) to generate the docs artifacts. We can trigger the build docs flow independently via dispatch (but the flow and the config will need to land on the target branch, i.e., |
SamWilsn
approved these changes
May 6, 2026
Contributor
SamWilsn
left a comment
SamWilsn
left a comment
There was a problem hiding this comment.
Looks fine as far as I care to check. I'd lean towards squash (with the eventual goal of merging forks/bpo3' into mainnet`).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🗒️ Description
Enable docs build and publishing for the
mainnetbranch by porting the docs toolchain, content, and CI workflows fromforks/amsterdam. Withmainnetalready listed in.github/configs/docs-branches.yaml(label:Mainnet (BPO2)), this PR makes the build actually green so the aggregator can publish it alongsideforks/amsterdam.Changes are split into atomic commits:
feat(tooling,ci): PortJustfile,mkdocs.yml,whitelist.txt,pyproject.tomldocs deps (docc>=0.5,pillow>=12,rich<15), and thedocc.listingplugin handler insrc/ethereum_spec_tools/docc.py.feat(docs): Portdocs/content (mkdocs site + spec writing guide). Drops--include-benchmarkfromdocs/scripts/gen_test_case_reference.pysincemainnet'sfillpredates that flag and already collectstests/benchmark/by default.feat(ci): Replacetest-docs.yamlwith the unifieddocs-build.yaml, updategh-pages.yaml, and add thesetup-uvcomposite action that the new docs workflows depend on.chore(docs): RefreshREADME.md/CONTRIBUTING.md, addCLAUDE.md, and dropEIP_AUTHORS_MANUAL.mdandRELEASING.md(replaced bydocs/specs/spec_releases.md).chore(tests): Add__init__.pyshims somkdocstringscan resolve generated test-case reference pages (e.g.,tests.benchmark.compute).feat(doc): usefully sort diff directory listings, and the follow-up suggestions commit).chore(test-cli): SetDocsConfig.TARGET_FORKandGENERATE_UNTIL_FORKtoBPO2, sincemainnet's latest fork isBPO2rather thanAmsterdam.Validated locally with
just docs(full mkdocs + spec doc): exit 0, 0 errors, 103s, 834 doc pages.🔗 Related Issues or PRs
Cherry-picks from #2756.
✅ Checklist
toxchecks to avoid unnecessary CI fails, see also Code Standards and Enabling Pre-commit Checks:uvx tox -e statictype(scope):.mkdocs servelocally and verified the auto-generated docs for new tests in the Test Case Reference are correctly formatted.Cute Animal Picture