chore: enable code coverage for e2e by dhyaniarun1993 · Pull Request #485 · op-rs/op-reth

dhyaniarun1993 · 2025-12-12T05:33:36Z

Closes #442

…y-impl

…into arun/chore/add-prune-e2e

…tion

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

codecov · 2025-12-12T06:39:00Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 38.24%. Comparing base (f7603f7) to head (19be4b6).
⚠️ Report is 1 commits behind head on unstable.

Additional details and impacted files

see 684 files with indirect coverage changes

Flag	Coverage Δ
e2e	`35.51% <ø> (+4.52%)`	⬆️
unit	`30.99% <ø> (?)`

Flags with carried forward coverage won't be shown. Click here to find out more.

Components	Coverage Δ
reth binary	`54.44% <ø> (+54.44%)`	⬆️
op historical proof	`89.57% <ø> (+4.85%)`	⬆️

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copilot

Pull request overview

This PR enables code coverage collection for end-to-end tests by integrating cargo-llvm-cov into the e2e test workflow. The changes configure the build and test environment to collect coverage data and upload it to Codecov, addressing issue #442.

Key Changes:

Integrated cargo-llvm-cov for code coverage collection in e2e tests
Switched from release to debug builds to support coverage instrumentation
Corrected the coverage flags in unit test workflow from "e2e" to "unit"

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

File	Description
`.github/workflows/e2e-op-historical-proof.yml`	Added cargo-llvm-cov installation, coverage collection steps, and Codecov upload for both verifier and sequencer jobs; updated matrix go_pkg_name references
`crates/optimism/tests/Makefile`	Changed build target from release to debug and updated the binary path accordingly to support coverage collection
`.github/workflows/cov-op-historical-proof.yml`	Corrected the coverage flag from "e2e" to "unit" for unit test coverage uploads

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

.github/workflows/e2e-op-historical-proof.yml

emhane · 2025-12-12T17:46:31Z

crates/optimism/tests/Makefile

 	export DEVSTACK_ORCHESTRATOR=sysgo; \
 	export OP_RETH_ENABLE_PROOF_HISTORY=true; \
-	export OP_RETH_EXEC_PATH="${SOURCE_DIR}/../../../target/release/op-reth"; \
+	export OP_RETH_EXEC_PATH="${SOURCE_DIR}/../../../target/debug/op-reth"; \


checkout the hive test profile

op-reth/Cargo.toml

Lines 304 to 308 in f7603f7

# Meant for testing - all optimizations, but with debug assertions and overflow checks.

[profile.hivetests]

inherits = "test"

opt-level = 3

lto = "thin"

Closes #442 --------- Co-authored-by: Emilia Hane <elsaemiliaevahane@gmail.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Closes op-rs/op-reth#442 --------- Co-authored-by: Emilia Hane <elsaemiliaevahane@gmail.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

@dhyaniarun1993

…19194) * refactor(db): derive Clone for DatabaseEnv (paradigmxyz/reth#21641) Co-authored-by: Amp <amp@ampcode.com> * chore(deps): breaking bumps (paradigmxyz/reth#21584) Co-authored-by: Georgios Konstantopoulos <me@gakonst.com> Co-authored-by: Amp <amp@ampcode.com> * refactor: add with_* compressor utility methods (paradigmxyz/reth#21680) * refactor: use alloy_primitives::map for all HashMap/HashSet types (paradigmxyz/reth#21686) Co-authored-by: Amp <amp@ampcode.com> * fix: support EIP-1559 params configuration for Optimism dev mode (paradigmxyz/reth#21855) * chore(rust/op-reth): pull latest changes and update dependencies * chore(op-exex): Add crate `reth-optimism-exex` and `reth-optimism-trie` (op-rs/op-reth#204) - Adds new crates `reth-optimism-exex` and `reth_optimism_trie` - Moves `reth_exex::external_proofs` -> `reth_optimism_exex` - Moves `reth_exex::external_proofs::storage` -> `reth_optimism_trie` * fix: ci lint and doc (op-rs/op-reth#234) Seems like when `reth-optimism-trie` depends on `reth-db-api` indirectly, the doctest crate couldn’t resolve `reth_db_api::…` and failing. Closes op-rs/op-reth#233 Co-authored-by: Emilia Hane <elsaemiliaevahane@gmail.com> * chore: separate storage and account cursors (op-rs/op-reth#229) This PR separates storage and account cursors which makes sense because these two cursors may have very different seek/next implementations. In the case of in-memory, a single cursor impl can still handle both, but for MDBX, it makes sense to implement separate cursors. Closes op-rs/op-reth#236 --------- Co-authored-by: Arun Dhyani <dhyaniarun7@gmail.com> * feat: add external state provider implementation (op-rs/op-reth#197) Based on op-rs/op-reth#203 , op-rs/op-reth#204 This PR implements `StateProvider` given a `OpProofsStorage` instance. It reads most data from the external database, but falls back on the latest provider for block hashes and code by hash similar to the existing historical provider in Reth. This is an important part to implementing live syncing since we're running the sync process on the DB being created. In the Reth implementation, `proof.rs` is contained in `trie/db`, so I think it makes sense to go in our DB crate. The `provider.rs` is in the `reth-provider` crate, but I don't think a separate provider crate helps us here, so I think we should also include that in the trie crate as well. --------- Co-authored-by: Emilia Hane <elsaemiliaevahane@gmail.com> * chore: add deletion tests for op proofs storage (op-rs/op-reth#230) Based on op-rs/op-reth#229 Adds tests to ensure that storing TrieUpdates that include deletions actually deletes the nodes at that block height, and that updates take precedence over deletions (same as `write_trie_updates` in Reth). Closes op-rs/op-reth#238 * feat: implement live state collector using external provider (op-rs/op-reth#198) Based on op-rs/op-reth#197 This PR implements the live state collector on top of the `OpProofsStateProvider` created in the previous PR. This sync process tries to re-execute all blocks from the current block of the external database to tip. --------- Co-authored-by: Emilia Hane <elsaemiliaevahane@gmail.com> * feat: add binary entry point for external proofs in OP (op-rs/op-reth#222) Closes op-rs/op-reth#164 --------- Co-authored-by: Arun Dhyani <dhyaniarun7@gmail.com> * feat: store hashed account and storage (op-rs/op-reth#235) Closes op-rs/op-reth#211 * feat: implement cursor read/write metrics (op-rs/op-reth#232) Fixes op-rs/op-reth#225 (does not track errors as these are fatal currently) * feat: store account and storage branches (op-rs/op-reth#255) Closes op-rs/op-reth#210 * feat: Concrete error variants for `reth-optimism-trie` (op-rs/op-reth#251) Fixes op-rs/op-reth#237 Adds more concrete error variants for `OpProofsStorageError` Co-authored-by: Arun Dhyani <dhyaniarun7@gmail.com> * feat: Remove block_number from initial sync store methods (op-rs/op-reth#248) Fixes op-rs/op-reth#247 Removes the `block_number` parameter from: - store_account_branches - store_storage_branches - store_hashed_accounts - store_hashed_storages Co-authored-by: Arun Dhyani <dhyaniarun7@gmail.com> * feat(MDBX): Implement earliest and latest block tracking in storage (op-rs/op-reth#253) Fixes op-rs/op-reth#215 --------- Co-authored-by: Arun Dhyani <dhyaniarun7@gmail.com> * feat(deps): Add `serde-bincode-compat` feature to `reth-trie` (op-rs/op-reth#250) Ref op-rs/op-reth#241 - Fixes comment from deps fix in op-rs/op-reth#229 (fix correct but comment misleading) - `serde-bincode-compat` is a feature which must be enabled when the `bincode` dependency is used. Asides for dev-deps, this only happens in `reth-stages`. `reth-exex` runs in `reth-stages`. In this wokrstream we use the `TrieUpdate` in exex. `serde-bincode-compat` is already implemented for `TrieUpdates` in `reth-trie-common`, this PR simply makes that feature accessible via `reth-trie` and in turn `reth-optimism-trie`, in order to enable it in `reth-optimism-exex`. * feat: store trie updates (op-rs/op-reth#258) Closes: op-rs/op-reth#214 * feat: implemented mdbx trie cursor (op-rs/op-reth#249) Closes op-rs/op-reth#212 --------- Co-authored-by: Emilia Hane <elsaemiliaevahane@gmail.com> * chore(deps): Rm redundant dep (op-rs/op-reth#260) Ref op-rs/op-reth#205 Removes redundant dep `reth-db-api` from `reth-optimism-trie`, since used types are re-exported via `reth-db` --------- Co-authored-by: Arun Dhyani <dhyaniarun7@gmail.com> * refactor: replace tuple with `BlockNumHash` (op-rs/op-reth#254) Resolves op-rs/op-reth#239 --------- Co-authored-by: Einar Rasmussen <einar@oplabs.co> * feat: Return ref to B256 (op-rs/op-reth#272) Fixes op-rs/op-reth#270 * chore(exex): Add missing crate header to `reth-optimism-exex` (op-rs/op-reth#271) Resolves op-rs/op-reth#243 --------- Co-authored-by: Einar Rasmussen <einar@oplabs.co> Co-authored-by: Emilia Hane <elsaemiliaevahane@gmail.com> * feat: implemented hashed cursors (op-rs/op-reth#268) Closes op-rs/op-reth#213 --------- Co-authored-by: Emilia Hane <elsaemiliaevahane@gmail.com> * feat: add support for `eth_getProof` (op-rs/op-reth#257) Adds support for `eth_getProof` RPC method. This required reworking the launch command to still work with an alternative DB provider. Closes op-rs/op-reth#173 --------- Co-authored-by: Emilia Hane <elsaemiliaevahane@gmail.com> * feat: implement proof collection e2e tests (op-rs/op-reth#231) Implements end to end tests for proof collection. These tests create a blockchain, run the backfill job, then run the live collector. The live collector executes on top of the backfilled state and it already checks that the state root matches, so the tests just ensure that it runs without errors. Fixes op-rs/op-reth#170 --------- Co-authored-by: Emilia Hane <elsaemiliaevahane@gmail.com> * perf(trie): feature gate `reth-optimism-trie` metrics (op-rs/op-reth#282) Closes op-rs/op-reth#281 - Feature gates `reth-optimism-trie` metrics - Moves cursor impls out of proofs module into new module `cursor` - Moves cursor factory impls into new module `cursor_factory` - Updates cursor factory impls to return cursor types with metrics wrapper if metrics feature is enabled * chore(docs): Add missing doclinks (op-rs/op-reth#277) Adds doc links to all types to make sure they are caught by lint if names of types change in the future * fix(docs): Fixes docs of `reth_optimism_trie::cursor_factory` (op-rs/op-reth#286) Fixes module docs of `reth_optimism_trie::cursor_factory`, which weren't updated on moving the code out of the proofs module * fix(test): Enable live collector tests with metrics feature (op-rs/op-reth#291) Closes op-rs/op-reth#283 Enable live collector tests when metrics feature is enabled * fix(docs): Fixes doc links (op-rs/op-reth#292) Fixes some doc links from op-rs/op-reth#277 Several places should have linked to `OpProofsStore` which linked to `OpProofsStorage` * feat: Track eth_getProof metrics (op-rs/op-reth#285) Adds latency, total requests, successful responses, and error responses metrics to the eth_getProof RPC method. closes op-rs/op-reth#226 Co-authored-by: Emilia Hane <elsaemiliaevahane@gmail.com> * chore: devnet setup (op-rs/op-reth#295) Closes op-rs/op-reth#200 * feat: implement `debug_executePayload` (op-rs/op-reth#276) Closes op-rs/op-reth#189 --------- Co-authored-by: Emilia Hane <elsaemiliaevahane@gmail.com> * fix: storage trie cursor (op-rs/op-reth#301) Closes op-rs/op-reth#299 * feat: latest block updates in `store_trie_updates` (op-rs/op-reth#304) Closes op-rs/op-reth#302 * feat: live collector integration (op-rs/op-reth#306) Closes op-rs/op-reth#296 --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Emilia Hane <elsaemiliaevahane@gmail.com> * feat: add mdbx to storage tests (op-rs/op-reth#308) Fixes a small issue where storage slots with a zero value weren't skipped. I implemented this above the BlockVersionedCursor since it only applies to the storage cursor. Also enables MDBX in the common storage tests and ensures that no storage tests insert block numbers out of order or insert duplicate keys. * fix: merge conflict caused by storage tests (op-rs/op-reth#313) Fix merge conflicts caused by storage API change. * feat: create index table to efficiently prune history tables (op-rs/op-reth#280) Closes op-rs/op-reth#219 --------- Co-authored-by: Emilia Hane <elsaemiliaevahane@gmail.com> Co-authored-by: Arun Dhyani <dhyaniarun7@gmail.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Julian Meyer <julianmeyer2000@gmail.com> * chore(trie): Move `OpProofsStorageError` to own module (op-rs/op-reth#317) Ref op-rs/op-reth#316 Moves `OpProofsStorageError` into own module for readability Co-authored-by: Arun Dhyani <dhyaniarun7@gmail.com> * feat: Named fields for OpProofsStorageError variants (op-rs/op-reth#319) Fixes op-rs/op-reth#315 --------- Co-authored-by: Emilia Hane <emiliaha95@gmail.com> Co-authored-by: Arun Dhyani <dhyaniarun7@gmail.com> * feat: implement `debug_executionWitness` (op-rs/op-reth#312) Add support for `debug_executionWitness` API. This hasn't been tested, but I'll be able to test it soon once the backfill job completes. This code is mostly copied from here and adapted to work with our state provider: https://github.com/op-rs/op-reth/blob/1470a9cc77af798c999da05d9541a7025e5d4924/crates/rpc/rpc/src/debug.rs#L643-L695 Fixes op-rs/op-reth#190 * chore(trie): Rename metrics scope (op-rs/op-reth#322) Ref op-rs/op-reth#321 Renames scope for `BlockMetrics` and `OperationMetrics` from `external_proofs` to `optimism_trie` * feat: implemented wiped storage handling on `store_trie_updates` (op-rs/op-reth#300) Closes op-rs/op-reth#259 --------- Co-authored-by: Arun Dhyani <dhyaniarun7@gmail.com> * chore(trie): Rename `OpProofsStorageError` variant (op-rs/op-reth#324) Ref op-rs/op-reth#315 Renames `OpProofsStorageError::BlockUpdateFailed` to `OpProofsStorageError::MissingParentBlock` which better matches the error message Co-authored-by: Arun Dhyani <dhyaniarun7@gmail.com> * fix(db): `is_parent check` in `store_trie_update` (op-rs/op-reth#309) Fixes op-rs/op-reth#305 * feat: fetch trie updates (op-rs/op-reth#327) Closes: op-rs/op-reth#256 --------- Co-authored-by: itschaindev <jagrutk@protonmail.com> Co-authored-by: Emilia Hane <elsaemiliaevahane@gmail.com> Co-authored-by: jagroot <4516586+itschaindev@users.noreply.github.com> * feat: implemented mdbx `replace_updates` (op-rs/op-reth#329) Closes op-rs/op-reth#193 Closes op-rs/op-reth#216 * feat: apply `BlockStateDiff` to the `prune_earliest_state` method (op-rs/op-reth#332) Closes op-rs/op-reth#217 Also fixed a bug where the `set_earliest_block_number_hash` was using `append` instead of `upsert`. --------- Co-authored-by: Arun Dhyani <dhyaniarun7@gmail.com> * chore: refactor storage trie cursor (op-rs/op-reth#338) Closes op-rs/op-reth#337 * feat: integrate metrics into live sync process (op-rs/op-reth#330) Fixes op-rs/op-reth#246 - Return `UpdateResult` from `store_trie_updates` with counts of items stored. - Log counts and latencies for live collector Signed-off-by: Julian Meyer <julian.meyer@coinbase.com> * test: basic storage proof (op-rs/op-reth#336) Part 1 for op-rs/op-reth#289 * test: `eth_getProof` for account proof validation (op-rs/op-reth#341) Closes op-rs/op-reth#288 --------- Co-authored-by: Arun Dhyani <dhyaniarun7@gmail.com> * test: `debug_executePayload` RPC (op-rs/op-reth#340) Closes op-rs/op-reth#325 * test: multi slot and zero testing (op-rs/op-reth#343) Closes op-rs/op-reth#289 * test: resyncing (op-rs/op-reth#350) Closes op-rs/op-reth#349 --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * test: e2e for `debug_executionWitness` (op-rs/op-reth#348) Closes op-rs/op-reth#326 Fixed the `nonce too low` issue with the way `execution_witness` was implemented for op. * test: doc test eg for exex setup (op-rs/op-reth#352) Closes op-rs/op-reth#342 * chore: enable remaining mdbx tests (op-rs/op-reth#354) Fixes op-rs/op-reth#252 * fix: ensure db tx is closed after backfill (op-rs/op-reth#357) The db tx isn't dropped ever for the backfill job, so create a temp scope to ensure it's dropped. * refactor: `MdbxProofStorage` (op-rs/op-reth#351) Closes op-rs/op-reth#335 --------- Co-authored-by: Arun Dhyani <dhyaniarun7@gmail.com> * fix: rebase conflicts (op-rs/op-reth#367) Fix conflicts rebasing onto latest upstream main * feat: handle reorg (op-rs/op-reth#356) Closes op-rs/op-reth#191 --------- * feat: add target debug logs to reth_optimism_exex (op-rs/op-reth#368) Fixes op-rs/op-reth#366 * test: basic reorg (op-rs/op-reth#369) Part of op-rs/op-reth#174 * fix: missing block after reorg (op-rs/op-reth#382) Closes op-rs/op-reth#379 * feat: add sync status RPC (op-rs/op-reth#353) Adds `debug_proofsSyncStatus` for fetching the proof window when enabled. Also adds metrics for the earliest and latest block numbers available. * feat: replace variant OpProofsStorageError::Other (op-rs/op-reth#355) Fixes op-rs/op-reth#316 I also noticed a few `DatabaseError` that were being mapped to `OpProofsStorageError::Other` and changed them to point to `OpProofsStorageError::DatabaseError` --------- Co-authored-by: Arun Dhyani <dhyaniarun7@gmail.com> * feat: Implemented `OpProofStoragePruner` (op-rs/op-reth#360) Co-authored-by: Emilia Hane <elsaemiliaevahane@gmail.com> * feat: handle `ChainReverted` notification (op-rs/op-reth#380) Closes op-rs/op-reth#192 --------- Co-authored-by: Arun Dhyani <dhyaniarun7@gmail.com> * chore: small lint (op-rs/op-reth#390) Find a small typo here. * test: reorg test scenarios (op-rs/op-reth#384) Closes op-rs/op-reth#174 * test: add new devnet for op-reth as sequencer (op-rs/op-reth#391) Closes op-rs/op-reth#389 * chore(trie): Simplify cursor interface (op-rs/op-reth#388) Ref op-rs/op-reth#371 Removes redundant `OpProofsTrieCursorRO` and `OpProofsHashedCursorRO` traits by enabling a `DatabaseError` variant that wraps an error trait object. This way we can have a bijective relation between `DatabaseError` and `OpProofsStorageError` in a type safe way, i.e. conversion both ways without matching on string. cc @dhyaniarun1993 * feat: add panels for `eth_getProof` in grafana dashboard (op-rs/op-reth#392) Closes op-rs/op-reth#318 <img width="2208" height="672" alt="Screenshot 2025-11-21 at 15 35 16" src="https://github.com/user-attachments/assets/62ba3b36-fd4f-498a-9bbc-7e4f33c9e3d2" /> The OP-Reth dashboard is not loaded by default when running the kurtosis devnet for e2e historical proofs. Once op-rs/op-reth#391 is merged, I'll update the devnet config to load it by default. * fix: add custom TrieType to separate metrics for op proofs storage (op-rs/op-reth#397) Closes op-rs/op-reth#321 * feat: implement methods for new traits for `inmemory` cursor (op-rs/op-reth#396) Closes op-rs/op-reth#400 * fix: use loop instead of recursing in hashed next (op-rs/op-reth#402) Fixes a potential DoS vector of overflowing the stack by writing zero values to storage. Resolves op-rs/op-reth#401, op-rs/op-reth#394 * fix: store block updates metric fixed (op-rs/op-reth#409) Closes op-rs/op-reth#410 * feat: implemented `OpProofStorage` Database metrics (op-rs/op-reth#407) Closes op-rs/op-reth#224 Closes op-rs/op-reth#387 --------- Co-authored-by: Arun Dhyani <dhyaniarun7@gmail.com> * feat: add `initialize-op-proofs` command (op-rs/op-reth#377) Adds `initialize-op-proofs` which is now required before running the node with the proof ExEx. Fixes op-rs/op-reth#376 --------- Co-authored-by: Emilia Hane <elsaemiliaevahane@gmail.com> Co-authored-by: Arun Dhyani <dhyaniarun7@gmail.com> * feat: integrate `initialize-op-proofs` to devnet (op-rs/op-reth#412) Closes op-rs/op-reth#411 --------- Co-authored-by: Julian Meyer <julian.meyer@coinbase.com> Co-authored-by: Emilia Hane <elsaemiliaevahane@gmail.com> * feat: dashboard for proof history related metrics (op-rs/op-reth#414) Closes op-rs/op-reth#413 Closes op-rs/op-reth#405 The new dashboard looks like following: <img width="2199" height="1086" alt="Screenshot 2025-11-27 at 18 19 36" src="https://github.com/user-attachments/assets/8d8f4f32-f1c3-4704-8ee4-ed67921ae039" /> <img width="2199" height="959" alt="Screenshot 2025-11-27 at 18 20 04" src="https://github.com/user-attachments/assets/33714ecc-5853-4a19-a519-4a35efff5cd9" /> --------- Co-authored-by: Arun Dhyani <dhyaniarun7@gmail.com> * fix: update metric on startup (op-rs/op-reth#419) Closes op-rs/op-reth#416 Issue: The issue was that CLI command `initialize-op-proofs` set the earliest block number but when the node starts, the metrics initialized are at default values. All other metrics get called soon after the start but not the earliest block number. Solution: Set the metric on startup, if available. * chore: code coverage for historical proof exex (op-rs/op-reth#415) Closes op-rs/op-reth#406 --------- Co-authored-by: Emilia Hane <elsaemiliaevahane@gmail.com> * chore: Add test for Display impl for PrunerOutput (op-rs/op-reth#435) Fixes op-rs/op-reth#432 * feat: add exex sync status in grafana (op-rs/op-reth#429) Closes op-rs/op-reth#426 <img width="1661" height="422" alt="Screenshot 2025-12-03 at 10 59 58" src="https://github.com/user-attachments/assets/91aeb374-a40a-45db-9c38-f9f36215747e" /> * feat: Consume TrieUpdates and HashedState from notification (op-rs/op-reth#439) ### Description This PR consumes trie updates and hashed state directly from notifications, falling back to full block execution when notification data is missing. ### Current limitations: There are known serialization/deserialization issues with the WAL. Before starting the node, ensure the WAL is cleaned. The Proof History ExEx is designed to tolerate missing notification data, but if you are running any additional ExEx components, ensure they can also safely handle gaps in notification data. Due to the above limitation, the resyncing test has been disabled. Closes op-rs/op-reth#420 Based on op-rs/op-reth#425 --------- Co-authored-by: Brian Picciano <me@mediocregopher.com> Co-authored-by: Emilia Hane <elsaemiliaevahane@gmail.com> * fix: fall back to block execution if state updates missing from notification (op-rs/op-reth#455) Closes op-rs/op-reth#453 * feat: Implemented `OpProofStoragePrunerTask` (op-rs/op-reth#375) Closes op-rs/op-reth#361 Closes op-rs/op-reth#395 --------- Co-authored-by: Emilia Hane <elsaemiliaevahane@gmail.com> * feat: implemented pruner metrics (op-rs/op-reth#454) Closes op-rs/op-reth#363 * fix: new earliest block after pruning (op-rs/op-reth#470) Fixes: op-rs/op-reth#469 * chore(exex): Add metrics feature in `reth-optimism-exex` (op-rs/op-reth#438) Closes op-rs/op-reth#427 --------- Co-authored-by: itschaindev <jagrutk@protonmail.com> * fix(trie): Add missing `ValueWithSubkey` impl for `VersionedValue<T>` (op-rs/op-reth#475) Closes op-rs/op-reth#474 * chore: add prune e2e test back (op-rs/op-reth#477) Co-authored-by: Emilia Hane <elsaemiliaevahane@gmail.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * chore: sysgo integration (op-rs/op-reth#478) Closes op-rs/op-reth#297 --------- Co-authored-by: Emilia Hane <elsaemiliaevahane@gmail.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * chore: enable code coverage for e2e (op-rs/op-reth#485) Closes op-rs/op-reth#442 --------- Co-authored-by: Emilia Hane <elsaemiliaevahane@gmail.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * chore(chain-state): Receive sorted updates to exex (op-rs/op-reth#464) Closes op-rs/op-reth#460 Makes use of the sorted updates which are now already available in the `ExecutedBlock`, instead of sorting out removed accounts/slots later in `reth-optimism-trie` --------- Co-authored-by: Arun Dhyani <dhyaniarun7@gmail.com> * feat(trie): Cover error branches of `execute_and_store_block_updates` (op-rs/op-reth#437) Fixes op-rs/op-reth#436 * feat: unwind command added (op-rs/op-reth#499) Closes op-rs/op-reth#451 --------- Co-authored-by: Emilia Hane <elsaemiliaevahane@gmail.com> * chore(test): use `reth_optimism_chainspec::BASE_SEPOLIA` in tests (op-rs/op-reth#505) Update to use `reth_optimism_chainspec::BASE_SEPOLIA` in engine tests, now that the jovian timestamp is known * feat: prune cli added (op-rs/op-reth#507) Closes op-rs/op-reth#452 * test: add unit test for OpProofsStateProviderRef Debug impl (op-rs/op-reth#493) Adds unit test for the `Debug` implementation of `OpProofsStateProviderRef`. Closes op-rs/op-reth#434 --------- Co-authored-by: Emilia Hane <emiliaha95@gmail.com> * test: extended prune e2e with `getProof` consistency (op-rs/op-reth#514) Closes op-rs/op-reth#461 * feat: used `OpProofsStorageWithMetrics` provider on pruner (op-rs/op-reth#498) Closes op-rs/op-reth#472 * feat: pruning progress and metrics improvements (op-rs/op-reth#504) Closes op-rs/op-reth#362 --------- Co-authored-by: Arun Dhyani <dhyaniarun7@gmail.com> * feat: implement prune batching (op-rs/op-reth#530) Closes op-rs/op-reth#364 * feat: address lookup table (op-rs/op-reth#509) Closes op-rs/op-reth#486 * feat: backfill address map (op-rs/op-reth#512) Closes op-rs/op-reth#510 * fix(trie): Fix broken `Debug` impl unit test (op-rs/op-reth#539) Fix broken unit test for verifying `Debug` impl, add missing map `address_mappings` to expected test output, introduced in op-rs/op-reth#512 * refactor(trie): return `OpProofsStorageError` from `store_block_updates` (op-rs/op-reth#537) Closes op-rs/op-reth#524 Co-authored-by: Emilia Hane <elsaemiliaevahane@gmail.com> * feat: metrics panel for debug api ext (op-rs/op-reth#533) Closes op-rs/op-reth#515 Closes op-rs/op-reth#516 <img width="1607" height="277" alt="Screenshot 2025-12-18 at 20 44 35" src="https://github.com/user-attachments/assets/d135ff3c-c837-4dc8-a097-b9f818a08a35" /> --------- Co-authored-by: Emilia Hane <elsaemiliaevahane@gmail.com> * refactor(trie): return `OpProofsStorageError` from `execute_and_store_block_updates` (op-rs/op-reth#535) Closes op-rs/op-reth#523 --------- Co-authored-by: Emilia Hane <elsaemiliaevahane@gmail.com> * fix: initialize-op-proofs needs chain parameter (op-rs/op-reth#544) * fix: support Send in test_op_proofs_state_provider_ref_debug (op-rs/op-reth#546) * chore(trie): return `OpProofsStorageError` from `unwind_history` (op-rs/op-reth#548) Cherry picks commit from op-rs/op-reth#540 Co-authored-by: Himess <semihcvlk53@gmail.com> * refactor(trie): return OpProofsStorageError from unwind_and_store_block_updates (op-rs/op-reth#541) Closes op-rs/op-reth#525 Co-authored-by: Emilia Hane <elsaemiliaevahane@gmail.com> * feat: integrated storing address mapping (op-rs/op-reth#534) Closes op-rs/op-reth#487 * chore: refactor op-proofs cli (op-rs/op-reth#554) Closes op-rs/op-reth#511 * feat: added startup safety check for massive pruning operation (op-rs/op-reth#556) Closes op-rs/op-reth#555 --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * chore: fix reorg e2e (op-rs/op-reth#558) Attempts to fix op-rs/op-reth#528 --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * chore: improve `append_or_delete_dup_sorted` (op-rs/op-reth#552) Closes op-rs/op-reth#501 * chore(trie): replace `eyre` with `OpProofsStorageError` (op-rs/op-reth#564) Closes op-rs/op-reth#527 * chore: revert hybrid storage experiment (op-rs/op-reth#568) Revert all the changes made as part of the hybrid storage experiment. * refactor(trie): extract save_hashed_accounts closure to method1 (op-rs/op-reth#570) Closes op-rs/op-reth#566 (1/5) Co-authored-by: Arun Dhyani <dhyaniarun7@gmail.com> * feat: optimized history deletions (op-rs/op-reth#565) Closes op-rs/op-reth#311 Performace comparison under load test: ``` previous: 4.873750666s sorted delete: 3.278444083s sorted delete + seperate read (current as per the PR): 2.980065583s ``` * ci: restrict codecov workflows to merge_group and workflow_dispatch (op-rs/op-reth#513) ## Summary - Remove `push` and `pull_request` triggers from coverage workflows - Add `merge_group` trigger to run codecov only in merge queue - Keep `workflow_dispatch` for manual triggering when needed This reduces CI resource usage by running codecov only when necessary. Closes op-rs/op-reth#500 * feat: optimised initial state update for pruning (op-rs/op-reth#563) Closes op-rs/op-reth#562 Validated the optimization by rewriting an initial state with N=50,000 updated entries. - Throughput: Increased by ~20.9% (326k → 394k entries/sec). - Latency: Reduced by ~17.2% (153ms → 126ms). ``` Old approach: Rewrite Duration: 153.26ms Throughput: 326,248 entries/sec New approach: Rewrite Duration: 126.77ms Throughput: 394,410 entries/sec ``` bench: op-rs/op-reth@274e544 * docs: OpProofStorage DB Schema (op-rs/op-reth#583) Closes op-rs/op-reth#240 * chore: push devstack support for op-reth upstream (op-rs/op-reth#585) Part of op-rs/op-reth#483. First wait for theo's PR [op-rs/op-reth#18754](#18754) to be merged. Then wait for our PR [op-rs/op-reth#18772](#18772) to be merged. After that, in this repo, remove the current submodule and replace it with optimism itself. * feat: add verification interval for integrity check (op-rs/op-reth#577) Closes op-rs/op-reth#449 The approach used is to perform full block verification after every N blocks to ensure the state is still correct. --------- Co-authored-by: Arun Dhyani <dhyaniarun7@gmail.com> * chore: backfill refactor (op-rs/op-reth#578) Closes op-rs/op-reth#566 --------- Co-authored-by: Himess <95512809+Himess@users.noreply.github.com> Co-authored-by: Himess <semihcvlk53@gmail.com> * chore: update optimism (op-rs/op-reth#590) * chore(tire): Improve test coverage for `OpProofsStorageError` (op-rs/op-reth#589) Ref op-rs/op-reth#588 Adds tests for conversions to `OpProofsStorageError` * fix: db_stat after upstream pull (op-rs/op-reth#601) * test: unit test for exex (op-rs/op-reth#599) Closes op-rs/op-reth#458 * fix: `MdbxStorageCursor` non first key issue (op-rs/op-reth#602) Closes op-rs/op-reth#600 --------- Co-authored-by: Arun Dhyani <dhyaniarun7@gmail.com> Co-authored-by: Sadiqur Rahman <sadiqurr8@gmail.com> * chore: added exex missing tests (op-rs/op-reth#605) Closes op-rs/op-reth#588 * chore: getProof benchmark utility added (op-rs/op-reth#550) Utility for op-rs/op-reth#446 * feat: refactor and optimise pruning (op-rs/op-reth#587) Closes op-rs/op-reth#567 This PR improves the performance of the historical state pruning logic. By refining how state updates are handled and written during pruning, we observe a significant reduction in write latency. Performance Benchmarks: Comparison run using [load_test](https://github.com/op-rs/op-reth/blob/8f0b61039a1af5f07793d5ea38e21e0439b7f0b3/crates/optimism/trie/src/db/store.rs#L3462) Metric | Current (Baseline) | Optimised (This PR) | Improvement -- | -- | -- | -- Write Time | ~107ms | ~50ms | ~2.1x Faster Total Duration | ~129ms | ~81ms | ~1.6x Faster * fix: prune redundant tombstones at history horizon (op-rs/op-reth#620) Closes op-rs/op-reth#619 * feat: backfill job restart-safe and idempotent (op-rs/op-reth#594) Closes op-rs/op-reth#244 * chore(trie): Use `reth-trie-common` to enable `serde-bincode-compat` in v1.10.1 (op-rs/op-reth#617) Ref op-rs/op-reth#611 Uses `reth-trie-common` dep explicitly in `reth-optimism-trie` to make sure `serde-bincode-compat` can be enabled while using reth release v1.10.1 * feat: backfill renamed to initialize (op-rs/op-reth#622) Closes op-rs/op-reth#606 * feat: refactored store (op-rs/op-reth#623) Closes op-rs/op-reth#626 * chore: mv proof args to rollup node (op-rs/op-reth#625) Closes op-rs/op-reth#613 * chore(trie): Revert redundant diff (op-rs/op-reth#610) Ref op-rs/op-reth#611 Uses getters instead of direct field access for trie updates to remove redundant diff with upstream * chore: use reth versioned deps (op-rs/op-reth#627) Closes op-rs/op-reth#621 This PR updates the workspace configuration to use reth as a git dependency instead of a local crate, including necessary fixes for the compiler, CI workflows, and test configurations. --------- Co-authored-by: Emilia Hane <elsaemiliaevahane@gmail.com> * chore(trie): Fix lint (op-rs/op-reth#636) Lint fix. Clippy is catching since yesterday's new nightly release. * fix: removed unused pruner metrics (op-rs/op-reth#638) Closes op-rs/op-reth#607 * chore(trie): Remove redundant generic lifetime for `InitializationJob` (op-rs/op-reth#632) Ref op-rs/op-reth#631 - Changes nested database tx type to owned type as has no need to be ref - Uses `AsyncFnOnce`(new in Rust 1.85) instead of generic future return type * chore(trie): Make `initialize` a method of `InitializationJob` (op-rs/op-reth#635) Ref op-rs/op-reth#631 - Removes `safe_fn` closure as arg from `initialize` by making `initialize` a method of `InitializationJob` - Moves save functions out of `InitializationJob` into impls of trait method of new trait `InitTable::save_entries` - Uses `size_hint` to avoid cost of heap re-allocations - Removes redundant clone * refactor(trie): use parking_lot::RwLock in InMemoryProofsStorage (op-rs/op-reth#637) Closes op-rs/op-reth#633 Co-authored-by: Emilia Hane <elsaemiliaevahane@gmail.com> * feat: async notification processing (op-rs/op-reth#614) This PR improves the robustness of the OP-Reth Proofs ExEx by process notifications asynchronously(to avoid WAL backpressure) when the gap between proof storage and the tip is big. This change ensures that notification pipeline is able to keep up and gracefully handle the gap. Closes op-rs/op-reth#597 --------- Co-authored-by: Emilia Hane <elsaemiliaevahane@gmail.com> * chore: simplify OpProofsStore (op-rs/op-reth#639) Closes: op-rs/op-reth#634 Blocked on: op-rs/op-reth#637 --------- Co-authored-by: Himess <semihcvlk53@gmail.com> * chore: ExEx config builder (op-rs/op-reth#642) Closes op-rs/op-reth#641 --------- Co-authored-by: Himess <semihcvlk53@gmail.com> * chore(trie): Fix redundant type annotations (op-rs/op-reth#645) Fixes lint `clippy::redundant_type_annotations` * chore(trie): Replace redundant clone of entries in `InitializationJob` (op-rs/op-reth#644) Closes op-rs/op-reth#631 - Puts off assigning variables until right when needed - Optimizes away redundant clone of entries on batch insertion - Updates symbol name to better show initialising is a batching algorithm * chore: moved proof initialization to `reth-optimism-node` (op-rs/op-reth#640) Closes op-rs/op-reth#612 --------- Co-authored-by: Arun Dhyani <dhyaniarun7@gmail.com> * chore: separate initial state store trait (op-rs/op-reth#647) Closes op-rs/op-reth#643 * chore(test): support reth as in sequencer and validator (op-rs/op-reth#650) This PR added support to run op-reth in sequencer and op-reth(with proof) in validator for the e2e test. Note: In this setup pruning test isn't supported(at this moment) since this test is falling outside the proof window of reth. --------- Co-authored-by: Emilia Hane <elsaemiliaevahane@gmail.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * chore: fix flaky e2e test (op-rs/op-reth#653) Closes op-rs/op-reth#528 --------- Co-authored-by: Emilia Hane <elsaemiliaevahane@gmail.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> * chore(test): dep dump (op-rs/op-reth#654) Co-authored-by: Sadiqur Rahman <sadiqurr8@gmail.com> Co-authored-by: Emilia Hane <elsaemiliaevahane@gmail.com> Co-authored-by: Himess <95512809+Himess@users.noreply.github.com> Co-authored-by: Himess <semihcvlk53@gmail.com> Co-authored-by: jagroot <4516586+itschaindev@users.noreply.github.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Julian Meyer <julianmeyer2000@gmail.com> * chore(op-reth/historical-proofs): fix compilation errors --------- Signed-off-by: Julian Meyer <julian.meyer@coinbase.com> Co-authored-by: Georgios Konstantopoulos <me@gakonst.com> Co-authored-by: Amp <amp@ampcode.com> Co-authored-by: DaniPopes <57450786+DaniPopes@users.noreply.github.com> Co-authored-by: Xzavier <xingqiang.yuan@okg.com> Co-authored-by: Emilia Hane <elsaemiliaevahane@gmail.com> Co-authored-by: Arun Dhyani <dhyaniarun7@gmail.com> Co-authored-by: Julian Meyer <julianmeyer2000@gmail.com> Co-authored-by: jagroot <4516586+itschaindev@users.noreply.github.com> Co-authored-by: Varun Doshi <doshivarun202@gmail.com> Co-authored-by: Sadiqur Rahman <sadiqurr8@gmail.com> Co-authored-by: einar-oplabs <oplabs@einar.io> Co-authored-by: Einar Rasmussen <einar@oplabs.co> Co-authored-by: op-will <232669456+op-will@users.noreply.github.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Emilia Hane <emiliaha95@gmail.com> Co-authored-by: itschaindev <jagrutk@protonmail.com> Co-authored-by: Jonas Warlike <mikmillarog@protonmail.com> Co-authored-by: Julian Meyer <julian.meyer@coinbase.com> Co-authored-by: Brian Picciano <me@mediocregopher.com> Co-authored-by: Himess <95512809+Himess@users.noreply.github.com> Co-authored-by: Himess <semihcvlk53@gmail.com> Co-authored-by: Sebastian Stammler <seb@oplabs.co>

Closes op-rs/op-reth#442 --------- Co-authored-by: Emilia Hane <elsaemiliaevahane@gmail.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

emhane and others added 20 commits December 10, 2025 16:44

Impl ValueWithSubKey for VersionedValue

605613f

Revert changes to runners

707d022

Merge branch 'emhane/revert-ci-runners' into emhane/fix-missing-subke…

7999191

…y-impl

Merge branch 'unstable' into emhane/fix-missing-subkey-impl

cb80fe9

chore: add prune e2e back

f3e5a9b

chore: sysgo integration

c73704c

fix code

fc1c8bd

Merge remote-tracking branch 'origin/emhane/fix-missing-subkey-impl' …

986c6f9

…into arun/chore/add-prune-e2e

Merge branch 'arun/chore/add-prune-e2e' into arun/chore/sysgo-integra…

a076494

…tion

arun/chore/add-prune-e2e Merged

b365ba9

fixes

95b0c0f

added missing changes

1fdd712

arun/chore/add-prune-e2e Merged

b2565e3

update go version

c76554c

Update crates/optimism/tests/proofs/prune/prune_test.go

8c7d628

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Apply suggestions from code review

87f01b9

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

unzip artifact before sysgo test

421ee8a

arun/chore/add-prune-e2e Merged

5e6ced8

unstable merged + conflict resolved

f56c993

chore: enable code coverage for e2e

6e8937d

dhyaniarun1993 self-assigned this Dec 12, 2025

dhyaniarun1993 added K-observability Kind: observability A-ci Area: CI K-test Kind: test W-historical-proofs Workstream: historical-proofs labels Dec 12, 2025

dhyaniarun1993 added 2 commits December 12, 2025 11:11

install-action@cargo-llvm-cov added

c9ad73a

fix build

46e601d

dhyaniarun1993 marked this pull request as ready for review December 12, 2025 07:11

dhyaniarun1993 requested a review from itschaindev as a code owner December 12, 2025 07:11

Base automatically changed from arun/chore/sysgo-integration to unstable December 12, 2025 12:54

dhyaniarun1993 requested review from BioMark3r and theochap as code owners December 12, 2025 12:54

unstable merged

af420e4

Copilot AI review requested due to automatic review settings December 12, 2025 13:44

Copilot started reviewing on behalf of dhyaniarun1993 December 12, 2025 13:45 View session

Copilot AI reviewed Dec 12, 2025

View reviewed changes

.github/workflows/e2e-op-historical-proof.yml Show resolved Hide resolved

.github/workflows/e2e-op-historical-proof.yml Show resolved Hide resolved

dhyaniarun1993 added 2 commits December 12, 2025 19:22

update build

843c0b6

revert build changes

19be4b6

emhane reviewed Dec 12, 2025

View reviewed changes

sadiq1971 approved these changes Dec 15, 2025

View reviewed changes

dhyaniarun1993 added this pull request to the merge queue Dec 15, 2025

Merged via the queue into unstable with commit 28f6b8d Dec 15, 2025
55 of 56 checks passed

dhyaniarun1993 deleted the arun/chore/enable-codecov branch December 15, 2025 12:12

emhane added a commit that referenced this pull request Dec 18, 2025

chore: enable code coverage for e2e (#485)

df3e0cf

Closes #442 --------- Co-authored-by: Emilia Hane <elsaemiliaevahane@gmail.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

emhane added a commit that referenced this pull request Dec 18, 2025

chore: enable code coverage for e2e (#485)

85987ee

Closes #442 --------- Co-authored-by: Emilia Hane <elsaemiliaevahane@gmail.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

emhane added a commit that referenced this pull request Jan 5, 2026

chore: enable code coverage for e2e (#485)

d020399

Closes #442 --------- Co-authored-by: Emilia Hane <elsaemiliaevahane@gmail.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

emhane added a commit that referenced this pull request Jan 13, 2026

chore: enable code coverage for e2e (#485)

949a305

Closes #442 --------- Co-authored-by: Emilia Hane <elsaemiliaevahane@gmail.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

emhane added a commit that referenced this pull request Jan 15, 2026

chore: enable code coverage for e2e (#485)

3cade9e

Closes #442 --------- Co-authored-by: Emilia Hane <elsaemiliaevahane@gmail.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

emhane added a commit that referenced this pull request Jan 20, 2026

chore: enable code coverage for e2e (#485)

633a9e2

Closes #442 --------- Co-authored-by: Emilia Hane <elsaemiliaevahane@gmail.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

emhane added a commit that referenced this pull request Jan 26, 2026

chore: enable code coverage for e2e (#485)

3b7d80e

Closes #442 --------- Co-authored-by: Emilia Hane <elsaemiliaevahane@gmail.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

emhane added a commit that referenced this pull request Jan 29, 2026

chore: enable code coverage for e2e (#485)

ef9c611

Closes #442 --------- Co-authored-by: Emilia Hane <elsaemiliaevahane@gmail.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Comments

chore: enable code coverage for e2e#485

chore: enable code coverage for e2e#485
dhyaniarun1993 merged 25 commits intounstablefrom
arun/chore/enable-codecov

dhyaniarun1993 commented Dec 12, 2025

Uh oh!

codecov bot commented Dec 12, 2025 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

emhane Dec 12, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

	# Meant for testing - all optimizations, but with debug assertions and overflow checks.
	[profile.hivetests]
	inherits = "test"
	opt-level = 3
	lto = "thin"

Comments

Conversation

dhyaniarun1993 commented Dec 12, 2025

Uh oh!

codecov bot commented Dec 12, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

emhane Dec 12, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

codecov bot commented Dec 12, 2025 •

edited

Loading