feat(storage): read headers and transactions only from static files#18788
Merged
feat(storage): read headers and transactions only from static files#18788
Conversation
shekhirin
force-pushed
the
alexey/static-file-only-read
branch
5 times, most recently
from
6ae5738 to
4e33cd6
Compare
shekhirin
force-pushed
the
alexey/static-file-only-read
branch
from
4e33cd6 to
ba3f65d
Compare
shekhirin
force-pushed
the
alexey/static-file-only-read
branch
3 times, most recently
from
68f2450 to
36d3a7c
Compare
shekhirin
force-pushed
the
alexey/static-file-only-read
branch
from
36d3a7c to
2e509c9
Compare
shekhirin
force-pushed
the
alexey/static-file-only-read
branch
from
b80665d to
de869c2
Compare
shekhirin
force-pushed
the
alexey/static-file-only-read
branch
from
de869c2 to
35c4984
Compare
shekhirin
requested review from
Rjected,
RomanHodulak,
fgimenez,
gakonst,
mattsse,
mediocregopher,
rakita,
rkrasiuk and
yongkangc
as code owners
yongkangc
reviewed
Oct 3, 2025
yongkangc
reviewed
Oct 3, 2025
Comment on lines
-79
to
-80
| "reth-db-api", | ||
| "reth-db/test-utils", |
Member
There was a problem hiding this comment.
curious why do we no longer need this?
Member
Author
There was a problem hiding this comment.
checking static files instead of the database now
yongkangc
approved these changes
Oct 3, 2025
| }, | ||
| |_| true, | ||
| ) | ||
| self.static_file_provider.transaction_hashes_by_range(tx_range) |
Member
There was a problem hiding this comment.
with this we can basically do the same thing but without the db?
this seems great, curious why did we not do this before?
Member
Author
There was a problem hiding this comment.
no good reason tbh, we were writing to both locations for a while now...
klkvr
reviewed
Oct 3, 2025
shekhirin
force-pushed
the
alexey/static-file-only-read
branch
from
c1a44f3 to
1322bdd
Compare
shekhirin
force-pushed
the
alexey/static-file-only-read
branch
from
1322bdd to
39623de
Compare
klkvr
approved these changes
Oct 3, 2025
Rjected
approved these changes
Oct 3, 2025
Member
Rjected
left a comment
Rjected
left a comment
There was a problem hiding this comment.
lgtm, I just have one question
Comment on lines
-1143
to
-1148
| warn!( | ||
| target: "provider::static_file", | ||
| ?segment, | ||
| ?number, | ||
| "Could not find block or tx number on a range request" | ||
| ); |
Member
There was a problem hiding this comment.
why do we go from warning and returning an err, to returning an Ok(result) now?
Member
Author
There was a problem hiding this comment.
we never hit this error because we always query only what we know is inside static files
reth/crates/storage/provider/src/providers/static_file/manager.rs
Lines 1288 to 1294 in 3338eb7
but now we're querying methods such as self.static_file_provider.headers_range directly without checking the ranges first, so I mirror the behaviour of database range querying
CarlBeek
added a commit
to CarlBeek/reth
that referenced
this pull request
Oct 10, 2025
* main: (280 commits) refactor: remove needless collect() calls in trie tests (paradigmxyz#18937) chore(grafana): use precompile address as legend (paradigmxyz#18913) perf(tree): worker pooling for storage in multiproof generation (paradigmxyz#18887) feat: wait for new blocks when build is in progress (paradigmxyz#18831) chore: align node_config threshold constant (paradigmxyz#18914) docs: duplicate comment in Eip4844PoolTransactionError (paradigmxyz#18858) ci: cache hive simulator images to reduce prepare-hive job time (paradigmxyz#18899) refactor: replace collect().is_empty() with next().is_none() in tests (paradigmxyz#18902) feat(provider): add get_account_before_block to ChangesetReader (paradigmxyz#18898) refactor(engine): separate concerns in on_forkchoice_updated for better maintainability (paradigmxyz#18661) chore(node): simplify EngineApiExt bounds by removing redundant constraints (paradigmxyz#18905) fix(trie): Reveal extension child when extension is last remaining child of a branch (paradigmxyz#18891) chore: make clippy happy (paradigmxyz#18900) chore: relax `ChainSpec` impls (paradigmxyz#18894) refactor: eliminate redundant allocation in precompile cache example (paradigmxyz#18886) fix(era-utils): fix off-by-one for Excluded end bound in process_iter (paradigmxyz#18731) docs: yellowpaper sections in consensus implementation (paradigmxyz#18881) feat(storage): read headers and transactions only from static files (paradigmxyz#18788) feat: Use generic `HeaderTy` for `reth db get static-file headers` (paradigmxyz#18870) fix: streamline payload conversion in custom engine API (paradigmxyz#18864) ...
theochap
pushed a commit
to theochap/reth
that referenced
this pull request
Oct 13, 2025
mattsse
pushed a commit
that referenced
this pull request
Oct 14, 2025
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #18684
This PR makes all provider methods that read headers or transactions to read only from static files, and never make a lookup in the database. Receipts are not touched yet, as they still can be written to the database.
This doesn't break existing nodes, because on startup we always copy to static files
reth/crates/node/builder/src/launch/engine.rs
Lines 159 to 160 in aa192c2
Hive passes https://github.com/paradigmxyz/reth/actions/runs/18220958085