Skip to content

feat(storage): write headers and transactions only to static files#18681

Merged
shekhirin merged 23 commits intomainfrom
alexey/static-files-only
Sep 29, 2025
Merged

feat(storage): write headers and transactions only to static files#18681
shekhirin merged 23 commits intomainfrom
alexey/static-files-only

Conversation

@shekhirin
Copy link
Member

@shekhirin shekhirin commented Sep 24, 2025

Closes #18683
Closes #11524

Removes the StorageLocation enum that was used to tell the provider where to write the data. Headers and transactions now are always written to static files only.

This doesn't break existing nodes, because on startup we always copy to static files

// The new engine writes directly to static files. This ensures that they're up to the tip.
pipeline.move_to_static_files()?;


We need to figure out what to do with receipts pruning. We support receipts_log_filter option that can prune everything but certain receipts that match the log filter, and use it for NOT pruning the deposit contract receipts. This type of pruning can't be done with static files, as they don't support deletions in the middle of the file.

For now, I'm still always writing receipts to the database, and to static files only if pruning is not enabled.

@shekhirin shekhirin self-assigned this Sep 24, 2025
@shekhirin shekhirin added C-enhancement New feature or request A-static-files Related to static files labels Sep 24, 2025
@github-project-automation github-project-automation bot moved this to Backlog in Reth Tracker Sep 24, 2025
@shekhirin shekhirin force-pushed the alexey/static-files-only branch from 6f9f855 to 4ec8c3c Compare September 24, 2025 16:04
@codspeed-hq
Copy link

codspeed-hq bot commented Sep 24, 2025

CodSpeed Performance Report

Merging #18681 will not alter performance

Comparing alexey/static-files-only (61dcd11) with main (564e3a6)

Summary

✅ 77 untouched

@shekhirin shekhirin force-pushed the alexey/static-files-only branch from 444e05e to a3603f3 Compare September 24, 2025 16:49
@shekhirin shekhirin force-pushed the alexey/static-files-only branch from 4efd278 to e293209 Compare September 24, 2025 17:29
@shekhirin shekhirin force-pushed the alexey/static-files-only branch from a8c4ff3 to 85600aa Compare September 24, 2025 17:46
@shekhirin shekhirin force-pushed the alexey/static-files-only branch from 00cad91 to 773da64 Compare September 25, 2025 10:53
@shekhirin shekhirin force-pushed the alexey/static-files-only branch from 773da64 to 61e162c Compare September 25, 2025 11:09
@shekhirin shekhirin force-pushed the alexey/static-files-only branch from 455978e to 8c0b19e Compare September 26, 2025 10:19
@shekhirin shekhirin force-pushed the alexey/static-files-only branch from 8c0b19e to a1fb6dc Compare September 26, 2025 10:20
@shekhirin shekhirin changed the title feat(storage): write headers, txs, and receipts only to static files feat(storage): write headers and transactions only to static files Sep 26, 2025
Copy link
Member

@mediocregopher mediocregopher left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a lot of changes but 80% looks like removing the StorageLocation argument from everything so pretty straightforward. LGTM

@github-project-automation github-project-automation bot moved this from Backlog to In Progress in Reth Tracker Sep 29, 2025
Ok(self.tx.commit()?)
/// Commit database transaction and static file if it exists.
pub fn commit(self) -> ProviderResult<()> {
UnifiedStorageWriter::commit(self)
Copy link
Collaborator

@joshieDo joshieDo Sep 29, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

unsure here, if this is called after/during an unwind this would technically be wrong right? UnifiedStorageWriter::commit_unwind exists

Copy link
Collaborator

@mattsse mattsse left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

makes sense,

lgtm!

@shekhirin shekhirin added this pull request to the merge queue Sep 29, 2025
Merged via the queue into main with commit 058ffdc Sep 29, 2025
42 checks passed
@shekhirin shekhirin deleted the alexey/static-files-only branch September 29, 2025 17:20
@github-project-automation github-project-automation bot moved this from In Progress to Done in Reth Tracker Sep 29, 2025
@jenpaff jenpaff moved this from Done to Completed in Reth Tracker Oct 14, 2025
theochap pushed a commit to ethereum-optimism/optimism that referenced this pull request Jan 22, 2026
theochap pushed a commit to ethereum-optimism/optimism that referenced this pull request Feb 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-static-files Related to static files C-enhancement New feature or request

Projects

Status: Completed

Development

Successfully merging this pull request may close these issues.

Write headers and transaction only to static files Write transactions to static files on insert_historical_block

4 participants