feat: add `Metadata` table and `StorageSettings` to `ProviderFactory` by joshieDo · Pull Request #19384 · paradigmxyz/reth

joshieDo · 2025-10-29T13:58:57Z

add generic Metadata table.
add StorageSettings entry to above table.
Initialize StorageSettings on init_genesis
Cache storage settings on ProviderFactory and DatabaseProvider and after init_genesis

#19384 <- #19399 <- #19401

mattsse · 2025-10-30T13:31:53Z

crates/storage/db-api/src/models/metadata.rs

+pub struct StorageSettings {
+    /// Whether this node always writes receipts to static files.
+    ///
+    /// If this is set to FALSE AND receipt pruning IS ENABLED, all receipts should be written to DB. Otherwise, they should be written to static files. This ensures that older nodes do not need to migrate their current DB tables to static files. For more, read: <https://github.com/paradigmxyz/reth/issues/18890#issuecomment-3457760097>
+    pub receipts_on_static_files: bool,
+}


can we make this so that we can easily extend this and store arbitrary info?

yeah I think we will also want something for snap sync here, maybe we could change receipts_on_static_files to static_files_v2_enabled since I think I'll just reuse this value to determine whether or not to write changesets in #18882

maybe this should just be a map?

couldnt we just add new booleans here as we need them and make the sf roll out changes more gradually?

reth/crates/storage/storage-api/src/metadata.rs

Lines 35 to 39 in 6429250

fn write_storage_settings(&self, settings: StorageSettings) -> ProviderResult<()> {

self.write_metadata(

keys::STORAGE_SETTINGS,

serde_json::to_vec(&settings).map_err(ProviderError::other)?,

)

this is also being encoded as serde_json so extending this should be fine

mattsse · 2025-10-30T13:32:33Z

crates/storage/db-api/src/tables/mod.rs

+    /// Stores generic node metadata as key-value pairs.
+    /// Can store feature flags, configuration markers, and other node-specific data.
+    table Metadata {
+        type Key = String;
+        type Value = Vec<u8>;
+    }


oh I see this also works because then we can store arbitrary metadata entries like key value storage

crates/storage/storage-api/src/metadata.rs

mattsse · 2025-10-30T13:36:47Z

crates/storage/provider/src/providers/database/provider.rs

        static_file_provider: StaticFileProvider<N::Primitives>,
        prune_modes: PruneModes,
        storage: Arc<N::Storage>,
+        storage_settings: Arc<RwLock<StorageSettings>>,


we might want to extend this in the future with things like snapsync node or smth, but for now i think this is okay

crates/storage/provider/src/providers/database/builder.rs

mattsse · 2025-10-30T13:40:00Z

crates/storage/provider/src/providers/database/mod.rs

+            Arc::new(RwLock::new(StorageSettings::default())),
+        )
+        .storage_settings()?
+        .unwrap_or_default();


this is correct here because this defaults to false, could we make this explicit with a constructor fn on StorageSettings

mattsse · 2025-10-30T13:40:45Z

crates/storage/provider/src/providers/database/mod.rs

+            Arc::new(RwLock::new(StorageSettings::default())),
+        )
+        .storage_settings()?


this feels a bit strange, why does this need StorageSettings as input in order to return it?

yeah it feels weird because its a bit cyclical, open to suggestions.

The overall idea is that both ProviderFactory and DatabaseProvider are instantiated with a cached storage settings object. But to obtain it in the first place, ProviderFactory needs to create a DatabaseProvider to fetch it from storage. That's what's happening here.

mattsse

smol style suggestion

mattsse · 2025-11-05T13:21:51Z

crates/storage/provider/src/providers/database/mod.rs

+            static_file_provider.clone(),
+            Default::default(),
+            Default::default(),
+            Arc::new(RwLock::new(legacy_settings)),


hmm, isn't this the same as creating Self first and then updating the storage_settings?

mattsse · 2025-11-05T13:23:22Z

crates/storage/provider/src/providers/database/mod.rs

+        let legacy_settings = StorageSettings::legacy();
+        let storage_settings = DatabaseProvider::<_, N>::new(


please document what's going on here, why are we doing this

crates/storage/db-api/src/models/metadata.rs

…#19384)

joshieDo added 2 commits October 29, 2025 13:20

add Metadata table and StorageSettings type

2cd3179

initialize storage_settings on provider factory

e295a4c

joshieDo self-assigned this Oct 29, 2025

joshieDo added C-enhancement New feature or request A-db Related to the database labels Oct 29, 2025

github-project-automation bot added this to Reth Tracker Oct 29, 2025

github-project-automation bot moved this to Backlog in Reth Tracker Oct 29, 2025

update factory cached settings on init_genesis

b4e32d6

joshieDo force-pushed the joshie/storage-settings branch from 3293ca2 to b4e32d6 Compare October 29, 2025 14:00

ci

5cd4a58

joshieDo marked this pull request as ready for review October 29, 2025 14:03

joshieDo requested review from Rjected, fgimenez, gakonst, klkvr, mattsse, mediocregopher, rakita and shekhirin as code owners October 29, 2025 14:03

update doctests

62b4443

joshieDo force-pushed the joshie/storage-settings branch from 49ada86 to 62b4443 Compare October 29, 2025 21:12

joshieDo added 2 commits October 29, 2025 21:34

add cached storagesettings to databaseprovider

eefe390

add cached_storage_settings to StorageSettingsCache

40a3cb2

joshieDo mentioned this pull request Oct 29, 2025

feat: always write receipts to static files on new nodes #19399

Merged

mattsse requested changes Oct 30, 2025

View reviewed changes

github-project-automation bot moved this from Backlog to In Progress in Reth Tracker Oct 30, 2025

joshieDo force-pushed the joshie/storage-settings branch 4 times, most recently from e5d730d to eea160a Compare October 31, 2025 12:48

use serde_json for storage settings

6429250

joshieDo force-pushed the joshie/storage-settings branch from eea160a to 6429250 Compare October 31, 2025 12:54

mattsse requested changes Nov 5, 2025

View reviewed changes

shekhirin reviewed Nov 5, 2025

View reviewed changes

crates/storage/db-api/src/models/metadata.rs Outdated Show resolved Hide resolved

joshieDo added 2 commits November 5, 2025 14:19

receipts_in_static_files

11dfd43

add docs

6b93241

mattsse approved these changes Nov 5, 2025

View reviewed changes

joshieDo added this pull request to the merge queue Nov 6, 2025

Merged via the queue into main with commit e20e56b Nov 6, 2025
41 checks passed

joshieDo deleted the joshie/storage-settings branch November 6, 2025 00:56

github-project-automation bot moved this from In Progress to Done in Reth Tracker Nov 6, 2025

yongkangc pushed a commit that referenced this pull request Nov 7, 2025

feat: add Metadata table and StorageSettings to ProviderFactory (…

5a0898f

…#19384)

shekhirin mentioned this pull request Nov 7, 2025

Configure storage settings using CLI #19580

Closed

	fn write_storage_settings(&self, settings: StorageSettings) -> ProviderResult<()> {
	self.write_metadata(
	keys::STORAGE_SETTINGS,
	serde_json::to_vec(&settings).map_err(ProviderError::other)?,
	)

		let legacy_settings = StorageSettings::legacy();
		let storage_settings = DatabaseProvider::<_, N>::new(

Conversation

joshieDo commented Oct 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Rjected Nov 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

joshieDo Nov 5, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mattsse left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

joshieDo commented Oct 29, 2025 •

edited

Loading

Rjected Nov 3, 2025 •

edited

Loading

joshieDo Nov 5, 2025 •

edited

Loading