-
Notifications
You must be signed in to change notification settings - Fork 115
fix(l1): use transactional database with RocksDB #4599
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
RocksDB is non-transactional by default, meaning writes logically interleave rather than behaving as a unit, which is prone to errors. The optimistic transaction mode implements ACID behavior, with conflict resolution at commit time to improve parallelism.
Lines of code reportTotal lines added: Detailed view |
|
Would be nice to see if this affects performance and how much |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR migrates the RocksDB implementation from non-transactional to optimistic transactional mode to provide ACID behavior and prevent data corruption from interleaved writes.
- Replaced
DBWithThreadModewithOptimisticTransactionDBthroughout the codebase - Updated write batch operations to use
WriteBatchWithTransactioninstead ofWriteBatch - Modified type signatures and imports to reflect the new transactional database types
Reviewed Changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| crates/storage/trie_db/rocksdb_locked.rs | Updated database and snapshot types to use OptimisticTransactionDB |
| crates/storage/trie_db/rocksdb.rs | Migrated TrieDB implementation and tests to use transactional database |
| crates/storage/store_db/rocksdb.rs | Updated main store implementation with transactional database and batch operations |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
|
Ran on Sepolia. Logs by the end of snap phase: 2025-09-22T16:57:11.186013Z INFO ethrex_p2p::network: P2P Snap Sync:
elapsed: 02h 34m 12s 09ms
466 peers.
\x1b[93mCurrent step:\x1b[0m Requesting Bytecodes
---
headers progress: 100.00% (total: 9257003, downloaded: 9257021, remaining: 0)
account leaves download: 90331863, elapsed: 00h 01m 56s 00ms
account leaves insertion: 100.00%, elapsed: 00h 15m 41s 00ms
storage leaves download: 557320764, elapsed: 00h 33m 43s 02ms, initially accounts with storage 12898384, healed accounts 25558
storage leaves insertion: 00h 56m 08s 03ms
healing: global accounts healed 28982 global storage slots healed 106561622, elapsed: 00h 29m 47s 01ms, current throttle \x1b[31mDatabase\x1b[0m
bytecodes progress: downloaded: 8276273, elapsed: 00h 10m 52s 00ms)
2025-09-22T16:57:11.719643Z INFO ethrex_p2p::sync: Sync cycle finished, time elapsed: 9242 secs
2025-09-22T16:57:12.681763Z INFO ethrex_rpc::engine::blobs: Received new engine request: Requested Blobs
2025-09-22T16:57:12.693448Z INFO ethrex_rpc::engine::payload: Received new payload with block hash: 0xd6d55a804f1e256cb95b93aceb9e27b59cc2a9b502d1f6c7dbcc7bebb174f162
2025-09-22T16:57:12.694628Z INFO ethrex_rpc::engine::payload: Executing payload with block hash: 0xd6d55a804f1e256cb95b93aceb9e27b59cc2a9b502d1f6c7dbcc7bebb174f162
2025-09-22T16:57:12.695669Z INFO ethrex_storage::store: Adding block to pending: 0xd6d5…f162
2025-09-22T16:57:12.696939Z INFO ethrex_p2p::sync: Syncing from current head 0x1c003e05ff7235b92caf72baee0a54a9bc2120ce5e48953ae624f92e3312ff34 to sync_head 0xd6d55a804f1e256cb95b93aceb9e27b59cc2a9b502d1f6c7dbcc7bebb174f162
2025-09-22T16:57:12.905816Z INFO ethrex_p2p::sync: Processing incoming headers full sync
2025-09-22T16:57:13.211321Z INFO ethrex_storage::store: get pending: 0xd6d5…f162
2025-09-22T16:57:13.211520Z INFO ethrex_p2p::sync: Executing 24 blocks for full sync. First block hash: 0xcc23297e1d15254ca7a1544c67aaffa1610edf1330dafae5281dd97110db3a7e Last block hash: 0xe4f3f172cb735d5c3454c08db47fc61a9c7c099abb044c3c1
7b4b7bfc576bd52
2025-09-22T16:57:16.818869Z INFO ethrex_blockchain: [SYNCING] 20% of batch processed
2025-09-22T16:57:18.330061Z INFO ethrex_blockchain: [SYNCING] 40% of batch processed
2025-09-22T16:57:19.920473Z INFO ethrex_blockchain: [SYNCING] 60% of batch processed
2025-09-22T16:57:21.688638Z INFO ethrex_blockchain: [SYNCING] 80% of batch processed
2025-09-22T16:57:24.537803Z INFO ethrex_blockchain: [METRICS] Executed and stored: Range: 24, Last block num: 9257026, Last block gas limit: 59999829, Total transactions: 3057, Total Gas: 654091593, Throughput: 0.05775683466784219 Gigagas/
s
2025-09-22T16:57:24.538274Z INFO ethrex_p2p::sync: [SYNCING] Executed & stored 24 blocks in 11.326 seconds.
Started at block with hash 0xcc23…3a7e (number 9257003).
Finished at block with hash 0xe4f3…bd52 (number 9257026).
Blocks per second: 2.119By the end of sync: 2025-09-22T16:58:28.512207Z INFO ethrex_p2p::sync: Executing 21 blocks for full sync. First block hash: 0x1ae3bcc8984d31c1eadfc4d996d94485f9c504933eca7d6fc734d4165e0cb3d7 Last block hash: 0x2cda080b25b47ead492d609ce5cbc063b9e28cc5ac8026fcc
3d4d91403249eaf
2025-09-22T16:58:29.049736Z INFO ethrex_blockchain: [METRIC] BLOCK EXECUTION THROUGHPUT (9257128): 0.110 Ggas/s TIME SPENT: 537 ms. Gas Used: 0.059 (98%), #Txs: 117. exec: 79% merkle: 15% store: 5%
2025-09-22T16:58:29.334177Z INFO ethrex_blockchain: [METRIC] BLOCK EXECUTION THROUGHPUT (9257129): 0.067 Ggas/s TIME SPENT: 284 ms. Gas Used: 0.019 (32%), #Txs: 88. exec: 70% merkle: 22% store: 7%
2025-09-22T16:58:29.697308Z INFO ethrex_blockchain: [METRIC] BLOCK EXECUTION THROUGHPUT (9257130): 0.095 Ggas/s TIME SPENT: 363 ms. Gas Used: 0.035 (58%), #Txs: 233. exec: 70% merkle: 22% store: 8%
2025-09-22T16:58:29.993102Z INFO ethrex_blockchain: [METRIC] BLOCK EXECUTION THROUGHPUT (9257131): 0.074 Ggas/s TIME SPENT: 295 ms. Gas Used: 0.022 (37%), #Txs: 139. exec: 69% merkle: 23% store: 7%
2025-09-22T16:58:30.420234Z INFO ethrex_blockchain: [METRIC] BLOCK EXECUTION THROUGHPUT (9257132): 0.069 Ggas/s TIME SPENT: 427 ms. Gas Used: 0.029 (49%), #Txs: 156. exec: 78% merkle: 17% store: 5%
2025-09-22T16:58:31.060910Z INFO ethrex_blockchain: [METRIC] BLOCK EXECUTION THROUGHPUT (9257133): 0.056 Ggas/s TIME SPENT: 640 ms. Gas Used: 0.036 (59%), #Txs: 100. exec: 83% merkle: 13% store: 4%
2025-09-22T16:58:31.391855Z INFO ethrex_blockchain: [METRIC] BLOCK EXECUTION THROUGHPUT (9257134): 0.079 Ggas/s TIME SPENT: 330 ms. Gas Used: 0.026 (44%), #Txs: 89. exec: 75% merkle: 21% store: 4%
2025-09-22T16:58:31.655650Z INFO ethrex_blockchain: [METRIC] BLOCK EXECUTION THROUGHPUT (9257135): 0.072 Ggas/s TIME SPENT: 263 ms. Gas Used: 0.019 (32%), #Txs: 92. exec: 71% merkle: 25% store: 5%
2025-09-22T16:58:31.975826Z INFO ethrex_blockchain: [METRIC] BLOCK EXECUTION THROUGHPUT (9257136): 0.077 Ggas/s TIME SPENT: 320 ms. Gas Used: 0.025 (41%), #Txs: 111. exec: 74% merkle: 21% store: 4%
2025-09-22T16:58:32.264650Z INFO ethrex_blockchain: [METRIC] BLOCK EXECUTION THROUGHPUT (9257137): 0.070 Ggas/s TIME SPENT: 288 ms. Gas Used: 0.020 (34%), #Txs: 100. exec: 72% merkle: 22% store: 7%
2025-09-22T16:58:32.625353Z INFO ethrex_blockchain: [METRIC] BLOCK EXECUTION THROUGHPUT (9257138): 0.074 Ggas/s TIME SPENT: 360 ms. Gas Used: 0.027 (44%), #Txs: 117. exec: 74% merkle: 19% store: 6%
2025-09-22T16:58:32.937694Z INFO ethrex_blockchain: [METRIC] BLOCK EXECUTION THROUGHPUT (9257139): 0.088 Ggas/s TIME SPENT: 312 ms. Gas Used: 0.028 (46%), #Txs: 198. exec: 69% merkle: 23% store: 7%
2025-09-22T16:58:33.279110Z INFO ethrex_blockchain: [METRIC] BLOCK EXECUTION THROUGHPUT (9257140): 0.086 Ggas/s TIME SPENT: 341 ms. Gas Used: 0.029 (49%), #Txs: 135. exec: 72% merkle: 21% store: 6%
2025-09-22T16:58:33.625591Z INFO ethrex_blockchain: [METRIC] BLOCK EXECUTION THROUGHPUT (9257141): 0.080 Ggas/s TIME SPENT: 346 ms. Gas Used: 0.028 (46%), #Txs: 128. exec: 74% merkle: 21% store: 5%
2025-09-22T16:58:33.908806Z INFO ethrex_blockchain: [METRIC] BLOCK EXECUTION THROUGHPUT (9257142): 0.095 Ggas/s TIME SPENT: 283 ms. Gas Used: 0.027 (45%), #Txs: 109. exec: 70% merkle: 24% store: 5%
2025-09-22T16:58:34.253566Z INFO ethrex_blockchain: [METRIC] BLOCK EXECUTION THROUGHPUT (9257143): 0.076 Ggas/s TIME SPENT: 344 ms. Gas Used: 0.026 (43%), #Txs: 90. exec: 76% merkle: 19% store: 6%
2025-09-22T16:58:34.629599Z INFO ethrex_blockchain: [METRIC] BLOCK EXECUTION THROUGHPUT (9257144): 0.102 Ggas/s TIME SPENT: 375 ms. Gas Used: 0.038 (64%), #Txs: 189. exec: 76% merkle: 19% store: 4%
2025-09-22T16:58:34.984441Z INFO ethrex_blockchain: [METRIC] BLOCK EXECUTION THROUGHPUT (9257145): 0.083 Ggas/s TIME SPENT: 354 ms. Gas Used: 0.029 (49%), #Txs: 138. exec: 74% merkle: 19% store: 6%
2025-09-22T16:58:35.324200Z INFO ethrex_blockchain: [METRIC] BLOCK EXECUTION THROUGHPUT (9257146): 0.075 Ggas/s TIME SPENT: 339 ms. Gas Used: 0.025 (42%), #Txs: 49. exec: 80% merkle: 14% store: 6%
2025-09-22T16:58:35.801310Z INFO ethrex_blockchain: [METRIC] BLOCK EXECUTION THROUGHPUT (9257147): 0.083 Ggas/s TIME SPENT: 477 ms. Gas Used: 0.040 (66%), #Txs: 147. exec: 78% merkle: 17% store: 5%
2025-09-22T16:58:36.139129Z INFO ethrex_blockchain: [METRIC] BLOCK EXECUTION THROUGHPUT (9257148): 0.075 Ggas/s TIME SPENT: 337 ms. Gas Used: 0.025 (42%), #Txs: 96. exec: 74% merkle: 20% store: 6%
2025-09-22T16:58:36.139325Z INFO ethrex_p2p::sync: [SYNCING] Executed & stored 21 blocks in 7.627 seconds.
Started at block with hash 0x1ae3…b3d7 (number 9257128).
Finished at block with hash 0x2cda…9eaf (number 9257148).
Blocks per second: 2.753
2025-09-22T16:58:36.139338Z INFO ethrex_p2p::sync: Sync cycle finished, time elapsed: 11 secs
2025-09-22T16:58:37.256245Z INFO ethrex_rpc::engine::blobs: Received new engine request: Requested Blobs
2025-09-22T16:58:37.277698Z INFO ethrex_rpc::engine::payload: Received new payload with block hash: 0x70ba5d54f1e94d2dbe31e7c97256484b73fbcf8fe8d48056ba2bbccea5ef5e70
2025-09-22T16:58:37.280094Z INFO ethrex_rpc::engine::payload: Executing payload with block hash: 0x70ba5d54f1e94d2dbe31e7c97256484b73fbcf8fe8d48056ba2bbccea5ef5e70
2025-09-22T16:58:37.770479Z INFO ethrex_blockchain: [METRIC] BLOCK EXECUTION THROUGHPUT (9257149): 0.095 Ggas/s TIME SPENT: 490 ms. Gas Used: 0.046 (77%), #Txs: 188. exec: 80% merkle: 16% store: 3%Store remains under 10% of the block time. |
This reverts commit 7c5315c.
This reverts commit 7c5315c.
…plicit configured cache (#4853) **Motivation** This PR reduce memory by 40~50% **Description** This PR sits is basically #4839 but instead of starting from #4821 it starts from main, and after different testing scenarios we came up with this cut in memory: - reverts #4599 - removes the rocksdb cache explicit setup The caveat is that it appears to be 50% slower, we need to make specific measures to validate the results seen in #4839 --------- Co-authored-by: Mario Rugiero <[email protected]> Co-authored-by: Javier Chatruc <[email protected]>
RocksDB is non-transactional by default, meaning writes logically
interleave rather than behaving as a unit, which is prone to errors.
The optimistic transaction mode implements ACID behavior, with conflict
resolution at commit time to improve parallelism.