-
Notifications
You must be signed in to change notification settings - Fork 110
perf(l1): revert optimistic transactions in rocksdb and remove the explicit configured cache #4853
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
Lines of code reportTotal lines added: Detailed view
|
Here we have some initial measurements for backing the 40~50% of memory reduction mentioned in this PR. We are right now testing it again without Node Memory (RSS) & Host memory in mages
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Pull Request Overview
This PR reverts optimistic transactions in RocksDB and removes explicit cache configurations to reduce memory usage by 40-50%. The changes replace OptimisticTransactionDB
with standard DBWithThreadMode
and remove cache-related options from RocksDB configuration.
- Replace
OptimisticTransactionDB
withDBWithThreadMode
across all RocksDB implementations - Remove explicit cache setup and cache-related block options to reduce memory footprint
- Update batch operations to use standard
WriteBatch
instead ofWriteBatchWithTransaction
Reviewed Changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.
File | Description |
---|---|
crates/storage/trie_db/rocksdb_locked.rs | Updated type signatures to use DBWithThreadMode instead of OptimisticTransactionDB |
crates/storage/trie_db/rocksdb.rs | Replaced optimistic transaction DB with standard DB and updated batch operations |
crates/storage/store_db/rocksdb.rs | Removed cache configuration and replaced optimistic transactions with standard DB operations |
CHANGELOG.md | Added entry documenting the memory optimization changes |
Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.
Benchmark Block Execution Results Comparison Against Main
|
We are going to make this changes behind a |
Motivation
This PR reduce memory by 40~50%
Description
This PR sits is basically #4839 but instead of starting from #4821 it starts from main, and after different testing scenarios we came up with this cut in memory:
The caveat is that it appears to be 50% slower, we need to make specific measures to validate the results seen in #4839