Skip to content

Conversation

@asdacap
Copy link
Contributor

@asdacap asdacap commented Jul 17, 2025

Require #8997

  • Modified the rocksdb config depending on the pruning config and available memory.
  • When available memory is more than 20GB, partitioned index is disabled and write buffer is increased.
  • When archive mode is on, level multiplier is set to 10, and write buffer is increased again, to reduce write amplification.
  • Also increase codedb write buffer size from 4MB to 16MB.

Types of changes

What types of changes does your code introduce?

  • New feature (a non-breaking change that adds functionality)
  • Optimization

Testing

Requires testing

  • Yes
  • No

If yes, did you write tests?

  • Yes
  • No

Notes on testing

  • Mainnet sync ok
  • Index size increased accordingly.
  • Available memory respond to cgroup limit.

@asdacap asdacap requested a review from rubo as a code owner July 17, 2025 11:36
@benaadams benaadams self-requested a review July 18, 2025 02:15
@benaadams benaadams requested a review from Copilot July 18, 2025 02:24
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This pull request implements dynamic RocksDB configuration optimization based on available system memory and pruning mode. The changes enable automatic adjustment of database settings to improve performance when sufficient memory is available (>32GB) or when running in archive mode, reducing write amplification and increasing write buffer sizes.

Key changes include:

  • Introduction of a RocksDbConfigFactory that dynamically adjusts database configurations based on hardware capabilities
  • Automatic optimization for high-memory systems (>32GB) and archive mode configurations
  • Increased default CodeDB write buffer size from 4MB to 16MB

Reviewed Changes

Copilot reviewed 22 out of 22 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
PruningTrieStateFactory.cs Refactored to move config advice logic to MainPruningTrieStoreFactory and updated dependency injection
RocksDbConfigFactory.cs New factory class implementing dynamic RocksDB configuration based on memory and pruning mode
DbConfig.cs Added new configuration options for large memory and archive mode optimizations
IRocksDbConfig.cs New interface abstracting RocksDB configuration properties
HardwareInfo.cs New service providing system memory information using GC memory APIs
Test files Updated to accommodate new factory pattern and dependency injection changes
Comments suppressed due to low confidence (2)

#endregion

#region read-write options
// TODO: These are not applied to column family
Copy link

Copilot AI Jul 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This TODO comment indicates incomplete functionality where options are not applied to column families. This should be addressed or properly documented with a plan for resolution.

Copilot uses AI. Check for mistakes.
Base automatically changed from refactor/move-init-db-to-di to master July 18, 2025 06:06
@LukaszRozmej
Copy link
Member

Please update with master

@asdacap asdacap force-pushed the feature/adjust-rocksdb-config-by-memory branch from 2797111 to c20f57a Compare July 21, 2025 12:07
@asdacap
Copy link
Contributor Author

asdacap commented Jul 21, 2025

Rebased

@asdacap asdacap merged commit 284c5ed into master Jul 23, 2025
77 checks passed
@asdacap asdacap deleted the feature/adjust-rocksdb-config-by-memory branch July 23, 2025 12:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants