Skip to content

perf(db): open MDBX DBIs only once at startup#18424

Merged
shekhirin merged 1 commit intoparadigmxyz:mainfrom
risechain:open-mdbx-dbis-once
Sep 15, 2025
Merged

perf(db): open MDBX DBIs only once at startup#18424
shekhirin merged 1 commit intoparadigmxyz:mainfrom
risechain:open-mdbx-dbis-once

Conversation

@hai-rise
Copy link
Contributor

Part 2 of #18292: Cache DBIs at startup time to reuse throughout the program's lifetime instead of just per transaction.

image image

reth-bench for the first 300 blocks of Shanghai:

$ reth-bench new-payload-fcu --advance 300

f66e197 (main):

2025-09-13T09:46:50.043037Z  INFO Total Ggas/s: 0.6499 total_duration=7.062721087s total_gas_used=4590139398 blocks_processed=300

This branch on f66e197:

2025-09-13T09:56:17.725214Z  INFO Total Ggas/s: 0.6967 total_duration=6.588003416s total_gas_used=4590139398 blocks_processed=300

Speedup:

  • Ggas/s: 0.6499 -> 0.6967 ~ 7.2% higher execution throughput
  • Duration: 7.062721087s -> 6.588003416s ~ 7.2% lower payload latency
  • Best win: From 39.2ms down to 13.3ms payload latency

/// More generally, do not dynamically create, re-open, or drop tables at
/// runtime. It's better to perform table creation and migration only once
/// at startup.
dbis: Arc<HashMap<&'static str, ffi::MDBX_dbi>>,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried &'static and it was ~0.3% faster than Arc, but reverted for consistency with the rest of the code base. We can review and make several things (like also the DB environment, etc.) &'static together later.

Copy link
Member

@shekhirin shekhirin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, nice! In theory we should even be able to make it a match, because tables are known during compile-time, but let's not bother with this for now.

@github-project-automation github-project-automation bot moved this from Backlog to In Progress in Reth Tracker Sep 15, 2025
@shekhirin shekhirin added C-perf A change motivated by improving speed, memory usage or disk footprint A-db Related to the database labels Sep 15, 2025
@shekhirin shekhirin added this pull request to the merge queue Sep 15, 2025
Merged via the queue into paradigmxyz:main with commit ef85d93 Sep 15, 2025
42 checks passed
@github-project-automation github-project-automation bot moved this from In Progress to Done in Reth Tracker Sep 15, 2025
@hai-rise hai-rise deleted the open-mdbx-dbis-once branch September 15, 2025 12:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

A-db Related to the database C-perf A change motivated by improving speed, memory usage or disk footprint

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

2 participants