perf(db): reuse MDBX DBIs for the same tx

[hai-rise]

MDBX_dbi is an index into the MDBX environment to a table handle. We can safely share this index around unless we manually close and remove table handles from the MDBX environment (we don't).

This PR takes the first step of sharing the DBIs across queries of the same DB transaction, which already yields lots of hits; each hit makes the query 6%-82% faster. For example, every single get was trying to create a new table handle (mdbx_dbi_open)...

reth/crates/storage/db/src/implementation/mdbx/tx.rs

Line 293 in 0cdd548

tx.get(self.get_dbi::<T>()?, key.as_ref())

A step further would be to pre-create and share the DBIs across all transactions.

No screenshot this one, as this path is literally everywhere 🫠.

[mattsse]

unfamiliar with the internals here, but can we safely assume that this will always exist for the duration of the transaction?

maybe @shekhirin knows

[hai-rise]

MDBX_dbi is an index/pointer to the actual table handle stored in the DB environment, which will live until the end of the environment, basically the end of the program for Reth, or until it is explicitly closed via mdbx_dbi_close, which we don't and shouldn't do.

Since we don't dynamically create new tables at runtime (right?), the best practice is to pre-create all these table handles at startup time, then share their MDBX_dbis across all transactions.

Otherwise, (repeatedly) calling mdbx_dbi_open at runtime is very expensive. In the code path I showed earlier in #18284, 2.5 / 5.1 ~ 49% of the get (stage checkpoint) is spent on open_db, which likely returns the same MDBX_dbi anyway. Just much more expensive as it needs to execute a transaction, creates contention at the DB-level, etc.

[shekhirin]

That sounds correct, confirmed with MDBX documentation https://libmdbx.dqdkfa.ru/group__c__dbi.html#ga9bef4a9fdf27655e9343bbbf8b6fc5a1

[gakonst]

What would be risks you foresee with this impl? Whether it is using data that's not up to date due to mvcc or any kind of deadlock? Very exciting work!

[hai-rise]

What would be risks you foresee with this impl? Whether it is using data that's not up to date due to mvcc or any kind of deadlock? Very exciting work!

@gakonst Not an MDBX expert, but it looks pretty straightforward from their docs and source, sir. Unless we dynamically create and close tables at runtime (not seeing why Reth would need to), the table handles and their DBIs (index/pointer) are essentially constant after initialisation, across the whole program's lifetime.

To be sure, I did a quick check at Erigon; they indeed open all DBIs once at startup time:
https://github.com/erigontech/erigon/blob/b5ba7ed89f10d0df10f9f7b22490f31756019dc6/db/kv/mdbx/kv_mdbx.go#L396-L398

Then look it up instead of calling mdbx_dbi_open every time afterwards (The tx.db.buckets[table].DBIs):
https://github.com/erigontech/erigon/blob/b5ba7ed89f10d0df10f9f7b22490f31756019dc6/db/kv/mdbx/kv_mdbx.go#L1116-L1129

To be even more sure, we should double-check existing code and write comments to warn against re-creating tables (like with different flags) and closing DBIs after initialisation.

[gakonst]

Would be awesome if you could reth-bench and see if there's e2e perf improvement given we heavily use opened transactions on a block by block basis -- does this carry over there too?

[hai-rise]

Would be awesome if you could reth-bench and see if there's e2e perf improvement given we heavily use opened transactions on a block by block basis -- does this carry over there too?

My observations yesterday:

• Heavy queries like flushing blocks with 10,000s of transactions "only" waste ~6% of their time on mdbx_dbi_open.
• Quick reads easily waste >50% of their time on mdbx_dbi_open.
• Each extra call doesn't only add latency to the query itself, but creates more contention at the DB-level, so it adds up and will scale badly when DB traffic spikes.
• Luckily, the heaviest reads already utilise cursors (albeit still not that well yet, e.g., Share a HashedStorages cursor among storage root calculations #18032), so we only waste an mdbx_dbi_open on cursor creation, not per disk read there.
• I added a sad dbg! to the Occupied arm, and my stderr exploded with DBI hit & reuse during a quick historical sync run.

I'm also very curious about E2E benchmark results (especially for block building & RPC), but I will need my colleagues' help and the kids to return to school first!

[erthink]

A step further would be to pre-create and share the DBIs across all transactions.

If I understand correctly, in Reth, the set of key-value tables changes rarely and only from version to version, but it is constant when the application is running. If so, then you should (actually must to avoid dummy overhead) open the DBI handles once immediately after opening and/or creating the database, or after completing the migration/upgrade of the data schema.

This approach allows to use a global variables for DBI-handles for all tables, without the need for any synchronization of access/using ones and concern about the lifetime (it is not even necessary to close). So this could be applied safe (without a risk of regressions).

Ask in the libmdbx' telegram group if in doubt.

[hai-rise]

Here are some quick DB benchmarks on my PC rebased on 119ed88:

$ cargo bench -p reth-db --features test-utils --bench get 

$ cargo bench -p reth-db --features test-utils --bench hash_keys

I'll try to run the stage benchmarks tomorrow. E2E may take more time. Meeting Monday was too brutal...

[hai-rise]

Stage benchmarks are mostly broken, so I only have this quick one:

@mattsse @gakonst Can someone from your team help with reth-bench? I'm still slowly syncing an Ethereum Mainnet node... Either way, this PR is ready for a quick improvement, and we can do big benches after the next big PR (setting DBIs once at startup and frictionlessly reusing at runtime).

[hai-rise]

@mattsse @gakonst Ok, I could run reth-bench for the first 300 blocks of Shanghai now:

$ reth-bench new-payload-fcu --advance 300

3e4c0cc:

2025-09-12T07:40:24.513522Z  INFO Total Ggas/s: 0.5774 total_duration=7.949905155s total_gas_used=4590139398 blocks_processed=300

This branch on 3e4c0cc:

2025-09-12T07:51:18.567650Z  INFO Total Ggas/s: 0.6385 total_duration=7.189380139s total_gas_used=4590139398 blocks_processed=300

Speedup:

• Ggas/s: 0.5774 -> 0.6385 ~ 10.6% higher execution throughput
• Duration: 7.949905155s -> 7.189380139s ~ 10.6% lower payload latency
• Best win: From 55.9ms down to 22.3ms payload latency

[shekhirin]

LGTM, this is such an obvious change to do 🙈 Thanks for this! I'm going to merge this and we will run our internal nodes on main for a while before releasing it, so I feel pretty confident.

In the future we can even get rid of DashMap overhead by creating all DBI handles on startup.

[hai-rise]

In the future we can even get rid of DashMap overhead by creating all DBI handles on startup.

I'll find some time next week!