feat(rpc): add transaction hash caching to EthStateCache#21180
feat(rpc): add transaction hash caching to EthStateCache#21180mattsse merged 27 commits intoparadigmxyz:mainfrom
Conversation
|
doesn't show results in practice |
mattsse
left a comment
There was a problem hiding this comment.
@figtracer I like this idea, but perhaps we can solve this on the rpc level, like monitor new blocks and track the index there, see ethrpccache
then we could ptimistaically use the cached block already
thanks, will reopen and expand into this |
Moving transaction hash caching to RPC layer instead, following maintainer suggestion to use EthStateCache pattern.
Add a transaction hash index to EthStateCache that maps TxHash to (BlockHash, TxIndex) for O(1) transaction lookups on recently cached blocks. Changes: - Add tx_hash_index HashMap to EthStateCacheService - Add block_tx_hashes HashMap to track txs per block for cleanup - Index transactions when blocks are cached via CacheNewCanonicalChain - Remove tx index entries when blocks are reorged via RemoveReorgedChain - Add get_transaction_by_hash() method to EthStateCache - Update LoadTransaction::transaction_by_hash() to check cache first
Replace HashMap with schnellru::LruMap for the transaction hash index. This eliminates the need for block_tx_hashes tracking since the LRU automatically evicts old entries when full. Benefits: - Saves ~32MB memory (no block_tx_hashes HashMap) - Self-managing via LRU eviction - Simpler code with same O(1) lookup performance - Graceful degradation on cache miss (falls back to DB)
mattsse
left a comment
There was a problem hiding this comment.
this makes a lot of sense actually and will be more important once we no longer have the index in mdbx directly
left some suggestions
| max_receipts: 2000, | ||
| max_headers: 1000, | ||
| max_concurrent_db_requests: 512, | ||
| max_cached_tx_hashes: 20_000, |
There was a problem hiding this comment.
I think we can bump this to 30k by default which should maybe be ~150MB or so
There was a problem hiding this comment.
how do we get this number?
| /// Cached data for a transaction lookup. | ||
| #[derive(Debug, Clone)] | ||
| pub struct CachedTransaction<B: Block, R> { | ||
| /// The block containing this transaction. | ||
| pub block: Arc<RecoveredBlock<B>>, | ||
| /// Index of the transaction within the block. | ||
| pub tx_index: usize, | ||
| /// Receipts for the block, if available. | ||
| pub receipts: Option<Arc<Vec<R>>>, | ||
| } |
There was a problem hiding this comment.
I'd be nice if this has a helper fn for creating the receipt directly
which could use:
reth/crates/rpc/rpc-eth-types/src/block.rs
Lines 75 to 86 in 4d0e06c
so we can move this type there as well I believe
| // `EthStateCacheService` task. It is created in `index_block_transactions`, accessed in the | ||
| // `GetTransactionByHash` handler, and removed in `remove_block_transactions` - all within | ||
| // the same task's poll loop. | ||
| unsafe impl<Provider, Tasks, LimitBlocks, LimitReceipts, LimitHeaders> Send |
There was a problem hiding this comment.
instead of this, let's do a warpper helper around rc
…, fix reorg bug Co-authored-by: Amp <amp@ampcode.com> Amp-Thread-ID: https://ampcode.com/threads/T-019bfa8e-74c2-73fd-b8ed-c38cd6a79b97
Amp-Thread-ID: https://ampcode.com/threads/T-019bfaa8-7d15-730d-a423-e49efe69369d Co-authored-by: Amp <amp@ampcode.com>
Co-authored-by: Matthias Seitz <matthias.seitz@outlook.de> Co-authored-by: Amp <amp@ampcode.com>
Summary
Adds O(1) transaction hash lookup caching to the RPC layer's
EthStateCache. This speeds upeth_getTransactionByHashandeth_getTransactionReceiptfor recently cached blocks by avoiding database lookups.Changes
Introduces
CachedTransaction<B, R>helper struct containing:block: Arc<RecoveredBlock<B>>- the cached block (shared with block cache)tx_index: usize- transaction index within the blockreceipts: Option<Arc<Vec<R>>>- optional receipts (shared with receipts cache)Adds
tx_hash_index: LruMap<TxHash, Rc(B256, usize)>toEthStateCacheServicethat indexes transaction hashes to their block hash and indexNew
get_transaction_by_hash()method returns the fullCachedTransactionwith block and optional receipts in a single lookupTransactions are indexed when new canonical blocks arrive via
CacheNewCanonicalChainand removed on reorgs viaRemoveReorgedChainAdds
--rpc-cache.max-cached-tx-hashesCLI option (default: 30,000) to configure the transaction hash cache size