Skip to content

triedb/pathdb: improve perf by separating nodes map#31306

Merged
rjl493456442 merged 2 commits intoethereum:masterfrom
weiihann:pathdb/separate-node-map
Apr 2, 2025
Merged

triedb/pathdb: improve perf by separating nodes map#31306
rjl493456442 merged 2 commits intoethereum:masterfrom
weiihann:pathdb/separate-node-map

Conversation

@weiihann
Copy link
Copy Markdown
Contributor

@weiihann weiihann commented Mar 3, 2025

Description

This PR refactors the nodeSet structure in the path database to use separate maps for account and storage trie nodes, resulting in performance improvements. The change maintains the same API while optimizing the internal data structure.

Motivation

The original implementation used a single nested map structure for both account and storage nodes:

nodes map[common.Hash]map[string]*trienode.Node // node set, mapped by owner and path

This required two map lookups for account nodes (first by owner hash, then by path). By separating account nodes into their own dedicated map, we can eliminate one level of indirection for account operations.

Changes

Refactored nodeSet to use two separate maps:

accountNodes map[string]*trienode.Node // for account trie nodes
storageNodes map[common.Hash]map[string]*trienode.Node // for storage trie nodes

This change maintains full backward compatibility as all method signatures remain unchanged. The refactoring is purely internal to the nodeSet structure.

Benchmark Results

goos: darwin
goarch: arm64
pkg: github.com/ethereum/go-ethereum/triedb/pathdb
cpu: Apple M3 Pro
                                                          │   old.txt    │               new.txt                │
                                                          │    sec/op    │    sec/op     vs base                │
Search128Layers-11                                          9.890µ ± 12%   6.651µ ± 16%  -32.75% (p=0.000 n=10)
Search512Layers-11                                          46.13µ ±  5%   32.45µ ±  2%  -29.66% (p=0.000 n=10)
Search1Layer-11                                             51.03n ±  1%   43.95n ±  5%  -13.86% (p=0.000 n=10)
Persist-11                                                  97.51m ±  9%   84.45m ± 11%  -13.39% (p=0.000 n=10)
AccountIteratorTraversal/binary_iterator_keys-11            9.184µ ±  1%   9.336µ ± 25%   +1.66% (p=0.005 n=10)
AccountIteratorTraversal/binary_iterator_values-11          14.05µ ± 12%   13.88µ ±  0%   -1.19% (p=0.000 n=10)
AccountIteratorTraversal/fast_iterator_keys-11              8.479µ ± 10%   7.881µ ±  1%   -7.05% (p=0.000 n=10)
AccountIteratorTraversal/fast_iterator_values-11            8.326µ ±  4%   8.066µ ±  2%   -3.13% (p=0.001 n=10)
AccountIteratorLargeBaselayer/binary_iterator_(keys)-11     11.05m ±  3%   10.59m ±  0%   -4.17% (p=0.000 n=10)
AccountIteratorLargeBaselayer/binary_iterator_(values)-11   17.45m ±  0%   16.83m ±  1%   -3.53% (p=0.000 n=10)
AccountIteratorLargeBaselayer/fast_iterator_(keys)-11       133.8µ ±  0%   131.2µ ±  0%   -1.96% (p=0.000 n=10)
AccountIteratorLargeBaselayer/fast_iterator_(values)-11     135.8µ ±  2%   133.2µ ±  3%   -1.89% (p=0.015 n=10)
geomean                                                     80.35µ         72.35µ         -9.96%

@weiihann weiihann requested a review from rjl493456442 as a code owner March 3, 2025 06:28
@weiihann weiihann changed the title pathdb: improve perf by separating nodes map triedb/pathdb: improve perf by separating nodes map Mar 3, 2025
@rjl493456442 rjl493456442 self-assigned this Mar 3, 2025
@rjl493456442
Copy link
Copy Markdown
Member

Let's merge the overall performance first before merging it.

@rjl493456442
Copy link
Copy Markdown
Member

rjl493456442 commented Mar 5, 2025

截屏2025-03-05 12 44 48 截屏2025-03-05 12 45 00 截屏2025-03-05 12 46 22

This PR (bench07) is slightly faster than master, due to a 1.8ms reduction in account trie update time (trie node loading).
I think we should accept this optimization.

@rjl493456442 rjl493456442 force-pushed the pathdb/separate-node-map branch from 0ec423a to 20830ec Compare April 2, 2025 02:25
@rjl493456442 rjl493456442 added this to the 1.15.8 milestone Apr 2, 2025
@rjl493456442 rjl493456442 merged commit a9e6c8d into ethereum:master Apr 2, 2025
3 of 4 checks passed
sivaratrisrinivas pushed a commit to sivaratrisrinivas/go-ethereum that referenced this pull request Apr 21, 2025
This PR refactors the `nodeSet` structure in the path database to use
separate maps for account and storage trie nodes, resulting in
performance improvements. The change maintains the same API while
optimizing the internal data structure.
jakub-freebit pushed a commit to fblch/go-ethereum that referenced this pull request Jul 3, 2025
This PR refactors the `nodeSet` structure in the path database to use
separate maps for account and storage trie nodes, resulting in
performance improvements. The change maintains the same API while
optimizing the internal data structure.
howjmay pushed a commit to iotaledger/go-ethereum that referenced this pull request Aug 27, 2025
This PR refactors the `nodeSet` structure in the path database to use
separate maps for account and storage trie nodes, resulting in
performance improvements. The change maintains the same API while
optimizing the internal data structure.
gballet pushed a commit to gballet/go-ethereum that referenced this pull request Sep 11, 2025
This PR refactors the `nodeSet` structure in the path database to use
separate maps for account and storage trie nodes, resulting in
performance improvements. The change maintains the same API while
optimizing the internal data structure.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants