Skip to content

Replace BytesTrieSet with HashSet#9641

Merged
siladu merged 4 commits intohyperledger:mainfrom
siladu:replace-BytesTrieSet
Jan 27, 2026
Merged

Replace BytesTrieSet with HashSet#9641
siladu merged 4 commits intohyperledger:mainfrom
siladu:replace-BytesTrieSet

Conversation

@siladu
Copy link
Contributor

@siladu siladu commented Jan 15, 2026

Related to #9165
(Replaces #9166)

This PR effectively reverts #6069 by wholesale removing BytesTrieSet and going back to using HashSet as was the case with this PR: #5537

Benchmarks show improvements across CREATE, CREATE2, SELFDESTRUCT and jumpdest analysis and no regressions including *CALL.


No mainnet improvement/regression detected, newPayload response time p50:

replace-BytesTrieSet-mainnet

Performed benchmarking using evm block-test and the eest benchmark compute suite as of ethereum/execution-specs@2c83b84

Ran https://github.com/ethereum/execution-specs/tree/forks/amsterdam/tests/benchmark/compute/instruction
and https://github.com/ethereum/execution-specs/tree/forks/amsterdam/tests/benchmark/compute/scenario
ignoring the precompile suite.

Instruction Benchmark: Control vs replace-BytesTrieSet

  Summary
  ┌────────────────────────────┬────────────────────────────┬────────────────────────────┬────────────────────┐
  │         Comparison         │          Control           │            Test            │     Difference     │
  ├────────────────────────────┼────────────────────────────┼────────────────────────────┼────────────────────┤
  │ All runs (7 ctrl, 3 test)  │ 463,541 ms @ 165.69 MGas/s │ 454,614 ms @ 168.77 MGas/s │ -8,927 ms (-1.93%) │
  ├────────────────────────────┼────────────────────────────┼────────────────────────────┼────────────────────┤
  │ repeat2 only (2 vs 3 runs) │ 461,082 ms @ 166.11 MGas/s │ 454,614 ms @ 168.77 MGas/s │ -6,468 ms (-1.40%) │
  └────────────────────────────┴────────────────────────────┴────────────────────────────┴────────────────────┘
  Top 10 Improvements
  ┌──────┬──────────────────┬─────────────────────────────────────────────────┐
  │ Rank │      Change      │                      Test                       │
  ├──────┼──────────────────┼─────────────────────────────────────────────────┤
  │ 1    │ -47.7% (-219 ms) │ test_create[0 bytes with value-CREATE2]         │
  ├──────┼──────────────────┼─────────────────────────────────────────────────┤
  │ 2    │ -46.3% (-206 ms) │ test_create[0 bytes without value-CREATE2]      │
  ├──────┼──────────────────┼─────────────────────────────────────────────────┤
  │ 3    │ -45.6% (-200 ms) │ test_create[0 bytes with value-CREATE]          │
  ├──────┼──────────────────┼─────────────────────────────────────────────────┤
  │ 4    │ -44.4% (-283 ms) │ test_selfdestruct_initcode[value_bearing_True]  │
  ├──────┼──────────────────┼─────────────────────────────────────────────────┤
  │ 5    │ -44.4% (-191 ms) │ test_create[0 bytes without value-CREATE]       │
  ├──────┼──────────────────┼─────────────────────────────────────────────────┤
  │ 6    │ -44.3% (-288 ms) │ test_selfdestruct_initcode[value_bearing_False] │
  ├──────┼──────────────────┼─────────────────────────────────────────────────┤
  │ 7    │ -38.8% (-2 ms)   │ test_create[max code size-CREATE2]              │
  ├──────┼──────────────────┼─────────────────────────────────────────────────┤
  │ 8    │ -35.8% (-420 ms) │ test_selfdestruct_created[value_bearing_True]   │
  └──────┴──────────────────┴─────────────────────────────────────────────────┘
  Top Regressions (small absolute values)
  ┌──────┬────────────────────┬────────────────────────────────────────────────┐
  │ Rank │       Change       │                      Test                      │
  ├──────┼────────────────────┼────────────────────────────────────────────────┤
  │ 1    │ +103.2% (+1.54 ms) │ test_log[zeros_topic-1_MiB_zeros_data-log1]    │
  ├──────┼────────────────────┼────────────────────────────────────────────────┤
  │ 2    │ +99.0% (+1.49 ms)  │ test_log[zeros_topic-1_MiB_non_zero_data-log2] │
  ├──────┼────────────────────┼────────────────────────────────────────────────┤
  │ 3    │ +84.7% (+1.43 ms)  │ test_log[zeros_topic-1_MiB_zeros_data-log4]    │
  └──────┴────────────────────┴────────────────────────────────────────────────┘
  Note: The regressions are all sub-2ms tests - the high percentages are misleading. The improvements are in the 200-420ms range, far outweighing the tiny regressions.

  Conclusion

  replace-BytesTrieSet is ~1.4-1.9% faster overall, with significant improvements in CREATE, CREATE2, and SELFDESTRUCT operations. The log test "regressions" are noise in sub-2ms tests.

Scenario Benchmark Results: Control vs replace-BytesTrieSet

Methodology: 10 iterations each, using evmtool block-test with execution-focused tests

Summary

Metric Control Test Difference
Total Time 6232.49 ms 5545.00 ms -687.49 ms (-11.0%)
Throughput 364.83 MGas/s 410.60 MGas/s +45.77 MGas/s
Tests 24 24 -
Iterations 10 10 -

Result: Test is 11.0% faster (687 ms saved)

Top Improvements

Test Control Test Change
test_jumpdest_analysis[benchmark-gas-value_100M-fork_Prag... 452.48 ms 336.74 ms -25.6%
test_jumpdest_analysis[benchmark-gas-value_100M-fork_Prag... 469.16 ms 352.35 ms -24.9%
test_jumpdest_analysis[benchmark-gas-value_100M-fork_Prag... 453.07 ms 346.33 ms -23.6%
test_jumpdest_analysis[benchmark-gas-value_100M-fork_Prag... 497.59 ms 383.95 ms -22.8%
test_jumpdest_analysis[benchmark-gas-value_100M-fork_Prag... 477.13 ms 375.30 ms -21.3%
test_jumpdest_analysis[benchmark-gas-value_100M-fork_Prag... 503.35 ms 404.36 ms -19.7%
test_block_full_of_ether_transfers[benchmark-gas-value_10... 104.38 ms 99.45 ms -4.7%
test_block_full_of_ether_transfers[benchmark-gas-value_10... 354.88 ms 339.69 ms -4.3%
test_block_full_of_ether_transfers[benchmark-gas-value_10... 210.20 ms 202.09 ms -3.9%
test_auth_transaction[benchmark-gas-value_100M-fork_Pragu... 327.28 ms 316.97 ms -3.2%

Potential Regressions

Test Control Test Change
test_block_full_of_ether_transfers[benchmark-gas-value_10... 211.63 ms 222.87 ms +5.3%
test_block_full_of_ether_transfers[benchmark-gas-value_10... 111.42 ms 114.89 ms +3.1%
test_block_full_access_list_and_data[benchmark-gas-value_... 53.18 ms 54.72 ms +2.9%
test_block_full_of_ether_transfers[benchmark-gas-value_10... 113.49 ms 115.54 ms +1.8%
test_block_full_data[benchmark-gas-value_100M-fork_Prague... 51.18 ms 52.02 ms +1.6%

Note: Small regressions (<5%) are within measurement variance (CV typically 5-15%)

All Tests (sorted by improvement)

# Test Control (ms) Test (ms) Diff Ctrl CV Test CV
1 test_jumpdest_analysis[benchmark-gas-value_100M... 452.48 336.74 -25.6% 1.4% 8.7%
2 test_jumpdest_analysis[benchmark-gas-value_100M... 469.16 352.35 -24.9% 0.8% 7.2%
3 test_jumpdest_analysis[benchmark-gas-value_100M... 453.07 346.33 -23.6% 1.6% 7.8%
4 test_jumpdest_analysis[benchmark-gas-value_100M... 497.59 383.95 -22.8% 1.1% 6.2%
5 test_jumpdest_analysis[benchmark-gas-value_100M... 477.13 375.30 -21.3% 1.0% 7.7%
6 test_jumpdest_analysis[benchmark-gas-value_100M... 503.35 404.36 -19.7% 1.0% 6.5%
7 test_block_full_of_ether_transfers[benchmark-ga... 104.38 99.45 -4.7% 16.2% 13.0%
8 test_block_full_of_ether_transfers[benchmark-ga... 354.88 339.69 -4.3% 8.3% 5.1%
9 test_block_full_of_ether_transfers[benchmark-ga... 210.20 202.09 -3.9% 17.5% 18.6%
10 test_auth_transaction[benchmark-gas-value_100M-... 327.28 316.97 -3.2% 3.9% 0.9%
11 test_block_full_of_ether_transfers[benchmark-ga... 111.68 108.33 -3.0% 13.5% 14.3%
12 test_block_full_data[benchmark-gas-value_100M-f... 15.12 14.67 -2.9% 9.7% 6.0%
13 test_block_full_of_ether_transfers[benchmark-ga... 228.92 223.80 -2.2% 14.2% 14.7%
14 test_block_full_of_ether_transfers[benchmark-ga... 338.85 332.40 -1.9% 11.2% 9.2%
15 test_block_full_of_ether_transfers[benchmark-ga... 232.31 228.66 -1.6% 11.8% 11.0%
16 test_empty_block[benchmark-gas-value_100M-fork_... 0.97 0.96 -0.5% 3.3% 3.3%
17 test_auth_transaction[benchmark-gas-value_100M-... 284.67 284.28 -0.1% 3.9% 4.3%
18 test_auth_transaction[benchmark-gas-value_100M-... 294.89 295.53 +0.2% 1.5% 4.4%
19 test_auth_transaction[benchmark-gas-value_100M-... 334.67 339.09 +1.3% 1.9% 2.6%
20 test_block_full_data[benchmark-gas-value_100M-f... 51.18 52.02 +1.6% 3.1% 2.2%
21 test_block_full_of_ether_transfers[benchmark-ga... 113.49 115.54 +1.8% 6.0% 11.3%
22 test_block_full_access_list_and_data[benchmark-... 53.18 54.72 +2.9% 8.0% 11.5%
23 test_block_full_of_ether_transfers[benchmark-ga... 111.42 114.89 +3.1% 7.6% 9.6%
24 test_block_full_of_ether_transfers[benchmark-ga... 211.63 222.87 +5.3% 9.0% 9.2%

Signed-off-by: Simon Dudley <simon.dudley@consensys.net>
Signed-off-by: Simon Dudley <simon.dudley@consensys.net>
@siladu siladu marked this pull request as ready for review January 23, 2026 03:58
Copy link
Member

@lu-pinto lu-pinto left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

@ahamlat ahamlat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚢

@siladu siladu enabled auto-merge (squash) January 27, 2026 06:02
@siladu siladu merged commit 1a199f1 into hyperledger:main Jan 27, 2026
46 of 61 checks passed
@github-project-automation github-project-automation bot moved this from In Progress to Done in Performance Jan 27, 2026
@github-project-automation github-project-automation bot moved this from Open PRs to Done in 26.1.0 Release Jan 27, 2026
@siladu siladu deleted the replace-BytesTrieSet branch January 27, 2026 06:39
macfarla pushed a commit to CPerezz/besu that referenced this pull request Feb 6, 2026
Improves performance of CREATE, CREATE2, SELFDESTRUCT and jumpdest analysis and no visible regressions including *CALL, according to https://github.com/ethereum/execution-specs/tree/c6c380acffa586cfd683384fb2e09348e9a97c03/tests/benchmark/compute benchmark suite
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: Done
Status: Done

Development

Successfully merging this pull request may close these issues.

4 participants