Skip to content

core: standardize slow block JSON output for cross-client metrics#33655

Merged
rjl493456442 merged 14 commits intoethereum:masterfrom
CPerezz:feature/execution-metrics-standardization
Jan 28, 2026
Merged

core: standardize slow block JSON output for cross-client metrics#33655
rjl493456442 merged 14 commits intoethereum:masterfrom
CPerezz:feature/execution-metrics-standardization

Conversation

@CPerezz
Copy link
Copy Markdown
Contributor

@CPerezz CPerezz commented Jan 21, 2026

Implement standardized JSON format for slow block logging to enable cross-client performance analysis and protocol research.

This change is part of the Cross-Client Execution Metrics initiative proposed by Gary Rong: https://hackmd.io/dg7rizTyTXuCf2LSa2LsyQ

The standardized metrics enabled data-driven analysis like the EIP-7907 research: https://ethresear.ch/t/data-driven-analysis-on-eip-7907/23850

JSON format includes:

  • block: number, hash, gas_used, tx_count
  • timing: execution_ms, total_ms
  • throughput: mgas_per_sec
  • state_reads: accounts, storage_slots, bytecodes, code_bytes
  • state_writes: accounts, storage_slots, bytecodes
  • cache: account/storage/code hits, misses, hit_rate

This should come after merging #33522

@CPerezz CPerezz requested a review from rjl493456442 as a code owner January 21, 2026 09:33
Comment thread core/blockchain_stats.go Outdated
Comment thread core/blockchain_stats.go Outdated
Comment thread core/blockchain_stats.go Outdated
Comment thread core/blockchain_stats.go Outdated
StateWrites: slowBlockWrites{
Accounts: s.AccountUpdated,
StorageSlots: s.StorageUpdated,
Bytecodes: 0, // Bytecodes are immutable once deployed
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can definitely track how many contract codes have been written within the state transition.

Unfortunately the code is mutable now with 7702 :(

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

9e90070 should address this.

Comment thread core/blockchain_stats.go Outdated
Comment thread core/blockchain_stats.go Outdated
CPerezz added a commit to CPerezz/go-ethereum that referenced this pull request Jan 22, 2026
Address review comments from ethereum#33655:
- Rename bytecodes -> code for cross-client consistency
- Use common.Hash instead of string for block hash
- Fix StateHashMs calculation to include AccountUpdates + StorageUpdates
- Fix CommitMs calculation using max() for parallel commits + BlockWrite
- Add code write tracking (CodeUpdated, CodeBytesWritten)
- Add EIP-7702 delegation tracking (delegations set/cleared)

The EIP-7702 fields will be 0 for pre-Pectra blocks per spec.
@CPerezz CPerezz force-pushed the feature/execution-metrics-standardization branch from 2692129 to ebb7821 Compare January 25, 2026 14:45
CPerezz added a commit to CPerezz/go-ethereum that referenced this pull request Jan 25, 2026
Address review comments from ethereum#33655:
- Rename bytecodes -> code for cross-client consistency
- Use common.Hash instead of string for block hash
- Fix StateHashMs calculation to include AccountUpdates + StorageUpdates
- Fix CommitMs calculation using max() for parallel commits + BlockWrite
- Add code write tracking (CodeUpdated, CodeBytesWritten)
- Add EIP-7702 delegation tracking (delegations set/cleared)

The EIP-7702 fields will be 0 for pre-Pectra blocks per spec.
@CPerezz CPerezz force-pushed the feature/execution-metrics-standardization branch from ebb7821 to a17ba89 Compare January 25, 2026 17:28
@CPerezz CPerezz requested a review from rjl493456442 January 25, 2026 21:01
@CPerezz CPerezz force-pushed the feature/execution-metrics-standardization branch from f04452c to f2f4d8d Compare January 26, 2026 09:48
@rjl493456442
Copy link
Copy Markdown
Member

Please rebase and resolve the conflicts

Implement standardized JSON format for slow block logging to enable
cross-client performance analysis and protocol research.

This change is part of the Cross-Client Execution Metrics initiative
proposed by Gary Rong: https://hackmd.io/dg7rizTyTXuCf2LSa2LsyQ

The standardized metrics enabled data-driven analysis like the EIP-7907
research: https://ethresear.ch/t/data-driven-analysis-on-eip-7907/23850

JSON format includes:
- block: number, hash, gas_used, tx_count
- timing: execution_ms, total_ms
- throughput: mgas_per_sec
- state_reads: accounts, storage_slots, bytecodes, code_bytes
- state_writes: accounts, storage_slots, bytecodes
- cache: account/storage/code hits, misses, hit_rate
Convert timing fields from int64 to float64 to preserve sub-millisecond
precision in slow block JSON output. Previously, durations under 1ms
would truncate to 0 due to integer division.

Changes:
- slowBlockTime struct fields: int64 -> float64
- Add durationToMs() helper for nanosecond to float ms conversion
- Update test assertions for float comparisons
Address review comments from ethereum#33655:
- Rename bytecodes -> code for cross-client consistency
- Use common.Hash instead of string for block hash
- Fix StateHashMs calculation to include AccountUpdates + StorageUpdates
- Fix CommitMs calculation using max() for parallel commits + BlockWrite
- Add code write tracking (CodeUpdated, CodeBytesWritten)
- Add EIP-7702 delegation tracking (delegations set/cleared)

The EIP-7702 fields will be 0 for pre-Pectra blocks per spec.
Add cross-client execution metrics for EIP-7702 delegations and code writes:
- Track Eip7702DelegationsSet/Cleared in SetCode based on CodeChangeReason
- Track CodeUpdated and CodeBytesWrite for contract deployments
- Collect metrics in ProcessBlock for slow block logging
Add integration tests for EIP-7702 delegation metrics:
- TestEIP7702DelegationMetricsSet
- TestEIP7702DelegationMetricsClear
- TestEIP7702DelegationMetricsMultiple
Add integration tests verifying cross-client execution metrics:
- Account metrics (loaded, updated, deleted)
- Storage metrics (loaded, updated, deleted)
- Code metrics (loaded, updated, bytes written)
Previously, --debug.logslowblock defaulted to 2 seconds (enabled)
and setting 0 would disable logging. This was confusing.

Now:
- Disabled by default (flag must be explicitly set)
- Value 0 logs ALL blocks
- Positive value filters by execution time threshold
Add deletion metrics to slow block JSON output:
- accounts_deleted: count of accounts destroyed (SELFDESTRUCT)
- storage_slots_deleted: count of storage slots set to zero

These metrics align with the cross-client test vectors for
standardized execution metrics.
Align struct field comments and json tags per goimports rules.
@CPerezz CPerezz force-pushed the feature/execution-metrics-standardization branch from f2f4d8d to 69aabdd Compare January 26, 2026 13:35
@CPerezz
Copy link
Copy Markdown
Contributor Author

CPerezz commented Jan 26, 2026

Please rebase and resolve the conflicts

Done! @rjl493456442

Comment thread core/state/statedb.go Outdated
func (s *StateDB) SetCode(addr common.Address, code []byte, reason tracing.CodeChangeReason) (prev []byte) {
stateObject := s.getOrNewStateObject(addr)
if stateObject != nil {
// Track code write and EIP-7702 delegation metrics
Copy link
Copy Markdown
Member

@rjl493456442 rjl493456442 Jan 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

First of all, the mutation counter may be inaccurate due to the potential inner callframe reverts, e.g., the call frame may fail after the code deployement.

We have two options: (a) explicitly revert the statistic (b) measure the code changes at the end of the state transition.

What's more, it's theoretical possible for multiple contracts deploying the same code within the same block. In this case, we are only interested on the de-duplicated data?

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Additionally, even after the de-duplication, the dirty contract code may already exists in the DB. In han's post, it turns out there are 90% duplicated code deployment.

I am not so sure how useful the measurements against the code mutations at this moment.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cb3122b. should address the first one

As for the seconf comment, we remove the metric if you feel is not useful.

CPerezz and others added 2 commits January 27, 2026 22:57
Count CodeUpdated and CodeBytesWrite in IntermediateRoot() instead of
SetCode(), consistent with how AccountUpdated, StorageUpdated, and
StorageDeleted are already measured. This ensures reverted CREATEs from
failed inner call frames are not counted.

EIP-7702 delegation counters remain in SetCode() as authorizations are
applied at the transaction level (applyAuthorization), outside
revertable EVM call frames.
@rjl493456442
Copy link
Copy Markdown
Member

rjl493456442 commented Jan 28, 2026

Example output:

WARN [01-28|08:27:13.305] {"level":"warn","msg":"Slow block","block":{"number":24332139,"hash":"0x715e77980e31b8971196f3ccd2b8228c23a6aad6aa799200f754e796ab5d4ca7","gas_used":34749600,"tx_count":288},"timing":{"execution_ms":65.407963,"state_read_ms":4.475989,"state_hash_ms":5.960876,"commit_ms":9.845399,"total_ms":88.240873},"throughput":{"mgas_per_sec":393.8039008294943},"state_reads":{"accounts":1083,"storage_slots":2177,"code":403,"code_bytes":2479332},"state_writes":{"accounts":697,"accounts_deleted":0,"storage_slots":766,"storage_slots_deleted":32,"code":26,"code_bytes":4298},"cache":{"account":{"hits":1067,"misses":16,"hit_rate":98.52262234533703},"storage":{"hits":2132,"misses":45,"hit_rate":97.9329352319706},"code":{"hits":402,"misses":1,"hit_rate":99.75186104218362,"hit_bytes":2457556,"miss_bytes":21776}}}
{
  "level": "warn",
  "msg": "Slow block",
  "block": {
    "number": 24332139,
    "hash": "0x715e77980e31b8971196f3ccd2b8228c23a6aad6aa799200f754e796ab5d4ca7",
    "gas_used": 34749600,
    "tx_count": 288
  },
  "timing": {
    "execution_ms": 65.407963,
    "state_read_ms": 4.475989,
    "state_hash_ms": 5.960876,
    "commit_ms": 9.845399,
    "total_ms": 88.240873
  },
  "throughput": {
    "mgas_per_sec": 393.8039008294943
  },
  "state_reads": {
    "accounts": 1083,
    "storage_slots": 2177,
    "code": 403,
    "code_bytes": 2479332
  },
  "state_writes": {
    "accounts": 697,
    "accounts_deleted": 0,
    "storage_slots": 766,
    "storage_slots_deleted": 32,
    "code": 26,
    "code_bytes": 4298
  },
  "cache": {
    "account": {
      "hits": 1067,
      "misses": 16,
      "hit_rate": 98.52262234533703
    },
    "storage": {
      "hits": 2132,
      "misses": 45,
      "hit_rate": 97.9329352319706
    },
    "code": {
      "hits": 402,
      "misses": 1,
      "hit_rate": 99.75186104218362,
      "hit_bytes": 2457556,
      "miss_bytes": 21776
    }
  }
}

@rjl493456442 rjl493456442 added this to the 1.17.0 milestone Jan 28, 2026
@rjl493456442 rjl493456442 merged commit 1e9dfd5 into ethereum:master Jan 28, 2026
7 of 8 checks passed
CPerezz added a commit to CPerezz/go-ethereum that referenced this pull request Feb 25, 2026
…hereum#33655)

Implement standardized JSON format for slow block logging to enable
cross-client performance analysis and protocol research.

This change is part of the Cross-Client Execution Metrics initiative
proposed by Gary Rong: https://hackmd.io/dg7rizTyTXuCf2LSa2LsyQ

The standardized metrics enabled data-driven analysis like the EIP-7907
research: https://ethresear.ch/t/data-driven-analysis-on-eip-7907/23850

JSON format includes:
- block: number, hash, gas_used, tx_count
- timing: execution_ms, total_ms
- throughput: mgas_per_sec
- state_reads: accounts, storage_slots, bytecodes, code_bytes
- state_writes: accounts, storage_slots, bytecodes
- cache: account/storage/code hits, misses, hit_rate

This should come after merging ethereum#33522

---------

Co-authored-by: Gary Rong <garyrong0905@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants