Batcher Metrics by sebastianst · Pull Request #5167 · ethereum-optimism/optimism

sebastianst · 2023-03-15T23:18:54Z

Description

This PR adds metrics to the batcher. The following metrics are recorded:

channel
- opened
- closed
- fully submitted
- timed out
L2 blocks loaded into queue
L2 blocks added to channel
batch transaction
- submitted
- success
- failed

Some metrics are implemented as simple events (like tx success), other contain many data dimensions, like RecordL2BlocksAdded.

Also added info logging of different stats when channels are closed. This has also long been overdue.

Open Tasks

Some open tasks were not done yet as to limit this PR's size.

CLI-3642 - Some reusable components were taken from the op-node metrics, namely RefMetrics and EventMetrics, and copied into package op-service/metrics. The duplicated code should be removed in a follow-up by making the op-node use these.
CLI-3641 - Record channel IDs and transaction IDs similar to how block hashes are recorded, to allow for a visual analysis of changing channels/transactions.
CLI-3640 - Record reasons for channel closing. A dependency cycle has to be resolved for that.

Metadata

Fixes CLI-3240

changeset-bot · 2023-03-15T23:18:57Z

⚠️ No Changeset found

Latest commit: 62003a9

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

netlify · 2023-03-15T23:19:23Z

✅ Deploy Preview for opstack-docs canceled.

Name	Link
🔨 Latest commit	`62003a9`
🔍 Latest deploy log	https://app.netlify.com/sites/opstack-docs/deploys/641832f05c6b8e00080dc899

trianglesphere

this is exciting

op-node/rollup/derive/l2block_util.go

op-batcher/batcher/batch_submitter.go

op-batcher/batcher/channel_manager.go

trianglesphere · 2023-03-16T19:37:12Z

@sebastianst needs a go lint on l2block_utils.go

trianglesphere · 2023-03-16T19:37:49Z

Also, all TODO's need linear tickets to pass semgrep.

mergify · 2023-03-16T20:12:29Z

Hey @sebastianst! This PR has merge conflicts. Please fix them before continuing review.

codecov · 2023-03-16T23:15:47Z

Codecov Report

Merging #5167 (62003a9) into develop (259868c) will decrease coverage by 4.34%.
The diff coverage is 21.13%.

Additional details and impacted files

@@             Coverage Diff             @@
##           develop    #5167      +/-   ##
===========================================
- Coverage    41.91%   37.58%   -4.34%     
===========================================
  Files          354      216     -138     
  Lines        21977    18192    -3785     
  Branches       776        0     -776     
===========================================
- Hits          9212     6837    -2375     
+ Misses       12062    10687    -1375     
+ Partials       703      668      -35

Flag	Coverage Δ
bedrock-go-tests	`37.58% <21.13%> (-0.40%)`	⬇️
contracts-bedrock-tests	`?`
contracts-tests	`?`
core-utils-tests	`?`
dtl-tests	`?`
fault-detector-tests	`?`
sdk-tests	`?`

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
op-batcher/batcher/batch_submitter.go	`0.00% <0.00%> (ø)`
op-batcher/batcher/config.go	`0.00% <ø> (ø)`
op-batcher/batcher/driver.go	`0.00% <0.00%> (ø)`
op-batcher/metrics/metrics.go	`0.00% <0.00%> (ø)`
op-node/rollup/derive/l2block_util.go	`0.00% <0.00%> (ø)`
op-service/metrics/event.go	`0.00% <0.00%> (ø)`
op-service/metrics/ref_metrics.go	`0.00% <0.00%> (ø)`
op-batcher/metrics/noop.go	`38.46% <38.46%> (ø)`
op-node/rollup/derive/channel_out.go	`77.84% <66.66%> (ø)`
op-batcher/batcher/channel_manager.go	`79.82% <83.54%> (+2.66%)`	⬆️
... and 1 more

... and 144 files with indirect coverage changes

tynes · 2023-03-16T23:36:41Z

This is looking good

op-batcher/batcher/channel_builder.go

refcell

Few minor nits

op-batcher/batcher/channel_builder.go

op-batcher/batcher/channel_builder_test.go

op-batcher/batcher/driver.go

op-batcher/batcher/channel_manager_test.go

op-batcher/batcher/channel_manager.go

This is similar to PayloadToBlockRef but takes an interface instead that e.g. a types.Block fulfills. This way, we can directly extract L2BlockRefs from type.Blocks that have a valid L1 Info Deposit tx as first tx. A block could also be converted into an ExecutionPayload, but this would make an inefficient roundtrip, marshalling and unmarshalling all transactions.

The caller of BlockToBatch is sometimes interested in the L1BlockInfo of the block, so instead of re-extracting it again, we just return it here.

RefMetrics are a reusable metrics module to enable block reference metrics.

Reusable event metrics modules.

By removing some code duplication and a more efficient implementation of the MaxRLPBytes test.

op-batcher/batcher/driver.go

refcell

Nice 🔥

sebastianst · 2023-03-20T13:15:27Z

Output at http://localhost:7301/ after letting the batcher running for some time: batcher-metrics-prometheus.raw.txt

mergify · 2023-03-20T14:15:13Z

This PR has been added to the merge queue, and will be merged soon.

mergify · 2023-03-20T14:15:15Z

This PR is next in line to be merged, and will be merged as soon as checks pass.

sebastianst requested review from a team, protolambda and trianglesphere as code owners March 15, 2023 23:18

trianglesphere reviewed Mar 16, 2023

View reviewed changes

op-node/rollup/derive/l2block_util.go Show resolved Hide resolved

op-batcher/batcher/batch_submitter.go Show resolved Hide resolved

op-batcher/batcher/channel_manager.go Show resolved Hide resolved

mergify bot added the conflict label Mar 16, 2023

sebastianst force-pushed the seb/batcher-metrics branch from 66bab91 to 8529542 Compare March 16, 2023 22:32

mergify bot removed the conflict label Mar 16, 2023

sebastianst force-pushed the seb/batcher-metrics branch from 8529542 to 2dbcdc2 Compare March 16, 2023 22:56

sebastianst force-pushed the seb/batcher-metrics branch 2 times, most recently from 9e30a15 to 17f13aa Compare March 17, 2023 17:15

refcell reviewed Mar 18, 2023

View reviewed changes

op-batcher/batcher/channel_builder.go Show resolved Hide resolved

refcell suggested changes Mar 18, 2023

View reviewed changes

sebastianst added 10 commits March 20, 2023 10:50

op-node/rollup/derive: Let BlockToBatch also return L1BlockInfo

2c2f2c1

The caller of BlockToBatch is sometimes interested in the L1BlockInfo of the block, so instead of re-extracting it again, we just return it here.

op-service/metrics: Add RefMetrics

4751b03

RefMetrics are a reusable metrics module to enable block reference metrics.

op-service/metrics: Add Event and EventVec metrics

cc2c0b4

Reusable event metrics modules.

op-batcher: Let channelBuilder.AddBlock return L1BlockInfo

b18b149

op-batcher: Add output bytes tracking to channelBuilder

5171e91

op-batcher/metrics: Add batcher metrics package

10d27cc

op-batcher: Add metrics to batcher

b4929fa

op-batcher: Improve unit tests

1a9e947

By removing some code duplication and a more efficient implementation of the MaxRLPBytes test.

op-batcher: Test channelBuilder.InputBytes

4c4bdc3

sebastianst force-pushed the seb/batcher-metrics branch from 61eb165 to 0803ea2 Compare March 20, 2023 09:50

semgrep-app bot reviewed Mar 20, 2023

View reviewed changes

op-batcher/batcher/driver.go Show resolved Hide resolved

sebastianst requested review from refcell and trianglesphere March 20, 2023 09:51

op-batcher: Test channelBuilder.OutputBytes

62003a9

sebastianst force-pushed the seb/batcher-metrics branch from 0803ea2 to 62003a9 Compare March 20, 2023 10:18

refcell approved these changes Mar 20, 2023

View reviewed changes

sebastianst mentioned this pull request Mar 20, 2023

op-batcher: enable graceful shutdown, closing current channel #5105

Merged

protolambda approved these changes Mar 20, 2023

View reviewed changes

mergify bot merged commit 1258b25 into develop Mar 20, 2023

mergify bot deleted the seb/batcher-metrics branch March 20, 2023 14:15

mergify bot added the on-merge-train label Mar 20, 2023

mergify bot removed the on-merge-train label Mar 20, 2023

shrimalmadhur mentioned this pull request Mar 20, 2023

op-proposer: add l2 block height metrics #5189

Merged

sebastianst mentioned this pull request Mar 20, 2023

fix(op-batcher): Fix PR #5186 Nits #5202

Merged

sebastianst mentioned this pull request Aug 11, 2023

fix: Use RefMetrics and Event from op-service/metrics package #6735

Closed

Conversation

sebastianst commented Mar 15, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

changeset-bot bot commented Mar 15, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

⚠️ No Changeset found

Uh oh!

netlify bot commented Mar 15, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Deploy Preview for opstack-docs canceled.

Uh oh!

trianglesphere left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

trianglesphere commented Mar 16, 2023

Uh oh!

trianglesphere commented Mar 16, 2023

Uh oh!

mergify bot commented Mar 16, 2023

Uh oh!

codecov bot commented Mar 16, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

tynes commented Mar 16, 2023

Uh oh!

Uh oh!

refcell left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

refcell left a comment

Choose a reason for hiding this comment

Uh oh!

sebastianst commented Mar 20, 2023

Uh oh!

mergify bot commented Mar 20, 2023

Uh oh!

mergify bot commented Mar 20, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

sebastianst commented Mar 15, 2023 •

edited

Loading

changeset-bot bot commented Mar 15, 2023 •

edited

Loading

netlify bot commented Mar 15, 2023 •

edited

Loading

codecov bot commented Mar 16, 2023 •

edited

Loading