fix: refactor blobs and data columns metrics #8198
Closed
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
KatyaRyazantseva
added
the
scope-metrics
| (blockInput.blockData.fork === ForkName.deneb || blockInput.blockData.fork === ForkName.electra) | ||
| ) { | ||
| const {blobsSource} = blockInput.blockData; | ||
| this.metrics?.importBlock.blobsBySource.inc({blobsSource}); |
Contributor
Author
There was a problem hiding this comment.
Moved to seenGossipBlockInput.ts.
Comment on lines
737
to
740
| validationTime: register.histogram({ | ||
| name: "lodestar_gossip_data_column_gossip_validate_time", | ||
| help: "Time elapsed for data column validation", | ||
| buckets: [0.05, 0.1, 0.2, 0.5, 1, 1.5, 2, 4], |
Contributor
Author
There was a problem hiding this comment.
At some point overlaps with the specs' beacon_data_column_sidecar_gossip_verification_seconds metric. Probably, should be deleted.
Member
There was a problem hiding this comment.
if it's already defined in the spec should prefer using those metrics if they track the same
| help: "Time elapsed between block slot time and the time data column sidecar reconstructed", | ||
| buckets: [2, 4, 6, 8, 10, 12], | ||
| gossipDataColumn: { | ||
| recvToValidation: register.histogram({ |
Contributor
Author
There was a problem hiding this comment.
Should be added to the PeerDAS dashboard?
Comment on lines
743
to
747
| blobs: { | ||
| bySource: register.gauge<{source: BlobsSource}>({ | ||
| name: "lodestar_blobs_by_source", | ||
| help: "Number of received blobs by source", | ||
| labelNames: ["source"], |
Contributor
Author
There was a problem hiding this comment.
Moved out from importBlock section - blobsBySource.
Contributor
Author
|
Are blob metrics from |
nflaig
reviewed
Aug 14, 2025
Member
nflaig
left a comment
nflaig
left a comment
There was a problem hiding this comment.
blobs metrics seem incomplete and data column likely as well (need to double check), at least for blobs we might also get them from engine or req/resp, not only gossip and api.
you can search for EventType.blobSidecar, also see changes in #7967
| } as BlockInputBlobs; | ||
|
|
||
| blockInputs.push(getBlockInput.availableData(config, block.data, blockSource, blockData)); | ||
| metrics?.blobs.bySource.inc({source: blobsSource}, allBlobSidecars.length); |
Contributor
Author
There was a problem hiding this comment.
This function also called from ByRoot.
|
|
||
| let allDataColumnsSidecars: fulu.DataColumnSidecar[]; | ||
| logger?.debug("allDataColumnsSidecars partialDownload", { | ||
| let allDataColumnSidecars: fulu.DataColumnSidecar[]; |
Contributor
Author
There was a problem hiding this comment.
Renamed for consistency, has been used twice in this module, as allDataColumnSidecars and allDataColumnsSidecars. Probably, this change should be moved into other PR.
| // for e.g. a blockInput that might be awaiting blobs promise fullfillment in | ||
| // verifyBlocksDataAvailability | ||
| cachedData.blobsCache.set(blobSidecar.index, blobSidecar); | ||
| metrics?.blobs.bySource.inc({source: BlobsSource.engine}); |
Contributor
Author
There was a problem hiding this comment.
Not sure how to count here: all the blobs we've got from EL or those from EL which are not cached yet. Same for data columns.
Member
There was a problem hiding this comment.
EL which are not cached yet.
otherwise we would get incorrect numbers, we care about from where we've received the data first
Member
There was a problem hiding this comment.
based on check cachedData.blobsCache.has(engineReqIdentifiers[j].index) === false it seems we already count this correctly, ie. we only increment if we've received a blob from EL which we haven't received yet from gossip or other sources
nflaig
reviewed
Aug 20, 2025
| }, | ||
| gossipDataColumn: { | ||
| recvToValidation: register.histogram({ | ||
| name: "lodestar_gossip_data_column_received_to_gossip_validate", |
Member
There was a problem hiding this comment.
the metric name might also suggest that it tracks time between receiving until before validation, not including validation time
Contributor
Author
There was a problem hiding this comment.
The [start]To[end] metric naming convention is a standard practice for measuring the total latency of a process. It is used to define the full time elapsed from the moment a process starts until the moment it completes. Adding unit makes sense.
| help: "Time elapsed between data column received and data column validation", | ||
| buckets: [0.05, 0.1, 0.2, 0.5, 1, 1.5, 2, 4], | ||
| }), | ||
| validationTime: register.histogram({ |
Member
There was a problem hiding this comment.
is there any difference between recvToValidation and validationTime? we shouldn't be doing a lot of work besides deserialzation which is pretty fast
Contributor
Author
There was a problem hiding this comment.
There is a difference. Deleted validationTime in favor of verificationTime from the metrics specs.
Comment on lines
737
to
740
| validationTime: register.histogram({ | ||
| name: "lodestar_gossip_data_column_gossip_validate_time", | ||
| help: "Time elapsed for data column validation", | ||
| buckets: [0.05, 0.1, 0.2, 0.5, 1, 1.5, 2, 4], |
Member
There was a problem hiding this comment.
if it's already defined in the spec should prefer using those metrics if they track the same
| help: "Time elapsed between block slot time and the time data column sidecar reconstructed", | ||
| buckets: [2, 4, 6, 8, 10, 12], | ||
| gossipDataColumn: { | ||
| recvToValidation: register.histogram({ |
| // for e.g. a blockInput that might be awaiting blobs promise fullfillment in | ||
| // verifyBlocksDataAvailability | ||
| cachedData.blobsCache.set(blobSidecar.index, blobSidecar); | ||
| metrics?.blobs.bySource.inc({source: BlobsSource.engine}); |
Member
There was a problem hiding this comment.
based on check cachedData.blobsCache.has(engineReqIdentifiers[j].index) === false it seems we already count this correctly, ie. we only increment if we've received a blob from EL which we haven't received yet from gossip or other sources
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## unstable #8198 +/- ##
=========================================
Coverage 54.24% 54.25%
=========================================
Files 847 847
Lines 63829 63839 +10
Branches 4833 4837 +4
=========================================
+ Hits 34626 34636 +10
Misses 29126 29126
Partials 77 77 🚀 New features to boost your workflow:
|
Contributor
Performance Report🚀🚀 Significant benchmark improvement detected
Full benchmark results
|
Contributor
Author
Member
|
@KatyaRyazantseva are you planning to continue working on this? needs to be updated against latest unstable since we did quite a lot of changes in block input refactor |
Contributor
Author
If the block input refactor has finished, then yes, I am going to finish this PR. Will merge the updates. |
Member
all changes have been merged, seems like we submitted the grafana dashboard changes already to unstable in #8443 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Motivation
This PR aims to solve inconsistencies and tech debt associated with the metrics for blobs and columns, as outlined in #8174.
Description
The metrics have been refactored for better consistency between blobs and data columns.
Steps to test or reproduce
Blobs metrics are not widely used on dashboards. For some data column metrics, use the PeerDAS dashboard.