Skip to content

refactor(metrics): use default prometheus go metrics collector, and implement standard Prometheus gauges for existing metrics#2219

Merged
timwu20 merged 23 commits intodevelopmentfrom
tim/metrics-refactor
Jan 28, 2022
Merged

refactor(metrics): use default prometheus go metrics collector, and implement standard Prometheus gauges for existing metrics#2219
timwu20 merged 23 commits intodevelopmentfrom
tim/metrics-refactor

Conversation

@timwu20
Copy link
Copy Markdown
Contributor

@timwu20 timwu20 commented Jan 20, 2022

Changes

  • Remove dot/metrics and the use of ethmetrics
  • Create internal/metrics which starts the http server with prometheus handler
  • Migrate all existing metrics to use github.com/prometheus/client_golang/prometheus metrics
    • All names of metrics have been revised with prometheus naming best practices applied
  • Unblocks the ability to tag metrics with given labels using prometheus.GaugeVec, prometheus.CounterVec, etc

Tests

go test ./...

Run node and ensure all go metrics and gossamer metrics are being outputted via the /metrics endpoint.

Issues

#2220

Primary Reviewer

@timwu20 timwu20 changed the title refactor(metrics): Use default prometheus go collector, and implement standard Prometheus gauges for existing metrics refactor(metrics): use default prometheus go metrics collector, and implement standard Prometheus gauges for existing metrics Jan 20, 2022
@codecov
Copy link
Copy Markdown

codecov bot commented Jan 20, 2022

Codecov Report

Merging #2219 (3f643c4) into development (be00a69) will decrease coverage by 0.59%.
The diff coverage is 74.77%.

Impacted file tree graph

@@               Coverage Diff               @@
##           development    #2219      +/-   ##
===============================================
- Coverage        60.41%   59.81%   -0.60%     
===============================================
  Files              210      212       +2     
  Lines            27365    27663     +298     
===============================================
+ Hits             16532    16548      +16     
- Misses            9081     9418     +337     
+ Partials          1752     1697      -55     
Flag Coverage Δ
unit-tests 59.81% <74.77%> (-0.60%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
dot/network/config.go 63.01% <ø> (-1.46%) ⬇️
dot/peerset/peerset.go 49.18% <ø> (ø)
dot/sync/chain_sync.go 54.84% <0.00%> (+1.15%) ⬆️
lib/trie/test_utils.go 87.75% <ø> (ø)
dot/network/service.go 56.17% <4.54%> (+1.24%) ⬆️
lib/grandpa/grandpa.go 59.80% <40.00%> (+0.29%) ⬆️
dot/state/block.go 44.92% <62.50%> (+0.91%) ⬆️
internal/metrics/metrics.go 68.88% <68.88%> (ø)
lib/common/common.go 37.33% <71.42%> (+0.32%) ⬆️
lib/trie/trie.go 96.30% <83.63%> (+4.78%) ⬆️
... and 53 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7d946fd...3f643c4. Read the comment docs.

@timwu20 timwu20 marked this pull request as ready for review January 20, 2022 19:57
Comment thread internal/metrics/metrics.go Outdated
Comment thread lib/grandpa/grandpa.go Outdated
Comment thread lib/grandpa/grandpa.go Outdated
Comment thread tests/stress/stress_test.go
Comment thread dot/state/service.go Outdated
Comment thread dot/state/service.go Outdated
Comment thread dot/state/service.go Outdated
Comment thread dot/state/service.go Outdated
Comment thread dot/network/service.go
Comment thread internal/metrics/metrics.go
Comment thread dot/network/service.go Outdated
Comment thread internal/metrics/metrics.go
Comment thread lib/grandpa/grandpa.go Outdated
Comment thread dot/sync/chain_sync.go

var pendingBlocksLimit = maxResponseSize * 32
var (
pendingBlocksLimit = maxResponseSize * 32
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit a bit out of scope, but I think this could be a constant

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I get an unparam lint error. Apparently there's a function newDisjointBlockSet(limit int) that only gets called with this variable.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Update your golangci-lint? 🤔

Comment thread dot/state/service.go Outdated
Comment thread dot/state/block.go Outdated
Comment thread dot/network/service.go
Copy link
Copy Markdown
Member

@EclesioMeloJunior EclesioMeloJunior left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just a nit

Comment thread internal/metrics/metrics.go Outdated
Co-authored-by: Eclésio Junior <eclesiomelo.1@gmail.com>
Comment thread dot/network/service.go
} else {
syncedBlocks.Update(num.Int64())
select {
case <-s.ctx.Done():
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe pass the context as a first argument to that function? Even if it means calling updateMetrics(s.ctx)? That way we can push context 'up' in the call stack and maybe one day not have them as service struct field

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have another issue to change the Service interface. With that refactor, I'm going to try and remove the contexts. I'll update this code then.

Comment thread dot/network/service.go
Comment thread dot/node.go Outdated
Comment thread dot/node.go Outdated
Comment thread dot/state/block.go Outdated
Comment thread internal/metrics/metrics.go
Comment thread internal/metrics/metrics.go Outdated
Comment thread internal/metrics/metrics.go Outdated
Comment thread internal/metrics/metrics.go Outdated
timwu20 and others added 6 commits January 26, 2022 23:25
Co-authored-by: Quentin McGaw <quentin.mcgaw@gmail.com>
Co-authored-by: Quentin McGaw <quentin.mcgaw@gmail.com>
Co-authored-by: Quentin McGaw <quentin.mcgaw@gmail.com>
Co-authored-by: Quentin McGaw <quentin.mcgaw@gmail.com>
@timwu20 timwu20 force-pushed the tim/metrics-refactor branch from db4e329 to 3919e7e Compare January 27, 2022 04:39
@timwu20 timwu20 merged commit e816e31 into development Jan 28, 2022
@timwu20 timwu20 deleted the tim/metrics-refactor branch January 28, 2022 18:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants