feat: add network tag to metrics

[virajbhartiya]

Closes #12715

Add network tag to metrics

[virajbhartiya]

@rvagg I am trying to setup a Grafana dashboard on my device to test this out. Can you please just let me know if I am going in the right direction with this PR

[rvagg]

Yep, I think this is the right direction. It's unfortunate that you need to add it to all of the views but that looks like the trickiest bit here.

You should be able to register the value at startup, like in here:

lotus/cmd/lotus/daemon.go

Lines 211 to 216 in 07f2f69

	ctx, _ := tag.New(context.Background(),
	tag.Insert(metrics.Version, build.NodeBuildVersion),
	tag.Insert(metrics.Commit, build.CurrentCommit),
	tag.Insert(metrics.NodeType, "chain"),
	)
	// Register all metric views

And in here:

lotus/cmd/lotus-miner/run.go

Lines 58 to 63 in 07f2f69

	ctx, _ := tag.New(lcli.DaemonContext(cctx),
	tag.Insert(metrics.Version, build.MinerBuildVersion),
	tag.Insert(metrics.Commit, build.CurrentCommit),
	tag.Insert(metrics.NodeType, "miner"),
	)
	// Register all metric views

Currently I have grafana setup for my node and I'm scraping with prometheus, in my prometheus.yml scape_configs I have:

  - job_name: lotus-mainnet
    scrape_interval: 10s
    metrics_path: '/debug/metrics'
    static_configs:
      - targets: ['localhost:1234']
        labels:
          network: 'mainnet'
  - job_name: lotus-calibnet
    scrape_interval: 10s
    metrics_path: '/debug/metrics'
    static_configs:
      - targets: ['localhost:1235']
        labels:
          network: 'calibnet'

you can see my mainnet and calibnet tags being inserted there so I can do what you're doing here, but after the fact.

Then in grafana, when I explore metrics I can see the labels showing up that get reported for that metric type. All of mine have network but some have additional tags. In this one, network is there, but prometheus has also added instance and job, but the metric itself (block/failure) also has its own custom tag, from here:

lotus/chain/sub/incoming.go

Lines 466 to 473 in 07f2f69

	func recordFailure(ctx context.Context, metric *stats.Int64Measure, failureType string) {
	ctx, _ = tag.New(
	ctx,
	tag.Upsert(metrics.FailureType, failureType),
	)
	stats.Record(ctx, metric.M(1))
	}

[virajbhartiya]

We could get the network name from build/buildconstants/params.go we'll have to split the + to use it