svm repo split: fork metrics

[buffalojoec]

The solana-metrics crate is a utility lib that's used across the validator to submit datapoints and other telemetry. Like solana-measure, it's not specific to any one component or domain of Agave, so it doesn't really make sense to live in SVM and have Agave import it.

One option, as this PR proposes, is to fork it just like we did with solana-measure in #7441. This is a pretty simple and reasonable route to take.

Another, more involved route is to just reimplement the datapoint module in the program-runtime locally, since it's the only SVM crate that uses solana-metrics (and just in one spot). However, this has two potential issues:

• We end up reimplementing most of the lib anyway
• If we need to use telemetry anywhere else in SVM, we would need to break it out of program-runtime anyway

Part of #7317

[mergify]

The Firedancer team maintains a line-for-line reimplementation of the
native programs, and until native programs are moved to BPF, those
implementations must exactly match their Agave counterparts.
If this PR represents a change to a native program implementation (not
tests), please include a reviewer from the Firedancer team. And please
keep refactors to a minimum.

[codecov-commenter]

Codecov Report

❌ Patch coverage is 75.11792% with 211 lines in your changes missing coverage. Please review.
✅ Project coverage is 83.2%. Comparing base (f6e6ff2) to head (1e22da1).
⚠️ Report is 2 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff            @@
##           master    #7442     +/-   ##
=========================================
- Coverage    83.2%    83.2%   -0.1%     
=========================================
  Files         800      804      +4     
  Lines      362783   363563    +780     
=========================================
+ Hits       302195   302774    +579     
- Misses      60588    60789    +201     

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[LucasSte]

I see most files are new. Did you simply copy solana-metrics into svm-metrics? Shouldn't we replace all usages in the validator by svm-metrics?

[buffalojoec]

I see most files are new. Did you simply copy solana-metrics into svm-metrics? Shouldn't we replace all usages in the validator by svm-metrics?

Yes, it's a fork of the crate. I detailed in the description why I thought it made more sense to fork it. Let me know if it's reasonable.

[LucasSte]

I see most files are new. Did you simply copy solana-metrics into svm-metrics? Shouldn't we replace all usages in the validator by svm-metrics?

Yes, it's a fork of the crate. I detailed in the description why I thought it made more sense to fork it. Let me know if it's reasonable.

Oh, I got it. Is the goal to maintain two separate implementations then? One for Agave and another one for SVM?

[buffalojoec]

Oh, I got it. Is the goal to maintain two separate implementations then? One for Agave and another one for SVM?

Yep, and we can thin each one out or evolve it based on requirements of each stack.

[steviez]

Like solana-measure, it's not specific to any one component or domain of Agave, so it doesn't really make sense to live in SVM and have Agave import it.

Remind me, why can't SVM just depend on the existing solana-metrics crate and why did we have to fork it ? solana-metrics doesn't depend on any other crates in Agave at the moment, so it is not like you're pulling in solana-core or agave-validator or anything:

agave/metrics/Cargo.toml

Lines 18 to 26 in bc650cf

	[dependencies]
	crossbeam-channel = { workspace = true }
	gethostname = { workspace = true }
	log = { workspace = true }
	reqwest = { workspace = true, features = ["blocking", "brotli", "deflate", "gzip", "rustls-tls", "json"] }
	solana-cluster-type = { workspace = true }
	solana-sha256-hasher = { workspace = true }
	solana-time-utils = { workspace = true }
	thiserror = { workspace = true }

[buffalojoec]

Remind me, why can't SVM just depend on the existing solana-metrics crate and why did we have to fork it ? solana-metrics doesn't depend on any other crates in Agave at the moment, so it is not like you're pulling in solana-core or agave-validator or anything:

I didn't want to end up in a situation like we have with spl-token, where SVM depends on some Agave libs and then Agave depends on SVM libs. This makes upgrades kind of annoying to manage.

We solved this for SPL with the SDK split-out. We could do the same here for metrics/telemetry, but I wasn't sure it was worth taking on that effort yet. As long as we're emitting the payloads, Grafana should pick it up, right?