bitswap/httpnet: limit metric cardinality#957
Conversation
Currently, we label the StatusCounter metric with the hostname of the http endpoint. If there was a high number of endpoints, this would cause high cardinality for this metric. This is a non-problem now as there are very few HTTP endpoints. This commit addresses it anyways by only labelling the metric hostnames when the Allowlist is enabled, therefore limiting labels to hostnames in the allowlist.
Codecov ReportAttention: Patch coverage is
@@ Coverage Diff @@
## main #957 +/- ##
==========================================
- Coverage 61.62% 61.56% -0.07%
==========================================
Files 254 254
Lines 31409 31425 +16
==========================================
- Hits 19357 19346 -11
- Misses 10480 10502 +22
- Partials 1572 1577 +5
... and 10 files with indirect coverage changes 🚀 New features to boost your workflow:
|
|
I don't know why I'm feeling that I should go the extra mile and have an option to log all endpoints. |
|
@hsanjuan perhaps a compromise is to have "HTTPRetrieval.Metriclist" in Kubo, and use it when HTTPRetrieval.Allowlist is not defined? This way we could track specific provider without limiting endpoints and without balooning cardinality globally. Logging all endpoints may be used in staging/testing, we could add |
Allows assigning host labels to metrics. To limit unwanted cardinality growth it is disabled by default.
|
I have added an option for this setting. It can be bubbled later to Kubo/Rainbow when merged. |
Currently, we label the StatusCounter metric with the hostname of the http endpoint. If there was a high number of endpoints, this would cause high cardinality for this metric.
This is a non-problem now as there are very few HTTP endpoints. This commit addresses it anyways by only labelling the metric hostnames when the Allowlist is enabled, therefore limiting labels to hostnames in the allowlist.