-
Notifications
You must be signed in to change notification settings - Fork 867
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Avoid conflicts in Micrometer description mapping #5452
Avoid conflicts in Micrometer description mapping #5452
Conversation
Signed-off-by: Fabian Stäber <[email protected]>
7725216
to
1c29dfa
Compare
Hi @fstab - a recent change to the OTel spec will stop throwing exceptions when registering metrics with the same name but different description. Do you know if that will be enough to address the issue without changes in this bridge? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm fine with merging this fix before the spec/SDK officially supports that use case - left one comment about adding a TODO for the future.
Thanks! 👍
...ter-1.5/library/src/main/java/io/opentelemetry/instrumentation/micrometer/v1_5/Bridging.java
Show resolved
Hide resolved
…a/io/opentelemetry/instrumentation/micrometer/v1_5/Bridging.java Co-authored-by: Mateusz Rzeszutek <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it might read easier with conventionName
-> name
everywhere.
(my brain kept reading it as the name of a convention, not the name of a meter)
thanks @fstab! |
* Avoid conflicts in Micrometer description mapping Signed-off-by: Fabian Stäber <[email protected]> * fix formatting * Update instrumentation/micrometer/micrometer-1.5/library/src/main/java/io/opentelemetry/instrumentation/micrometer/v1_5/Bridging.java Co-authored-by: Mateusz Rzeszutek <[email protected]> Co-authored-by: Mateusz Rzeszutek <[email protected]>
Background: How PrometheusMeterRegistry Maps Descriptions
The following shows an example of metrics in Prometheus format as produced by Micrometer's
PrometheusMeterRegistry
:If you look closely at the
HELP
text, you will see a slight mismatch: The description is specific tolevel="error"
and not to other attribute values.The reason is that in Micrometer you can define descriptions per set of attributes. It is not very common to have different descriptions per set of attributes, but there are a few real-world examples like Micrometer's built-in
LogbackMetrics
https://github.com/micrometer-metrics/micrometer/blob/fc9c4eed843f02f00cc6f3c9621aed5d4797e29d/micrometer-core/src/main/java/io/micrometer/core/instrument/binder/logging/LogbackMetrics.java#L139-L169
As Prometheus supports only one description per metric, Micrometer's
PrometheusMeterRegistry
maintains acollectorMap
that caches the description per metric name, so metrics with the same name will get the same description. As a result, all metrics namedlogback_events_total
will get the first description that was registered, which happens to be the description forlevel="error"
in the example above.https://github.com/fstab/micrometer/blob/31488cc4ca1232f6599f9479ddb6b8445ece3d06/implementations/micrometer-registry-prometheus/src/main/java/io/micrometer/prometheus/PrometheusMeterRegistry.java#L482-L502
This PR: Make OpenTelemetryMeterRegistry use The Same Strategy as PrometheusMeterRegistry
The
OpenTelemetryMeterRegistry
has the same mismatch with Micrometer, as OTel metrics also don't support different descriptions for different attribute values. However, the current implementation does not handle this, and eventually the SDK throws aDuplicateMetricStorageException
when Micrometer'sLogbackMetrics
register metrics with the same name but different descriptions:https://github.com/open-telemetry/opentelemetry-java/blob/b19157ed98ffc81d5083752631b04e4074ea71f6/sdk/metrics/src/main/java/io/opentelemetry/sdk/metrics/internal/state/MetricStorageRegistry.java#L61-L67
This will happen whenever you use the
OpenTelemetryMeterRegistry
in a Spring application and Spring's Micrometer auto-detectsLogback
.This PR introduces a
descriptionsCache
to mimic the behavior of the Micrometer -> Prometheus mapping. We cache the first description seen for each metric name and re-use it for all attributes.This will result in descriptions that don't fit to the attributes (as in the example above). However, in practice this slight mismatch in the description has never been reported as an issue by Prometheus users (to my knowledge), and I think it is better than throwing a
DuplicateMetricStorageException
when Spring detectsLogback
.