Change the default buckets for Explicit Bucket Histogram

[reyang]

Fixes #2759

Changes

@jmacd and I did some research offline after we received #2759, it seems that Prometheus official clients don't have consistency regarding the default buckets. This PR tries to find a good balance that would cover all the official prom-clients.

[jack-berg]

Is changing bucket boundaries not a breaking change? There was discussion in the spec related to this here.

Java was not aligned with the spec (issue here) until recently. We changed to align with the spec, but took special care to provide tools for users to revert to the previous behavior. Note that these changes are unreleased. Although our last released bucket boundaries are aligned with this PR, we definitely don't want to release code that changes the defaults then have to flip flop on users again.

It's important to resolve whether changes to the default bucket boundaries is a breaking change. I say that it is a breaking change.

[reyang]

Is changing bucket boundaries not a breaking change? There was discussion in the spec related to this here.

Good question - here is my personal thinking: introducing more buckets is not a breaking change (it might cause more resource, which is fine), removing buckets (reducing the number of buckets) could be a breaking change since it drops the accuracy (which could cause existing alerts being unavailable/wrong).

[jack-berg]

It's difficult to say whether a particular backend or dashboard is operating in a way that is relying on the existing bucket boundaries in a strict way. My employer New Relic would not be impacted by this change. However I'm inclined to take the conservative approach and consider any change to the default values as a breaking change, even adding new buckets. I don't feel strongly enough to block this PR.

[bogdandrutu]

Isn't this breaking change?

...

[jsuereth]

I believe this is a breaking change. I agree with it, but can we give SDKs guidance on how to accept this change?

[jmacd]

Quoting the OpenMetrics specification on histogram buckets,

Buckets are cumulative to allow monitoring systems to drop any non-+Inf bucket for performance/anti-denial-of-service reasons in a way that loses granularity but is still a valid Histogram.

This to me clarifies that adding buckets is a non-breaking change.

[jack-berg]

If there is any doubt about the backwards compatibility of this change, one option is follow the pattern used when changing OTLP exporters default enpdoint from https://localhost:4317 to http://localhost:4317, where it was phrased as:

SDKs SHOULD default endpoint variables to use http scheme unless they have good reasons to choose
https scheme for the default (e.g., for backward compatibility reasons in a stable SDK release).

In a similar manner, we could say that SDKs should use the default bucket boundaries defined in this PR unless they have good reason to use the bucket boundaries previously defined, such as maintaining backwards compatibility.

This would allow most SDKs to be aligned on these bucket boundaries while leaving the door open for SDK implementations to use the old values if they must. Taking this approach does codify that we do think of changing bucket boundaries as a breaking change, which is reasonable and conservative. The stable metrics SDKs are java, dotnet, and python. Java wouldn't have to do anything to be compatible with this change since because of an error, these new bounds match our current defaults. It looks like dotnet is in favor of this change and would make the change. @open-telemetry/python-maintainers how would this change impact you?

[reyang]

If there is any doubt about the backwards compatibility of this change, one option is follow the pattern used when changing OTLP exporters default enpdoint from https://localhost:4317 to http://localhost:4317, where it was phrased as:

SDKs SHOULD default endpoint variables to use http scheme unless they have good reasons to choose
https scheme for the default (e.g., for backward compatibility reasons in a stable SDK release).

I've updated the PR, PTAL.