Use sync.Map for exponential histogram aggregations#8077
Use sync.Map for exponential histogram aggregations#8077dashpole wants to merge 8 commits intoopen-telemetry:mainfrom
Conversation
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #8077 +/- ##
=======================================
- Coverage 82.0% 82.0% -0.1%
=======================================
Files 308 308
Lines 24060 24228 +168
=======================================
+ Hits 19748 19882 +134
- Misses 3936 3961 +25
- Partials 376 385 +9
🚀 New features to boost your workflow:
|
4a958eb to
fc3a34b
Compare
3054c75 to
d5f3996
Compare
|
One part if this that is challenging to resolve is dealing with underflow for cumulative metrics. The current design mirrors how the histogram implementation works: Collection swaps hot and cold, reads the cold, and then merges the cold back into the hot point. The issue comes during the merge process. It is possible that an observation made to the hot point should underflow, but we don't find that out until we try to merge the cold point into the hot one. For example: attrs := attribute.NewSet()
maxSize := 2
h := newExpoHistogram(maxsize, ...)
h.measure(ctx, math.MaxFloat64, attrs, ...)
go h.collect(...)
h.measure(ctx, math.SmallestNonzeroFloat64, attrs)
// assume collect() finishes after measure, and tries to merge
// an exp histogram with math.MaxFloat64 into an exp histogram
// with math.SmallestNonzeroFloat64. This will underflow, but we
// can't remove the underflowed measurement after it has been
// aggregated.This is an extremely rare case: Underflow is only possible with maxSize <= 2, and when making measurements where one is 2^1024 times greater than the other. Some options i've come up with to deal with it:
I'm planning to implement the proper fix (option 1.i), but I wanted to document this in-case it comes up later. Option 2.i is also appealing given how extremely rare this should be in-practice. |
fa32a82 to
906aa6e
Compare
Some small testing improvements forked from #8077. This also fixes a flake where the order in which sums are added can change the resulting sum. Use assertSumEqual to handle this similar to other places in the test. Co-authored-by: Tyler Yahn <MrAlias@users.noreply.github.com>
906aa6e to
5ad698a
Compare
Part of #7796
This applies the same approach as I did for fixed-bucket histograms (#7474) to exponential histograms.
Changes
This does not make the buckets concurrent-safe. That will be done in subsequent PRs.