Prototype exposing the exemplar reservoir for upcoming spec changes #5960

jsuereth · 2023-11-03T14:21:33Z

Create ExemplarReservoirFactory to help abstract over Double/Long hell
Create AggregationExtension that allows specifying an exemplar reservoir factory
Add a test which hacks RNG to make sure this all works correctly.

This is for open discussion on how we want to expose customization of ExemplarReservoir to users. One of the two remaining blockers for stabilizing Exemplar specification.

cc @jack-berg

- Create ExemplarReservoirFactory to help abstract over Double/Long hell - Create AggregationExtension that allows specifying an exemplar reservoir factory - Add a test which hacks RNG to make sure this all works correctly.

codecov · 2023-11-03T14:33:01Z

Codecov Report

Attention: Patch coverage is 81.57895% with 14 lines in your changes missing coverage. Please review.

Project coverage is 91.03%. Comparing base (efa46a5) to head (1f7225f).
Report is 460 commits behind head on main.

Files	Patch %	Lines
.../sdk/metrics/internal/view/DefaultAggregation.java	66.66%	5 Missing and 1 partial ⚠️
...cs/internal/exemplar/ExemplarReservoirFactory.java	66.66%	4 Missing ⚠️
...nal/view/Base2ExponentialHistogramAggregation.java	88.88%	1 Missing ⚠️
...try/sdk/metrics/internal/view/DropAggregation.java	0.00%	1 Missing ⚠️
...ernal/view/ExplicitBucketHistogramAggregation.java	87.50%	1 Missing ⚠️
...dk/metrics/internal/view/LastValueAggregation.java	91.66%	1 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##               main    #5960      +/-   ##
============================================
- Coverage     91.09%   91.03%   -0.06%     
- Complexity     5571     5579       +8     
============================================
  Files           612      613       +1     
  Lines         16425    16467      +42     
  Branches       1637     1637              
============================================
+ Hits          14962    14991      +29     
- Misses         1012     1024      +12     
- Partials        451      452       +1

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

jack-berg · 2023-11-03T15:29:34Z

sdk/metrics/src/test/java/io/opentelemetry/sdk/metrics/SdkCustomExemplarReservoirTest.java

+                View.builder()
+                    .setAggregation(
+                        ((AggregationExtension) Aggregation.sum())
+                            .setExemplarReservoirFactory(reservoirFactory))


Should exemplar reservoir be a configuration parameter that hangs off an aggregation or a top level view config option? I.e. View.builder.setExemplarReservoirFactory(...).setAggregation(..)

This is one of the things we need to decide.

Right now, ExemplarReservoir (defaults) are decided by aggregation.

If a user wanted to leverage "default" aggregation, but different reservoir by-instrument type or aggregator choice, we're actually lacking an interplay between those two. E.g. trying to ensure Explicit buckets are the same between exemplars + histograms.

Personally, I think we could go either way here. I'm happy to extract it out further if we want.

Another interesting side affect of having exemplar reservoir be a configurable property of Aggregation is that it causes the reservoir be something that MetricReader / MetricExporter have influence over, since MetricReader / MetricExporter can dictate the default Aggregation as a function of instrument kind.

I actually really like that interaction. It allows exporters to optimise exemplar choice for the backend they speak to. Do you think we should encode this in the specification?

I think it ought to be in the spec if we're going to allow that type of configuration in the java implementation.

Another thought: I think we probably should introduce an AggregationBuilder instead of allowing Aggregation instances to be mutated after instantiation with setAggregation.

I propose we add new method static AggregationBuilder Aggregation#builder() for accessing an AggregationBuilder, defined as follows:

public interface AggregationBuilder { // Configure aggregation AggregationBuilder setDrop(); AggregationBuilder setSum(); AggregationBuilder setLastValue(); AggregationBuilder setExplicitBucketHistogram(); AggregationBuilder setExplicitBucketHistogram(List<Double> bucketBoundaries); AggregationBuilder setBase2ExponentialHistogram(); AggregationBuilder setBase2ExponentialHistogram(int maxBuckets, int maxScale); // Configure reservoir AggregationBuilder setExemplarReservoirFactory(Supplier<ExemplarReservoir<?>> factory); Aggregation build(); }

Usage would be something like:

SdkMeterProvider.builder() .registerView( InstrumentSelector.builder().build(), View.builder() .setAggregation(Aggregation.builder() .setSum() .setExemplarReservoirFactory(ExemplarReservoirFactory.noSamples()) .build()) .build());

jack-berg · 2023-11-03T15:30:01Z

Just the one comment but conceptually I think this makes sense.

jsuereth · 2023-11-03T15:55:49Z

Wanted to add another comment about things we should likely be exposing to users with this change (once specification is written and publicized):

ExemplarReservoirFactory would be exposed in some fashion. We can play with its final API to limit it if we'd like, or even expose it to InstrumentationDescriptor if we think that's necessary. I like keeping it simple, as it is today.
ExemplarReservoir the interface would have to be exposed so you can implement it.
I also believe that due to the complexities in making an efficient reservoir, we should expose:
- FixedSizeExemplarReservoir as the baseline abstraction users can use to build their own reservoirs. This would remain abstract.
- ReservoirCellSelector as the key component a user would be overriding in most implementations.
- ReservoirCell as the key mechanism to pre-allocate memory and avoid memory issues in reservoirs. This would be marked final.

jack-berg · 2023-11-30T21:05:38Z

...s/src/main/java/io/opentelemetry/sdk/metrics/internal/exemplar/ExemplarReservoirFactory.java

+
+      @Override
+      public String toString() {
+        return "fixedSize(" + size + ")";


I'd like to see this split out into a class so we can provide a more conventional toString implementation:

Suggested change

return "fixedSize(" + size + ")";

return "FixedSizeReservoirFactory(" + size + ")";

jack-berg · 2023-11-30T21:07:36Z

sdk/metrics/src/main/java/io/opentelemetry/sdk/metrics/internal/view/SumAggregation.java

+    this(
+        ExemplarReservoirFactory.fixedSize(
+            Clock.getDefault(),
+            Runtime.getRuntime().availableProcessors(),


Do we still think Runtime.getRuntime().availableProcessors() is best if the spec provides the new language:

the default size MAY be
the number of possible concurrent threads (e.g. numer of CPUs) to help reduce
contention. Otherwise, a default size of 1 SHOULD be used.

I wrote the specification with this bit of code in mind.... What would you instead like to see?

The rationale here was a measurable difference in contention.

jack-berg · 2023-11-30T21:08:45Z

...main/java/io/opentelemetry/sdk/metrics/internal/view/ExplicitBucketHistogramAggregation.java

-    return new ExplicitBucketHistogramAggregation(bucketBoundaries);
+    return new ExplicitBucketHistogramAggregation(
+        bucketBoundaries,
+        ExemplarReservoirFactory.histogramBucket(Clock.getDefault(), bucketBoundaries));


I realized that we need to use fixed size by default if bucketBoundaries.size() == 0.

jack-berg · 2023-11-30T21:18:03Z

sdk/metrics/src/test/java/io/opentelemetry/sdk/metrics/SdkCustomExemplarReservoirTest.java

+                View.builder()
+                    .setAggregation(
+                        ((AggregationExtension) Aggregation.sum())
+                            .setExemplarReservoirFactory(reservoirFactory))


I think it ought to be in the spec if we're going to allow that type of configuration in the java implementation.

Another thought: I think we probably should introduce an AggregationBuilder instead of allowing Aggregation instances to be mutated after instantiation with setAggregation.

I propose we add new method static AggregationBuilder Aggregation#builder() for accessing an AggregationBuilder, defined as follows:

public interface AggregationBuilder { // Configure aggregation AggregationBuilder setDrop(); AggregationBuilder setSum(); AggregationBuilder setLastValue(); AggregationBuilder setExplicitBucketHistogram(); AggregationBuilder setExplicitBucketHistogram(List<Double> bucketBoundaries); AggregationBuilder setBase2ExponentialHistogram(); AggregationBuilder setBase2ExponentialHistogram(int maxBuckets, int maxScale); // Configure reservoir AggregationBuilder setExemplarReservoirFactory(Supplier<ExemplarReservoir<?>> factory); Aggregation build(); }

Usage would be something like:

SdkMeterProvider.builder() .registerView( InstrumentSelector.builder().build(), View.builder() .setAggregation(Aggregation.builder() .setSum() .setExemplarReservoirFactory(ExemplarReservoirFactory.noSamples()) .build()) .build());

Fixes #2922 Fixes #3812 Related to #3756 ## Changes - Cleans up language around exposing `ExemplarReservoir`. (Remove TODO, e.g.) - Remove `ExemplarFilter` interface but keep configuration options. (see #3812) - Clarify / simplify Spec Compliance matrix for existing state of the Exemplar Specification. Prototype in java: open-telemetry/opentelemetry-java#5960

jack-berg · 2024-08-26T16:00:06Z

Closing because this is stale. Please re-open when ready to pursue this work.

…emetry#3820) Fixes open-telemetry#2922 Fixes open-telemetry#3812 Related to open-telemetry#3756 ## Changes - Cleans up language around exposing `ExemplarReservoir`. (Remove TODO, e.g.) - Remove `ExemplarFilter` interface but keep configuration options. (see open-telemetry#3812) - Clarify / simplify Spec Compliance matrix for existing state of the Exemplar Specification. Prototype in java: open-telemetry/opentelemetry-java#5960

jack-berg reviewed Nov 3, 2023

View reviewed changes

jsuereth mentioned this pull request Nov 3, 2023

Attribute to indicate whether a Span is used as an Exemplar open-telemetry/opentelemetry-specification#2922

Closed

jack-berg reviewed Nov 30, 2023

View reviewed changes

jsuereth mentioned this pull request Jan 12, 2024

Expose ExemplarReservoir as configuration option of View. open-telemetry/opentelemetry-specification#3820

Merged

jack-berg closed this Aug 26, 2024

dashpole mentioned this pull request Oct 17, 2024

Add selector of exemplar reservoir providers to metric.Stream configuration open-telemetry/opentelemetry-go#5861

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Prototype exposing the exemplar reservoir for upcoming spec changes #5960

Prototype exposing the exemplar reservoir for upcoming spec changes #5960

jsuereth commented Nov 3, 2023

codecov bot commented Nov 3, 2023 •

edited

Loading

jack-berg Nov 3, 2023

jsuereth Nov 3, 2023

jack-berg Nov 3, 2023

jsuereth Nov 15, 2023

jack-berg Nov 30, 2023

jack-berg commented Nov 3, 2023

jsuereth commented Nov 3, 2023

jack-berg Nov 30, 2023

jack-berg Nov 30, 2023

jsuereth Jan 3, 2024

jack-berg Nov 30, 2023

jack-berg Nov 30, 2023

jack-berg commented Aug 26, 2024

	return "fixedSize(" + size + ")";
	return "FixedSizeReservoirFactory(" + size + ")";

Prototype exposing the exemplar reservoir for upcoming spec changes #5960

Prototype exposing the exemplar reservoir for upcoming spec changes #5960

Conversation

jsuereth commented Nov 3, 2023

codecov bot commented Nov 3, 2023 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jack-berg commented Nov 3, 2023

jsuereth commented Nov 3, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jack-berg commented Aug 26, 2024

codecov bot commented Nov 3, 2023 •

edited

Loading