-
Notifications
You must be signed in to change notification settings - Fork 403
[Extensions] Add consistent probability sampler #4629
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. Weβll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
26 commits
Select commit
Hold shift + click to select a range
2a0c502
[Extensions] Add consistent probability sampler
martincostello bf363c3
[Extensions] Add fuzz tests
martincostello 5814b0c
[Extensions] Add benchmarks
martincostello 1e4e725
[Extensions] Update CHANGELOG
martincostello 3c09555
[Extensions] Extend test coverage
martincostello 4d1430f
[Extensions] Address feedback
martincostello 45f4705
[Extensions] Fix flaky test
martincostello 7967ef3
[Extensions] Rename field
martincostello bff4142
[Extensions] Update comments
martincostello aee453d
[Extensions] Use built-in method where available
martincostello 66d5d13
[Extensions] Add end-to-end test
martincostello fe5ec67
Merge branch 'main' into gh-3678
martincostello 68ccf30
Merge branch 'main' into gh-3678
martincostello 91a4b36
[Extensions] Update CHANGELOG
martincostello 2067452
Merge branch 'main' into gh-3678
martincostello 10fa391
[Extensions] Update ConsistentProbability
martincostello 9bf9754
[Extensions] Add constant
martincostello 41c1834
[Extensions] Extend test coverage
martincostello 4d075d5
Merge branch 'main' into gh-3678
martincostello 4ec87a4
[Extensions] Address feedback
martincostello a559073
Merge branch 'main' into gh-3678
martincostello 8e090ec
[Extensions] Reject malformed OpenTelemetry tracestate
Kielek 35f56f3
[Extensions] Enforce tracestate member limit
Kielek ea5805d
[Extensions] Preserve OpenTelemetry tracestate at size limit
Kielek b97af74
Merge branch 'main' into gh-3678
martincostello 19e6de1
[Extensions] Add char polyfills
martincostello File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
243 changes: 243 additions & 0 deletions
243
src/OpenTelemetry.Extensions/Internal/ConsistentProbability.cs
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,243 @@ | ||
| // Copyright The OpenTelemetry Authors | ||
| // SPDX-License-Identifier: Apache-2.0 | ||
|
|
||
| using System.Globalization; | ||
|
|
||
| namespace OpenTelemetry.Extensions.Internal; | ||
|
|
||
| /// <summary> | ||
| /// Helpers for converting between sampling probabilities, 56-bit rejection thresholds and their | ||
| /// hexadecimal <c>th</c>/<c>rv</c> encodings, following the OpenTelemetry | ||
| /// <see href="https://opentelemetry.io/docs/specs/otel/trace/tracestate-probability-sampling/"> | ||
| /// probability sampling</see> and | ||
| /// <see href="https://opentelemetry.io/docs/specs/otel/trace/tracestate-handling/">tracestate handling</see> | ||
| /// specifications. | ||
| /// </summary> | ||
| internal static class ConsistentProbability | ||
| { | ||
| /// <summary> | ||
| /// The maximum number of hexadecimal digits used to encode a 56-bit value. | ||
| /// </summary> | ||
| public const int MaxHexDigits = 14; | ||
|
|
||
| /// <summary> | ||
| /// The default encoding precision recommended by the specification. | ||
| /// </summary> | ||
| public const int DefaultPrecision = 4; | ||
|
|
||
| /// <summary> | ||
| /// <c>2^56</c>, the number of distinct 56-bit values (the maximum adjusted count). | ||
| /// </summary> | ||
| public const long MaxAdjustedCount = 1L << 56; | ||
|
|
||
| /// <summary> | ||
| /// The largest valid randomness value, <c>2^56 - 1</c>. | ||
| /// </summary> | ||
| public const long MaxRandomValue = MaxAdjustedCount - 1; | ||
|
|
||
| /// <summary> | ||
| /// Encodes a sampling probability as a <c>th</c> value using the specified precision. | ||
| /// </summary> | ||
| /// <param name="probability">The sampling probability, in the range <c>(0, 1]</c>.</param> | ||
| /// <param name="precision">The number of significant hexadecimal digits, in the range <c>[1, 14]</c>.</param> | ||
| /// <returns>The threshold encoded with trailing zeros removed (for example <c>fd70a</c>).</returns> | ||
| /// <remarks> | ||
| /// This computes the exact 56-bit rejection threshold directly from the probability, matching the | ||
| /// OpenTelemetry Collector implementation rather than the (less accurate) floating-point reference | ||
| /// pseudocode in the specification, so that values near <c>0</c> and <c>1</c> are encoded exactly: | ||
| /// <see href="https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/6d20534d0a232acaa8cf7161ddbaeab6915e0c01/pkg/sampling/probability.go#L33-L77"> | ||
| /// ProbabilityToThresholdWithPrecision</see>. | ||
| /// </remarks> | ||
| public static string EncodeThreshold(double probability, int precision) | ||
| { | ||
| if (probability >= 1.0) | ||
| { | ||
| // Special case: 100% sampling has a rejection threshold of zero. | ||
| return "0"; | ||
| } | ||
|
|
||
| const int HexBits = 4; // 4 bits per hex digit | ||
|
|
||
| // Raise the precision by the number of leading '0' or 'f' digits so the configured precision | ||
| // applies to the significant digits of the threshold near both 0 and 1. frexp returns an | ||
| // exponent <= 0; every multiple of -4 corresponds to another leading '0' or 'f' hex digit. | ||
| var exponentFraction = FrexpExponent(probability); | ||
| var exponentRejection = FrexpExponent(1.0 - probability); | ||
|
|
||
| precision = Math.Min( | ||
| MaxHexDigits, | ||
| Math.Max(precision + (exponentFraction / -HexBits), precision + (exponentRejection / -HexBits))); | ||
|
|
||
| // Compute the rejection threshold as a 56-bit integer: T = 2^56 - round(probability * 2^56). | ||
| var scaled = (long)Math.Round(probability * MaxAdjustedCount, MidpointRounding.AwayFromZero); | ||
| var threshold = MaxAdjustedCount - scaled; | ||
|
|
||
| // Round to the requested precision by dropping the low hex digits, rounding to nearest. | ||
| var shift = HexBits * (MaxHexDigits - precision); | ||
|
|
||
| if (shift > 0) | ||
| { | ||
| var half = 1L << (shift - 1); | ||
| threshold = ((threshold + half) >> shift) << shift; | ||
| } | ||
|
|
||
| return EncodeThresholdInteger(threshold); | ||
| } | ||
|
|
||
| /// <summary> | ||
| /// Encodes a 56-bit integer rejection threshold as a <c>th</c> value, with trailing zeros removed. | ||
| /// </summary> | ||
| /// <param name="threshold">The rejection threshold, in the range <c>[0, 2^56)</c>.</param> | ||
| /// <returns>The encoded threshold (for example <c>8</c> for 50% sampling).</returns> | ||
| public static string EncodeThresholdInteger(long threshold) | ||
| { | ||
| if (threshold <= 0) | ||
| { | ||
| return "0"; | ||
| } | ||
|
|
||
| const string Format = "x14"; // 14 hex digits, no leading "0x" | ||
|
|
||
| #if NET | ||
| Span<char> buffer = stackalloc char[MaxHexDigits]; | ||
|
|
||
| _ = threshold.TryFormat(buffer, out var written, Format, CultureInfo.InvariantCulture); | ||
|
|
||
| var trimmed = buffer.Slice(0, written).TrimEnd('0'); | ||
|
|
||
| return trimmed.IsEmpty ? "0" : new string(trimmed); | ||
| #else | ||
| var hex = threshold.ToString(Format, CultureInfo.InvariantCulture).TrimEnd('0'); | ||
| return hex.Length == 0 ? "0" : hex; | ||
| #endif | ||
| } | ||
|
|
||
| /// <summary> | ||
| /// Decodes a <c>th</c> value into a 56-bit integer rejection threshold by extending it with | ||
| /// trailing zeros to 14 digits and parsing the result. | ||
| /// </summary> | ||
| /// <param name="threshold">The encoded threshold (1 to 14 lowercase hexadecimal digits).</param> | ||
| /// <returns>The rejection threshold, in the range <c>[0, 2^56)</c>.</returns> | ||
| public static long DecodeThreshold(string threshold) | ||
| { | ||
| _ = TryDecodeThreshold(threshold.AsSpan(), out var value); | ||
| return value; | ||
| } | ||
|
|
||
| /// <summary> | ||
| /// Attempts to decode a <c>th</c> value into a 56-bit integer rejection threshold. | ||
| /// </summary> | ||
| /// <param name="threshold">The encoded threshold (1 to 14 lowercase hexadecimal digits).</param> | ||
| /// <param name="value">The rejection threshold when successful; otherwise zero.</param> | ||
| /// <returns><see langword="true"/> if the value was decoded; otherwise <see langword="false"/>.</returns> | ||
| public static bool TryDecodeThreshold(ReadOnlySpan<char> threshold, out long value) | ||
| { | ||
| if (threshold.IsEmpty || threshold.Length > MaxHexDigits || !TryParseHex56(threshold, out var parsed)) | ||
| { | ||
| value = 0; | ||
| return false; | ||
| } | ||
|
|
||
| // Extend the value with trailing zeros to 14 digits, i.e. shift left by 4 bits per omitted digit. | ||
| var shift = 4 * (MaxHexDigits - threshold.Length); | ||
| value = shift > 0 ? parsed << shift : parsed; | ||
|
|
||
| return true; | ||
| } | ||
|
|
||
| /// <summary> | ||
| /// Parses a lowercase hexadecimal string of 1 to 14 digits into its integer value. | ||
| /// </summary> | ||
| /// <param name="value">The hexadecimal string.</param> | ||
| /// <param name="result">The parsed value when successful; otherwise zero.</param> | ||
| /// <returns><see langword="true"/> if the value was parsed; otherwise <see langword="false"/>.</returns> | ||
| public static bool TryParseHex56(string? value, out long result) | ||
| => TryParseHex56(value.AsSpan(), out result); | ||
|
|
||
| /// <summary> | ||
| /// Parses a lowercase hexadecimal span of 1 to 14 digits into its integer value. | ||
| /// </summary> | ||
| /// <param name="value">The hexadecimal characters.</param> | ||
| /// <param name="result">The parsed value when successful; otherwise zero.</param> | ||
| /// <returns><see langword="true"/> if the value was parsed; otherwise <see langword="false"/>.</returns> | ||
| /// <remarks> | ||
| /// Uppercase digits are rejected: the specification requires both <c>th</c> and <c>rv</c> to be | ||
| /// encoded with lowercase hexadecimal digits, the same as <see cref="System.Diagnostics.ActivityTraceId"/>. | ||
| /// </remarks> | ||
| public static bool TryParseHex56(ReadOnlySpan<char> value, out long result) | ||
| { | ||
| result = 0; | ||
|
|
||
| if (value.IsEmpty || value.Length > MaxHexDigits) | ||
| { | ||
| return false; | ||
| } | ||
|
|
||
| long parsed = 0; | ||
|
|
||
| foreach (var ch in value) | ||
| { | ||
| var digit = ch switch | ||
| { | ||
| >= '0' and <= '9' => ch - '0', | ||
| >= 'a' and <= 'f' => ch - 'a' + 10, | ||
| _ => -1, | ||
| }; | ||
|
|
||
| if (digit < 0) | ||
| { | ||
| return false; | ||
| } | ||
|
|
||
| parsed = (parsed << 4) | (long)digit; | ||
| } | ||
|
|
||
| result = parsed; | ||
| return true; | ||
| } | ||
|
|
||
| /// <summary> | ||
| /// Calculates the sampling probability represented by a rejection threshold. | ||
| /// </summary> | ||
| /// <param name="threshold">The rejection threshold, in the range <c>[0, 2^56)</c>.</param> | ||
| /// <returns> | ||
| /// The sampling probability, in the range <c>(0, 1]</c>. | ||
| /// </returns> | ||
| /// <remarks> | ||
| /// Per the specification: <c>Probability = (MaxAdjustedCount - Threshold) / MaxAdjustedCount</c>. | ||
| /// </remarks> | ||
| public static double ThresholdToProbability(long threshold) | ||
| => (double)(MaxAdjustedCount - threshold) / MaxAdjustedCount; | ||
|
|
||
| /// <summary> | ||
| /// Calculates the adjusted count (inverse sampling probability) for a rejection threshold. | ||
| /// </summary> | ||
| /// <param name="threshold">The rejection threshold, in the range <c>[0, 2^56)</c>.</param> | ||
| /// <returns> | ||
| /// The adjusted count. | ||
| /// </returns> | ||
| /// <remarks> | ||
| /// Per the specification: <c>AdjustedCount = MaxAdjustedCount / (MaxAdjustedCount - Threshold)</c>. | ||
| /// </remarks> | ||
| public static double ThresholdToAdjustedCount(long threshold) | ||
| => (double)MaxAdjustedCount / (MaxAdjustedCount - threshold); | ||
|
martincostello marked this conversation as resolved.
|
||
|
|
||
| /// <summary> | ||
| /// Returns the exponent that <c>math.frexp</c> would produce for a positive value in <c>(0, 1]</c>, | ||
| /// i.e. the value <c>e</c> such that <c>value = m * 2^e</c> with <c>0.5 <= m < 1</c>. | ||
| /// </summary> | ||
| private static int FrexpExponent(double value) | ||
| { | ||
| // value is a positive, normal double in (0, 1] (1.0 arises when 1 - probability rounds up). | ||
| #if NET | ||
| return Math.ILogB(value) + 1; | ||
| #else | ||
| var bits = BitConverter.DoubleToInt64Bits(value); | ||
| var biasedExponent = (int)((bits >> 52) & 0x7FF); | ||
|
|
||
| // frexp normalises the mantissa to [0.5, 1) rather than [1, 2), so its exponent is one | ||
| // greater than the unbiased IEEE-754 exponent (biasedExponent - 1023). | ||
| return biasedExponent - 1022; | ||
| #endif | ||
| } | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Given that the package is beta and that the new public API exposed here is effectively an implementation of a stable abstract class and otherwise only accepts a
doublevalue to set the probability for sampling, this is OK to not be experimental?If the spec were to change that would just be internal implementation details and we'd just ship a new version right?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, since the package is not stable, we do not need to do the experimental API dance.