Add realistic data pattern benchmarks for TSDB codec#140390
Merged
salvatore-campagna merged 46 commits intoelastic:mainfrom Jan 14, 2026
Merged
Conversation
The previous benchmarks only measured the bit-packing step using DocValuesForUtil. This change switches to TSDBDocValuesEncoder to measure the full encoding pipeline including delta encoding, offset removal, GCD compression, and bit packing. Changes: - Replace DocValuesForUtil with TSDBDocValuesEncoder - Add MetricsConfig for setup configuration - Add CompressionMetrics (@AuxCounters) for benchmark metrics reporting - Switch to SampleTime mode for latency distribution - Test at encoding boundaries (1,4,8,9,16,17,24,25,32,33,40,48,56,57,64)
Add comprehensive javadoc documentation to internal benchmark classes: - CompressionMetrics: document all metrics fields, usage pattern, and JMH auxiliary counters integration - MetricsConfig: document configuration parameters and injection pattern - AbstractTSDBCodecBenchmark: document template method pattern and encoding pipeline stages - EncodeBenchmark/DecodeBenchmark: add class-level documentation
MetricsConfig uses @State(Scope.Benchmark), meaning a single instance is shared across all benchmark threads. While JMH ensures @setup completes before @benchmark methods run, the happens-before relationship requires volatile to guarantee visibility of writes to reader threads. Added volatile modifier to all fields with documentation explaining the JMH lifecycle and why volatile is sufficient (no synchronization needed since there is no write contention).
…vadoc Extract the magic number 64 into a named constant EXTRA_METADATA_SIZE with documentation explaining its purpose: buffer headroom for encoding metadata written by TSDBDocValuesEncoder during delta, offset, and GCD compression steps. Also simplify and align javadoc across EncodeBenchmark and DecodeBenchmark for consistency, removing redundant details while keeping essential information about what each class measures.
…arity Rename getEncodedBytes() to getEncodedSize() across the benchmark API to better communicate that this method returns a size value (number of bytes) rather than the bytes themselves. Also rename getEncodedBytesPerBlock() to getEncodedSizePerBlock() in MetricsConfig and CompressionMetrics for consistency.
…y null check We always pass a non-null reference, hence the null check is not needed.
JMH guarantees @teardown(Level.Iteration) runs after benchmark operations complete. Since recordOperation is called on every operation, config will always be set before computeMetrics runs.
…er passing Remove the MetricsConfig intermediary class and simplify the benchmark architecture by passing values directly to CompressionMetrics.recordOperation(). Changes: - Delete MetricsConfig.java entirely - Update CompressionMetrics.recordOperation() to accept blockSize, encodedBytes, and nominalBits parameters directly instead of a MetricsConfig object - Simplify all 8 benchmark classes by removing MetricsConfig from method signatures and passing values directly from the benchmark context This eliminates an unnecessary abstraction layer and makes the data flow more explicit.
…etrics Change metric fields from public to private and expose them via public getter methods for better encapsulation. JMH @AuxCounters supports both public fields and public getters for metric discovery.
Remove throughput metrics that only report raw iteration totals. These cannot be converted to per-operation metrics without breaking the compression efficiency metrics. Keep only the meaningful compression metrics: encodedBytesPerValue, compressionRatio, encodedBitsPerValue, and overheadRatio.
Add ThroughputMetrics class using JMH @AuxCounters with Type.OPERATIONS to track bytes/s and values/s throughput rates. Update all TSDB codec benchmarks to use Mode.Throughput for compatibility with both metric types. Encode benchmarks provide two methods: - throughput() reports encodedBytes and valuesProcessed rates - compression() reports compressionRatio, encodedBitsPerValue, etc. Decode benchmarks provide only throughput() since compression metrics are a property of the encoded data, not the decoding process. This allows running benchmarks selectively: - ./gradlew :benchmarks:jmh -Pjmh.includes='.*Encode.*throughput' - ./gradlew :benchmarks:jmh -Pjmh.includes='.*Encode.*compression' - ./gradlew :benchmarks:jmh -Pjmh.includes='.*Decode.*throughput'
salvatore-campagna
force-pushed
the
benchmarks/tsdb-codec-realistic-data-patterns
branch
2 times, most recently
from
98570e3 to
628c57e
Compare
The TSDB encoder mutates the input array in-place during encoding (subtracting min offset, dividing by GCD, computing deltas). This caused incorrect compression metrics because after the first encoding, subsequent operations encoded already-zeroed data. Changes: - Store original input array and restore via System.arraycopy - Make EncodeBenchmark and DecodeBenchmark final classes since they use composition pattern and are not designed for inheritance
…ration Add method-level @WarmUp(iterations=0) and @measurement(iterations=1) to compression benchmarks. Compression metrics are deterministic since the same input data always produces the same encoded size, unlike throughput measurements which vary due to JIT compilation and CPU state.
salvatore-campagna
force-pushed
the
benchmarks/tsdb-codec-realistic-data-patterns
branch
from
f86ae7c to
4c411be
Compare
Change benchmark setup from Level.Iteration to Level.Trial since the input data is deterministic (fixed seed) and does not need to be regenerated before each iteration. This reduces setup overhead while maintaining correct behavior. The setupInvocation() method continues to restore the input array via System.arraycopy before each benchmark invocation.
salvatore-campagna
force-pushed
the
benchmarks/tsdb-codec-realistic-data-patterns
branch
from
4c411be to
be30ee0
Compare
Java shift semantics cause 1L << 64 to wrap to 1 instead of producing a 64-bit range. This made bitsPerValue=64 benchmarks generate only zeros, skewing results. Use unbounded nextLong() for 64-bit values to get proper full-range random numbers.
Add six new benchmark classes for realistic time series data patterns: - TimestampLike: monotonically increasing with jitter - GaugeLike: oscillating values around a baseline - CounterWithResets: monotonic counters that periodically reset to zero - NearConstant: constant values with occasional outliers - LowCardinality: few distinct values with Zipf distribution - GcdFriendly: values that are multiples of a common divisor Each pattern includes both encode and decode benchmarks with configurable parameters to test codec behavior across different data characteristics.
Add method-level @WarmUp(iterations = 0) and @measurement(iterations = 1) annotations to compression benchmark methods in the realistic pattern encode benchmarks. Compression metrics are deterministic: the same input data always produces the same encoded size. Unlike throughput measurements which vary due to JIT compilation and CPU state, compression ratios are constant across runs.
salvatore-campagna
force-pushed
the
benchmarks/tsdb-codec-realistic-data-patterns
branch
from
ace6fd5 to
0da20cb
Compare
elasticsearchmachine
added
the
needs:triage
salvatore-campagna
added
>non-issue
:StorageEngine/TSDB
Collaborator
|
Pinging @elastic/es-storage-engine (Team:StorageEngine) |
spinscale
pushed a commit
to spinscale/elasticsearch
that referenced
this pull request
Jan 21, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The existing TSDB codec benchmarks rely on synthetic patterns (constants, monotonic sequences, random values) that do not reflect real metrics behavior. This PR adds benchmarks with realistic data patterns to better evaluate compression efficiency and encoding throughput.
Real metrics exhibit characteristic shapes: timestamps advance at near-constant intervals, counters increase and occasionally reset, gauges fluctuate around a baseline, and error metrics are mostly zero with rare spikes. To model this, the PR introduces six data generators:
jitterProbabilityto vary regularityresetProbabilityvarianceRatiooutlierProbabilitygcddistinctValuesandskewEach supplier exposes
getNominalBitsPerValue(), computing the theoretical bit width from the value range (max–min) after offset removal. This allows compression ratios to be compared against a meaningful lower bound rather than a fixed 64-bit baseline.Encode benchmarks expose
throughput()(encoding speed) andcompression()(compression efficiency). Decode benchmarks reportthroughput()only, since compression is an encode-time property.Unit tests validate determinism, value bounds, and nominal bit calculations for all suppliers.
Running the benchmarks