Avoid OtapPdata::num_items in batch_processor#2885
Merged
jmacd merged 11 commits intoopen-telemetry:mainfrom May 8, 2026
Merged
Avoid OtapPdata::num_items in batch_processor#2885jmacd merged 11 commits intoopen-telemetry:mainfrom
jmacd merged 11 commits intoopen-telemetry:mainfrom
Conversation
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #2885 +/- ##
========================================
Coverage 86.17% 86.18%
========================================
Files 714 714
Lines 270290 270546 +256
========================================
+ Hits 232929 233168 +239
- Misses 36837 36854 +17
Partials 524 524
🚀 New features to boost your workflow:
|
JakeDern
approved these changes
May 7, 2026
utpilla
reviewed
May 7, 2026
lalitb
reviewed
May 7, 2026
Contributor
|
In general, I think the intention behind this change is related to issue #2859. If the
|
Contributor
Author
|
@drewrelmas I filed #2884 to talk about the same topic -- I would not add items/bytes counters to the flow measurement/stopwatch thing, I would add them to individual nodes, allow us to enable on a node-by-node basis at detailed metric level. |
Contributor
|
@drewrelmas Determining the number of signals/items is only costly for pdata with pure OTLP protobytes backend. For regular OTAP based pdata the cost is probably negligeable. |
jmacd
added a commit
to jmacd/otel-arrow
that referenced
this pull request
May 8, 2026
…metry#2885 The batch_processor num_items/weight refactor and the matching liveness-test assertion update were extracted into upstream PR open-telemetry#2885. Restore both files to upstream/main state on this branch so the changes flow back in via the upstream merge rather than living in two places. Files restored to upstream/main: - crates/core-nodes/src/processors/batch_processor/mod.rs - crates/otap/tests/core_node_liveness_tests.rs Verified: cargo build (profiling, df_engine + obsday_logger) clean; batch_processor unit tests 42/42 pass. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Merged
via the queue into
open-telemetry:main
with commit May 8, 2026
c19ac24
84 of 85 checks passed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Change Summary
Fixes #2882. Uses the sizer's natural weight which is O(1) for batch processor bookkeeping. OTLP batching pays a penalty otherwise, due to an (not strictly necessary) allocation in addition to the traversal.