Skip to content

Conversation

@p-datadog
Copy link
Member

@p-datadog p-datadog commented Nov 24, 2025

What does this PR do?
Implements chunking of dynamic instrumentation snapshot payloads to stay under the 10 MB intake limit.

Motivation:
Without chunking, if a probe (or multiple probes) produce many events, all of the events will be dropped by the intake.

Change log entry
Yes: dynamic instrumentation: fix sending large quantities of snapshots

Additional Notes:
There is also "snapshot pruning" which is supposed to be done to reduce the size of individual snapshot events below the 1 MB limit for a "log line". This is not in scope for this PR (or Ruby Live Debugger GA).

This PR ensures that if many events are generated by Live Debugger, they all get sent to the backend, but does not help with huge individual snapshots getting dropped.

This PR does however add diagnostics when snapshots are dropped due to excessive size.

How to test the change?
Unit tests added

@github-actions
Copy link

github-actions bot commented Nov 24, 2025

Thank you for updating Change log entry section 👏

Visited at: 2025-11-24 18:05:19 UTC

@pr-commenter
Copy link

pr-commenter bot commented Nov 24, 2025

Benchmarks

Benchmark execution time: 2025-12-01 21:28:30

Comparing candidate commit 5390e30 in PR branch di-chunk with baseline commit 18e70e4 in branch master.

Found 1 performance improvements and 1 performance regressions! Performance is the same for 42 metrics, 2 unstable metrics.

scenario:profiling - Allocations ()

  • 🟩 throughput [+239921.346op/s; +276924.446op/s] or [+8.071%; +9.315%]

scenario:tracing - Propagation - Trace Context

  • 🟥 throughput [-4060.371op/s; -3952.700op/s] or [-10.860%; -10.572%]

@p-datadog p-datadog changed the title DI: chunk snapshot payloads DEBUG-3558 DI: chunk snapshot payloads Nov 24, 2025
@github-actions
Copy link

github-actions bot commented Nov 24, 2025

Typing analysis

Note: Ignored files are excluded from the next sections.

Untyped methods

This PR introduces 4 untyped methods, and clears 3 untyped methods. It decreases the percentage of typed methods from 54.94% to 54.93% (-0.01%).

Untyped methods (+4-3)Introduced:
sig/datadog/di/transport/input.rbs:37
└── def initialize: (untyped apis, untyped default_api, logger: untyped) -> void
sig/datadog/di/transport/input.rbs:39
└── def current_api: () -> untyped
sig/datadog/di/transport/input.rbs:41
└── def send_input: (untyped payload, untyped tags) -> untyped
sig/datadog/di/transport/input.rbs:43
└── def send_input_chunk: (untyped chunked_payload, untyped serialized_tags) -> untyped
Cleared:
sig/datadog/di/transport/input.rbs:34
└── def initialize: (untyped apis, untyped default_api, logger: untyped) -> void
sig/datadog/di/transport/input.rbs:36
└── def current_api: () -> untyped
sig/datadog/di/transport/input.rbs:38
└── def send_input: (untyped payload, untyped tags) -> untyped

If you believe a method or an attribute is rightfully untyped or partially typed, you can add # untyped:accept to the end of the line to remove it from the stats.

@datadog-official
Copy link

datadog-official bot commented Nov 24, 2025

✅ Tests

🎉 All green!

❄️ No new flaky tests detected
🧪 All tests passed

🎯 Code Coverage
Patch Coverage: 93.98%
Total Coverage: 95.16% (+0.00%)

View detailed report

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 5390e30 | Docs | Datadog PR Page | Was this helpful? Give us feedback!

@p-datadog p-datadog marked this pull request as ready for review November 24, 2025 18:31
@p-datadog p-datadog requested a review from a team as a code owner November 24, 2025 18:31
@p-datadog p-datadog requested review from a team as code owners November 26, 2025 20:57
@p-datadog p-datadog requested a review from mabdinur November 26, 2025 20:57
@github-actions github-actions bot added core Involves Datadog core libraries tracing labels Nov 26, 2025
@p-datadog
Copy link
Member Author

I got the limits wrong in the initial implementation of this PR, this has now been corrected.

p and others added 2 commits November 27, 2025 11:51
* master:
  Transports: DRY HTTP client code (#5095)
  DI: extract instance_double_agent_settings_with_stubs to DRY tests (#5087)
  [PROF-13115] Fix profiler ractor specs failing on Ruby 4
  [PROF-13115] Bootstrap installing dependencies on Ruby 4.0.0-preview2
  Clarify support for `rb_obj_info` and why it's OK to not have it
  Tweak pending to not apply to all Ruby preview versions
  Do not try to use `rb_obj_info` on Ruby 4.0
  Adjust stack collector spec to account for changed Ruby 4 behavior
  [PROF-13115] Disable heap profiling on Ruby 4 preview due to incompatibility
  Stub sampling in integration tests
  Rewrite security response tests
  Bump the gh-actions-packages group across 3 directories with 13 updates
Copy link
Member

@Strech Strech left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have a tiny suggestion over performance of the filter_map

Copy link
Member

@Strech Strech left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've left the suggestion about the filter_map path when it's not lazy enumerator

@p-datadog p-datadog merged commit 895d80e into master Dec 2, 2025
553 checks passed
@p-datadog p-datadog deleted the di-chunk branch December 2, 2025 15:10
@github-actions github-actions bot added this to the 2.23.0 milestone Dec 2, 2025
p-datadog pushed a commit that referenced this pull request Dec 8, 2025
* master: (523 commits)
  DI: relax upper bound for reported long method duration (#5113)
  Change development guide to use Ruby 3.4 by default (#5112)
  Fix exception in RC on OpenFeature::Engine absence
  [🤖] Update Latest Dependency: https://github.com/DataDog/dd-trace-rb/actions/runs/19996135509
  [🤖] Update System Tests: https://github.com/DataDog/dd-trace-rb/actions/runs/19996155319
  Add ruby guikld to all dsm files
  add DSM to codeowners
  feat: add process tags to traces (#5033)
  DEBUG-3558 DI: chunk snapshot payloads (#5086)
  CI: enable push_to_test_optimization unless this is a pull request from a fork
  fix steep checks
  [🤖] Lock Dependency: https://github.com/DataDog/dd-trace-rb/actions/runs/19827679824
  [🤖] Lock Dependency: https://github.com/DataDog/dd-trace-rb/actions/runs/19827296218
  Add test to verify consumer backlog serialization with kafka_commit tag
  Remove one diagnostic output
  Install rubygems
  Add diagnostic steps
  Use default rubygems+bundler for each Ruby version
  Revert "Disable profiling specs for 4.0"
  Use git `datadog-ruby_core_source` having 4.0
  ...
@Strech Strech mentioned this pull request Dec 11, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

core Involves Datadog core libraries tracing

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants