Skip to content

feat(controlplane): always sample batch subgraph publish traces - #2957

Merged
gausie merged 10 commits into
mainfrom
controlplane/full-sample-batch-publish-traces
Jun 12, 2026
Merged

feat(controlplane): always sample batch subgraph publish traces#2957
gausie merged 10 commits into
mainfrom
controlplane/full-sample-batch-publish-traces

Conversation

@gausie

@gausie gausie commented Jun 11, 2026

Copy link
Copy Markdown
Contributor

Problem

Batch subgraph publishes go through the PublishFederatedSubgraphs RPC, which is rare, slow, and frequently hits the ~90s request timeout. In prod, SENTRY_TRACES_SAMPLE_RATE is below 1.0, so the flat tracesSampleRate drops most successful batch publishes — only the timed-out ones reliably show up in Sentry. That makes it hard to track the end-to-end composition cost of a publish over time.

Change

Replace the flat tracesSampleRate with a tracesSampler that:

  • returns 1.0 for the batch publish RPC (/wg.cosmo.platform.v1.PlatformService/PublishFederatedSubgraphs), matched against the span name / http.route / http.target / url.path / url.full;
  • falls back to SENTRY_TRACES_SAMPLE_RATE for everything else (no change for other transactions);
  • honors an upstream sampling decision (parentSampled) so distributed traces stay intact.

Child composition spans (e.g. ComposeGraphsWorker.composeGraphsInWorker) inherit the parent's sampling decision, so the full trace — including per-federated-graph compose timings — is captured.

The forced-sample list is a single constant (ALWAYS_SAMPLE_PATHS), easy to extend to other operations later.

Notes

  • tracesSampler takes precedence over tracesSampleRate; the sampler returns the configured rate as its default, so overall sampling behaviour is unchanged except for the batch publish path.
  • Takes effect after a control plane deploy.

Summary by CodeRabbit

  • Chores
    • Improved distributed tracing: selected RPC paths are now always fully sampled, existing upstream parent/child sampling decisions are preserved when present, and all other traces follow the configured sample rate—resulting in more accurate end-to-end trace coverage and clearer diagnostics for publish-related operations.

Batch publishes (PublishFederatedSubgraphs) are rare, slow, and frequently
hit the request timeout, but the flat SENTRY_TRACES_SAMPLE_RATE drops most of
the successful ones so their end-to-end composition cost can't be tracked.

Add a tracesSampler that forces 100% sampling for the batch publish RPC and
falls back to the configured rate for everything else. Child composition spans
inherit the parent decision, and upstream sampling decisions are preserved.
@gausie
gausie marked this pull request as ready for review June 11, 2026 10:21
@gausie
gausie requested a review from a team as a code owner June 11, 2026 10:21
@coderabbitai

coderabbitai Bot commented Jun 11, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 9c7d851f-9b0d-4468-89f8-84cbf21aaa67

📥 Commits

Reviewing files that changed from the base of the PR and between 070e28a and abf4bab.

📒 Files selected for processing (1)
  • controlplane/src/core/sentry.config.ts
🚧 Files skipped from review as they are similar to previous changes (1)
  • controlplane/src/core/sentry.config.ts

Walkthrough

Sentry distributed-tracing sampling is now customized via a new tracesSampler. The configuration introduces path-based sampling rules and preserves upstream sampling decisions from distributed traces, replacing the previous static tracesSampleRate approach.

Changes

Sentry tracing sampling customization

Layer / File(s) Summary
Distributed-tracing sampling logic and configuration
controlplane/src/core/sentry.config.ts
ALWAYS_SAMPLE_PATHS lists RPC endpoints that require 100% trace sampling. publishAwareTracesSampler checks ctx.name and span attributes (http.route, http.target, url.path, url.full) for substring matches to always-sampled paths, returns 1/0 when ctx.parentSampled is a boolean to preserve upstream sampling, and otherwise falls back to SENTRY_TRACES_SAMPLE_RATE. Sentry.init is updated to use tracesSampler: publishAwareTracesSampler (removing tracesSampleRate).

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

  • wundergraph/cosmo#2905: Updates Sentry configuration to parse sentryEnvVariables, both PRs touch Sentry init and tracing configuration.
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the main change: introducing selective tracing for batch subgraph publish operations to ensure these critical traces are always sampled.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@controlplane/src/core/sentry.config.ts`:
- Around line 28-30: The current check using ctx.parentSampled wrongly returns
SENTRY_TRACES_SAMPLE_RATE when ctx.parentSampled is false; update the sampling
decision so that if typeof ctx.parentSampled === 'boolean' you return 1 when
ctx.parentSampled is true and 0 when false (do not fall back to
SENTRY_TRACES_SAMPLE_RATE), i.e. honor ctx.parentSampled exactly; update the
branch that references ctx.parentSampled and SENTRY_TRACES_SAMPLE_RATE
accordingly so child spans are not sampled when parentSampled === false.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 117d1f6e-006b-4eee-8e49-b21addcf17ec

📥 Commits

Reviewing files that changed from the base of the PR and between ce8f3c5 and 936d085.

📒 Files selected for processing (1)
  • controlplane/src/core/sentry.config.ts

Comment thread controlplane/src/core/sentry.config.ts Outdated
@codecov

codecov Bot commented Jun 11, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 0% with 29 lines in your changes missing coverage. Please review.
✅ Project coverage is 65.63%. Comparing base (9aef4ca) to head (1da4058).
⚠️ Report is 3 commits behind head on main.

Files with missing lines Patch % Lines
controlplane/src/core/sentry.config.ts 0.00% 29 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##             main    #2957       +/-   ##
===========================================
+ Coverage   41.64%   65.63%   +23.99%     
===========================================
  Files        1053      329      -724     
  Lines      133865    47321    -86544     
  Branches     6472     5300     -1172     
===========================================
- Hits        55742    31059    -24683     
+ Misses      76332    16238    -60094     
+ Partials     1791       24     -1767     
Files with missing lines Coverage Δ
controlplane/src/core/sentry.config.ts 0.00% <0.00%> (ø)

... and 725 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

gausie added 3 commits June 11, 2026 13:56
When an upstream span opted out of sampling (parentSampled === false), the
sampler fell back to SENTRY_TRACES_SAMPLE_RATE instead of 0, which can leave
orphaned child spans in a trace whose root was not sampled.

Check the always-sample paths first, then honor parentSampled exactly
(true -> 1, false -> 0), and only fall back to the configured rate when there
is no upstream decision.

@comatory comatory left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am worried that this code is in the hot path and I'd try to optimize here. Maybe I'm paranoid? I know we can't be sure if we don't benchmark but still... wDYT?

Comment thread controlplane/src/core/sentry.config.ts Outdated
Comment thread controlplane/src/core/sentry.config.ts Outdated
@gausie
gausie requested a review from comatory June 11, 2026 13:47
@gausie
gausie enabled auto-merge (squash) June 12, 2026 13:42
@gausie
gausie merged commit 2665d95 into main Jun 12, 2026
10 checks passed
@gausie
gausie deleted the controlplane/full-sample-batch-publish-traces branch June 12, 2026 14:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants