test(sglang): cover disaggregated OTLP tracing

[knarangN]

Overview:

Centralize OTLP span-attribute and engine-role parsing used by the existing frontend tracing tests.

Details:

• add reusable helpers for typed OTLP span attributes
• add a shared helper for extracting engine generate roles
• update the unified-worker OTLP test to use the shared helpers
• keep the final change focused on supported tracing coverage

Where should reviewer start?

1. tests/utils/otel.py for the shared OTLP helpers
2. tests/frontend/test_unified_worker_otlp_export.py for their use in the existing test

Related Issues

• No linked GitHub issue
• Internal tracking: OPS-7596

Validation

• Current head: 0e3d4beabe17ad33387afa6ed10a20a5c6182675
• Protected workflow run 30029349218 completed successfully with 40 successful and 30 skipped jobs.
• Current PR checks showed 54 passing and 41 skipped checks when refreshed.
• Runtime evidence boundary: the final diff does not add a native disaggregated SGLang end-to-end test; that coverage remains deferred until the supported gRPC path is available.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[knarangN]

/ok to test f4fe8c3

[knarangN]

/ok to test b2f44e6

[knarangN]

/ok to test 704d836

[devin-ai-integration]

Devin Review found 2 potential issues.

[coderabbitai]

Walkthrough

Changes

OTLP span validation

Layer / File(s)	Summary
OTLP collector and role helpers tests/utils/otel.py	The in-process collector records service names, exposes span attributes, and provides engine-generation role polling helpers.
Frontend OTLP assertion migration tests/frontend/test_unified_worker_otlp_export.py	Frontend tests use shared OTEL attribute and role helpers instead of local extraction logic.
Disaggregated SGLang OTLP coverage tests/serve/test_sglang.py	Adds a disaggregated SGLang OTLP test that validates prefill and decode spans and their worker service names.

Estimated code review effort: 3 (Moderate) | ~25 minutes

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Title check	✅ Passed	The title clearly matches the main change: adding disaggregated OTLP tracing coverage for SGLang tests.
Description check	✅ Passed	The description includes Overview, Details, reviewer start points, and a no-linked-issue declaration, so it is mostly complete.

---

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@tests/serve/test_sglang.py`:
- Around line 28-31: Update the assertions in test_sglang.py, including the
additional affected range, to validate each engine.generate span as an expected
(service.name, disagg_role) pair rather than comparing service names and roles
as independent sets. Reuse the existing wait_for_engine_generate_count and
wait_for_engine_generate_roles helpers where appropriate, preserving the
expected prefill/decode role-to-service mapping.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 82c93965-6938-4281-87e4-75430b19bba0

📥 Commits

Reviewing files that changed from the base of the PR and between d168e63 and 704d836.

📒 Files selected for processing (3)

• tests/frontend/test_unified_worker_otlp_export.py
• tests/serve/test_sglang.py
• tests/utils/otel.py

[knarangN]

/ok to test 37c57ec

[knarangN]

/ok to test 4fb4124

[knarangN]

/ok to test 09ddca6

[knarangN]

/ok to test ab67a74

[knarangN]

/ok to test 213b1db

[knarangN]

Follow-up on the --unified review:

I removed the disaggregated SGLang OTLP E2E from this PR and restored the sidecar launcher. Validation showed that the legacy dynamo.sglang path serves requests but does not emit the common-backend engine.generate spans, while the native sidecar path cannot run in the supported CI image because its SGLang build rejects --grpc-port. The sidecar disaggregated launcher also depends on upstream sgl-project/sglang#25185 plus forwarding support; that change is currently closed and unmerged.

The SGLang E2E should be reintroduced after native disaggregated gRPC support is available in the supported runtime image. The remaining PR diff is limited to shared OTLP attribute/role helpers used by existing frontend tests.

Validation evidence:

• legacy path: https://github.com/ai-dynamo/dynamo/actions/runs/29964327641/job/89074077145
• native sidecar packaging attempt: https://github.com/ai-dynamo/dynamo/actions/runs/29966874961/job/89080675436
• packaged sidecar with current SGLang runtime: https://github.com/ai-dynamo/dynamo/actions/runs/30027481113/job/89276944811

[knarangN]

/ok to test 0e3d4be

[nv-tusharma]

LGTM