[nv-yna]

Dynamo recipes currently require explicit lifecycle and OTLP environment settings plus a separately configured collector to save native request-lifecycle traces. Enable this capture automatically for the Dynamo stack, including when observability.enabled is false or omitted.

• Imply a managed, host-native OTel collector before Dynamo workers start; retain it until serving processes have shut down and flushed traces. Save OTLP JSONL under logs/otel/ for DSight.
• Supply lifecycle/export defaults to Dynamo frontends and workers, preserving explicit environment and external-endpoint overrides. Direct engine stacks receive no new default collector or lifecycle environment.
• Install checksum-verified collector v0.146.1 with make setup ARCH=<compute_arch>; show it in dry-run output. Document service overrides and update the affected launch snapshots.

Requires a Dynamo build containing native lifecycle instrumentation (ai-dynamo/dynamo#14101). This does not upgrade a recipe's pinned Dynamo version. Existing installations need make setup to install the new collector binary.

Validation: make check passed (4,437 tests), a real OTLP/gRPC export and shutdown-flush smoke test passed, and all PR CI checks passed.

Two successful AgentX runs on Lyris GB200 (Qwen3.5-9B, SGLang TP2, AIPerf 0.12.0, Dynamo 1.6.0.dev20261007), each with a 30-minute walltime limit. Both recipes omit enable_otel, otel_endpoint and DYN_LIFECYCLE_TRACE_ENABLED:

observability.enabled	Job	Scheduler result	Measured client → lifecycle join
false	3316476	COMPLETED 0:0,9m28s	7/7 requests,70 spans
true	3316539	COMPLETED 0:0,11m46s	6/6 requests,60 spans

Both runs reported zero client errors. Real Chromium browser checks passed for both portable DSight reports: lifecycle expansion, TTFT zoom, native worker admission/dispatch/streaming activities, pinned GPU metrics (402 metric families), and zero JavaScript errors. The enabled case also rendered aligned frontend Nsight ranges beside the selected request and OTel spans: two profile catalogs imported, 31 selected frontend NVTX ranges, 5 ranges in the checked TTFT window. The worker profile had no selected DSight ranges; CPU samples were not collected.

Validation limitation: the initial enabled run 3316479 completed its requests but timed out during Nsight CPU-capture shutdown. The successful retry retained NVTX profiling with observability.nsys.cpu_sampling: none and a 120s report timeout. These are short diagnostic AgentX runs (submission_valid=false), not a performance or overhead comparison. No profiler implementation changes are included.

Run artifacts and per-run reports: /lustre/fsw/coreai_comparch_aarwlt/yna/lifecycle-otel-default-20261012/outputs/{3316476,3316539}/. Reproduction script, browser assertions/screenshots and saved report views: /home/yna/code/tmp_context/observability/dsight/20261012-always-on-lifecycle-otel/.

[codecov-commenter]

Codecov Report

❌ Patch coverage is 90.76923% with 6 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (main@0019e8a). Learn more about missing BASE report.

Files with missing lines	Patch %	Lines
src/srtctl/services/otel.py	89.28%	6 Missing ⚠️

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #598   +/-   ##
=======================================
  Coverage        ?   85.23%           
=======================================
  Files           ?      162           
  Lines           ?    23247           
  Branches        ?        0           
=======================================
  Hits            ?    19814           
  Misses          ?     3433           
  Partials        ?        0           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[github-actions]

src/srtctl diff: +143 -11