[michaelzhang-ai]

What

Pass telemetry={"tracing": False, "metrics": False, "logs": False, "auto_configure": False} to FastAPI() on the main, encoder and diffusion HTTP servers. The new helper fastapi_telemetry_disabled() in observability/trace.py returns these arguments, and returns nothing on FastAPI versions without the telemetry parameter.

Why

FastAPI 0.142.0 (fastapi/fastapi#16403, released 2026-09-29) turned on native OpenTelemetry by default. It now records POST /generate, fastapi.dependencies, fastapi.endpoint and fastapi.serialization spans through the global tracer provider. When tracing is enabled, SGLang installs that same provider for its own request tracing, so FastAPI's spans get exported whatever /set_trace_level says. fastapi is unpinned, so CI picked up 0.142.2 on 2026-09-30, and observability/test_tracing.py::test_trace_level_0 has failed since then:

AssertionError: 6 != 0 : Spans collected but expected none: ['GET /set_trace_level', 'POST /generate', 'fastapi.dependencies', 'fastapi.endpoint']

Example: https://github.com/sgl-project/sglang/actions/runs/37395620023/job/112050816802

FastAPI's environment auto-configuration would also add a second OTLP exporter to SGLang's provider whenever OTEL_EXPORTER_OTLP_* is set. This PR turns all four settings off, which puts the servers back to their behavior before 0.142.

Test plan

• Standalone repro on FastAPI 0.142.2 with an in-memory span exporter on the global provider. Default app: ['POST /generate', 'fastapi.dependencies', 'fastapi.endpoint', 'fastapi.serialization']. With fastapi_telemetry_disabled(): no spans.
• FastAPI 0.141.1: the helper returns {}, so the app is constructed exactly as before.
• observability/test_tracing.py on MI300 in stage-b-test-1-gpu-small-amd part 8: passed ("passed": true, "elapsed": 202), no spans collected at trace level 0.
• Control: the same partition on a branch without this change still fails with AssertionError: 6 != 0 : Spans collected but expected none: ['GET /set_trace_level', 'POST /generate', 'fastapi.dependencies', 'fastapi.endpoint'] (run).

---

CI States

Latest PR Test (Base): ❌ Run #38317178141
Latest PR Test (Extra): ❌ Run #38317177695
Latest PR Test (AMD ROCm 10): ❌ Run #38317177892

[michaelzhang-ai]

/tag-and-rerun-ci

[michaelzhang-ai]

/rerun-failed-ci

[yichiche]

@michaelzhang-ai

Affected Scope
Line: python/sglang/srt/entrypoints/http_server.py:483
Why: This line spreads a telemetry-disabled dict into FastAPI() on every backend, so NVIDIA stops exporting the native request spans the previous constructor recorded by default on FastAPI 0.142.
Fix: Remove **fastapi_telemetry_disabled() from the unguarded FastAPI() calls at http_server.py:483, encoder http_server.py:104, and multimodal create_app line 434 so NVIDIA keeps the previous constructor arguments and span export.

[michaelzhang-ai]

Thanks for looking. This is now superseded: #43021 (merged 10/13) landed the same change on main at five call sites, including the three here, so I'm closing this one.

On NVIDIA, though, the issue isn't backend-specific. FastAPI emits these spans into whatever global tracer provider exists, and --enable-trace installs one on any backend. #43021's own validation shows test_trace_level_0 failing on FastAPI 0.143 with 5 FastAPI spans exported, and NVIDIA CI currently runs 0.143.0. FastAPI also had no native telemetry before 0.142.0, so turning it off restores the earlier constructor behavior rather than dropping spans sglang used to export.

If FastAPI's request spans are wanted when tracing is on, gating them on the trace level (FastAPI's exclude hook takes the ASGI scope) would be a better axis than the backend — but that'd be a change to what's on main now.