Skip to content

✨ Add native OpenTelemetry support - #16403

Merged
tiangolo merged 6 commits into
masterfrom
codex/native-opentelemetry-support
Sep 29, 2026
Merged

tiangolo merged 6 commits into
masterfrom
codex/native-opentelemetry-support

Conversation

@tiangolo

@tiangolo tiangolo commented Sep 26, 2026 •

Copy link
Copy Markdown
Member

✨ Add native OpenTelemetry support

This is designed from the beginning considering OpenTelemetry's contrib current implementation, Logfire, and Sentry.

Several features and data extraction parts in this implementation are there so that these tools can get the data they need, but would currently acquire by monkeypatching FastAPI.

Ideally, no third-party package should need/want to patch FastAPI internals (I'll change and refactor a lot of those internals soon 😅).


2026-09-29 update: checked with Sentry and Logfire, this doesn't block or limit their work, we can go with this. A follow up is probably worth with OpenTelemetry contrib to make use the new built-in integration.

AI Disclaimer

Made with the help of AI, using Codex with gpt-6-astra, manually reviewed.

@tiangolo tiangolo added the feature New feature or request label Sep 26, 2026
@codspeed

codspeed Bot commented Sep 26, 2026 •

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 24 untouched benchmarks


Comparing codex/native-opentelemetry-support (daba280) with master (79d42b2)1

Open in CodSpeed

Footnotes

  1. No successful run was found on master (c30032a) during the generation of this report, so 79d42b2 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

@github-actions

github-actions Bot commented Sep 26, 2026 •

Copy link
Copy Markdown
Contributor

@tiangolo
tiangolo marked this pull request as ready for review September 29, 2026 16:27
@tiangolo
tiangolo merged commit 4b3949c into master Sep 29, 2026
53 checks passed
@tiangolo
tiangolo deleted the codex/native-opentelemetry-support branch September 29, 2026 17:25
@claudiu7t

Copy link
Copy Markdown

People can configure OpenTelemetry for other purposes that have nothing to do with FastAPI. For example, I use it with Arize Phoenix to collect LLM/agent traces. I was surprised and bewildered this morning to see API calls in my Arize instance. I understand that I can disable FastAPI's traces, but in my opinion this should be an opt-in, not opt-out.

yaoyu-33 pushed a commit to yaoyu-33/Gym that referenced this pull request Oct 8, 2026
NVIDIA-NeMo#4216)

## Problem

FastAPI 0.142.0 to 0.142.4 ([release
notes](https://github.com/fastapi/fastapi/releases/tag/0.142.0),
[fastapi/fastapi#16403](fastapi/fastapi#16403))
configure OpenTelemetry export by themselves at ASGI startup whenever
`OTEL_EXPORTER_OTLP_ENDPOINT` is set. Gym sets that variable for every
server process so that nemo-lens finds the collector, and nemo-lens has
already registered providers with their own exporters. FastAPI attaches
a second `BatchSpanProcessor` and a second
`PeriodicExportingMetricReader` to those providers. Gym does not pin
FastAPI, so every job environment resolved since 2026-09-29 picked this
up.

Observed in every gym-native run since then:

- every span is exported twice: each trace appears twice in Tempo and
the collector's span-metrics counts are doubled;
- every metric is exported by two readers with separate aggregation
state, so the cumulative `_sum` of the Gym histograms carries last-digit
dips that Prometheus treats as counter resets; `increase()`/`rate()` on
a sum then adds the whole running value (mean rollout duration showed
hours instead of minutes, 60 of 60 runs affected).

## Change

FastAPI 0.143.0
([fastapi/fastapi#16476](fastapi/fastapi#16476),
from [discussion
#16473](fastapi/fastapi#16473)) makes the
automatic exporter setup opt-in, so only the 0.142 series
double-exports. This PR excludes that series in `pyproject.toml`
(`fastapi!=0.142.*`) and refreshes `uv.lock`; the locked version stays
0.135.2. An earlier revision wrapped the six `FastAPI()` sites in a
factory that switched the native telemetry off; the exclusion replaces
it because it also covers the bare `FastAPI()` apps outside the server
bases and needs nothing from future servers.

Environments already built with FastAPI 0.142 keep exporting twice until
they are reinstalled.

## Verification

A gym-native run with the factory revision as the job-side Gym
(terminal-bench miniswe, 8 rollouts, 1 node) showed one
`gym.rollout`/`gym.verify` span per rollout, one metric datapoint per 10
s with consistent sums, and
`resets(gym_rollout_duration_ms_milliseconds_sum[1d]) = 0`, against 2
spans per rollout and up to 684 resets per run on `main`. The exclusion
produces the same single export by resolving FastAPI to 0.143.0 instead
of switching the feature off.

Signed-off-by: Ingo Guehring <iguehring@nvidia.com>
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
yashverma2110 added a commit to yashverma2110/codesloth that referenced this pull request Oct 11, 2026
…Try demo

- GitHub-style accessible tabs (tablist/tab/tabpanel, arrow/Home/End), remembered
- Graph view: layered SVG layout (no deps, no innerHTML), import + test edges,
  layer legend/filter, search, pan/zoom, hover neighbours, click-to-jump,
  generated collapsed, directory clustering for big PRs, Go package/cycle boxes,
  expanded overlay and full-tab graph page (plan from SW cache, no token)
- Review: per-PR reviewed snapshots with blob SHAs -> 'changed since reviewed' flag
- First-run 5-step coach-mark tour, '?' help menu, Try demo with bundled
  real plan (fastapi/fastapi#16403), works without a token
- Tests: graph build/layout, review state, tour placement, widget DOM (170 total)
- README: How to use the Review tab, Graph view, headless renders
yashverma2110 added a commit to yashverma2110/codesloth that referenced this pull request Oct 11, 2026
- Background: read-only GETs for PR commits (cap 250) and each commit's files,
  per-file history with rename following, merge commits skipped, soft cap of
  the latest 100 commits, quota reserve, cached by owner/repo/number/headSha.
- Review tab: N commits chip, Latest line, selected-file timeline (initials,
  relative time, per-commit +/-, linked short SHA, newest tag), step line
  'Touched in X of Y commits' with dots, After review / Late change chips.
  Loaded lazily; failures only add a note to the menu. Hidden for 1-commit PRs.
- Demo: bundled real commit story for fastapi/fastapi#16403.
- Headings: duplicated file names get their folder (Then services/x.go), longer
  folder paths tried before numbering.
- Panel reveal scrolls only the Review pane (never the panel frame).
- Tests for history building, renames, merges, signals, caps, fetch, headings.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feature New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants