fix(gateway): keep the Prometheus /metrics Mount in the gateway route trim

[yassin-berriai]

Relevant issues

Fixes #30291

Linear ticket

Resolves LIT-4236

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Both runs use the componentized gateway entrypoint with the same config, no load balancer or ingress in the path (this is what a kubectl port-forward of the gateway service sees):

model_list:
  - model_name: gpt-5.4-mini
    litellm_params:
      model: openai/gpt-5.4-mini
      api_key: os.environ/OPENAI_API_KEY

litellm_settings:
  success_callback: ["prometheus"]
  require_auth_for_metrics_endpoint: false

uvicorn gateway.main:app --host 127.0.0.1 --port 4236

Before the fix (base litellm_internal_staging), /metrics 404s while /health works, which is the exact symptom from the issue:

$ curl -sL -w "\nHTTP %{http_code}\n" http://127.0.0.1:4236/metrics
{"detail":"Not Found"}
HTTP 404

$ curl -s -o /dev/null -w "HTTP %{http_code}\n" http://127.0.0.1:4236/health/liveliness
HTTP 200

Control on the same base commit and config: the monolithic entrypoint (uvicorn litellm.proxy.proxy_server:app) serves /metrics with HTTP 200, confirming the regression is specific to the gateway trim

After the fix, on the same gateway entrypoint, /metrics serves the Prometheus exposition and the management/UI surfaces stay trimmed:

$ curl -sL -w "HTTP %{http_code}\n" http://127.0.0.1:4236/metrics | head -4
# HELP python_gc_objects_collected_total Objects collected during gc
# TYPE python_gc_objects_collected_total counter
python_gc_objects_collected_total{generation="0"} 2465.0
HTTP 200

$ curl -s -o /dev/null -w "HTTP %{http_code}\n" http://127.0.0.1:4236/ui
HTTP 404

$ curl -s -o /dev/null -w "HTTP %{http_code}\n" -X POST http://127.0.0.1:4236/key/generate
HTTP 404

End to end with a real provider call through the fixed gateway, the per-user request metrics the endpoint exists for show up on the scrape:

$ curl -s http://127.0.0.1:4236/v1/chat/completions \
    -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" \
    -d '{"model":"gpt-5.4-mini","messages":[{"role":"user","content":"Say metrics-ok and nothing else"}]}'
-> "metrics-ok", usage: {"prompt_tokens": 12, "completion_tokens": 5}

$ curl -sL http://127.0.0.1:4236/metrics | grep -E "^litellm_" | head -3
litellm_spend_metric_total{api_provider="openai",model="gpt-5.4-mini",requested_model="gpt-5.4-mini",user="default_user_id",...} 3.15e-05
litellm_total_tokens_metric_total{api_provider="openai",model="gpt-5.4-mini",...} 17.0
litellm_requests_metric_total{api_provider="openai",model="gpt-5.4-mini",...} 1.0

Independent e2e run

Reproduced independently on the componentized gateway entrypoint with the config and commands above. BEFORE is the merge-base of this branch with litellm_internal_staging (git merge-base origin/litellm_internal_staging origin/litellm_gateway_metrics_mount, commit 5b93ba0), where _is_gateway_route drops every Mount; AFTER is this branch (ad0a456), where GATEWAY_MOUNT_PATHS keeps /metrics. Both ran uvicorn gateway.main:app, BEFORE on port 4000 and AFTER on port 4001

Before and after curl sequence against the running gateway:

BEFORE, on the merge-base, /metrics is 404 on the gateway even though /health/liveliness is 200:

AFTER, on this branch, /metrics returns 200 with the Prometheus exposition text (# HELP python_gc_...), and /ui plus POST /key/generate stay 404:

The regression tests in tests/test_litellm/proxy/test_component_allowlists.py pass on the branch, 9 passed:

One environment note for anyone reproducing this locally: FastAPI 0.137+ changed app.include_router to attach a lazy _IncludedRouter object that has no .path, so on those versions the lifespan trim drops the whole included-route surface (health, models, chat, everything), not just Mounts, and the fixed /metrics cannot even be observed because nothing else is served. This run pinned fastapi==0.136.3, the lower bound of the fastapi>=0.136.3,<1.0 range in pyproject.toml, where include_router still flattens routes so the trim behaves as designed. A reasonable follow-up would be to make _is_gateway_route recurse into _IncludedRouter so the gateway keeps working on newer FastAPI, since the resolved lockfile currently picks 0.139.0

Type

🐛 Bug Fix

Changes

The componentized gateway entrypoint (gateway/main.py) trims the shared proxy route table at startup. Its _is_gateway_route predicate rejected every starlette Mount before consulting the allowlist, but Prometheus registers /metrics as a Mount via app.mount("/metrics", make_asgi_app()) (litellm/integrations/prometheus.py). The trim runs inside the wrapped lifespan after the proxy's startup hooks have mounted /metrics, so the route was created and then deleted, and the gateway returned 404 even though /metrics is listed in gateway/routes/allowlist.py and the helm ingress routes /metrics to the gateway service

The fix mirrors the pattern the backend component already uses for its /swagger Mount (BACKEND_MOUNT_PATHS): gateway/routes/allowlist.py gains a GATEWAY_MOUNT_PATHS frozenset containing /metrics, and the Mount branch of _is_gateway_route now keeps Mounts whose path is in that set instead of returning False unconditionally. UI static mounts (/ui, /_next, /litellm-asset-prefix/_next) and /swagger remain trimmed

Tests in tests/test_litellm/proxy/test_component_allowlists.py previously excluded Mounts from the union-coverage assertion, which is why CI never saw this. The new tests import the real _is_gateway_route (restoring the lifespan wrapper and DATABASE_* env vars its import mutates) and assert that a production-shaped /metrics Mount survives the trim, that UI and swagger Mounts are still dropped, and that every Mount on the app is assigned to the gateway, the backend, or the UI container. The core regression test fails on the pre-fix code

Link to Devin session: https://app.devin.ai/sessions/90969e1591404372a4a9f57fcad4e023

[yassin-berriai]

@greptileai

[greptile-apps]

Greptile Summary

This PR fixes a regression in the componentized gateway entrypoint where the Prometheus /metrics endpoint returned 404. Starlette registers /metrics as a Mount (via app.mount()), and the old _is_gateway_route predicate rejected all Mount objects unconditionally before checking the allowlist. Since the route trim runs after the proxy lifespan mounts /metrics, the endpoint was created and then immediately deleted.

• gateway/routes/allowlist.py gains GATEWAY_MOUNT_PATHS = frozenset({"/metrics"}), mirroring the existing BACKEND_MOUNT_PATHS pattern used for /swagger. gateway/main.py consults this set for Mount instances instead of always returning False.
• Four new tests cover the regression directly: one asserts the /metrics Mount survives _is_gateway_route, one asserts UI/swagger Mounts are still dropped, one validates the constant exists, and one asserts every app Mount is assigned to at least one component. Module-level test setup safely imports _is_gateway_route by snapshotting/restoring the lifespan wrapper and DB env vars that gateway.main mutates at import time.

Confidence Score: 4/5

The fix is isolated to the gateway route-trim predicate and allowlist; no production request path, auth layer, or database access is touched.

The core change is a one-line predicate update and a new frozenset constant — both easy to verify and matching the pattern already established for the backend. The new tests exercise the regression directly and the module-level test setup is carefully documented. The only finding is a stale docstring on a helper that is not called from any new code paths.

No files require special attention; test_component_allowlists.py has a minor docstring inconsistency in _component_paths but it does not affect test correctness.

Important Files Changed

Filename	Overview
gateway/main.py	_is_gateway_route now consults GATEWAY_MOUNT_PATHS for Starlette Mounts instead of returning False unconditionally; the logic is correct and the fix is minimal
gateway/routes/allowlist.py	Adds GATEWAY_MOUNT_PATHS frozenset containing "/metrics"; also fixes a missing trailing comma on "/watsonx". Structure mirrors the existing BACKEND_MOUNT_PATHS pattern.
tests/test_litellm/proxy/test_component_allowlists.py	Adds four new Mount-specific tests and complex module-level setup to import _is_gateway_route safely; the _component_paths docstring is now slightly stale relative to the full gateway predicate

Comments Outside Diff (1)

1. tests/test_litellm/proxy/test_component_allowlists.py, line 89-100 (link)

   _component_paths docstring diverged from the gateway predicate

   The docstring says this helper reproduces gateway.main._is_gateway_route, but after this PR that predicate also handles Mounts via GATEWAY_MOUNT_PATHS. The helper still skips every Mount unconditionally (if isinstance(r, Mount): continue), so it only reproduces the non-Mount half of _is_gateway_route. The helper is only called in test_gateway_plus_backend_covers_full_app, which intentionally excludes Mounts from its union-coverage check, so behaviour is correct — but the docstring implies full fidelity and will mislead future readers who try to extend this test for Mounts.

   Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

_{Reviews (1): Last reviewed commit: "fix(gateway): keep the Prometheus /metri..." | Re-trigger Greptile}

[greptile-apps]

Greptile Summary

This PR fixes a 404 regression on /metrics in the componentized gateway entrypoint. When Prometheus is enabled, it registers /metrics as a Starlette Mount during the proxy startup lifespan; the gateway's _is_gateway_route predicate previously rejected all Mount instances before consulting the allowlist, silently removing the mount after startup and returning 404 on every scrape.

• gateway/routes/allowlist.py introduces GATEWAY_MOUNT_PATHS = frozenset({"/metrics"}), mirroring the existing BACKEND_MOUNT_PATHS pattern used for /swagger.
• gateway/main.py changes the Mount branch of _is_gateway_route from an unconditional return False to return path in GATEWAY_MOUNT_PATHS, so the Prometheus mount survives the trim while UI/asset mounts remain excluded.
• Four new tests in test_component_allowlists.py directly import and exercise the real predicate, with careful env and lifespan cleanup to avoid corrupting sibling tests in the same xdist worker.

Confidence Score: 5/5

Safe to merge — the change is minimal, well-scoped, and backed by a clear before/after demonstration and dedicated regression tests.

The two-line predicate change in _is_gateway_route is exactly targeted at the failure mode, the new GATEWAY_MOUNT_PATHS constant mirrors an already-proven pattern, and the tests directly exercise the pre-fix failure path. No existing logic is altered; UI and asset mounts continue to be excluded.

No files require special attention; the one style nit is in the test helper docstring.

Important Files Changed

Filename	Overview
gateway/main.py	Core fix: _is_gateway_route now consults GATEWAY_MOUNT_PATHS for Mount instances instead of unconditionally returning False, allowing the Prometheus /metrics Mount to survive the gateway route trim.
gateway/routes/allowlist.py	Adds GATEWAY_MOUNT_PATHS = frozenset({"/metrics"}) for Mount-specific allowlisting; also fixes a missing trailing comma after /watsonx in GATEWAY_PATH_PREFIXES.
tests/test_litellm/proxy/test_component_allowlists.py	Adds four new tests covering Mount-specific gateway behaviour; imports _is_gateway_route directly with careful env/lifespan teardown. The _component_paths docstring now diverges from the real predicate it claims to reproduce.

_{Reviews (2): Last reviewed commit: "fix(gateway): keep the Prometheus /metri..." | Re-trigger Greptile}

[greptile-apps]

The _component_paths docstring still claims to reproduce gateway.main._is_gateway_route, but since the fix it no longer does: _is_gateway_route now returns path in GATEWAY_MOUNT_PATHS for Mount instances, while _component_paths skips every Mount unconditionally. A future developer leaning on this docstring to understand the predicate will get a subtly wrong picture.

Suggested change

	def _component_paths(routes, exact_paths, path_prefixes) -> set[str]:
	"""Reproduce ``gateway.main._is_gateway_route`` / ``backend.main._is_backend_route``."""
	def _component_paths(routes, exact_paths, path_prefixes) -> set[str]:
	"""Reproduce the non-Mount portion of ``gateway.main._is_gateway_route`` / ``backend.main._is_backend_route``.
	Mount handling is intentionally omitted here; it is covered by the dedicated
	Mount tests that import ``_is_gateway_route`` directly.
	"""

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[yassin-berriai]

CI status: 122 of 123 checks pass. The one failure is ci/circleci: llm_responses_api_testing, which runs tests/llm_responses_api_testing/** against real provider APIs (Azure, Google AI Studio, Anthropic). That suite is disjoint from this diff, which only touches gateway/main.py, gateway/routes/allowlist.py, and tests/test_litellm/proxy/test_component_allowlists.py; nothing under litellm/ changed, and gateway/ is not imported by any code that suite exercises

I ran the suite locally on both this branch and the base litellm_internal_staging with identical results (the same provider-credential-gated tests fail identically on both within seconds, which is local cred noise, not a behavioral delta), and the same job currently passes on other open PRs against the same base. This looks like the known per-run flakiness of the real-API CircleCI jobs. I cannot re-trigger CircleCI from here; a maintainer clicking Rerun failed on workflow https://circleci.com/gh/BerriAI/litellm/2004033 should clear it