Skip to content

feat(proxy): admin-only /debug/report sharing the bug report environment - #42440

Merged
ryan-crabbe-berri merged 8 commits into
mainfrom
litellm_debug_report_endpoint
Sep 22, 2026
Merged

ryan-crabbe-berri merged 8 commits into
mainfrom
litellm_debug_report_endpoint

Conversation

@devin-ai-integration

@devin-ai-integration devin-ai-integration Bot commented Sep 22, 2026 •

Copy link
Copy Markdown
Contributor

TLDR

Problem this solves:

  • Support asks operators five questions to learn version, deployment and config shape
  • The bug report link already computes those facts but only inside an issue URL
  • Nothing lets an admin pull them as JSON without pasting config by hand

How it solves it:

  • New admin-only GET /debug/report returns the bug report environment fields as JSON
  • Same EnvironmentReport builder feeds the issue URL and the endpoint, one allowlist for both
  • Only LiteLLM-owned values leave the box: versions, bools, enum words, counts
  • Credential-shaped keys never render, even when the secret equals a LiteLLM enum word
  • Verbose config walk is deferred to a follow-up PR

User Flow

Before: an admin asking support for help has no way to hand over version and config shape without redacting it by hand

  1. They send GET https://litellm-domain/debug/report with their proxy admin key
  2. They get 404 {"detail":"Not Found"}
  3. Support asks for version, Python version, deploy method and the config section by section

After: one curl gives support the same facts the bug report link would put in a GitHub issue, with nothing operator-authored in it

  1. They send GET https://litellm-domain/debug/report with their proxy admin key
  2. They get 200 with surface, litellm_version, python_version, deployment and config_lines such as litellm_settings.cache_params.type = local and model_list[*].provider = [openai]
  3. Model aliases, api bases, callback module paths, env var names and any other string LiteLLM does not define are absent from the body
  4. A key with any other role (including view-only admin) gets 401 or 403 and no report body

Relevant issues

Follow-up to #42065

Affected release

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

  • I have added meaningful tests
  • The handful of test files covering my change pass locally, e.g. uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
  • My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
  • My PR's scope is as isolated as possible; it only solves 1 specific problem
  • I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Screenshots / Proof of Fix

Video walkthrough of this exact run at tip 892d9b4 (main 404, tip safe report for the admin, hostile config strings absent, viewer 403, internal user and no auth 401, real gpt-4o-mini call) is in the Slack thread: https://berriaillm.slack.com/archives/C0B302ZJU05/p1790098365878969?thread_ts=1789865287.481439&cid=C0B302ZJU05

Two proxies per side, one with 4 uvicorn workers and one with 1, started from main and from the tip with the same config, master key and OPENAI_API_KEY. Base ports 18431 (4 workers) and 18441 (1 worker), head ports 18432 and 18442. $K is the proxy admin master key, $NA is a virtual key with the internal_user role. Config, with placeholder values shaped like the strings an operator would not want in a report:

model_list:
  - model_name: acme-prod-gpt4
    litellm_params:
      model: openai/gpt-4o-mini
      api_key: os.environ/OPENAI_API_KEY
      api_base: https://acme-eastus.example.com/v1
  - model_name: gpt-4o-mini
    litellm_params:
      model: openai/gpt-4o-mini
      api_key: os.environ/OPENAI_API_KEY
litellm_settings:
  drop_params: true
  callbacks: ["acme_hooks.audit_logger", "prometheus"]
  cache: true
  cache_params:
    type: local
router_settings:
  routing_strategy: least-busy
  model_group_alias:
    acme-gpt4: acme-prod-gpt4
  fallbacks:
    - acme-prod-gpt4: [gpt-4o-mini]
environment_variables:
  ACME_TENANT: acme-eu-tenant
general_settings:
  master_key: os.environ/LITELLM_MASTER_KEY
  store_model_in_db: false
  acme_sso_tenant: acme-prod

Each status line below is for i in $(seq 8); do curl -s -o /dev/null -w "%{http_code} " ... ; done, run against both the 4-worker and the 1-worker proxy of that side; both legs printed identical codes

Before (deba473, main at the time of the run)

Proxy admin

  1. curl -s -o /dev/null -w "%{http_code} " -H "Authorization: Bearer $K" http://127.0.0.1:18431/debug/report x8
  2. 404 404 404 404 404 404 404 404

Internal user key

  1. curl -s -o /dev/null -w "%{http_code} " -H "Authorization: Bearer $NA" http://127.0.0.1:18431/debug/report x8
  2. 404 404 404 404 404 404 404 404

No auth

  1. curl -s -o /dev/null -w "%{http_code} " http://127.0.0.1:18431/debug/report x8
  2. 404 404 404 404 404 404 404 404

Not in the OpenAPI schema

  1. curl -s -H "Authorization: Bearer $K" http://127.0.0.1:18431/openapi.json | grep -c '"/debug/report"'
  2. 0

Real provider call

  1. curl -s -o /dev/null -w "%{http_code} " -H "Authorization: Bearer $K" -H "Content-Type: application/json" -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"reply with ok"}],"max_tokens":3}' http://127.0.0.1:18431/v1/chat/completions x8 in parallel
  2. 200 200 200 200 200 200 200 200, all 4 workers (and the single worker on 18441) show CPU time consumed

After (892d9b4)

Proxy admin

  1. curl -s -o /dev/null -w "%{http_code} " -H "Authorization: Bearer $K" http://127.0.0.1:18432/debug/report x8
  2. 200 200 200 200 200 200 200 200
  3. curl -s -H "Authorization: Bearer $K" http://127.0.0.1:18432/debug/report | python3 -m json.tool
{
    "surface": "proxy",
    "litellm_version": "1.103.0",
    "python_version": "3.12.13",
    "deployment": null,
    "config_lines": [
        "general_settings.store_model_in_db = false",
        "litellm_settings.drop_params = true",
        "litellm_settings.callbacks = [prometheus]",
        "litellm_settings.cache = true",
        "litellm_settings.cache_params.type = local",
        "router_settings.routing_strategy = least-busy",
        "model_list[*].provider = [openai]"
    ]
}
  1. for s in acme-prod sk- postgres 10.0.0 hooks.example internal.corp; do echo -n "$s: "; curl -s -H "Authorization: Bearer $K" http://127.0.0.1:18432/debug/report | grep -ci "$s"; done
acme-prod: 0
sk-: 0
postgres: 0
10.0.0: 0
hooks.example: 0
internal.corp: 0
  1. curl -s -H "Authorization: Bearer $K" "http://127.0.0.1:18432/debug/report?verbose=true" returns 200 with a body byte-identical to step 3: the query is not part of the route and is ignored

Internal user key

  1. curl -s -o /dev/null -w "%{http_code} " -H "Authorization: Bearer $NA" http://127.0.0.1:18432/debug/report x8
  2. 401 401 401 401 401 401 401 401 (the proxy admin route gate rejects before the handler; a proxy_admin_viewer key reaches the handler and gets 403, shown in the video and covered in test_debug_utils.py)

No auth

  1. curl -s -o /dev/null -w "%{http_code} " http://127.0.0.1:18432/debug/report x8
  2. 401 401 401 401 401 401 401 401

Not in the OpenAPI schema

  1. curl -s -H "Authorization: Bearer $K" http://127.0.0.1:18432/openapi.json | grep -c '"/debug/report"'
  2. 0

Real provider call

  1. curl -s -o /dev/null -w "%{http_code} " -H "Authorization: Bearer $K" -H "Content-Type: application/json" -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"reply with ok"}],"max_tokens":3}' http://127.0.0.1:18432/v1/chat/completions x8 in parallel
  2. 200 200 200 200 200 200 200 200, all 4 workers (and the single worker on 18442) show CPU time consumed

Type

🆕 New Feature

Caveats (if any)

Low

  • config_lines come from proxy_config.config as loaded from YAML, so models added later through the DB or UI are not counted in model_list[*].provider (same scope as the bug report link today)
  • The endpoint is hidden from /openapi.json on purpose; schema.d.ts still carries its type because the generator includes hidden routes
  • A verbose mode that walks every config key was removed from this PR and will be proposed separately

Final Attestation

  • The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

Link to Devin session: https://app.devin.ai/sessions/b7cc1b2b8d9d48a8b6a62d6b3a5e01bc
Open in Devin Desktop: https://app.devin.ai/desktop/session/b7cc1b2b8d9d48a8b6a62d6b3a5e01bc?variant=devin


Note

Low Risk
Adds a gated debug read path with existing allowlisted config sanitization; BugReport is a structural refactor with tests asserting parity with issue URLs.

Overview
Introduces a shared EnvironmentReport (versions, surface, deployment hint, allowlisted config_lines) and nests it under BugReport, so GitHub issue URLs and a new proxy endpoint use the same builder.

Proxy: Adds admin-only GET /debug/report (hidden from public OpenAPI) returning that environment JSON via build_proxy_environment_report(), reusing the same safe config line extraction as bug reports. Non–proxy-admin callers get 403.

Bug report: Deployment metadata moves into EnvironmentReport (SDK always reports pip; proxy reports Docker only when /.dockerenv exists). Issue URL generation reads nested report.environment fields; behavior is covered by new/updated tests including hostile-config leakage checks.

Reviewed by Cursor Bugbot for commit 892d9b4. Bugbot is set up for automated code reviews on this repo. Configure here.

ryan-crabbe-berri and others added 2 commits September 22, 2026 06:28
…ronment fields

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
…ey with typed values

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@devin-ai-integration
devin-ai-integration Bot requested a review from a team September 22, 2026 06:49
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

  • Disable automatic comment, CI, and merge conflict monitoring

@mateo-berri

Copy link
Copy Markdown
Contributor

bugbot run

@greptile-apps

greptile-apps Bot commented Sep 22, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 5/5

The PR appears safe to merge; no actionable correctness, security, or repository-rule violations remain.

Summary

Adds a hidden, proxy-admin-only /debug/report endpoint that exposes the same allowlisted environment information used by bug-report issue URLs.

  • Introduces EnvironmentReport and shares its construction between SDK/proxy bug reports and the new endpoint.
  • Restricts configuration output to LiteLLM-defined booleans, enum values, and provider identifiers while suppressing credential-shaped string fields.
  • Removes the previously proposed verbose configuration walk and adds authorization, parity, and hostile-input tests.
  • Updates the generated dashboard HTTP schema for the new response type and endpoint.

Reviews (4) · Last reviewed commit: "chore(ui): regenerate schema.d.ts after ..."

Comment thread tests/test_litellm/proxy/common_utils/test_debug_utils.py
Comment thread tests/test_litellm/proxy/common_utils/test_debug_utils.py

@cursor cursor Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

Comment thread litellm/proxy/bug_report_config.py
@codecov

codecov Bot commented Sep 22, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

…the B008 budget flat

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@chilkotiKartik

Copy link
Copy Markdown

Review for PR #42440: Feat(proxy): Admin-only /debug/report sharing the bug report environment, with verbose config walk

Security & Sanitization Analysis

  1. Secret Redaction Pipeline:
    When walking active proxy configurations for the diagnostic report, all environment variables matching *KEY*, *SECRET*, *TOKEN*, and *PASSWORD* must be replaced with [REDACTED] prior to JSON serialization.

  2. RBAC Guard:
    Ensure the endpoint strictly verifies the caller possesses proxy_admin role credentials before rendering environment dumps.

… from the safe renderer, regenerate schema.d.ts

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Comment thread litellm/proxy/bug_report_config.py Fixed
Comment thread litellm/proxy/bug_report_config.py Fixed
@codspeed

codspeed Bot commented Sep 22, 2026 •

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 31 untouched benchmarks


Comparing litellm_debug_report_endpoint (892d9b4) with main (deba473)

Open in CodSpeed

…d budget maps in verbose /debug/report, single-exit scalar renderers

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

@greptileai

@mateo-berri

Copy link
Copy Markdown
Contributor

bugbot run

@cursor cursor Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

…ines detector allowlist entry

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@mateo-berri

Copy link
Copy Markdown
Contributor

bugbot run

@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

@greptileai

@cursor cursor Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stale Bugbot comment from a previous run.

ryan-crabbe-berri and others added 2 commits September 22, 2026 17:24
…merge main

Keeps GET /debug/report limited to the same allowlist-only environment block the bug report link carries, so the endpoint lands low risk. The verbose walk (every key path, numbers, count-only operator maps) is deferred to a follow-up PR. Takes main's key-aware credential filter for the safe renderer

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
…om /debug/report

Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
@devin-ai-integration devin-ai-integration Bot changed the title feat(proxy): admin-only /debug/report sharing the bug report environment, with verbose config walk feat(proxy): admin-only /debug/report sharing the bug report environment Sep 22, 2026
@mateo-berri

Copy link
Copy Markdown
Contributor

bugbot run

@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

@greptileai

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

Reviewed by Cursor Bugbot for commit 892d9b4. Configure here.

@ryan-crabbe-berri
ryan-crabbe-berri enabled auto-merge (squash) September 22, 2026 19:05
@ryan-crabbe-berri
ryan-crabbe-berri merged commit 5035c45 into main Sep 22, 2026
100 checks passed
@ryan-crabbe-berri
ryan-crabbe-berri deleted the litellm_debug_report_endpoint branch September 22, 2026 19:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants