Repository navigation
fix(proxy): gate disable_global_guardrails on keys and teams to proxy admins - #42699
Conversation
… admins Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
|
I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".
|
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
|
…guardrail opt-out helper Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
…ot server defaults Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
|
@greptileai please re-review at b2c3fad, the server-default metadata finding is addressed there |
There was a problem hiding this comment.
Devin Review found 1 potential issue.
1 flag not posted on this PR by your GitHub settings — view it in Devin Review. (Configure)
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
|
@greptileai review latest head |
|
bugbot run |
…y admins Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
…to the sink delay Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
|
@greptileai review latest head 0ce0fdb |
|
bugbot run |
|
@greptileai review latest head |
|
bugbot run |
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit 0ce0fdb. Configure here.
TLDR
This fixes a bypass where non-admins could set disable_global_guardrails: true on a key or team and skip global guardrails.
After this PR, only proxy admins can enable disable_global_guardrails: true, including through metadata. Non-admins can keep an exemption that was already granted by an admin, but they can’t create a new one. They can only flip true → false
The UI now hides the option for non-admins, and attempts to enable it return 403.
User Flow
Before: an internal user who is a team admin creates a key that skips the default-on guardrail
{"team_id": "<team>", "disable_global_guardrails": true}sk-...key whosemetadatashows"disable_global_guardrails": trueAfter: the same request is refused and only a proxy admin can grant the exemption
{"team_id": "<team>", "disable_global_guardrails": true}Only proxy admins can set disable_global_guardrails on a key.metadata, or on POST /key/update, POST /key/regenerate, POST /key/service-account/generate and POST /team/newsynthetic policy denialfrom the default-on guardrail on POST /v1/chat/completionsRelevant issues
Affected release
Linear ticket
Resolves LIT-7702
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*,make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more@greptileaito re-request a review after pushing changes)Delays in PR merge?
If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).
Screenshots / Proof of Fix
Setup, identical on both sides: real proxy with Postgres and Redis, master key creates an
internal_user, a team with that user as team admin, a personal key for that user (the caller, allowed routes/key/generate,/key/update,/key/regenerate) and a plain team key. Chat cases useopenai/gpt-4o-miniagainst the real OpenAI API through a local forwarding recorder that counts upstream calls, plus adefault_ongeneric_guardrail_apiguardrail pointed at a local sink that always returnsBLOCKED. Tokens are redactedBefore (40ec84c)
Non-admin sets the flag on /key/generate
curl -X POST http://127.0.0.1:14010/key/generate -H 'Authorization: Bearer <caller>' -d '{"team_id":"<team>","disable_global_guardrails":true}'HTTP_STATUS=200, body"metadata":{"disable_global_guardrails":true}Non-admin smuggles the flag under metadata
curl -X POST http://127.0.0.1:14010/key/generate -H 'Authorization: Bearer <caller>' -d '{"team_id":"<team>","metadata":{"disable_global_guardrails":true}}'HTTP_STATUS=200Non-admin sets the flag on /key/update
curl -X POST http://127.0.0.1:14010/key/update -H 'Authorization: Bearer <caller>' -d '{"key":"<team-key>","disable_global_guardrails":true}'HTTP_STATUS=200Non-admin sets the flag on /key/regenerate
curl -X POST http://127.0.0.1:14010/key/regenerate -H 'Authorization: Bearer <caller>' -d '{"key":"<team-key>","disable_global_guardrails":true}'HTTP_STATUS=200Non-admin sets the flag on /key/service-account/generate and /team/new
curl -X POST http://127.0.0.1:14010/key/service-account/generate -H 'Authorization: Bearer <caller>' -d '{"team_id":"<team>","service_account_id":"svc","disable_global_guardrails":true}'HTTP_STATUS=200curl -X POST http://127.0.0.1:14010/team/new -H 'Authorization: Bearer <caller>' -d '{"team_alias":"t2","disable_global_guardrails":true}'HTTP_STATUS=200Admin-exempted key, then non-admin re-saves it with the stored metadata echoed back
curl -X POST http://127.0.0.1:14010/key/generate -d '{"team_id":"<team>","disable_global_guardrails":true}'returnsHTTP_STATUS=200curl -X POST http://127.0.0.1:14010/key/update -H 'Authorization: Bearer <caller>' -d '{"key":"<exempt-key>","key_alias":"renamed","metadata":{"disable_global_guardrails":true}}'HTTP_STATUS=200, stored flag stilltruecurl -X POST http://127.0.0.1:14010/v1/chat/completions -H 'Authorization: Bearer <exempt-key>' -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"synthetic denied marker"}]}'returnsHTTP_STATUS=200with real usage, recorder shows 1 upstream callPlain team key hits the default-on guardrail
curl -X POST http://127.0.0.1:14010/v1/chat/completions -H 'Authorization: Bearer <team-key>' -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"synthetic denied marker"}]}'HTTP_STATUS=400,{"error":{"message":"synthetic policy denial",...,"code":"400"}}, recorder shows 0 upstream callsAfter (2cfb50e)
Non-admin sets the flag on /key/generate
curl -X POST http://127.0.0.1:14000/key/generate -H 'Authorization: Bearer <caller>' -d '{"team_id":"<team>","disable_global_guardrails":true}'HTTP_STATUS=403,{"error":{"message":"{'error': 'Only proxy admins can setdisable_global_guardrailson a key.'}",...,"code":"403"}}Non-admin smuggles the flag under metadata
curl -X POST http://127.0.0.1:14000/key/generate -H 'Authorization: Bearer <caller>' -d '{"team_id":"<team>","metadata":{"disable_global_guardrails":true}}'HTTP_STATUS=403, same messageNon-admin sets the flag on /key/update
curl -X POST http://127.0.0.1:14000/key/update -H 'Authorization: Bearer <caller>' -d '{"key":"<team-key>","disable_global_guardrails":true}'HTTP_STATUS=403, same messageNon-admin sets the flag on /key/regenerate
curl -X POST http://127.0.0.1:14000/key/regenerate -H 'Authorization: Bearer <caller>' -d '{"key":"<team-key>","disable_global_guardrails":true}'HTTP_STATUS=403, same messageNon-admin sets the flag on /key/service-account/generate and /team/new
curl -X POST http://127.0.0.1:14000/key/service-account/generate -H 'Authorization: Bearer <caller>' -d '{"team_id":"<team>","service_account_id":"svc","disable_global_guardrails":true}'HTTP_STATUS=403,{"detail":{"error":"Only proxy admins can setdisable_global_guardrailson a key."}}curl -X POST http://127.0.0.1:14000/team/new -H 'Authorization: Bearer <caller>' -d '{"team_alias":"t2","disable_global_guardrails":true}'HTTP_STATUS=403,Only proxy admins can setdisable_global_guardrailson a team.Admin-exempted key, then non-admin re-saves it with the stored metadata echoed back
curl -X POST http://127.0.0.1:14000/key/generate -d '{"team_id":"<team>","disable_global_guardrails":true}'returnsHTTP_STATUS=200,"metadata":{"disable_global_guardrails":true}curl -X POST http://127.0.0.1:14000/key/update -H 'Authorization: Bearer <caller>' -d '{"key":"<exempt-key>","key_alias":"lr-head-2cfb50e902-ex-renamed","metadata":{"disable_global_guardrails":true}}'HTTP_STATUS=200,"key_alias":"lr-head-2cfb50e902-ex-renamed", stored flag stilltruecurl -X POST http://127.0.0.1:14000/v1/chat/completions -H 'Authorization: Bearer <exempt-key>' -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"synthetic denied marker"}]}'returnsHTTP_STATUS=200,"model":"gpt-4o-mini","usage":{"completion_tokens":5,"prompt_tokens":19,...}, recorder shows 1 upstream callPlain team key hits the default-on guardrail
curl -X POST http://127.0.0.1:14000/v1/chat/completions -H 'Authorization: Bearer <team-key>' -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"synthetic denied marker"}]}'HTTP_STATUS=400,{"error":{"message":"synthetic policy denial","type":"invalid_request_error","param":null,"code":"400"}}, recorder shows 0 upstream callsIntegration regression
tests/integration/authorization/test_key_guardrail_opt_out.py::test_non_admin_cannot_opt_key_out_of_default_on_guardrail(selected through theauthorizationgroup intests/integration/run.py) runs a real proxy with Postgres, Redis and a local scripted guardrail sink. Against a proxy started from40ec84caa2it fails atassert generated.status_code == 403with 200. Against0ce0fdbd5bit passesAudit at 0ce0fdb
17 deterministic cells across
tests/integration/authorization/test_key_guardrail_opt_out.py,test_key_guardrail_opt_out_runtime.pyandtest_key_guardrail_opt_out_chaos.py(nocoversmarkers, since thecontracts.jsonmanifest was dropped on main in #42711 and this branch merged that at80ba786dc3; the slow sink check bounds elapsed time by the configured sink delay rather than a fixed constant): every key and team write route with the flag top-level and smuggled, hostile inputs, server-default metadata, resave, omit and revoke sequences, runtime denial and exemption on chat, messages and responses across streaming and the OpenAI, AsyncOpenAI and Anthropic SDKs, team-level flag, spend logs, 40 concurrent writes, cross-worker revocation, guardrail sink outage and slow sink, and a SIGKILLed uvicorn worker. Real proxy, Postgres and Redis, scripted upstream, no real providers. Basee26a6450c8(current merge base): 9 failed, all atassert 200 == 403on flag writes, 8 passed. Head0ce0fdbd5btwice: 17 passed, 17 passed. Not reachable as written: a team admin hitting the new/team/updatecheck, because the team-admin editable-fields gate returns 403 first on both base and head since the flag is not inSUPPORTED_TEAM_ADMIN_EDITABLE_TEAM_FIELDSThe
managementintegration shard (tests/integration/run.py management, seed 4106601) was also run locally at2cfb50e902against a fresh proxy pair, Postgres and Redis: 46 passed, 2 failed, all 17 nodes from this PR passed. The 2 failures areowned_redisstartup in two unrelated Redis-outage tests because the local Redis is 6.0.16 and the harness passes--set-proc-title, a Redis 7 flag; CircleCI publishes no check on this PR head so that local run stands in for the shardMutation check
Stubbing
_check_disable_global_guardrails_caller_permissiontoreturnmakes 11 of the 18 new unit tests fail (every 403 case across generate, update, regenerate, team new and team update, plus the stored-false and explicit-false-with-metadata-true cases); the 7 allow cases stay green. Restoring the helper: 18 passed. RemovingisProxyAdminRole(userRole)from either UI form fails the new "hides the switch from a non-admin" testsType
🐛 Bug Fix
Caveats (if any)
Severe
disable_global_guardrailstoday now get 403 on key and team write routesMedium
Low
/key/regeneratea non-premium non-admin sees the enterprise error before the 403, since the row load sits behind the premium gate0ce0fdbd5bvse26a6450c8(identical results to the earlier2cfb50e902vs40ec84caa2run): only the intended 403s differ;/key/bulk_update(already admin-only),/team/key/bulk_update(schema forbids the flag) andPATCH /team/{id}(team-admin editable-fields gate fires first) are unchangedFinal Attestation
Link to Devin session: https://app.devin.ai/sessions/b4c09d500c5c4c8ab366e8eae782ac40
Open in Devin Desktop: https://app.devin.ai/desktop/session/b4c09d500c5c4c8ab366e8eae782ac40?variant=devin
Requested by: @yucheng-berri
Note
High Risk
Changes authorization on guardrail opt-out for keys and teams; non-admins lose a previously allowed path to bypass default-on guardrails, so deploys may break workflows that relied on team admins setting the flag.
Overview
Restricts who can opt keys and teams out of default-on global guardrails. Non–proxy-admins now get 403 when they try to set
disable_global_guardrails(top-level or viametadata) on key generate, update, regenerate (including path-based regenerate), service-account generate, and team new / update. Proxy admins are unchanged.A shared
_check_disable_global_guardrails_caller_permissionhelper mirrors the existing passthrough-routes admin gate: truthy values are blocked (including smuggled metadata), key generate checks caller-supplied metadata before server defaults apply, and updates allow non-admins to re-send an already-storedtrueso admin UI edits do not break.The dashboard hides the “Disable Global Guardrails” switch for non-admins on key create/edit and team create/edit; API docs/schema strings now say proxy admin only. Broad unit and integration coverage exercises denial paths, admin success, resave/revoke, concurrency, and runtime guardrail skip behavior.
Reviewed by Cursor Bugbot for commit 0ce0fdb. Bugbot is set up for automated code reviews on this repo. Configure here.