Repository navigation
fix(guardrails): stop the Javelin api_version default leaking into Azure Content Safety - #41941
Merged
mateo-berri merged 3 commits intoSep 19, 2026
Merged
Conversation
…ure Content Safety LitellmParams mixes every provider config model into one class, so the Javelin api_version default of "v1" reached the Azure Content Safety guardrails whenever config.yaml omitted api_version and Azure answered 404. The shared field now defaults to None, Javelin keeps filling in "v1" itself, and the Azure guardrails fall back to the documented 2024-09-01 at request time so a DB update that omits api_version stays on the default too.
Contributor
Author
|
I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".
|
mateo-berri
added
run-ci
backport-stable
Contributor
Contributor
 |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
…t for Azure Content Safety Guardrails created through POST /guardrails on older releases have api_version "v1" saved in the database, because the writer persists every default. Azure Content Safety never accepts that value, so those guardrails kept answering 404 after the default moved to None. The Azure base now resolves "v1" to 2024-09-01 the same way it resolves a missing value. Also restores the OpenAPI snapshot line that a Python 3.14 regeneration had dedented
Contributor
Contributor
|
bugbot run |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
✅ Bugbot reviewed your changes and found no new issues!
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit e327a6a. Configure here.
mateo-berri
deleted the
litellm_azure_content_safety_api_version_default
branch
September 19, 2026 10:09
mateo-berri
removed
the
backport-stable
3 of 5 tasks
keppy
pushed a commit
to keppy/litellm
that referenced
this pull request
Oct 10, 2026
…ure Content Safety (BerriAI#41941) (cherry picked from commit 385932b)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
TLDR
Problem this solves:
api_versioncall Azure withapi-version=v1default_on: trueevery chat completion failsv1default lives in the shared guardrail params model and leaks into AzurePOST /guardrailshave thatv1saved in the databaseHow it solves it:
api_versionfield now defaults toNone; Javelin already fills inv1itself2024-09-01at request timev1on an Azure guardrail is treated as unset, so existing rows heal on upgradeapi_versionand with a storedv1, and assert the URLDecision worth a reviewer's eye:
POST /guardrailsandPUT /guardrails/{id}save every default, so guardrails created on older releases carryapi_version: "v1"that nobody typed. Azure Content Safety has never acceptedv1(the control below is a 404), so the Azure guardrails now read a storedv1as "not set" and use2024-09-01. The alternatives were a data migration that nulls the stored value (it would not reach deployments that boot with--use_prisma_db_push, and migrations here should not rewrite rows), leaving those rows broken with a documented workaround, or changing the writer to save only the fields the admin set (the right long-term fix, but it touches every guardrail provider, so it is a follow-up)User Flow
Before: a proxy admin adds an Azure Content Safety guardrail without
api_version, in config.yaml or through the API, and every chat completion through the proxy comes back 404azure/text_moderationsandazure/prompt_shieldguardrails in config.yaml withapi_key,api_base,mode: pre_call,default_on: true, and noapi_version, then starts the proxy{"model": "gpt-5.6", "messages": [{"role": "user", "content": "Say hello in five words."}]}{"detail":{"error":"{\"error\":{\"code\":\"404\",\"message\": \"Resource not found\"}}"}}, and the Azure resource's request log shows a call to/contentsafety/text:analyze?api-version=v1not_found_error, until the admin addsapi_version: "2024-09-01"to each Azure guardrail and restartsapi_versionout sees"api_version": "v1"on it in GET https://litellm-domain/v2/guardrails/list, a value they never typedAfter: the same guardrails with no
api_versionwork, because they call Azure at its documented default version, and a guardrail saved by an older release starts working after the upgrade without being editedapi_versionand starts the proxy/contentsafety/text:analyze?api-version=2024-09-01and/contentsafety/text:shieldPrompt?api-version=2024-09-01api_version; an explicit value such as2024-02-15-previewstill wins"api_version": "v1"in GET https://litellm-domain/v2/guardrails/list, and a guardrail created after the upgrade showsnullRelevant issues
Affected release
Regression in v1.77.7 (the shared
v1default landed in that release; the Azure guardrails predate it)Linear ticket
Resolves LIT-7686
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*,make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more@greptileaito re-request a review after pushing changes)Delays in PR merge?
If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).
Screenshots / Proof of Fix
Every case ran on both sides from its own worktree and venv, booted with
--num_workers 2on a random free port, against the real Azure Content Safety resource, real OpenAIgpt-5.6, and real Anthropicclaude-opus-5. The guardrailapi_basepoints at a small forwarding recorder (http://127.0.0.1:46846) that logs the path and body keys of every request and forwards it unchanged to the real Azure resource, so both sides show exactly what LiteLLM sent Azure. Same configs and same requests on both sides; the two database cases share one Postgresconfig.yaml for the first case (no
api_versionon either guardrail). The explicit case addsapi_version: "2024-02-15-preview"to both guardrails, and the database case drops theguardrails:block and addsstore_model_in_db: true:Boot:
python litellm/proxy/proxy_cli.py --config <config> --port <port> --num_workers 2 --detailed_debug(the database cases add--use_prisma_db_pushand aDATABASE_URL)Requests, three of each per case:
The stored guardrail in the database case was created once, on the merge base, with this request and no
api_version(sent by a four-line script that reads the key from an env file, so the key never reaches the shell history):Control, the same call sent straight through the recorder to Azure at both versions, to show the 404 is the version and not the resource:
Before (5f1268c, merge base)
config.yaml guardrails with no api_version
POST /v1/chat/completions, all three attemptsPOST /v1/responses, all three attemptsPOST /v1/messages, all three attemptsWhat reached Azure for the whole case: six calls, every one at
api-version=v1, every one 404, and prompt shield never reached because text moderation failed firstGuardrail saved by POST /guardrails on the older release
GET /v2/guardrails/listshows the guardrail with a version nobody typed:azure-text-moderation-ui ... "api_version": "v1"POST /v1/chat/completions, all three attemptsPOST /v1/responses, all three attemptsPOST /v1/messages, all three attemptsWhat reached Azure for the whole case: six calls, all at
api-version=v1, all 404Explicit api_version in config.yaml
POST /v1/chat/completions, all three attemptsPOST /v1/responses, all three attemptsPOST /v1/messages, all three attemptsWhat reached Azure for the whole case: twelve calls, all at the configured
2024-02-15-preview, all 200GET /v2/guardrails/list for the config.yaml guardrails
Both guardrails report
"api_version": "v1"although config.yaml never set itAfter (24064e3)
The tip since then is e327a6a, which only removes test docstrings, so it cannot change behavior and the proof above stands at 24064e3
config.yaml guardrails with no api_version
POST /v1/chat/completions, all three attemptsPOST /v1/responses, all three attemptsPOST /v1/messages, all three attemptsWhat reached Azure for the whole case: twelve calls, every one at
api-version=2024-09-01, every one 200, both guardrails reached, and thetext:analyzebody keys match the before side, so the version is the only thing that changed on the wireGuardrail saved by POST /guardrails on the older release
GET /v2/guardrails/liston the upgraded proxy still shows the same row with"api_version": "v1"; nothing rewrote itPOST /v1/chat/completions, all three attemptsPOST /v1/responses, all three attemptsPOST /v1/messages, all three attemptsWhat reached Azure for the whole case: six calls, all at
api-version=2024-09-01, all 200, from the same database row, which was not editedA second guardrail created on the PR head with the same
POST /guardrailslists asazure-text-moderation-created-at-tip api_version= None, and four more chat completions with both guardrails active are all 200, with eighttext:analyze?api-version=2024-09-01 -> 200calls at AzureExplicit api_version in config.yaml
POST /v1/chat/completions, all three attemptsPOST /v1/responses, all three attemptsPOST /v1/messages, all three attemptsWhat reached Azure for the whole case: twelve calls, all at the configured
2024-02-15-preview, all 200, so an explicit version still winsGET /v2/guardrails/list for the config.yaml guardrails
The only difference in the whole response (keys and host redacted, then
diff base tip) is that the two guardrails stop reporting av1they never had.GET /guardrails/listis byte-identical on both sidesObserved alongside the fix (this PR leaves every one of these alone):
/v1/responsesinput entirely, both sides2024-09-01for a guardrail stored with nullv1default is unit-tested only, no live credentialsType
🐛 Bug Fix
Caveats (if any)
Backward incompatible
Each of these is an observable change /live-pr-risk found at 24064e3. All of them are inherent to the fix shape LIT-7686 names in its "Done looks like" (make the Azure default win over
v1), so the ticket author approved shipping them there and none has a fixing commitapi_version: "v1"typed on purpose on an Azure guardrail now calls Azure at2024-09-01v1all 404 at base, 6 calls at2024-09-01all 200 at the tip)GET /v2/guardrails/liststops showing"api_version": "v1"on guardrails that never set itGET /guardrails/listis identicalapi_version="v1"kwarg when the config never set onemodel_dump(exclude_none=True), not driven live; no in-repo class reads it besides Azure and Javelin"v1"; the guardrail writer still saves every default (follow-up)2024-09-01while the listing still showsv1Low
v1default; only the runtime model changedv1fallback is unit-tested only, no live Javelin credentialsci/circleci: litellm_utils_testingis red at the tip ontest_models_by_provideralone, the same test that fails on main's own pipelines 89834 and 89818; nothing in this diff touches itcore-utils / Run testsfailed once on the wall-clock flaketest_detailed_timing_pre_processing_uses_receive_anchor, green at the two earlier tips and on main, fix in flight in test(response_metadata): anchor detailed-timing test on a fixed instant instead of wall clock #41946; the re-run of that job passed, so every required check is green at the tipFinal Attestation