fix: enforce tag budgets on x-litellm-tags header requests - #27573
Conversation
The x-litellm-tags header was merged into request metadata only after the auth chain completed, so _tag_max_budget_check (which reads tags from the request body) silently failed open for header-tagged requests — spend accumulated past max_budget without any 400 budget_exceeded response. Move the client-tag policy (strip-or-merge gated on allow_client_tags) to run before common_checks so header tags are visible to budget enforcement. The post-auth strip+merge in add_litellm_data_to_request stays as defense-in-depth; the new pre-auth helper is idempotent with it. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
Greptile SummaryThis PR fixes a silent budget-enforcement bypass where
Confidence Score: 5/5Safe to merge — the change is a targeted pre-auth merge that correctly closes a header-tag budget bypass, the double-merge is idempotent via deduplication, and new tests cover the end-to-end flow. The logic is sound: No files require special attention.
|
| Filename | Overview |
|---|---|
| litellm/proxy/litellm_pre_call_utils.py | Adds apply_client_tag_policy_pre_auth to merge x-litellm-tags header into request_data before common_checks runs; correctly deduplicates with _merge_tags and handles JSON-string metadata. The user_api_key_dict parameter is declared but never used. |
| litellm/proxy/auth/user_api_key_auth.py | Adds top-level import of LiteLLMProxyRequestSetup and calls apply_client_tag_policy_pre_auth in _run_centralized_common_checks immediately before common_checks, correctly closing the header-tag budget-enforcement gap. |
| tests/test_litellm/proxy/test_litellm_pre_call_utils.py | Adds 8 new mock-only tests for the new helper; two pre-existing single-line assertion reformats are formatting-only with no change to assertions or coverage. |
Reviews (6): Last reviewed commit: "fix(proxy): parse string metadata before..." | Re-trigger Greptile
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
![veria-ai[bot]](https://avatars.githubusercontent.com/in/1723776?s=60&v=4){kind=small}
Header tag budget enforcement addedThis PR merges Status: 1 open |
Stripping body-supplied tags in apply_client_tag_policy_pre_auth silently disabled per-tag budget enforcement for non-opted-in keys — pre-PR behavior was that those tags reached _tag_max_budget_check inside common_checks. The post-auth strip in add_litellm_data_to_request continues to remove unauthorized tags before they leave the proxy. Also moves the LiteLLMProxyRequestSetup import to module-level (no circular dep with auth/user_api_key_auth.py). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
@greptile review again with the new commit |
DALL-E 2 and DALL-E 3 were removed from the OpenAI API on 2026-05-12, causing e2e image-generation tests to fail with "model does not exist". Swap all live-API DALL-E references in proxy-backed tests to gpt-image-1 and update the dall-e-2 alias in proxy_server_config.yaml to point at openai/gpt-image-1 (preserves any historical dall-e-2 callers).
… gpt-image-1 Second wave of failures from the 2026-05-12 DALL-E shutdown: - tests/image_gen_tests/test_image_edits.py::TestOpenAIImageEditDallE2 and tests/image_gen_tests/test_image_generation.py::TestOpenAIDalle3 are explicitly named for the deprecated models and can't pass; remove. gpt-image-1 coverage already exists in sibling classes. - tests/local_testing/test_router.py image gen tests use dall-e-3 only as a routing example; swap to gpt-image-1. - tests/local_testing/test_custom_callback_input.py image_generation success/failure paths swapped to gpt-image-1.
Caller-supplied tags (`x-litellm-tags` header, body `tags`, `metadata.tags`) were silently dropped unless the key/team had `metadata.allow_client_tags: true` set. Restore the documented behavior: tags from the request always flow into `metadata.tags` and union with any admin-configured static tags from key/team/project metadata. Removes the `allow_client_tags` opt-in flag from the pre-call pipeline. The flag was only ever read here; it has no schema or endpoint footprint, so leftover values in existing key metadata are inert. Test cleanup mirrors the simplification: drop the three tests that verified the strip-when-not-opted-in path, drop the `allow_client_tags` fixture lines from the merge/union tests.
The tag-strip block was removed in the parent commit but two surrounding comments still referenced "tags without opt-in" and "runs AFTER the strip". Update them to describe the remaining user_api_key_* and _pipeline_managed_guardrails strip that the snapshot/merge ordering actually protects against.
|
@greptile review again |
|
@greptile allow_client_tags gate removed is intended because we reverted that PR and merged that base into this PR. Update the score based on this info. |
|
Acknowledged. With that context — the Re-evaluated score: the actual delta in this PR is the new Updated assessment: no blocking issues from this PR's changes. The implementation is correct for the stated fix scope. |
|
@greptile allow_client_tags gate removed is intended because we reverted that PR and merged that base into this PR. Update the score based on this info and that we have removed its mention from the pr description. |
| _metadata_variable_name = get_metadata_variable_name_from_kwargs(request_data) | ||
| metadata = request_data.get(_metadata_variable_name) | ||
| if not isinstance(metadata, dict): | ||
| metadata = {} |
There was a problem hiding this comment.
Medium: Tag-budget bypass with string metadata
get_tags_from_request_body() parses string metadata for budget checks, but this new pre-auth merge replaces a string metadata or litellm_metadata value with {} before that parser runs. An authenticated caller whose key allows client tags can send metadata='{"tags":["over-budget-tag"]}' plus any x-litellm-tags header, causing _tag_max_budget_check to see only the header tags and skip the over-budget metadata tag. Parse string metadata here, or preserve the existing value and merge into the parsed dict instead of overwriting it.
…name cascade tests Removes the allow_client_tags metadata check from apply_client_tag_policy_pre_auth so x-litellm-tags headers are always merged into request metadata, matching the post-auth behavior in add_litellm_data_to_request. Updates pre-call tests accordingly and adds a new test suite covering cascading credential renames into model rows. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
@greptile review again with new commits |
`apply_client_tag_policy_pre_auth` overwrote string-typed metadata
with `{}` before merging header tags, dropping any tags inside. A
caller could send `metadata='{"tags":["over-budget"]}'` plus
`x-litellm-tags: within-budget` and bypass `_tag_max_budget_check`
on the body tag. Parse the string via `safe_json_loads` first so
existing tags survive the merge.
Also drop the empty `tests/test_litellm/proxy/credential_endpoints/`
directory — the cascade-rename tests it held imported a function
that was never implemented (out of scope for this PR).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
@greptile review again with new commit that resolves the p1 issue as that file was out of scope |
…er_enforcement fix: enforce tag budgets on x-litellm-tags header requests
3 participants
Fixes: #27480
Description
When a request passes tags via the x-litellm-tags HTTP header, the per-tag budget gate _tag_max_budget_check silently fails open. Header-tagged spend grows past max_budget and requests continue returning HTTP 200 instead of 400 budget_exceeded.
Body-supplied tags ({"tags": [...]} or {"metadata": {"tags": [...]}}) work correctly — only the header path is broken.
Severity: high for deployments using tag-based budget enforcement. Tag budgets configured against header-supplied tags silently fail open; spend grows unbounded and operators only discover the issue from billing.
Cause
common_checks runs _tag_max_budget_check, which calls get_tags_from_request_body(request_body) (litellm/proxy/common_utils/http_parsing_utils.py:418). That helper only reads:
request_body["tags"]
request_body["metadata"]["tags"]
The x-litellm-tags header is merged into request metadata by LiteLLMProxyRequestSetup.add_request_tag_to_metadata (litellm/proxy/litellm_pre_call_utils.py:1160), but that logic runs inside add_litellm_data_to_request after the auth chain has already completed.
Net effect on header-tagged requests:
get_tags_from_request_body() returns []
the budget loop iterates over nothing
_tag_max_budget_check returns successfully
spend tracking still runs post-call, so tag spend continues accumulating without enforcement
The existing coverage only tested body-supplied tags (metadata={"tags": [...]}), which is why CI did not catch the issue.
Fix
Adds LiteLLMProxyRequestSetup.apply_client_tag_policy_pre_auth() in litellm/proxy/litellm_pre_call_utils.py and invokes it before common_checks in _run_centralized_common_checks.
The helper:
resolves tag policy from key/team metadata
strips unauthorized tags before budget checks run
parses x-litellm-tags
merges header tags into the correct metadata namespace using get_metadata_variable_name_from_kwargs
reuses _merge_tags for deterministic dedupe/order preservation
This ensures _tag_max_budget_check sees header-supplied tags before enforcement executes.
The existing post-auth merge/strip logic in add_litellm_data_to_request is preserved as defense-in-depth.
Tests
Added TestApplyClientTagPolicyPreAuth (8 tests) in tests/test_litellm/proxy/test_litellm_pre_call_utils.py covering:
header merge before budget enforcement
union with existing metadata.tags
litellm_metadata precedence
stripping unauthorized tags
no merge when unauthorized
no-op behavior without headers
end-to-end budget enforcement for header-supplied tags
Also verified:
all test_litellm_pre_call_utils.py tests pass
all auth tag enforcement tests pass
all user_api_key_auth tests pass
All 129 tests in test_litellm_pre_call_utils.py, all auth_checks tag tests, and all 58
user_api_key_auth tests pass. Black-formatted.
Before:
After:
