feat(proxy): add disable_budget_reservation general setting (#27639)

[Ar-maan05]

Relevant issues

#27639

Pre-Submission checklist

• I have added meaningful tests
• My PR passes all unit tests on make test-unit
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

Type

🐛 Bug Fix

Changes

Background

Since the optimistic budget reservation feature shipped in v1.84.0, operators running multiple replicas with a Redis spend counter have reported intermittent BudgetExceededError (HTTP 429) for keys and end users whose actual database spend is well under budget. The reported shape is a phantom reservation: the per-request reservation increments the cross-pod counter (spend:key:<token>, spend:end_user:<id>) up front, and several reporters observe counters that climb to clean integer multiples of the real spend, which is consistent with increments that are never decremented on some terminal paths. The phantom value accumulates over the budget window and eventually rejects legitimate requests.

The lifecycle that produces this: reserve_budget_for_request runs during auth (in _reserve_budget_after_common_checks) and increments the counter, while the matching decrement happens later, either in the cost-tracking success callback or in the post-call failure hook. A request that reserves budget but then terminates without reaching either of those, for example a client disconnect mid-stream, an upstream timeout, or a pod restart, leaves its reservation behind.

Hardening that lifecycle so every terminal path reconciles is a larger, separate effort. In the meantime, reporters on #27639 asked for a way to turn the feature off so they can fall back to the enforcement behavior that worked before v1.84.0.

What this PR does

This adds a disable_budget_reservation general setting. When it is set, _reserve_budget_after_common_checks returns before reserving, so no reservation entry is created and the reservation counter is never incremented. Nothing downstream needs to unwind because there is nothing to unwind: the request metadata writer only attaches a reservation when one exists, and reconcile_budget_reservation(None) is a no-op.

Budget enforcement is preserved. common_checks already evaluates the team, key, organization, user, and end-user budgets at read time, before the reservation step, so an over-budget caller is still rejected with the flag on. Disabling the reservation only removes the optimistic pre-charge; it reverts to the read-time enforcement that predates v1.84.0. It does not disable budgets.

The tradeoff is the one that existed before v1.84.0, and it is called out in the setting description. Without the reservation, a burst of concurrent requests from one caller can each pass the read-time spend check before any of them is charged, so a configured budget can be briefly exceeded under high concurrency. An operator enabling this flag is accepting that looser concurrency-time enforcement in exchange for not receiving phantom BudgetExceededError responses from leaked reservations. The strict disable_budget_reservation is True gate is deliberate so that an ambiguous or string-typed config value fails safe by leaving the reservation enabled rather than silently weakening enforcement.

The default is unchanged. Reservation stays on unless an operator opts out:

general_settings:
  disable_budget_reservation: true

The setting is registered on ConfigGeneralSettings so it appears in the config schema and can be toggled through the dynamic /config/field/update API, not only by editing config.yaml and restarting. It is dependency-injected into the helper rather than read from a module global, so the behavior is unit-testable without patching globals or proxy state.

Scope

This is the opt-out the issue thread requested. It is intentionally limited to that and does not modify the reservation lifecycle itself, which is being hardened separately. A related but distinct report (#28283) attributes some counter drift to the cross-pod spend counter rather than to reservations; that is out of scope here.

Tests

tests/test_litellm/proxy/auth/test_user_api_key_auth.py gets two regression tests on the helper. With disable_budget_reservation set, reserve_budget_for_request is asserted to never be called and no reservation is stored. With the flag absent, the reservation still runs and is stored on the auth object. Deleting the gate fails the first test, so the tests pin the behavior rather than just exercising it.

Screenshots / Proof of Fix

Start a proxy with the flag enabled and a Redis cache, against a config that has a real model:

python litellm/proxy/proxy_cli.py --config litellm/proxy/dev_config.yaml --detailed_debug --reload --use_v2_migration_resolver 2>&1 | tee litellm.log

with disable_budget_reservation: true under general_settings in that config.

A normal request still succeeds (hits a live provider, costs a few cents):

curl -i http://localhost:4000/v1/chat/completions \
  -H "Authorization: Bearer sk-1234" -H "Content-Type: application/json" \
  -d '{"model": "<a-model-in-your-config>", "messages": [{"role": "user", "content": "hi"}]}'

Budget enforcement is still active. Create a key with a tiny budget, exhaust it with one real call, then confirm inference is rejected at read time:

KEY=$(curl -s http://localhost:4000/key/generate \
  -H "Authorization: Bearer sk-1234" -H "Content-Type: application/json" \
  -d '{"max_budget": 0.00001, "models": ["<a-model-in-your-config>"]}' | jq -r .key)

curl -s http://localhost:4000/v1/chat/completions \
  -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
  -d '{"model": "<a-model-in-your-config>", "messages": [{"role": "user", "content": "hi"}]}' > /dev/null

# still returns 429 Budget Exceeded, so disabling reservation does not disable enforcement
curl -i http://localhost:4000/v1/chat/completions \
  -H "Authorization: Bearer $KEY" -H "Content-Type: application/json" \
  -d '{"model": "<a-model-in-your-config>", "messages": [{"role": "user", "content": "hi"}]}'

With the flag on, grepping the log for the reserved counter keys shows no reservation increments while requests are in flight:

grep -i "spend:key:\|spend:end_user:\|reserve" litellm.log

[Ar-maan05]

@greptile-apps

[greptile-apps]

Greptile Summary

This PR adds a disable_budget_reservation general setting that lets operators opt out of the optimistic per-request budget reservation introduced in v1.84.0, which has been causing phantom BudgetExceededError responses from leaked reservations on multi-replica Redis deployments. When the flag is set, _reserve_budget_after_common_checks returns early before any reservation is created; read-time budget enforcement via common_checks is unaffected.

• ConfigGeneralSettings gains an Optional[bool] field with a description that clearly documents the concurrency-under-budget trade-off operators accept when enabling the flag.
• _reserve_budget_after_common_checks accepts a new general_settings: dict parameter (dependency-injected from the module global in _run_centralized_common_checks) and short-circuits on disable_budget_reservation is True.
• Two new mock-only regression tests cover both the disabled and enabled paths; the existing stale-reservation test is updated to supply the new required argument.

Confidence Score: 5/5

Safe to merge — the change is a targeted opt-out flag with a deliberately conservative default; existing reservation behavior is unchanged unless the operator explicitly sets the flag.

The diff is small and self-contained: one new field on a config model, one early-return guard in a single helper, and matching tests. Budget enforcement at read time is untouched, the new parameter follows the same general_settings.get(...) pattern used throughout the surrounding code, and all tests are properly isolated mocks with no network calls.

No files require special attention.

Important Files Changed

Filename	Overview
litellm/proxy/_types.py	Adds disable_budget_reservation: Optional[bool] field to ConfigGeneralSettings with a clear description of the trade-off. No issues.
litellm/proxy/auth/user_api_key_auth.py	Threads general_settings through to _reserve_budget_after_common_checks and adds an early return when disable_budget_reservation is True. Change is minimal and consistent with how surrounding general_settings.get() calls are written elsewhere in the same function.
tests/test_litellm/proxy/auth/test_user_api_key_auth.py	Adds two focused regression tests (flag on skips reservation; flag absent runs reservation), and updates the existing stale-reservation test to pass the new required general_settings arg. Tests are mock-only, no real network calls.

_{Reviews (4): Last reviewed commit: "ci: re-trigger flaky docker build (prism..." | Re-trigger Greptile}

[codecov]

Codecov Report

❌ Patch coverage is 83.33333% with 1 line in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
litellm/proxy/auth/user_api_key_auth.py	50.00%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!

[greptile-apps]

Greptile Summary

Introduces a disable_budget_reservation opt-out flag under general_settings that lets operators disable the optimistic Redis reservation introduced in v1.84.0, which has been causing phantom BudgetExceededError responses on multi-replica deployments. The flag is dependency-injected into _reserve_budget_after_common_checks rather than read from a global, so it is unit-testable without patching module state.

• Adds general_settings: dict as a required parameter to _reserve_budget_after_common_checks and inserts an early-return guard that skips the reservation (and leaves budget_reservation = None) when the flag is set, while leaving budget enforcement unchanged.
• Two new mock-only regression tests pin both code paths: flag-on verifies reserve_budget_for_request is never called, flag-off verifies the reservation is awaited and stored on the auth object.

Confidence Score: 4/5

Safe to merge; the change is a minimal opt-out guard on a non-default code path with no impact on the existing reservation lifecycle or budget enforcement.

The implementation correctly threads general_settings through the call chain, the early-return is positioned after budget_reservation is cleared to None so no stale state can leak, and general_settings is always a dict in production. The only notable issue is the strict is True identity check, which could silently miss a string true from non-YAML config sources. Both new tests are mock-only and correctly pin both code paths.

No files require special attention beyond the single-line guard in user_api_key_auth.py.

Important Files Changed

Filename	Overview
litellm/proxy/auth/user_api_key_auth.py	Adds general_settings: dict parameter to _reserve_budget_after_common_checks and an early-return guard that skips the Redis reservation when disable_budget_reservation is set; the change is minimal and correctly threaded from the module-level import in _run_centralized_common_checks.
tests/test_litellm/proxy/auth/test_user_api_key_auth.py	Adds general_settings={} to the existing stale-reservation test and two new mock-only tests that pin the flag-on and flag-off code paths; no real network calls, correctly structured.

_{Reviews (2): Last reviewed commit: "feat(proxy): add disable_budget_reservat..." | Re-trigger Greptile}

[Ar-maan05]

@greptile-apps

[veria-ai]

Medium: Budget limit bypass through concurrent requests

When this returns early, the remaining budget checks only compare current spend against the limit; they do not reserve estimated cost, and actual spend counters are incremented after the LLM call completes. A key holder can start many concurrent expensive requests just below a max budget and have each request pass the read-time check, then exceed the configured budget after the responses are charged. If this escape hatch must exist, treat it as disabling hard budget enforcement in configuration/docs, or keep an atomic admission check that accounts for estimated request cost without leaking counters.

[Sameerlite]

@Ar-maan05 Please fixx

[Ar-maan05]

I have addressed the concern in two ways:

1. Add a verbose_proxy_logger.warning() at the early-return point in _reserve_budget_after_common_checks so operators see an explicit warning in proxy logs on every request while the flag is active.

2. Expand the disable_budget_reservation description in ConfigGeneralSettings to prominently call out that this weakens hard budget enforcement and document the concurrency tradeoff.

[veria-ai]

PR overview

This PR adds a general proxy setting to disable budget reservation behavior during API key authentication and request admission. The changed code alters how budget enforcement proceeds when that setting is enabled.

There is one open security concern remaining: disabling budget reservation can let an authorized key holder launch concurrent requests that all pass the current-spend check before usage is charged, allowing configured budgets to be exceeded. The impact is limited to budget enforcement and spend control rather than broader data access or code execution, but it is a concrete attacker-actionable bypass of a configured limit. No issues have been marked fixed yet.

Open issues (1)

• Medium: Budget limit bypass through concurrent requests — litellm/proxy/auth/user_api_key_auth.py:2134

Fixed/addressed: 0 · PR risk: 6/10

[Ar-maan05]

Some CI tests seem to be flaky, but my branch is passing all the required tests.

[Sameerlite]

@Ar-maan05 Please resolve the comments from greptile.Please get the score 5/5. You can tag it again for a new review after a commit.

[Ar-maan05]

@greptile-apps

[Ar-maan05]

@Sameerlite seems like there are two greptile review comments. I have resolved the issues stated by greptile and received a 5/5 in the latest review.

[Ar-maan05]

@Sameerlite friendly ping! Let me know if any changes are needed!

[Ar-maan05]

test_tokenizers in Unit Tests: Core Utilities / core-utils / Run tests (pull_request) failed because the CI runner couldn't reach HuggingFace Hub to download a tokenizer file: a network connectivity issue on the runner, not a code issue. This is a pre-existing flaky test that has nothing to do with my changes. Let me know if any further changes are required!

[Ar-maan05]

@Sameerlite friendly ping, I have made the required changes, let me know if any further changes are required.