feat(proxy)!: gate all mock testing request params behind a single config flag

[yuneng-berri]

TLDR

Problem this solves:

• Six mock testing params, three different behaviors
• Two reached the router ungated, three were dropped
• A dropped param returned a normal success
• So a fallback drill could pass without running

How it solves it:

• One config flag now gates all six
• Unset is the default and rejects with 400
• The rejection names the params and the key
• Enabling it logs a startup warning

Relevant issues

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

All runs below are against a live proxy at commit 2a9aa966ce, hitting the real Anthropic API

Config used, two variants differing only by the general_settings line:

model_list:
  - model_name: primary-model
    litellm_params:
      model: anthropic/claude-haiku-4-5
      api_key: os.environ/ANTHROPIC_API_KEY
  - model_name: fallback-model
    litellm_params:
      model: anthropic/claude-sonnet-4-5
      api_key: os.environ/ANTHROPIC_API_KEY

general_settings:
  master_key: sk-1234
  # dangerously_allow_mock_testing_request_params: true   <- present only in the second run

1. Flag not set (the default): gated params are refused

curl -s -w "\nHTTP %{http_code}\n" http://127.0.0.1:4020/v1/chat/completions -H "Authorization: Bearer sk-1234" -H "Content-Type: application/json" -d '{"model":"primary-model","messages":[{"role":"user","content":"hi"}],"mock_testing_fallbacks":true}'

{"error":{"message":"Mock testing request params are disabled on this proxy: mock_testing_fallbacks. An admin can enable them by setting `general_settings.dangerously_allow_mock_testing_request_params: true` in config.yaml. This setting cannot be changed from the Admin UI or the API.","type":"None","param":"None","code":"400",...}}
HTTP 400

Several at once are all named, so fixing one does not surprise you with the next:

curl -s http://127.0.0.1:4020/v1/chat/completions -H "Authorization: Bearer sk-1234" -H "Content-Type: application/json" -d '{"model":"primary-model","messages":[{"role":"user","content":"hi"}],"mock_timeout":true,"mock_delay":30}'

Mock testing request params are disabled on this proxy: mock_timeout, mock_delay. An admin can enable them by setting `general_settings.dangerously_allow_mock_testing_request_params: true` in config.yaml. This setting cannot be changed from the Admin UI or the API.

2. Flag not set: ordinary traffic is untouched

curl -s -w "HTTP %{http_code}\n" http://127.0.0.1:4020/v1/chat/completions -H "Authorization: Bearer sk-1234" -H "Content-Type: application/json" -d '{"model":"primary-model","messages":[{"role":"user","content":"Reply with exactly: ok"}],"max_tokens":10}'

content: ok | model: primary-model
HTTP 200

No startup warning is emitted in this run

3. Flag set: startup warning

========================================================================
 DANGEROUS SETTING ENABLED
 general_settings.dangerously_allow_mock_testing_request_params = true

 Any caller with a valid key on this proxy can now inject synthetic
 failures and latency into their own requests using these body params:
   mock_testing_fallbacks
   mock_testing_context_fallbacks
   mock_testing_content_policy_fallbacks
   mock_testing_rate_limit_error
   mock_timeout
   mock_delay

 A request using them consumes a connection and a concurrency slot
 without reaching a provider, and returns an error the caller chose.

 Intended for testing fallback chains. Do not leave enabled.
========================================================================

The warning lists every param the flag unlocks, because the key name says mock_testing while the gate also covers mock_timeout and mock_delay

4. Flag set: a forced fallback actually runs

curl -s -D headers.txt http://127.0.0.1:4021/v1/chat/completions -H "Authorization: Bearer sk-1234" -H "Content-Type: application/json" -d '{"model":"primary-model","messages":[{"role":"user","content":"Reply with exactly: fell back"}],"max_tokens":10,"mock_testing_fallbacks":true,"fallbacks":["fallback-model"]}'
grep -i x-litellm-attempted-fallbacks headers.txt

content: fell back
model that served it: claude-sonnet-4-5-20250929
x-litellm-attempted-fallbacks: 1

The request named primary-model (haiku) and was served by claude-sonnet-4-5, so the fallback chain was exercised for real rather than reported as exercised

Type

🆕 New Feature

Changes

The six params are mock_testing_fallbacks, mock_testing_context_fallbacks, mock_testing_content_policy_fallbacks, mock_testing_rate_limit_error, mock_timeout and mock_delay. They now share one gate in route_request, which is where the dispatch chokepoint is; the Responses WebSocket route calls it directly and never runs add_litellm_data_to_request, so putting the gate with its siblings in litellm_pre_call_utils would have missed that route

Rejecting rather than dropping is the behavioral change that matters. A dropped param used to produce an ordinary 200 from the real model, which reads as a fallback test that passed when nothing was mocked at all. The 400 names the params it saw and the config key to set, which also answers anyone following the older docs

The flag is config-file only. It is deliberately absent from ConfigGeneralSettings, and that absence is what makes /config/update drop it on parse and /config/field/update reject it. Two tests pin that behavior so the field cannot be added back as a tidy-up without the reason surfacing

mock_response and mock_tool_calls are unchanged and keep their existing per key or team allow_client_mock_response opt-in

Breaking change

mock_timeout and mock_testing_rate_limit_error were accepted unconditionally before this and now require the flag. Anyone sending them to a proxy today gets a 400 until an admin sets it in config.yaml

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

[greptile-apps]

Greptile Summary

The PR introduces a config-file-only gate for six mock-testing request parameters.

• Rejects gated parameters with HTTP 400 unless explicitly enabled.
• Emits a startup warning when the dangerous opt-in is active.
• Adds coverage for request gating, pass-through behavior, configuration API restrictions, and startup warnings.
• Removes the inaccurate WebSocket request-flow comment identified previously.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains.

Important Files Changed

Filename	Overview
litellm/proxy/proxy_server.py	Adds the startup warning and tests that prevent the opt-in from being enabled through configuration APIs.
litellm/proxy/route_llm_request.py	Replaces unconditional mock-parameter stripping with a centralized config-controlled rejection or pass-through gate.
tests/test_litellm/proxy/test_proxy_server.py	Covers startup warning contents and verifies that management endpoints cannot persist the config-file-only flag.
tests/test_litellm/proxy/test_route_llm_request.py	Covers all six gated parameters, default rejection, enabled forwarding, multi-parameter errors, and ordinary requests.

_{Reviews (2): Last reviewed commit: "refactor(proxy): drop an inaccurate comm..." | Re-trigger Greptile}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[yuneng-berri]

@greptile review again and give an updated score

[devin-ai-integration]

QA verdict: PASS

The single config flag general_settings.dangerously_allow_mock_testing_request_params gates all six mock-testing request params exactly as described. With the flag absent every gated param is rejected with a 400 that names each detected param and the config key, ordinary traffic is untouched, the flag cannot be set through the Admin UI or the config API, and a startup warning fires only when it is enabled. Enabling it lets mock_testing_fallbacks, mock_testing_context_fallbacks and mock_testing_content_policy_fallbacks drive real fallback chains and mock_timeout raise a synthetic timeout. Two pre-existing per-param quirks (mock_delay, mock_testing_rate_limit_error) are noted at the bottom as non-blockers; neither is introduced or changed by this PR

Everything below hit a live proxy backed by real Anthropic calls (primary anthropic/claude-haiku-4-5, fallback anthropic/claude-sonnet-4-5), plus a Postgres-backed proxy for the config/API paths

Config used

Two proxies, identical except for the one flag. Disabled proxy on :4020, enabled proxy on :4021

model_list:
  - model_name: primary-model
    litellm_params:
      model: anthropic/claude-haiku-4-5
      api_key: os.environ/ANTHROPIC_API_KEY
  - model_name: fallback-model
    litellm_params:
      model: anthropic/claude-sonnet-4-5
      api_key: os.environ/ANTHROPIC_API_KEY

general_settings:
  master_key: sk-1234
  # dangerously_allow_mock_testing_request_params: true   # only in the enabled proxy

The config/API paths were exercised against a third proxy on :4030 with store_model_in_db: true and DATABASE_URL pointed at a local Postgres

Sad path: flag absent, every gated param rejected (:4020)

Each of the six is rejected on its own with a 400 that names it and points at the config key. Only the first is shown; the other five are identical in shape

$ curl -s -w 'HTTP %{http_code}' http://127.0.0.1:4020/v1/chat/completions \
    -H 'Authorization: Bearer sk-1234' -H 'Content-Type: application/json' \
    -d '{"model":"primary-model","messages":[{"role":"user","content":"hi"}],"mock_testing_fallbacks":true}'
{"error":{"message":"Mock testing request params are disabled on this proxy: mock_testing_fallbacks. An admin can enable them by setting `general_settings.dangerously_allow_mock_testing_request_params: true` in config.yaml. This setting cannot be changed from the Admin UI or the API.", ... ,"code":"400"}}
HTTP 400

All present params are named in one error when several are sent together

$ ... -d '{... ,"mock_timeout":true,"mock_delay":30}'
"...disabled on this proxy: mock_timeout, mock_delay. ..."
HTTP 400

$ ... -d '{... all six ...}'
"...disabled on this proxy: mock_testing_fallbacks, mock_testing_context_fallbacks, mock_testing_content_policy_fallbacks, mock_testing_rate_limit_error, mock_timeout, mock_delay. ..."
HTTP 400

Edge case: the gate keys on presence, not truthiness, so a param sent with false is still rejected. This is the safe choice; worth a note only so it is intentional

$ ... -d '{... ,"mock_testing_fallbacks":false}'
"...disabled on this proxy: mock_testing_fallbacks. ..."
HTTP 400

Ordinary traffic is untouched, and the non-gated mock_response path still works

$ ... -d '{"model":"primary-model","messages":[{"role":"user","content":"Reply with exactly: ok"}],"max_tokens":10}'
{... "content":"ok" ...}
HTTP 200

$ ... -d '{... ,"mock_response":"canned reply"}'
{... "content":"Hey! How's it going? ..." ...}
HTTP 200

Happy path: flag enabled (:4021)

mock_testing_fallbacks forces a real fallback: the request is served by the sonnet fallback and the response carries x-litellm-attempted-fallbacks: 1

$ curl -D- -s http://127.0.0.1:4021/v1/chat/completions -H 'Authorization: Bearer sk-1234' \
    -H 'Content-Type: application/json' \
    -d '{"model":"primary-model","messages":[{"role":"user","content":"say: fell back"}],"fallbacks":["fallback-model"],"mock_testing_fallbacks":true}'
x-litellm-attempted-fallbacks: 1
served model: claude-sonnet-4-5-20250929
content: fell back

mock_testing_context_fallbacks and mock_testing_content_policy_fallbacks behave the same against their respective fallback maps (both served by sonnet, x-litellm-attempted-fallbacks: 1)

mock_timeout raises a synthetic timeout end to end

$ ... -d '{... ,"mock_timeout":true,"timeout":1}'
{"error":{"message":"litellm.Timeout: This is a mock timeout error ...","code":"408"}}
HTTP 408

Ordinary traffic still returns 200 with the flag on

Cannot be set from the Admin UI or the config API

/config/field/update rejects the flag as an unknown field

$ curl -s -w 'HTTP %{http_code}' http://127.0.0.1:4030/config/field/update -H 'Authorization: Bearer sk-1234' \
    -H 'Content-Type: application/json' \
    -d '{"field_name":"dangerously_allow_mock_testing_request_params","field_value":true,"config_type":"general_settings"}'
{"detail":{"error":"Invalid field=dangerously_allow_mock_testing_request_params passed in."}}
HTTP 400

/config/update returns success but silently drops the undeclared key, and a gated request stays rejected afterward, confirming it never reached runtime

$ curl -s http://127.0.0.1:4030/config/update ... -d '{"general_settings":{"dangerously_allow_mock_testing_request_params":true}}'
{"message":"Config updated successfully"}

$ curl -s -w 'HTTP %{http_code}' http://127.0.0.1:4030/v1/chat/completions ... -d '{... ,"mock_testing_fallbacks":true}'
"...disabled on this proxy: mock_testing_fallbacks. ..."
HTTP 400

The Admin UI matches: the General tab under Router Settings lists every editable general_settings field and the flag is not among them

Startup warning

Absent when the flag is off. When on, it names the setting and all six params

DANGEROUS SETTING ENABLED
general_settings.dangerously_allow_mock_testing_request_params = true

Any caller with a valid key on this proxy can now inject synthetic
failures and latency into their own requests using these body params:
  mock_testing_fallbacks
  mock_testing_context_fallbacks
  mock_testing_content_policy_fallbacks
  mock_testing_rate_limit_error
  mock_timeout
  mock_delay
...
Intended for testing fallback chains. Do not leave enabled.

Unit tests

The PR's own tests pass: tests/test_litellm/proxy/test_route_llm_request.py and tests/test_litellm/proxy/test_proxy_server.py, 346 passed

Non-blockers (pre-existing, not introduced by this PR)

mock_delay sent on its own with the flag enabled is forwarded to the provider and 400s with AnthropicException ... "mock_delay: Extra inputs are not permitted", and applies no delay. It is only consumed on the mock-completion path (alongside mock_response, mock_tool_calls or mock_timeout, via should_run_mock_completion in litellm/main.py). This param was never stripped before this PR, so the behavior is unchanged; the gate just makes it reachable only when the flag is on

mock_testing_rate_limit_error on a single-deployment group with num_retries >= 1 raises the synthetic RateLimitError once (visible in the router log, _mock_rate_limit_error() - Raising mock RateLimitError), then the automatic retry makes a real call and the request returns 200. To see a surfaced failure you need a fallback chain or num_retries: 0. Again this is existing router retry behavior, unchanged here

[codspeed-hq]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_/mock-testing-feature-flag-6e30ad (764b233) with litellm_internal_staging (fa56283)¹}

Footnotes

1. No successful run was found on litellm_internal_staging (b5cfc2c) during the generation of this report, so fa56283 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩