fix(e2e): stop tests from breaking the shared proxy for every suite after them by mubashir1osmani · Pull Request #34664 · BerriAI/litellm

mubashir1osmani · 2026-07-25T22:28:22Z

TLDR

Problem this solves:

A single e2e test permanently broke Redis on the deployment it ran against, which failed 60 of 72 tests in one run
A second test applied a Bedrock guardrail to all proxy traffic, so an upstream 403 failed four unrelated tests
The OpenAI passthrough body sent a parameter current OpenAI models reject

How it solves it:

Round-trip the cache settings blob verbatim instead of a three-field subset, and refuse to write when the round-trip cannot be lossless
Register the Bedrock guardrail opted out of default_on and select it per request
Send max_completion_tokens on the passthrough route, which forwards the body untranslated

Relevant issues

Product defects found while diagnosing this, filed separately: LIT-4816 (cache settings cannot round-trip transport fields), LIT-4817 (Redis-only budgets fail open), LIT-4818 (probe timing), plus evidence added to LIT-3802 (consolidate Redis configuration)

Linear ticket

The cache settings bug

TestCacheSettings.test_update_persists_cache_backend_to_get read the live cache settings and wrote them back, intending a no-op. Its capture modelled only type/host/port:

class CacheSettingsValue(BaseModel):
    type: str
    host: str = ""
    port: str = ""

On a TLS cluster that write-back dropped ssl and redis_startup_nodes. /cache/settings persists what it receives into LiteLLM_CacheConfig, that row outranks YAML cache_params, and init_cache_settings_in_db re-applies it on a timer, so restarts do not clear it. The proxy then drove a TLS-only cluster endpoint as a plaintext standalone node and every Redis call blocked to socket timeout

Verified on the affected deployment:

plaintext PING              -> FAIL TimeoutError
TLS PING                    -> +PONG
get_redis_client(ssl=True)  -> RedisCluster, PING=True, SET/GET=ok

Downstream that took out rate limiting entirely (the v3 limiter is a Lua script on Redis with no DB fallback), Redis-only budget levels, spend tracking, ResetBudgetJob (self-starved at 54 skipped runs per 15 min), and ProxyConfig.add_deployment, whose last statement syncs guardrails and never ran

The teardown "restore" wrote the same lossy payload, so the safety net was part of the problem. GET /cache/settings also never reads YAML; it resolves the stored row overlaid with REDIS_* env, so on a fresh deploy it cannot even see YAML's ssl to echo back

Changes

CacheSettingsValue is now a RootModel over an exhaustive value union matching litellm's CACHE_SETTINGS_FIELDS types, so a subset cannot be written. Two guards make a regression fail at this test rather than silently downstream:

refuse to write when GET reports redis_type=cluster but omits redis_startup_nodes, the exact precondition for persisting a downgrade
compare /cache/ping before and after, so a write that breaks connectivity fails here

create_bedrock_guardrail defaults to default_on=False and the test selects the guardrail per request via the selector the harness already supports

OpenAIChatBody sends max_completion_tokens. vllm_chat keeps max_tokens, which vLLM accepts

QA runbook

Redis was repaired on the affected deployment by writing the full config through the supported route, then restarting both planes:

POST /cache/settings   {"cache_settings": {..., "ssl": true, "redis_startup_nodes": [...]}}
kubectl rollout restart deploy/litellm-gateway deploy/litellm-backend -n litellm

Verified after:

GET /cache/settings -> keys=[host, namespace, port, redis_startup_nodes, redis_type, ssl, ttl, type]
                       ssl=True  redis_type=cluster
redis "cannot be connected" errors since restart : 0   (was 64 per 10 min)
reset_budget "maximum running instances" skips    : 0   (was 54 per 15 min)

Type

✅ Test

Known remaining work

This PR stops the suite from breaking itself. It does not make the suite fully green, and I would rather say so than imply otherwise:

test_uploaded_file_appears_in_list is skipped, not fixed. GET /v1/files does not include a just-uploaded file even though GET /v1/files/{id} resolves it; the listing stays fixed at 27 entries whose newest created_at is ~10h older than the upload, on both the managed and provider-scoped routes. Filed as LIT-4820, and removing the skip is part of that ticket's definition of done. The assertion is untouched deliberately, since relaxing it would delete the signal that anything enumerating files depends on
test_budget_fallback_reroutes_anthropic_messages_to_openai needed no change. model_max_budget is a Redis-only counter with no DB row, so with Redis unreachable the near-zero budget never registered as exhausted and the fallback never fired. Verified after the Redis repair: call 0 served by claude-haiku-4-5, call 1 by gpt-5.5
test_openai_chat_prompt_cache_hits_on_repeat needed no change either; cached_tokens returns 3615 of 3618 prompt tokens against a dedicated deployment. An earlier reading of 0 was an artifact of probing a fan-out alias
the throughput SLO test failed with "no requests completed in 60s", which was the wedged gateway rather than a throughput regression

Two things I asserted earlier and then disproved, recorded here so nobody re-derives them:

The Bedrock 403 is not an IAM problem. ApplyGuardrail succeeds from the gateway pod under its pod-identity role, the configured identifier and version are valid, and driving the guardrail through the proxy returns a correct 400 Violated guardrail policy. 403 Bedrock guardrail request failed is litellm's unparsed-error fallback (bedrock_guardrails.py:1251) and no longer reproduces; it was most likely a consequence of the starved guardrail sync described above, which left the gateway holding an incompletely synced guardrail.

The presidio failures do not need #34570 either. That PR expands os.environ/ inside guardrail params, but these tests resolve PRESIDIO_ANALYZER_API_BASE in Python and send literal URLs, so there is nothing for it to expand. They failed because _init_non_llm_objects_in_db() never ran, which is the same Redis root cause.

Final Attestation

The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

…is config TestCacheSettings.test_update_persists_cache_backend_to_get read the live cache settings and wrote them back, intending a no-op. Its capture modelled only type/host/port, so on a TLS cluster the write-back silently dropped `ssl` and `redis_startup_nodes`. That is not recoverable on its own. `/cache/settings` persists what it receives into LiteLLM_CacheConfig, that row outranks the YAML `cache_params`, and init_cache_settings_in_db re-applies it on a timer, so a restart does not clear it. The proxy ends up driving a TLS-only cluster endpoint as a plaintext standalone node and every Redis call blocks to socket timeout. On the affected deployment that took out rate limiting entirely (the v3 limiter is a Lua script on Redis with no DB fallback), Redis-only budget levels (tag, per-model, team-member, per-window), spend tracking, `ResetBudgetJob` (which self-starved at 54 skipped runs per 15 min), and `ProxyConfig.add_deployment`, whose last statement syncs guardrails and never ran. 60 of 72 failures in one run traced back here. The settings blob is now round-tripped verbatim via a RootModel over an exhaustive value union, so a subset cannot be written. Two guards make a regression fail loudly at this test instead of silently downstream: - refuse to write when GET reports redis_type=cluster but omits redis_startup_nodes, which is the exact precondition for persisting a downgrade. GET resolves the stored row overlaid with REDIS_* env and never reads YAML, so a cluster configured only in YAML cannot round-trip here - compare /cache/ping before and after, so a write that breaks connectivity fails this test rather than every suite that follows The underlying product defect is filed as LIT-4816: GET cannot express the effective config, and a partial POST is allowed to downgrade transport. This change only stops the suite from triggering it; the Admin UI can still do so. basedpyright clean (0 errors) under the e2e gate.

…urrent token param Two failures that had nothing to do with the guardrail or route under test. create_bedrock_guardrail registered with default_on=True, which applies the guardrail to every request the proxy serves. The upstream ApplyGuardrail call was answering 403, and that came back to unrelated traffic as `403 Bedrock guardrail request failed`, failing three a2a tests and a passthrough headers test alongside the bedrock one. The harness already supports the per-request `guardrails` selector, so the guardrail is now registered opted out of default_on and selected by the test that wants it. A broken upstream guardrail fails its own test instead of whatever else is running. Note this only contains the blast radius; the 403 itself still needs the bedrock:ApplyGuardrail permission (or a valid guardrail identifier) on the deployment, so test_bedrock_pre_call_blocks_harmful_prompt can still fail on its own until that is sorted. The OpenAI passthrough body sent `max_tokens`, which newer models reject with "Unsupported parameter: 'max_tokens' is not supported with this model. Use 'max_completion_tokens' instead." Passthrough forwards the body untranslated, so drop_params does not apply and the body has to satisfy OpenAI's contract directly. vllm_chat keeps max_tokens, which vLLM accepts. basedpyright clean (0 errors) under the e2e gate.

greptile-apps · 2026-07-25T22:30:34Z

Greptile Summary

This PR hardens shared-proxy E2E tests by preserving cache settings, selecting Bedrock guardrails per request, and updating the OpenAI passthrough token parameter.

Changes Bedrock guardrail registration to opt out of global default application and selects the guardrail on the test request.
Sends max_completion_tokens through the untranslated OpenAI passthrough route.
Replaces the fixed cache-settings capture model with a full-blob model and adds transport and connectivity checks around the write.

Confidence Score: 4/5

The Sentinel cache-settings validation gap should be fixed before merging because it makes the updated E2E test fail on a supported deployment configuration.

The new RootModel improves round-trip safety, but its value union cannot validate nested Sentinel node pairs returned by the management endpoint; the other guardrail and passthrough changes remain correctly scoped to their current callers.

Files Needing Attention: tests/e2e/management/test_config_misc_endpoints_e2e.py

Important Files Changed

Filename	Overview
tests/e2e/guardrails/guardrails_client.py	Adds an explicit default_on option that safely defaults Bedrock test guardrails to request-scoped use.
tests/e2e/guardrails/test_bedrock_guardrail_e2e.py	Selects the newly registered Bedrock guardrail by name on the request while retaining assertions that fail if it does not execute.
tests/e2e/llm_translation/passthrough_client.py	Updates the sole OpenAI passthrough test helper to send max_completion_tokens for its current reasoning model.
tests/e2e/management/test_config_misc_endpoints_e2e.py	Adds safer cache round-trip checks, but the purportedly exhaustive settings value union rejects the supported nested sentinel_nodes shape.

_{Reviews (1): Last reviewed commit: "fix(e2e): scope the bedrock guardrail pe..." | Re-trigger Greptile}

codecov · 2026-07-25T22:36:40Z

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

#34512 pinned `api_key="os.environ/ANTHROPIC_API_KEY"` on the a2a bridge agent. The a2a bridge forwards the agent's litellm_params straight into litellm.acompletion() without expanding "os.environ/" indirection, so that literal string was sent upstream as x-api-key and every message/send failed with `AnthropicException - {"type":"authentication_error","message":"invalid x-api-key"}`. Omitting api_key restores the normal provider resolution: litellm reads ANTHROPIC_API_KEY from the proxy's own environment for this provider, which is what the agent-owner flow depends on and what the suite did before #34512. Verified against a live proxy, same agent shape each time: api_key omitted -> message/send 200 api_key "os.environ/ANTHROPIC_API_KEY" -> message/send 500 invalid x-api-key api_key <literal key> -> message/send 200 and the key itself is valid (direct call to api.anthropic.com returns 200), so this was indirection that never got expanded rather than a bad credential. This accounts for four failures (test_semver_protocol_version_registers_and_serves, test_message_send_runs_completion_bridge, test_pinned_v0_3_serves_flat_message_shape, test_pinned_v1_0_serves_nested_message_shape). They were previously reported as `403 Bedrock guardrail request failed`, because a default_on Bedrock guardrail short-circuited the request before it ever reached the bridge and hid this. The bridge silently ignoring "os.environ/" in agent params is a product defect in its own right, filed separately; anyone configuring an agent credential that way through the UI hits the same wall. basedpyright clean (0 errors) under the e2e gate.

750 users at spawn rate 50 saturated the request path hard enough to distort the latency-sensitive suites sharing the same proxy, and it spends real provider money at that rate. Drop to 200 users at spawn rate 20. The RPS floor moves with the user count rather than staying put, so the assertion keeps its meaning instead of becoming a formality: 355 RPS over 750 users is ~0.47 RPS/user, and 90 over 200 holds that same per-user expectation with a similar pass margin. A request-path regression still trips it. All four knobs stay env-overridable (E2E_LOAD_USERS, E2E_LOAD_SPAWN_RATE, E2E_LOAD_DURATION_SECONDS, E2E_LOAD_MIN_RPS) for a deliberate load run. Note the recorded failure for this test was "no requests completed in 60s", which was the gateway wedged on unreachable Redis rather than a throughput regression; this change is about not perturbing its neighbours, not about that failure.

… reasons test_openai_chat_reasoning_reports_reasoning_tokens asked "A train travels 60 miles in 1.5 hours. What is its average speed in mph?" at reasoning_effort="low", then asserted reasoning_tokens > 0. The model answers that directly without reasoning, so 0 is correct behavior and the assertion was testing the model's discretion rather than litellm's reporting. Verified against a live proxy on a dedicated openai/gpt-5.6 deployment, matching how the test provisions its model: reasoning_effort=low, one-step arithmetic -> reasoning_tokens=0 reasoning_effort=high, the prompt used here -> reasoning_tokens=114 Raised to high effort with a prompt that requires a proof plus a search, so the field under test is actually populated and the assertion fails only if litellm stops surfacing it. While confirming this I also checked prompt caching, which needed no change: cached_tokens comes back 3615 of 3618 prompt tokens on a repeated large prefix against a dedicated deployment. An earlier reading of 0 was an artifact of probing a fan-out alias whose requests land on different deployments, not a caching defect.

GET /v1/files does not include a just-uploaded file. The upload returns 200 and GET /v1/files/{id} resolves it, but the listing never contains it: the returned set stays fixed at 27 entries whose newest created_at is roughly ten hours older than the upload, on both the managed (/v1/files?model=) and provider-scoped (/openai/v1/files) routes. Polled for 40s, so not an eventual-consistency window. Filed as LIT-4820. Skipping keeps a known, ticketed product bug from holding the suite red and masking a new regression somewhere else in the same test. The assertion is left exactly as it was on purpose. It encodes the contract we actually want, that a file retrievable by id is also enumerable, and anything that lists files (a UI picker, cleanup tooling that lists then deletes and would therefore leak provider-side files) depends on it. Relaxing it to get green would delete the signal. The skip reason says so and links the ticket, and the ticket records that removing this marker is part of its definition of done. Matches the existing pattern in this file, where test_unified_file_and_batch_create skips with a reason citing LIT-3266. While skipped, the registry cell llm.files.openai.list.nonstream.works has no passing covering test, so files-list coverage reports as uncovered rather than passing, which is the honest state.

The value union covered scalar lists and lists of mappings, but not lists of lists. `redis_startup_nodes` holds host/port mappings while `sentinel_nodes` holds positional pairs (CACHE_SETTINGS_FIELDS documents `[['localhost', 26379]]`), so on a Sentinel deployment pydantic rejected the response: sentinel_nodes.list[dict[str,...]].1 Input should be a valid dictionary [input_value=['localhost', 26380]] The round-trip test reads GET /cache/settings before it writes anything, so that rejection failed the test at the read, before any assertion ran. A Sentinel deployment would have looked like a broken cache-settings route rather than a model too narrow to parse a documented shape. A list element may now be a scalar, a list or a mapping, which covers both node shapes without special-casing either and tolerates a heterogeneous list instead of rejecting the whole response. Adds TestCacheSettingsModel, harness-level with no `e2e` marker so it runs without a proxy, covering all four backend shapes (cluster mappings, sentinel pairs, plain node, url mode with a null discrete field) plus transport() key selection. Confirmed it fails on the previous union and passes on this one: old union -> 1 failed, 4 passed (the sentinel case) new union -> 5 passed

The test could not fail for the thing it claimed to test, and could break the deployment it ran against. Both halves of that are worth stating. It read the live settings, wrote back identical values, and asserted the read-back matched. If POST /cache/settings were a complete no-op that returned 200 and touched nothing, GET would still return the values read a moment earlier and the test would pass. It verified that GET is stable, not that the route persists anything. Against that, /cache/settings persists what it receives into LiteLLM_CacheConfig, that row outranks YAML cache_params, and init_cache_settings_in_db re-applies it on a timer. A write that omits ssl or redis_startup_nodes converts a TLS cluster into a plaintext standalone client and every later Redis call blocks to socket timeout. On 2026-07-25 that failed 60 of 72 tests in one run: rate limiting stopped enforcing, Redis-only budgets admitted billable over-budget spend, ResetBudgetJob self-starved, and guardrail sync never ran. Guarding the previous shape was not sufficient. Writing the blob verbatim plus a cluster precondition and a /cache/ping check narrowed the hazard but did not remove it, because GET cannot express the effective config: it resolves the stored row overlaid with REDIS_* env and never reads YAML. On a fresh deploy it cannot see YAML's ssl to echo back, so a TLS non-cluster deployment could still have a row written that drops it. No round-trip through this route is safe on a shared proxy. Removed with the models and helpers it owned, and TestCacheSettingsModel with them since it existed only to protect that parsing. The registry row mgmt.cache_settings.update.happy_path stays, now carrying the rationale for why it is deliberately uncovered and what a safe test would require (an isolated proxy, or LIT-4816 fixed so a partial write cannot downgrade transport). Coverage therefore reports this cell as a gap, which is the honest state. Collector passes --strict; the module still collects 11 tests.

…fter them (BerriAI#34664) * fix(e2e): stop the cache-settings test from persisting a degraded Redis config TestCacheSettings.test_update_persists_cache_backend_to_get read the live cache settings and wrote them back, intending a no-op. Its capture modelled only type/host/port, so on a TLS cluster the write-back silently dropped `ssl` and `redis_startup_nodes`. That is not recoverable on its own. `/cache/settings` persists what it receives into LiteLLM_CacheConfig, that row outranks the YAML `cache_params`, and init_cache_settings_in_db re-applies it on a timer, so a restart does not clear it. The proxy ends up driving a TLS-only cluster endpoint as a plaintext standalone node and every Redis call blocks to socket timeout. On the affected deployment that took out rate limiting entirely (the v3 limiter is a Lua script on Redis with no DB fallback), Redis-only budget levels (tag, per-model, team-member, per-window), spend tracking, `ResetBudgetJob` (which self-starved at 54 skipped runs per 15 min), and `ProxyConfig.add_deployment`, whose last statement syncs guardrails and never ran. 60 of 72 failures in one run traced back here. The settings blob is now round-tripped verbatim via a RootModel over an exhaustive value union, so a subset cannot be written. Two guards make a regression fail loudly at this test instead of silently downstream: - refuse to write when GET reports redis_type=cluster but omits redis_startup_nodes, which is the exact precondition for persisting a downgrade. GET resolves the stored row overlaid with REDIS_* env and never reads YAML, so a cluster configured only in YAML cannot round-trip here - compare /cache/ping before and after, so a write that breaks connectivity fails this test rather than every suite that follows The underlying product defect is filed as LIT-4816: GET cannot express the effective config, and a partial POST is allowed to downgrade transport. This change only stops the suite from triggering it; the Admin UI can still do so. basedpyright clean (0 errors) under the e2e gate. * fix(e2e): scope the bedrock guardrail per request and send OpenAI's current token param Two failures that had nothing to do with the guardrail or route under test. create_bedrock_guardrail registered with default_on=True, which applies the guardrail to every request the proxy serves. The upstream ApplyGuardrail call was answering 403, and that came back to unrelated traffic as `403 Bedrock guardrail request failed`, failing three a2a tests and a passthrough headers test alongside the bedrock one. The harness already supports the per-request `guardrails` selector, so the guardrail is now registered opted out of default_on and selected by the test that wants it. A broken upstream guardrail fails its own test instead of whatever else is running. Note this only contains the blast radius; the 403 itself still needs the bedrock:ApplyGuardrail permission (or a valid guardrail identifier) on the deployment, so test_bedrock_pre_call_blocks_harmful_prompt can still fail on its own until that is sorted. The OpenAI passthrough body sent `max_tokens`, which newer models reject with "Unsupported parameter: 'max_tokens' is not supported with this model. Use 'max_completion_tokens' instead." Passthrough forwards the body untranslated, so drop_params does not apply and the body has to satisfy OpenAI's contract directly. vllm_chat keeps max_tokens, which vLLM accepts. basedpyright clean (0 errors) under the e2e gate. * fix(e2e): drop the pinned a2a api_key that broke every message/send BerriAI#34512 pinned `api_key="os.environ/ANTHROPIC_API_KEY"` on the a2a bridge agent. The a2a bridge forwards the agent's litellm_params straight into litellm.acompletion() without expanding "os.environ/" indirection, so that literal string was sent upstream as x-api-key and every message/send failed with `AnthropicException - {"type":"authentication_error","message":"invalid x-api-key"}`. Omitting api_key restores the normal provider resolution: litellm reads ANTHROPIC_API_KEY from the proxy's own environment for this provider, which is what the agent-owner flow depends on and what the suite did before BerriAI#34512. Verified against a live proxy, same agent shape each time: api_key omitted -> message/send 200 api_key "os.environ/ANTHROPIC_API_KEY" -> message/send 500 invalid x-api-key api_key <literal key> -> message/send 200 and the key itself is valid (direct call to api.anthropic.com returns 200), so this was indirection that never got expanded rather than a bad credential. This accounts for four failures (test_semver_protocol_version_registers_and_serves, test_message_send_runs_completion_bridge, test_pinned_v0_3_serves_flat_message_shape, test_pinned_v1_0_serves_nested_message_shape). They were previously reported as `403 Bedrock guardrail request failed`, because a default_on Bedrock guardrail short-circuited the request before it ever reached the bridge and hid this. The bridge silently ignoring "os.environ/" in agent params is a product defect in its own right, filed separately; anyone configuring an agent credential that way through the UI hits the same wall. basedpyright clean (0 errors) under the e2e gate. * test(e2e): make the load suite less aggressive against a shared proxy 750 users at spawn rate 50 saturated the request path hard enough to distort the latency-sensitive suites sharing the same proxy, and it spends real provider money at that rate. Drop to 200 users at spawn rate 20. The RPS floor moves with the user count rather than staying put, so the assertion keeps its meaning instead of becoming a formality: 355 RPS over 750 users is ~0.47 RPS/user, and 90 over 200 holds that same per-user expectation with a similar pass margin. A request-path regression still trips it. All four knobs stay env-overridable (E2E_LOAD_USERS, E2E_LOAD_SPAWN_RATE, E2E_LOAD_DURATION_SECONDS, E2E_LOAD_MIN_RPS) for a deliberate load run. Note the recorded failure for this test was "no requests completed in 60s", which was the gateway wedged on unreachable Redis rather than a throughput regression; this change is about not perturbing its neighbours, not about that failure. * fix(e2e): make the reasoning-tokens assertion exercise a request that reasons test_openai_chat_reasoning_reports_reasoning_tokens asked "A train travels 60 miles in 1.5 hours. What is its average speed in mph?" at reasoning_effort="low", then asserted reasoning_tokens > 0. The model answers that directly without reasoning, so 0 is correct behavior and the assertion was testing the model's discretion rather than litellm's reporting. Verified against a live proxy on a dedicated openai/gpt-5.6 deployment, matching how the test provisions its model: reasoning_effort=low, one-step arithmetic -> reasoning_tokens=0 reasoning_effort=high, the prompt used here -> reasoning_tokens=114 Raised to high effort with a prompt that requires a proof plus a search, so the field under test is actually populated and the assertion fails only if litellm stops surfacing it. While confirming this I also checked prompt caching, which needed no change: cached_tokens comes back 3615 of 3618 prompt tokens on a repeated large prefix against a dedicated deployment. An earlier reading of 0 was an artifact of probing a fan-out alias whose requests land on different deployments, not a caching defect. * test(e2e): skip the files-list test while LIT-4820 is open GET /v1/files does not include a just-uploaded file. The upload returns 200 and GET /v1/files/{id} resolves it, but the listing never contains it: the returned set stays fixed at 27 entries whose newest created_at is roughly ten hours older than the upload, on both the managed (/v1/files?model=) and provider-scoped (/openai/v1/files) routes. Polled for 40s, so not an eventual-consistency window. Filed as LIT-4820. Skipping keeps a known, ticketed product bug from holding the suite red and masking a new regression somewhere else in the same test. The assertion is left exactly as it was on purpose. It encodes the contract we actually want, that a file retrievable by id is also enumerable, and anything that lists files (a UI picker, cleanup tooling that lists then deletes and would therefore leak provider-side files) depends on it. Relaxing it to get green would delete the signal. The skip reason says so and links the ticket, and the ticket records that removing this marker is part of its definition of done. Matches the existing pattern in this file, where test_unified_file_and_batch_create skips with a reason citing LIT-3266. While skipped, the registry cell llm.files.openai.list.nonstream.works has no passing covering test, so files-list coverage reports as uncovered rather than passing, which is the honest state. * fix(e2e): parse Sentinel node lists in the cache-settings model The value union covered scalar lists and lists of mappings, but not lists of lists. `redis_startup_nodes` holds host/port mappings while `sentinel_nodes` holds positional pairs (CACHE_SETTINGS_FIELDS documents `[['localhost', 26379]]`), so on a Sentinel deployment pydantic rejected the response: sentinel_nodes.list[dict[str,...]].1 Input should be a valid dictionary [input_value=['localhost', 26380]] The round-trip test reads GET /cache/settings before it writes anything, so that rejection failed the test at the read, before any assertion ran. A Sentinel deployment would have looked like a broken cache-settings route rather than a model too narrow to parse a documented shape. A list element may now be a scalar, a list or a mapping, which covers both node shapes without special-casing either and tolerates a heterogeneous list instead of rejecting the whole response. Adds TestCacheSettingsModel, harness-level with no `e2e` marker so it runs without a proxy, covering all four backend shapes (cluster mappings, sentinel pairs, plain node, url mode with a null discrete field) plus transport() key selection. Confirmed it fails on the previous union and passes on this one: old union -> 1 failed, 4 passed (the sentinel case) new union -> 5 passed * test(e2e): remove the cache-settings round-trip test The test could not fail for the thing it claimed to test, and could break the deployment it ran against. Both halves of that are worth stating. It read the live settings, wrote back identical values, and asserted the read-back matched. If POST /cache/settings were a complete no-op that returned 200 and touched nothing, GET would still return the values read a moment earlier and the test would pass. It verified that GET is stable, not that the route persists anything. Against that, /cache/settings persists what it receives into LiteLLM_CacheConfig, that row outranks YAML cache_params, and init_cache_settings_in_db re-applies it on a timer. A write that omits ssl or redis_startup_nodes converts a TLS cluster into a plaintext standalone client and every later Redis call blocks to socket timeout. On 2026-07-25 that failed 60 of 72 tests in one run: rate limiting stopped enforcing, Redis-only budgets admitted billable over-budget spend, ResetBudgetJob self-starved, and guardrail sync never ran. Guarding the previous shape was not sufficient. Writing the blob verbatim plus a cluster precondition and a /cache/ping check narrowed the hazard but did not remove it, because GET cannot express the effective config: it resolves the stored row overlaid with REDIS_* env and never reads YAML. On a fresh deploy it cannot see YAML's ssl to echo back, so a TLS non-cluster deployment could still have a row written that drops it. No round-trip through this route is safe on a shared proxy. Removed with the models and helpers it owned, and TestCacheSettingsModel with them since it existed only to protect that parsing. The registry row mgmt.cache_settings.update.happy_path stays, now carrying the rationale for why it is deliberately uncovered and what a safe test would require (an isolated proxy, or LIT-4816 fixed so a partial write cannot downgrade transport). Coverage therefore reports this cell as a gap, which is the honest state. Collector passes --strict; the module still collects 11 tests.

mubashir1osmani added 2 commits July 25, 2026 15:23

greptile-apps Bot reviewed Jul 25, 2026

View reviewed changes

Comment thread tests/e2e/management/test_config_misc_endpoints_e2e.py Outdated

mubashir1osmani added 6 commits July 25, 2026 15:38

mubashir1osmani enabled auto-merge (squash) July 25, 2026 23:01

yucheng-berri approved these changes Jul 25, 2026

View reviewed changes

mubashir1osmani merged commit 64fc19d into litellm_internal_staging Jul 25, 2026
71 of 72 checks passed

mubashir1osmani deleted the litellm_e2e_redis_config_poisoning branch July 25, 2026 23:12

devin-ai-integration Bot mentioned this pull request Jul 25, 2026

test(e2e): prove the global guardrail is live before asserting the team bypass #34641

Open

9 tasks

github-actions Bot mentioned this pull request Jul 25, 2026

🦞 OpenClaw 生态日报 2026-07-26 junlinzhao327-oss/big_model_radar#161

Open

Navigation Menu

Uh oh!

Uh oh!

fix(e2e): stop tests from breaking the shared proxy for every suite after them - #34664

fix(e2e): stop tests from breaking the shared proxy for every suite after them#34664
mubashir1osmani merged 8 commits into
litellm_internal_stagingfrom
litellm_e2e_redis_config_poisoning

mubashir1osmani commented Jul 25, 2026 •

edited

Loading

Uh oh!

greptile-apps Bot commented Jul 25, 2026

Important Files Changed

Uh oh!

Uh oh!

codecov Bot commented Jul 25, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Uh oh!

Conversation

mubashir1osmani commented Jul 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

TLDR

Relevant issues

Linear ticket

The cache settings bug

Changes

QA runbook

Type

Known remaining work

Final Attestation

Uh oh!

greptile-apps Bot commented Jul 25, 2026

Greptile Summary

Confidence Score: 4/5

Important Files Changed

Uh oh!

Uh oh!

codecov Bot commented Jul 25, 2026

Codecov Report

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

mubashir1osmani commented Jul 25, 2026 •

edited

Loading