test(e2e): bypass the proxy response cache in the mid-conversation system and fallback tests

[yuneng-berri]

TLDR

Problem this solves:

• litellm-e2e has been red on every build since 2026-08-20: the three test_unflagged_model_converts_system_reminder_and_succeeds tests (Azure Foundry, Vertex, Bedrock Invoke) fail deterministically in _prime_prompt_cache with prompt cache never became readable in full within 60.0s
• Build 50 also lost both TestReliabilityFallbacks tests to a single empty gpt-5.5 completion

How it solves it:

• The priming helper re-sends an identical /v1/messages body until the usage shows the full prefix read back three times in a row. The e2e stack runs with the litellm response cache on, so every resend after the first is served from redis with the first call's usage (cache_read_input_tokens never grows) and the streak can never form. RichMessagesRequest now sends cache: {"no-cache": true}, the same bypass test_cache_control.py already uses; the key is honored by the @client cache check and filtered out before the provider request
• The two fallback tests sent the same "say hi" / max_tokens=16 body to the gpt-5.5 fallback, so one finish_reason=length completion with empty content served the second test from the response cache and failed both. Each test now uses a unique prompt, and chat_override leaves gpt-5.5 64 tokens to emit text

Test-only change; no product code touched.

Relevant issues

Follows up #36968, which added the consecutive-read priming check. Failing builds: litellm-e2e 43–50 (e.g. https://buildkite.com/berriai-1/litellm-e2e/builds/50).

Pre-Submission checklist

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review

[greptile-apps]

Greptile Summary

This test-only PR prevents proxy response-cache entries from interfering with provider prompt-cache and reliability fallback E2E tests.

• Adds a response-cache bypass to rich Messages API requests.
• Gives fallback tests unique prompts so responses cannot collide in cache.
• Raises the shared reliability helper’s output allowance to avoid empty, length-truncated completions.

Confidence Score: 5/5

The PR appears safe to merge, with no actionable regressions identified in the changed E2E test paths.

The cache bypass targets the proxy response cache rather than the provider prompt cache under test, while unique prompts and the larger output allowance remove the documented cross-test collision and empty-completion conditions without invalidating existing callers.

Important Files Changed

Filename	Overview
tests/e2e/llm_translation/endpoints_client.py	Adds the established no-cache request control so repeated prompt-cache probes reach the provider instead of reusing a proxy response.
tests/e2e/router/reliability_support.py	Raises the completion limit for reliability requests; existing callers do not rely on the previous token count.
tests/e2e/router/test_reliability_fallbacks_e2e.py	Makes each fallback prompt unique without changing the fallback assertions or routing behavior.

_{Reviews (1): Last reviewed commit: "test(e2e): bypass the proxy response cac..." | Re-trigger Greptile}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!