test(ci): repair stale tests and move retired OpenAI text-completion fixtures

[yuneng-berri]

TLDR

Problem this solves:

• 13 tests red on main in GHA and CircleCI after intended product changes
• 10 CircleCI text-completion tests pinned to retired OpenAI models
• No product behavior is wrong; the tests encode stale contracts, fakes, or fixtures

How it solves it:

• Request fakes carry the scope every real Starlette request has
• GCS pub/sub spend-log golden gains the agent identity keys from feat(agents): authenticate Entra identities and delegated requests #43722
• Auto-router session tests follow the baseline_models contract from fix(autorouter): compare historical and new savings consistently #43348
• Interactions spec checks find the create body and resource paths by shape
• Single-prompt native text completions move to gpt-5.4-nano, which OpenAI still serves on /v1/completions
• Batched prompts and echo with logprobs move to Fireworks on the same transport
• The optional-params test now asserts the request body actually sent
• The integration-security route sweep skips GET /roi-calculator/repositories from feat(proxy): add native ROI calculator for gateway spend vs merged PRs #43669, which lists repositories from api.github.com

User Flow

Before: a contributor's PR inherits red GHA and CircleCI unit jobs that have nothing to do with their change

1. They open a PR against main and GHA Unit Tests runs proxy-endpoints and misc
2. Both jobs fail on batches metadata and Google Interactions spec tests
3. The scheduled CircleCI pipeline also fails unit, local_testing_part1, local_testing_part2, and logging_testing, including 404s for retired OpenAI models

After: those jobs only go red when LiteLLM behavior actually changes

1. They open a PR against main and GHA Unit Tests runs proxy-endpoints and misc
2. The repaired tests pass, and still fail when the behavior they guard is broken

Screenshots / Proof of Fix

Tests-only PR, so there is no customer-facing request to curl. Product code is identical in Before and After; the proof is that each repaired test passes on current code and fails when the guarded behavior is mutated

Before (38b0762)

1. pytest tests/test_litellm/proxy/batches_endpoints/test_endpoints.py -k non_object tests/local_testing/test_http_parsing_utils.py tests/unit/models/test_models.py::TestAutoRouterSession tests/unit/interactions/test_openapi_compliance.py tests/logging_callback_tests/test_gcs_pub_sub.py::test_async_gcs_pub_sub_v1
2. 13 failed, matching CI: batches metadata 400 x2 (assert 'input_file_id' == 'metadata'), http parsing x3 ('MockRequest' object has no attribute 'scope'), auto-router label x3, Interactions spec x4 (KeyError: 'CreateModelInteractionParams', GET /interactions/{id} endpoint not found), GCS golden x1 (Extra key in actual: billing_agent_id and 5 metadata keys)
3. test_read_request_body_empty_body and test_read_request_body_unexpected_error were also passing vacuously: both hit the missing-scope fallback, not the path they name

Retired text-completion fixtures

4. CircleCI local_testing_part1 and local_testing_part2 fail 10 tests with HTTP 404 The model gpt-3.5-turbo-instruct / davinci-002 / gpt-3.5-turbo-1106 has been deprecated
5. test_completion_openai_with_optional_params was also vacuous: its success callback still asserted model == "gpt-3.5-turbo-1106" after I pointed the call at gpt-6-luna, and the test passed, because callback failures never reach pytest

After (ac9e725)

1. Same command plus the full mapped files: 357 passed
2. Product mutations, each restored after the run:
   • Valid JSON body parsed to {}: 3 http parsing tests fail
   • Invalid JSON swallowed instead of raising: 2 http parsing tests fail
   • Non-object batch metadata coerced to {} instead of 400: both batches params fail
   • Partial-comparison guard removed from baseline_model: the multi-baseline partial test fails
   • Guard applied to a single baseline too: the single-baseline partial test fails
   • Tie broken by insertion order: the tie test fails
   • billing_agent_id dropped from the spend log: GCS golden fails with Missing key: billing_agent_id
   • Spec without tools on the model create body, without DELETE on the resource path, or without model required: the matching spec test fails
3. Live OpenAI, real spend: the 6 OpenAI-backed text-completion tests pass (test_completion_text_openai, _async, test_qwen_text_completion, test_completion_openai_with_optional_params, test_completion_gpt_instruct, test_text_completion_basic)
4. Mutations for those: GPT-5 config dropping seed or user fails the optional-params test with KeyError; text-completion config dropping logprobs fails test_qwen_text_completion
5. CircleCI integration-security on main (ed4caeb): 31 tests fail with Owned proxy tried to reach external hosts: [b'CONNECT api.github.com:443 HTTP/1.1'], because the S2 sweep GETs every route and the new ROI route calls https://api.github.com/user/repos under default settings
6. Fireworks-backed tests (test_completion_openai_prompt, test_completion_text_003_prompt_array, test_text_completion_with_echo[True/False]): with a mocked transport, LiteLLM sends the 2 string prompts, the 2 token-ID prompts, and echo plus logprobs unchanged to https://api.fireworks.ai/inference/v1/completions. Live verification runs in CircleCI local_testing_part2, since no Fireworks key exists outside CI

Why these vehicles

I probed every model on our OpenAI key against /v1/completions. The gpt-5.4 family and gpt-5.1 still accept it; gpt-5.6-terra and gpt-6-luna are chat-only. On every model that accepts it, a single prompt, logprobs up to 10, streaming, and echo alone work, but a prompt list with more than one entry and echo combined with logprobs return HTTP 500. Fireworks documents string and token-ID prompt batches and echo with logprobs, so those cases keep the text-completion-openai code path and only change the upstream

Type

Test

Caveats (if any)

Medium

• The 4 Fireworks-backed tests are unverified live until CircleCI runs them
  • Fireworks' serverless catalog rotates, so glm-5p3-flash may need a swap later
• OpenAI's 500 on batches and echo with logprobs is read as unsupported, not as an outage
  • Consistent across 6 models and retries, but OpenAI returned no 4xx naming the limit
• Interactions tests still read Google's live spec, by design per the file
  • Google renamed {id} to {interactionsId} and folded CreateModelInteractionParams into a ModelInteraction oneOf variant
  • input is no longer marked required upstream, so the test now checks it is accepted rather than required
• Left out on purpose, still red on main:
  • CircleCI e2e_ui_testing call-id tooltip spec, unresolved and needs its own investigation
  • CircleCI integration-extensions H4: a product regression from perf(proxy): one post-call Redis pipeline per backend for spend, rate-limit, routing and response-cache writes #43779, where the response-cache write waits for every success callback, so an immediate identical request misses the cache
  • GHA LiteLLM Rust stubtest: NativeTraceStorage.query stub says sql, runtime says query
  • GHA documentation / code-quality: fixed in litellm-docs ([FIX] s3 cache proxy - fix notImplemented error #1966 merged), plus Codecov signature failures in proxy-behavior

[greptile-apps]

Confidence Score: 5/5

[Low risk] Test files repaired to match current code and specs.

The PR appears safe to merge; no actionable issue was established in the changed tests.

Summary

This test-only PR updates request fakes, the spend-log golden payload, auto-router baseline assertions, and Google Interactions spec lookups to match current contracts. No actionable regression was established.

_{Reviews (1) · Last reviewed commit: "test(ci): repair stale request fakes, sp..."}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!