chore(release): backport part of #43902 and #43983 to rc/1.104.0

[yuneng-berri]

TLDR

Problem this solves:

• Two rc/1.104.0 Buildkite e2e tests fail on every run
• One calls the deprecated gpt-3.5-turbo-instruct, OpenAI returns 404
• One picks a Together reasoning-only model that returns no content

How it solves it:

• Cherry-picks the matching test hunks from test(e2e): repair completion, SAIL, and spend-log fixtures #43902 and test(ci): repair stale tests and flaky CI infrastructure #43983 with -x
• Both files now match main exactly

User Flow

Before: the rc/1.104.0 e2e suite stays red on two tests no retry can fix

1. A release engineer runs the litellm-e2e Buildkite pipeline at the rc/1.104.0 tip
2. test_text_completion_returns_text fails with a 404 saying gpt-3.5-turbo-instruct has been deprecated
3. test_response_format_json_schema_shapes_the_reply fails with content=None from together_ai/zai-org/GLM-5.3-Flash
4. Both fail again in the retry round (Buildkite build 352)

After: the same pipeline no longer hits either failure

1. A release engineer runs the litellm-e2e Buildkite pipeline at the rc/1.104.0 tip
2. The text completion test calls openai/gpt-5.4-nano with reasoning off and gets text back
3. The Together structured-output test runs on together_ai/Qwen/Qwen3.5-9B with reasoning off and gets schema-shaped JSON

Relevant issues

Backports part of #43902 and part of #43983 to rc/1.104.0 so the e2e suite can go green before the next 1.104.0 release candidate. Test-only, no version bump

What is included, in order:

• test(e2e): repair completion, SAIL, and spend-log fixtures #43902 test(e2e): align completion, SAIL, and spend-log fixtures with supported contracts (only tests/e2e/llm_translation/test_completions_endpoint_e2e.py)
• test(ci): repair stale tests and flaky CI infrastructure #43983 test(ci): repair stale tests and flaky CI infrastructure (only tests/e2e/llm_translation/test_together_ai_e2e.py)

Pre-Submission checklist

• I have added meaningful tests
• The handful of test files covering my change pass locally
• My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review

Screenshots / Proof of Fix

Local proxy started from this branch with this config (master key redacted), hitting real OpenAI:

model_list:
  - model_name: instruct-before
    litellm_params:
      model: text-completion-openai/gpt-3.5-turbo-instruct
  - model_name: nano-after
    litellm_params:
      model: openai/gpt-5.4-nano

Before (31d6c60)

Text completion

1. Buildkite litellm-e2e build 352, round 1 retry: 2 failed, 2 passed, the two failures are test_text_completion_returns_text and test_response_format_json_schema_shapes_the_reply
2. The model the old test uses, called through the proxy:

curl -s localhost:4517/v1/completions -H "Authorization: Bearer $MASTER_KEY" -H 'Content-Type: application/json' \
  -d '{"model": "instruct-before", "prompt": "Say hello in one word.", "max_tokens": 16}'

{"error":{"message":"litellm.NotFoundError: OpenAIException - Error code: 404 - {'error': {'message': 'The model `gpt-3.5-turbo-instruct` has been deprecated, learn more here: https://platform.openai.com/docs/deprecations', 'type': 'invalid_request_error', 'param': None, 'code': 'model_not_found'}}\n\nLiteLLM: model group 'instruct-before' failed with the error above. No fallback was attempted.","type":null,"param":null,"code":"404"}}

Together structured output

1. Buildkite build 352: AssertionError: together_ai/zai-org/GLM-5.3-Flash returned no content: role='assistant' content=None reasoning_content='The user wants me to invent a fictional person...'

After (ddd3c48)

Text completion

1. The model the new test uses, same request shape as the test:

curl -s localhost:4517/v1/completions -H "Authorization: Bearer $MASTER_KEY" -H 'Content-Type: application/json' \
  -d '{"model": "nano-after", "prompt": "Say hello in one word.", "max_tokens": 16, "reasoning_effort": "none"}'

{"id":"chatcmpl-EUmM5t067ggVYftZC4FjztakyCwU8","object":"text_completion","created":1791004425,"model":"nano-after","choices":[{"finish_reason":"stop","index":0,"text":"Hello","logprobs":null}],"usage":{"completion_tokens":4,"prompt_tokens":12,"total_tokens":16,...}}

Together structured output

1. Not run locally, there is no Together key on this machine. It will be proven by the Buildkite run at the rc tip after merge

Type

Test

Caveats (if any)

Low

• Together after-proof is pending the post-merge Buildkite run
• Bedrock 5m prompt caching and RSS memory also failed in build 352
  • Both passed in the retry round, so they are flakes and are not touched here
• The kept hunks are identical to upstream, both files match main byte for byte

QA runbook

• tests/e2e/llm_translation/test_completions_endpoint_e2e.py::TestCompletionsEndpoint::test_text_completion_returns_text - /v1/completions on an OpenAI deployment returns text
  • Add a deployment with model: openai/gpt-5.4-nano (needs OPENAI_API_KEY)
  • POST /v1/completions with a short prompt, max_tokens: 16 and reasoning_effort: "none"
  • Expect 200 with object: "text_completion" and non-empty choices[0].text
  • Sanity check: this test makes sense to add and is not hand-wavey or potentially flaky
• tests/e2e/llm_translation/test_together_ai_e2e.py::TestTogetherChatCompletions::test_response_format_json_schema_shapes_the_reply - a json_schema response_format shapes the Together reply
  • Add a deployment with model: together_ai/Qwen/Qwen3.5-9B (needs TOGETHERAI_API_KEY)
  • POST /v1/chat/completions asking for a fictional person, max_tokens: 1024, reasoning_effort: "none" and a json_schema response_format with a name field
  • Expect choices[0].message.content to be JSON matching the schema with a non-empty name
  • Sanity check: this test makes sense to add and is not hand-wavey or potentially flaky

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

[greptile-apps]

Confidence Score: 4/5

[Low risk] Updates test fixtures and documentation for LLM endpoint tests.

The live native text-completion coverage should be restored before merging to satisfy the repository’s test-coverage requirement.

Findings

1.  Native completion coverage lost ▶

Summary

This test-only backport replaces a deprecated OpenAI instruct deployment and a Together model that returned no content in two live E2E tests.

• The OpenAI test now uses a chat model with reasoning disabled.
• The Together structured-output test now pins a schema-capable Qwen hybrid model with reasoning disabled.
• The OpenAI substitution leaves the native text-completion adapter without live E2E coverage.

_{Reviews (1) · Last reviewed commit: "test(ci): repair stale tests and flaky C..."}

[greptile-apps]

Native completion coverage lost

Registering openai/gpt-5.4-nano makes this test use the chat-model adapter behind /v1/completions, not the native text-completion-openai adapter it previously tested. No live E2E test now covers that native path, so a regression there could pass the suite. The repository requires changes to existing tests not to weaken regression coverage; please preserve a live check for the native path before merging.

Rule Used: What: Flag any modifications to existing tests and verify they don't weaken test coverage or mask regressions. Why: Developers may alter tests to make failing code pass rather than fix the actual bug, hiding regressions. Good: ``` // Test updated t... (source)

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!