test(e2e): backport Together structured-output and text-completion fixes to stable/1.103.x

[devin-ai-integration]

TLDR

Problem this solves:

• Two e2e tests fail on every stable/1.103.x Buildkite e2e run
• Together structured output picks GLM-5.3-Flash, which returns no content
• Text completion uses gpt-3.5-turbo-instruct, which OpenAI retired (404)

How it solves it:

• Backports the main fixes from test(ci): repair stale tests and flaky CI infrastructure #43983 and test(e2e): repair completion, SAIL, and spend-log fixtures #43902 for these tests only
• Together case pins HYBRID_REASONING_BACKEND with reasoning_effort="none"
• Text completion moves to openai/gpt-5.4-nano with reasoning off

User Flow

Before: a maintainer re-running the stable/1.103.x e2e suite sees two red tests unrelated to litellm code

1. They run the Buildkite e2e lane for stable/1.103.x
2. test_response_format_json_schema_shapes_the_reply fails because the reply content is None
3. test_text_completion_returns_text fails with a 404 from OpenAI for gpt-3.5-turbo-instruct

After: the same run exercises both behaviors on models that still answer

1. They run the Buildkite e2e lane for stable/1.103.x
2. The Together case runs on the hybrid Qwen backend with reasoning off and gets JSON content back
3. The text completion case runs on openai/gpt-5.4-nano with reasoning off and gets text back

Relevant issues

Backports part of #43983 and #43902

Affected release

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• The handful of test files covering my change pass locally, e.g. uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
• My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

No live provider runs were made for this test-only backport. The maintainer re-runs Buildkite e2e on the line after merge. The proof here is that both files collect cleanly and that the Together hunks match main exactly

Before (8a7280e)

1. At the base, both tests fail on every Buildkite e2e run of the line for the reasons in the TLDR

After (ff65781)

Collect-only

1. PYTHONPATH=tests/e2e:tests/e2e/llm_translation uv run --no-sync pytest tests/e2e/llm_translation/test_together_ai_e2e.py tests/e2e/llm_translation/test_completions_endpoint_e2e.py --collect-only -q -c tests/e2e/pytest.ini
2. Output ends with:

llm_translation/test_together_ai_e2e.py::TestTogetherChatCompletions::test_response_format_json_schema_shapes_the_reply
...
llm_translation/test_completions_endpoint_e2e.py::TestCompletionsEndpoint::test_text_completion_returns_text

15 tests collected in 0.85s

3. uv run --no-sync basedpyright on the three touched files: 0 errors, 0 warnings, 0 notes

Hunk equality, Together

1. f=tests/e2e/llm_translation/test_together_ai_e2e.py; diff <(git diff 6ca90b927c8^ 6ca90b927c8 -- $f | grep '^[+-][^+-]') <(git diff origin/stable/1.103.x..HEAD -- $f | grep '^[+-][^+-]')
2. Output is empty

Hunk equality, text completion

1. Same command against e61733b1701 and tests/e2e/llm_translation/test_completions_endpoint_e2e.py
2. The model and docstring lines match main. Only the reasoning_effort line differs, as expected:

5,6c5,9
< -            extra_body=NO_PROXY_CACHE,
< +            extra_body={**NO_PROXY_CACHE, "reasoning_effort": "none"},
---
> -            key, model, "Finish this sentence in a few words: the capital of France is"
> +            key,
> +            model,
> +            "Finish this sentence in a few words: the capital of France is",
> +            reasoning_effort="none",

This line predates the OpenAI SDK rewrite of the test (LIT-4577), so the test still calls EndpointsClient.text_completions. reasoning_effort is added there as an optional field on CompletionsRequest, and the e2e transport drops it from the body when unset, so other callers send the same request as before

Type

✅ Test

Caveats (if any)

Low

• endpoints_client.py is a third file, needed because the line has no sdk_clients.py
• Not run live here; the post-merge Buildkite e2e run is the live check

QA runbook

• tests/e2e/llm_translation/test_together_ai_e2e.py::TestTogetherChatCompletions::test_response_format_json_schema_shapes_the_reply - a json_schema response_format on Together comes back as JSON matching the schema

  • POST /model/new with together_ai/Qwen/Qwen3.5-9B and TOGETHERAI_API_KEY
  • POST /v1/chat/completions with reasoning_effort: "none" and a json_schema response_format
  • Expect 200 with message content that parses as JSON matching the schema
  • Sanity check: this test makes sense to add and is not hand-wavey or potentially flaky

• tests/e2e/llm_translation/test_completions_endpoint_e2e.py::TestCompletionsEndpoint::test_text_completion_returns_text - /v1/completions returns real generated text

  • POST /model/new with openai/gpt-5.4-nano and OPENAI_API_KEY
  • POST /v1/completions with a short prompt, max_tokens: 32, reasoning_effort: "none"
  • Expect 200 with a non-empty choices[0].text
  • Sanity check: this test makes sense to add and is not hand-wavey or potentially flaky

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

Link to Devin session: https://app.devin.ai/sessions/86e3e785e8564ef5b53fabc96ae7a4c6
Open in Devin Desktop: https://app.devin.ai/desktop/session/86e3e785e8564ef5b53fabc96ae7a4c6?variant=devin

[devin-ai-integration]

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

• Disable automatic comment, CI, and merge conflict monitoring

[greptile-apps]

Confidence Score: 5/5

[Low risk] Updates end-to-end test fixtures and documentation.

The PR appears safe to merge; no actionable issue was identified in the changed code.

Summary

This test-only backport replaces two failing live-e2e model choices and adds an optional reasoning-effort field to the e2e text-completion client.

• Pins the Together structured-output test to a schema-capable hybrid Qwen model with reasoning disabled.
• Moves the text-completion test to gpt-5.4-nano with reasoning disabled.

_{Reviews (1) · Last reviewed commit: "test(e2e): align completion, SAIL, and s..."}