fix(chatgpt): keep text and accept string input for the Codex backend - #38086
fix(chatgpt): keep text and accept string input for the Codex backend#38086alekc wants to merge 3 commits into
Conversation
Two request-side defects stop chatgpt/* serving structured output or a string
input.
The allowlist at the end of transform_responses_api_request omits "text". The
chat-to-responses bridge translates response_format into text.format, so the
allowlist discards strict schemas silently and the backend answers with prose
instead of JSON. The backend does honour text.format when it receives it.
Separately the backend rejects a bare string input with
{"detail": "Input must be a list"}, while the Responses API itself accepts
either shape, so the string needs wrapping before it is sent.
Both are request-shaping only, so neither depends on the streaming work in
BerriAI#31332 or BerriAI#34095, though those two block the same call and have to be in place
to observe this one end to end.
The "text" case was reported in BerriAI#24356 and closed by the stale bot without a
fix.
Signed-off-by: Alexander Chernov <alexander@chernov.it>
Greptile SummaryThis PR updates the ChatGPT Responses adapter to preserve structured-output configuration and normalize plain string inputs for the Codex backend
Confidence Score: 4/5The PR appears safe to merge after addressing the non-blocking source-convention violation The request-shaping changes have focused regression and live-backend evidence, with only parameter-rebinding and unnecessary-comment cleanup remaining Files Needing Attention: litellm/llms/chatgpt/responses/transformation.py
|
| Filename | Overview |
|---|---|
| litellm/llms/chatgpt/responses/transformation.py | Corrects request shaping, but the new implementation violates repository conventions on parameter rebinding and routine explanatory comments |
| tests/test_litellm/llms/chatgpt/responses/test_chatgpt_responses_transformation.py | Adds isolated regression tests covering text preservation, string normalization, and unchanged list behavior without network calls |
Reviews (1): Last reviewed commit: "fix(chatgpt): keep text and accept strin..." | Re-trigger Greptile
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| # The Responses API accepts a string or a list, but this backend | ||
| # rejects a string with {"detail": "Input must be a list"}. | ||
| if isinstance(input, str): | ||
| input = [{"role": "user", "content": input}] |
There was a problem hiding this comment.
These routine comments and the input reassignment violate repository source conventions, adding maintenance overhead and potentially failing lint-budget checks
Context Used: CLAUDE.md (source)
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
The LIT002 budget counts mutable-collection construction, and wrapping the string input builds a list and a dict, which put the total two over its ceiling. Both constructions are the wire payload this backend requires, so the list cannot be replaced with an immutable equivalent; annotate the line with a reason instead, matching the convention used elsewhere in the tree. LIT005 freezes reasonless suppressions at zero, so the reason is required rather than decorative. Signed-off-by: Alexander Chernov <alexander@chernov.it>
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
…rameter Greptile flagged the parameter rebinding and the explanatory comments against the repo conventions in CLAUDE.md, which bans rebinding a function parameter (LIT011) and keeps comments to suppressions, TODOs and genuinely complex logic Bind a Final local for the coerced input rather than reassigning `input`, and drop both prose comments; the rationale lives in the commit and the PR body rather than duplicated at the call site. The mutable-ok suppression stays, since it is the allowed kind and the list is still constructed Behaviour is unchanged: 24 tests pass, and the same 4 fail with only the provider file reverted to base Signed-off-by: Alexander Chernov <alexander@chernov.it>
|
|
1 participant
TLDR
Problem this solves:
chatgpt/*answers prose when you ask for a strict JSON schemachatgpt/*rejects a plain stringinputoutrightHow it solves it:
textin the outgoing request so the schema survivesinputin a list before sendingUser Flow
Before: a developer asking a ChatGPT subscription model for structured output silently gets prose, so their parser fails
chatgpt/*model and a strictjson_schemaintext.format, asking for a city and countryParis, Francerather than an object"input": "say hi"as a plain string{"detail":"Input must be a list"}, even though the same string works against OpenAIAfter: the same two requests behave the way the API contract says
json_schema{"city": "Paris", "country": "France"}, which their parser accepts"input": "say hi"as a plain stringRelevant issues
Fixes #38085
Refs #24356, refs #27851
Linear ticket
Pre-Submission checklist
uv run pytest tests/test_litellm/<your_test_file>.py -vScreenshots / Proof of Fix
Shared setup for both cases, run against the live Codex backend with no mocks. Model
chatgpt/gpt-5.6-sol, promptParis is the capital of France. Return the city and country., schema{"city": string, "country": string}withstrict: trueandadditionalProperties: false.One thing a reviewer should know about how this was captured. Two other known defects on this provider, the forced streaming flag in #34094 / #34095 and the drained stream in #25429 / #31332, block the same call from the response side. Both arms below have equivalents of those two applied, identically, so the only difference between Before and After is this PR's request-side change. Without that, neither arm returns anything to compare.
Before (f005afa)
case 1, strict json_schema
text.formatset to the strict schema'Paris, France', prose rather than JSON, no error and no warningcase 2, string input
inputas the plain stringsay hilitellm.BadRequestError: ChatgptException - {"detail":"Input must be a list"}After (3ee1ccf)
case 1, strict json_schema
{"city": "Paris", "country": "France"}, parses against the schemacase 2, string input
status=completedUnit tests, and confirmation that they fail without the production change:
uv run pytest tests/test_litellm/llms/chatgpt/responses/test_chatgpt_responses_transformation.py -qgives24 passedlitellm/llms/chatgpt/responses/transformation.pyand rerunning gives4 failed, 20 passed, the 4 being the new cases across both parametrised modelsType
🐛 Bug Fix
Caveats (if any)
/v1/chat/completionsneeds fix(streaming): backfill response.completed output from output_item.done events #31332 or fix(responses): honor caller stream flag when provider forces SSE #34095 too, same calltexthalf was diagnosed in [Bug]: ChatGPT responses allowlist drops text, so response_format/json_schema never reaches ChatGPT Codex backend #24356, closed by the stale botcode-qualityfails on the base branch too, no workflow files touched hereFinal Attestation