fix: send whisper timestamp_granularities as bracketed array field - #36036
Open
richboyneedcash wants to merge 18 commits into
Open
Conversation
chore(ci): promote internal staging to main
chore(ci): promote internal staging to main
chore(ci): promote internal staging to main
chore(ci): promote internal staging to main
_await_model_servable used poll_timeout (120s), the spend/log read-back budget. A stuck model reload therefore stalled every suite that creates a deployment for two minutes before failing Give create_model a fixed harness middle ground: model_servable_timeout=40s, polled every 2s, with each /v1/models call capped at 5s and clamped to the remaining deadline so one slow GET cannot overrun the wait. Happy path still returns on the first listing. Not derived from proxy general_settings or env Transport.get accepts an optional per-call timeout for that clamp. Unit tests cover the deadline arithmetic and clamp without a live proxy (cherry picked from commit c082a0e)
create_model returned after the first /v1/models hit that listed the model, so chat could still land on a cold gateway worker (numWorkers>1 / peer pod) and 400 Invalid model name. Require continuous listing for the product default add_deployment interval (30s) after first sight so every worker has synced from the DB; first listing still bounded at 40s (cherry picked from commit 7d1ee2f)
Keep the create_model DB-sync wait in the harness; the pure-function unit file is not needed for this PR (cherry picked from commit 8920465)
When less than one full poll interval remained in the first-listing budget, the pre-sleep check returned NotServable without another /v1/models call. Sleep only min(interval, time left) so a model that becomes listable in the last seconds of the timeout still gets a clamped final poll (cherry picked from commit 8439195)
A poll may start with remaining budget and still return after started+timeout if the transport overruns its clamp. Recheck the first-listing deadline after the response so a late listing does not open the continuous DB-sync phase (cherry picked from commit 7ff2bcb)
…l_servable_timeout test(e2e): bound the post-/model/new servable wait at 40s
* fix(mcp): resolve call_tool by registry without requiring tool map Multi-worker reloads put MCP servers in the registry from the DB but do not re-run tools/list on every process. Gating call_tool on tool_name_to_mcp_server_name_mapping made cold workers 500 with Tool not found after another worker had already listed the tool. Treat a registry match on server id/name/alias as enough; upstream rejects unknown tools * test(e2e): poll MCP register, tools/list, and tools/call across multi-worker lag Stage multi-worker gateways only load MCP servers and tool maps on the process that handled the request. Poll until the server is listed, the tool appears on tools/list, and tools/call is not a cold-worker 500 so key-access and Datadog MCP e2e stop racing the LB * Revert "fix(mcp): resolve call_tool by registry without requiring tool map" This reverts commit 8b56e51. * test(e2e): tighten MCP multi-worker lag classifier Only retry tools/call on gateway shapes Tool <name> not found and server_not_found, not any 500 that mentions tool/server not found, so upstream failures are not retried until the poll deadline * test(e2e): drop unit file for MCP lag classifier The live await_call_tool polls already cover multi-worker lag; a separate string-match unit module is not worth keeping (cherry picked from commit c274cf3)
…p_e2e_poll test(e2e): poll MCP tools across multi-worker lag (BerriAI#35047)
chore(ci): promote internal staging to main
chore: promote staging to main
chore(ci): promote internal staging to main
chore(ci): promote internal staging to main
chore(ci): promote internal staging to main
OpenAI's multipart transcription API expects array params as repeated bracketed fields (timestamp_granularities[]=segment & timestamp_granularities[]=word). LiteLLM stored the list under the bare timestamp_granularities key, so OpenAI kept only the last value and a combined ["segment", "word"] request silently returned just one granularity (words OR segments, never both). Rename the key to timestamp_granularities[] in transform_audio_transcription_request so httpx encodes it as the repeated bracketed fields OpenAI expects. This matches how the official openai-python client serializes array form fields (array_format="brackets"). Fixes the multi-value case; single-value arrays already worked and continue to. Co-authored-by: TRAE CLI <noreply@bytedance.com>
|
|
Contributor
Greptile SummaryThis PR changes OpenAI Whisper timestamp granularity parameters to use the bracketed multipart-array field name and adds regression coverage for multiple, single, and absent values
Confidence Score: 4/5The functional change appears safe to merge after removing the newly added comments required by repository guidance The OpenAI Whisper transformation now emits the bracketed array key on the established SDK path, and the only accepted concern is a non-blocking repository convention violation Files Needing Attention: litellm/llms/openai/transcriptions/whisper_transformation.py, tests/test_litellm/llms/openai/transcriptions/test_whisper_transformation.py
|
| Filename | Overview |
|---|---|
| litellm/llms/openai/transcriptions/whisper_transformation.py | Corrects the OpenAI-specific multipart field name, with a non-blocking repository-guidance violation from newly added comments |
| tests/test_litellm/llms/openai/transcriptions/test_whisper_transformation.py | Adds focused regression cases for key renaming and omission, with one additional newly introduced inline comment |
Reviews (1): Last reviewed commit: "fix: send whisper timestamp_granularitie..." | Re-trigger Greptile
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
Comment on lines
+112
to
+118
| # OpenAI's multipart form API expects array params as repeated | ||
| # bracketed fields (`timestamp_granularities[]=segment` + | ||
| # `timestamp_granularities[]=word`). Sending the list under the bare | ||
| # `timestamp_granularities` key makes OpenAI keep only the last value, | ||
| # so a combined `["segment", "word"]` request silently returns just one | ||
| # granularity. Rename the key so the list is encoded as `[]` fields. | ||
| granularities = data.pop("timestamp_granularities", None) |
Contributor
There was a problem hiding this comment.
The new production comment and the inline comment in test_whisper_transformation.py:78 violate the repository instruction prohibiting new comments unless explicitly requested, adding prose that must be removed to conform to project standards
Context Used: CLAUDE.md (source)
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
Contributor
Author
|
recheck |
5 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #35937
Problem
Calling
/v1/audio/transcriptions(whisper-1) withresponse_format="verbose_json"and a two-elementtimestamp_granularitiesarray only honors the last element:Calling OpenAI directly with
["segment", "word"]returns both.Root cause
In
whisper_transformation.py,transform_audio_transcription_requeststores the list under the bare keytimestamp_granularities. OpenAI's multipart form API expects array params as repeated bracketed fields (timestamp_granularities[]=segment+timestamp_granularities[]=word). With the bare key, OpenAI keeps only the last value.This matches how the official
openai-pythonclient serializes array form fields — it usesarray_format="brackets".Fix
Rename the key to
timestamp_granularities[]so httpx encodes it as the repeated bracketed fields OpenAI expects.Verification
Added regression tests in
test_whisper_transformation.py(multi-value, single-value, and absent cases). Confirmed end-to-end that the resulting multipart body now contains twotimestamp_granularities[]fields (bothsegmentandword), with no bare-key residual.Single-value arrays already worked and continue to. Note: this is distinct from the previously-fixed #12407 / #10282 (param dropped entirely); this addresses multi-value arrays being truncated to one element.