fix(transcription): tolerate non-conforming usage in json response_format

[devin-ai-integration]

Relevant issues

Fixes #33764

Linear ticket

Pre-Submission checklist

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem

Type

🐛 Bug Fix

Changes

Transcription requests with response_format=json against OpenAI-compatible servers that are not OpenAI (the reporter hit this with llama.cpp) could fail even though the server returned a perfectly good transcription. Those servers return a usage object of type: "tokens" whose input_token_details is null (or absent), while OpenAI always populates it, so TranscriptionUsageTokensObject(**usage) raised a pydantic ValidationError at convert_dict_to_response.py:837 and the whole call surfaced as an APIConnectionError

Two changes make usage parsing tolerant without losing data:

TranscriptionUsageInputTokenDetailsObject on the tokens usage model is now Optional[...] = None, so a null or missing input_token_details still parses and the token counts are preserved

Usage construction now goes through a small _parse_transcription_usage helper that validates the dict against the duration or tokens model and returns None (logging at debug) when it does not conform, instead of throwing. A malformed or unrecognized usage shape now drops just the usage field rather than sinking a successful transcription. usage with the expected OpenAI shape is unchanged

Screenshots / Proof of Fix

Reproduced end to end against a live proxy pointed at a local server that emulates llama.cpp's response_format=json transcription response ({"text": ..., "usage": {"type": "tokens", ..., "input_token_details": null}}), which is exactly the shape the reporter's server returns

Before the fix (parsing code at pre-fix HEAD~1, proxy on :4001):

$ curl -s -w "\nHTTP %{http_code}\n" http://127.0.0.1:4001/v1/audio/transcriptions \
    -H "Authorization: Bearer sk-1234" \
    -F "model=gemma-transcribe" -F "file=@sample.wav;type=audio/wav" -F "response_format=json"
{"error":{"message":"litellm.APIConnectionError: APIConnectionError: OpenAIException - Invalid response object Traceback (most recent call last):
  File ".../convert_dict_to_response.py", line 837, in convert_to_model_response_object
    tr_usage_object = TranscriptionUsageTokensObject(**response_object[\"usage\"])
pydantic_core._pydantic_core.ValidationError: 1 validation error for TranscriptionUsageTokensObject
input_token_details
  Input should be a valid dictionary or instance of TranscriptionUsageInputTokenDetailsObject [type=model_type, input_value=None, input_type=NoneType]
...","code":"500"}}
HTTP 500

After the fix (commit a38f050944, proxy on :4000):

$ curl -s -w "\nHTTP %{http_code}\n" http://127.0.0.1:4000/v1/audio/transcriptions \
    -H "Authorization: Bearer sk-1234" \
    -F "model=gemma-transcribe" -F "file=@sample.wav;type=audio/wav" -F "response_format=json"
{"text":"Four score and seven years ago","usage":{"type":"tokens","input_tokens":12,"output_tokens":7,"total_tokens":19,"input_token_details":null}}
HTTP 200

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

Link to Devin session: https://app.devin.ai/sessions/c11c456e42f74f77ba41e0697049d2d9

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment, CI, and merge conflict monitoring

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[greptile-apps]

Greptile Summary

This PR fixes a crash where transcription requests against non-OpenAI OpenAI-compatible servers (e.g. llama.cpp) would surface as an APIConnectionError because those servers return "input_token_details": null in their usage object, which failed pydantic validation.

• TranscriptionUsageInputTokenDetailsObject is made Optional in TranscriptionUsageTokensObject, so a null or absent field parses cleanly and token counts are preserved.
• A new _parse_transcription_usage helper wraps model_validate in a ValidationError try/except, dropping only the usage field (with a debug-level log) on any parse failure rather than sinking an otherwise-successful transcription; this also applies to the type == "duration" path as a side effect.
• Five new unit tests cover the exact failure scenario plus boundary cases (missing field, full valid payload, completely invalid dict, unknown type).

Confidence Score: 5/5

Safe to merge — the change is isolated to transcription usage parsing, is strictly more permissive than before, and all five new tests pass in-process with no network dependency.

The fix is minimal and targeted: one field made optional, one helper added with a narrow exception catch, and the new behavior (drop usage on parse failure) is explicitly tested for every relevant shape. Existing OpenAI-shaped responses are covered by a regression test and continue to parse identically. No auth paths, no schema changes, and no backwards-incompatible behavior for callers that already received a working usage object.

No files require special attention.

Important Files Changed

Filename	Overview
litellm/types/utils.py	Makes input_token_details optional (defaulting to None) in TranscriptionUsageTokensObject, directly fixing the pydantic ValidationError when non-OpenAI servers omit or null the field.
litellm/litellm_core_utils/llm_response_utils/convert_dict_to_response.py	Introduces _parse_transcription_usage helper that uses model_validate with a ValidationError catch, so a malformed usage dict drops the usage field at debug-log level instead of propagating an APIConnectionError.
tests/test_litellm/llms/openai/transcriptions/test_transcription_duration_hidden.py	Adds TestTokensUsageParsingIsResilient with five new unit tests covering null input_token_details, missing field, full valid payload, completely invalid dict, and unknown type — all using in-process mock data with no real network calls.

_{Reviews (1): Last reviewed commit: "fix(transcription): tolerate non-conform..." | Re-trigger Greptile}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[codspeed-hq]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_fix_transcription_json_usage_33764 (a38f050) with litellm_internal_staging (442fdc1)}

[Aayush-engineer]

Reviewed this PR against the reported crash and also dug into the follow-up data @niosHD posted (the input_tokens_details plural key with real cached_tokens data).

CI failures: The 3 failing checks (osv-scan, misc, proxy-infra) are pre-existing and unrelated to this diff. Verified by running the failing tests directly against 442fdc181e (this PR's actual base commit, before any of its changes) — the same failures reproduce there. osv-scan is a pre-existing mcp==1.26.0 CVE in the lockfile. Should be safe to merge without waiting on these.

On the input_tokens_details (plural) data: Traced this against both the OpenAI SDK's actual generated types (transcription_text_done_event.py) and litellm's TranscriptionUsageInputTokenDetailsObject. OpenAI's real schema only defines input_token_details (singular) with audio_tokens/text_tokens — there's no input_tokens_details in their documented contract. The plural key in the reporter's payload looks like llama.cpp/llama-swap emitting a non-standard extra field (shaped more like chat completions' cached_tokens), which the OpenAI SDK only tolerates because its base model uses extra="allow".

So this isn't a second real schema litellm needs to parse — it's provider-specific extra data. Rather than adding fragile logic to map/recover that specific key, it'd be more robust (and consistent with the OpenAI SDK's own behavior) if TranscriptionUsageTokensObject allowed extra fields instead of erroring/dropping them. That would passively preserve this kind of vendor-specific data for anyone inspecting the raw response, without litellm hard-coding meaning onto a key that isn't part of any documented API. Happy to open a small follow-up PR for that if useful — separate from this one, since it's an enhancement rather than part of the crash fix.