test(integration): audio, image and per-unit cost cases

[devin-ai-integration]

TLDR

Problem this solves:

• Cost shard has no audio, speech or image cases
• Per second, per character and per image pricing never asserted
• Multipart uploads and binary responses unsupported by the harness

How it solves it:

• 16 scripted cases: 5 transcription, 3 speech, 7 image generation, 1 image edit
• Harness generates WAV and PNG uploads and serves binary audio bodies
• Cost map schema learns the per unit rate fields litellm reads
• Cases can opt out of a persisted cost_breakdown with breakdown_persisted: false

User Flow

Before: a proxy admin has no shard proving what transcriptions, speech or images cost

1. They send POST https://litellm-domain/v1/audio/transcriptions as multipart with a 3.5 second WAV and get {"text": "hello"} back
2. They send POST https://litellm-domain/v1/audio/speech with "input": "hello world" and get MP3 bytes back
3. They send POST https://litellm-domain/v1/images/generations with "size": "1024x1024", "quality": "hd" and get one image URL back
4. They open https://litellm-domain/ui/?page=logs and hand check each spend row, because tests/integration/run.py cost only covers text endpoints

After: each of those calls has a scripted case with the exact spend and row pinned

1. They run tests/integration/run.py cost
2. Each new case posts the same request to /v1/audio/transcriptions, /v1/audio/speech, /v1/images/generations or /v1/images/edits, receives a literal provider body, and asserts the persisted spend, tokens and, where litellm sends one, the response cost header
3. A change to per second, per character or per image billing fails the shard before release

Relevant issues

Builds on #41999 (harness fields this data needs). Diff shows both until that one lands, then this rebases to main

Affected release

Linear ticket

Resolves LIT-8193

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• The handful of test files covering my change pass locally, e.g. uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
• My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Test data only, the proxy code path is unchanged. Proof is the shard against the scripted upstream that the shard itself starts (no provider spend by design)

Before (9bd648b)

1. tests/integration/run.py cost -k "transcriptions or speech or images" selects nothing

After (5b97d98)

1. tests/integration/run.py cost -k "transcriptions or speech or images" against a fresh proxy and empty Redis: 16 passed, 366 deselected
2. Smoke of 8 existing chat and responses cases on the same proxy: 8 passed
3. make lint passes

Type

✅ Test

Caveats (if any)

Medium

• Image rows (DALL-E, GPT image, Imagen, Nova Canvas) persist spend but no cost_breakdown, so those cases opt out of the breakdown checks
• Speech responses carry no x-litellm-response-cost header, so binary cases only check it when present

Low

• Verbose Whisper bills the body's duration, not the uploaded audio length
• GPT image generation prices from the bare model entry, image edits from the low/1024-x-1024/ entry
• TTS counts hello world as 10 billable characters

Link to Devin session: https://app.devin.ai/sessions/5425681f498543a19de84adcf8daeed8
Open in Devin Desktop: https://app.devin.ai/desktop/session/5425681f498543a19de84adcf8daeed8?variant=devin

[devin-ai-integration]

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

• Disable automatic comment, CI, and merge conflict monitoring

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[greptile-apps]

Confidence Score: 5/5

The PR appears safe to merge, with no concrete correctness, security, or repository-rule violations identified

Summary

This PR extends the integration cost harness to exercise audio transcription, speech, image generation, and image editing pricing

• Adds generated WAV and PNG multipart uploads plus scripted binary responses
• Adds per-second, per-character, per-image, and image-token rate fields
• Adds 16 cost cases across OpenAI-compatible, Azure, Deepgram, Vertex AI, and Bedrock paths
• Keeps cost-breakdown assertions enabled by default while allowing image cases to reflect current persistence behavior

_{Reviews (1) · Last reviewed commit: "Merge remote-tracking branch 'origin/lit..."}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!