test(integration): passthrough route cost cases

[devin-ai-integration]

TLDR

Problem this solves:

• Passthrough routes bill through their own cost path with zero shard coverage
• No proof that /gemini/*, /anthropic/* or /bedrock/* spend rows are right

How it solves it:

• 6 scripted cases: Gemini generateContent and SSE, Anthropic messages and cache read, Bedrock converse and converse-stream
• Cases can name a passthrough path as their endpoint, with $MODEL in the path
• Cases can opt out of the response cost header with cost_header: false
• Scripted upstream picks the scenario from a forwarded header when the path carries none
• Cost proxy in CI gets GEMINI_API_BASE and ANTHROPIC_API_BASE pointed at the upstream

User Flow

Before: a proxy admin has no shard proving what passthrough calls cost

1. They send POST https://litellm-domain/gemini/v1beta/models/gemini-3.1-pro:generateContent with x-goog-api-key set to their virtual key and get Google's body back
2. They send POST https://litellm-domain/anthropic/v1/messages and get Anthropic's body back
3. They send POST https://litellm-domain/bedrock/model/my-claude/converse and get the Converse body back
4. They open https://litellm-domain/ui/?page=logs and hand check each pass_through_endpoint row, because tests/integration/run.py cost only covers native endpoints

After: each of those calls has a scripted case with the exact spend and row pinned

1. They run tests/integration/run.py cost
2. Each new case posts the same request to its passthrough path, receives a literal provider body, and asserts the persisted spend and tokens, plus the breakdown where litellm writes one
3. A change to passthrough billing fails the shard before release

Relevant issues

Builds on #42024 (needs breakdown_persisted). Diff shows both until that one lands, then this rebases to main

Affected release

Linear ticket

Resolves LIT-8187

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• The handful of test files covering my change pass locally, e.g. uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
• My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Test data only, the proxy code path is unchanged. Proof is the shard against the scripted upstream that the shard itself starts (no provider spend by design)

Before (247c4dd)

1. tests/integration/run.py cost -k passthrough selects nothing

After (110d4c2)

1. tests/integration/run.py cost -k passthrough against a fresh proxy and empty Redis: 6 passed, 403 deselected
2. Regression of the image cases plus the native Gemini, Anthropic and Bedrock cases on the same proxy: 12 passed
3. make lint passes

Type

✅ Test

Caveats (if any)

High

• Gemini passthrough prices the same model from two different map entries. Non streaming generateContent prices via gemini/gemini-3.1-pro, streaming via the bare gemini-3.1-pro (vertex) entry. Both cases are first principles against their own entry, and the names say which key each one hits. With the real map the two rates differ, so a user pays a different price for the same tokens depending on stream

Medium

• Anthropic streaming and every /openai/* passthrough are excluded. Their logging only kicks in when the upstream hostname is api.anthropic.com or *.openai.com, so with the scripted upstream they persist a $0 row. Vertex passthrough has no api base override at all, so it is excluded too
• Passthrough rows return no x-litellm-response-cost header, so those cases only check it when present. Gemini and Anthropic rows also persist no cost_breakdown

Low

• Gemini passthrough authenticates with x-goog-api-key carrying the virtual key, not the Bearer header

Link to Devin session: https://app.devin.ai/sessions/5425681f498543a19de84adcf8daeed8
Open in Devin Desktop: https://app.devin.ai/desktop/session/5425681f498543a19de84adcf8daeed8?variant=devin

[devin-ai-integration]

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

• Disable automatic comment, CI, and merge conflict monitoring

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[greptile-apps]

Confidence Score: 4/5

The test coverage is broadly sound, but the repository’s external-fact attribution requirement must be satisfied before merging

Findings

1.  Uncited vendor prices ▶
2.  Requests remain unverified ▶

Summary

This PR extends the integration cost harness to cover Gemini, Anthropic, and Bedrock passthrough billing against a local scripted upstream

• Adds six passthrough cost cases covering JSON, SSE, cache reads, Converse, and ConverseStream
• Adds passthrough endpoint selection, model substitution, forwarded scenario headers, and optional cost-header assertions
• Configures the cost-shard proxy to route Gemini and Anthropic passthrough traffic to the scripted upstream
• Keeps provider calls local, consistent with the repository’s prior work to isolate spend tests from provider outages and paid requests

_{Reviews (1) · Last reviewed commit: "test(integration): add passthrough cost ..."}

[greptile-apps]

Uncited vendor prices

These cases pin vendor prices without a source and date, violating a repository requirement that must be satisfied before merging

Context Used: AGENTS.md (source)

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

[devin-ai-integration]

These are not vendor prices. Every expected value is arithmetic over the test-owned cost_map in this same file, served at /_cost_map, e.g. 1840 x 2e-6 = 0.00368

[greptile-apps]

You're right — I misclassified these values. They are deterministic test expectations derived from the test-owned cost_map in cost_tracking_cases.json, served by the scripted upstream via /_cost_map; they are not uncited vendor-price assertions. For example, 1840 × 2e-6 = 0.00368. This comment does not apply and can be dismissed.

[greptile-apps]

You're right. These expected costs are derived from the test-owned cost_map served by the scripted upstream, not from vendor pricing. Since the fixture rates are defined and controlled within this test harness, an external price source and date are not required. I retract this comment.

[greptile-apps]

You're right — these are test-owned fixture prices, not uncited vendor prices. The scripted upstream's cost_map() loads cost_tracking_cases.json and serves its cost_map section at /_cost_map, so the expected values are intentionally derived from the same versioned test data. I retract the finding; no change is needed for this comment.

[greptile-apps]

Requests remain unverified

The upstream selects only by scenario ID, so malformed passthrough paths or bodies can still pass these tests

[devin-ai-integration]

This suite asserts what the gateway bills for a scripted upstream response, matching every existing case here. Forwarded path and body fidelity is a separate concern, out of scope

[greptile-apps]

That scope distinction is fair. These cases specifically validate gateway billing and persistence for scripted passthrough responses, consistent with the existing integration cases; path/body fidelity would require separate forwarding-contract tests. I’m withdrawing this concern for this PR.

[greptile-apps]

That scope distinction is fair. This shard validates passthrough billing and persisted spend, while request forwarding is covered by tests/integration/providers/test_request_boundary.py through /__observations, consistent with the native cost cases. I withdraw the request about malformed paths and bodies.

[greptile-apps]

That distinction is fair. Since this shard specifically validates passthrough billing persistence, and request forwarding/shape is covered by tests/integration/providers/test_request_boundary.py (including analogous native 400 cost cases), I withdraw the request to add forwarding assertions here. No further change needed.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!