fix(xai): bill web_search from server_side_tool_usage_details

[geraint0923]

TLDR

Problem this solves:

• xAI web search cost read legacy num_sources_used at $25/1k sources
• xAI now reports server_side_tool_usage_details.web_search_calls instead
• Result: xAI web search calls were billed at $0
• Responses API usage normalization also dropped the tool usage details

How it solves it:

• Bill web_search_calls at xAI tools pricing, $5/1k calls
• Prefer search_context_cost_per_query from the model map when set
• Keep the /v1/responses usage schema untouched: tool details already survive as extra fields
• The shared usage bridge carries provider extras into cost tracking generically
• Count web search output items even when they stay plain dicts
• Bill chat searches even when answers carry no citations
• Applies on /v1/chat/completions, /v1/responses, and /v1/messages

User Flow

Before: a developer who gives Grok the web_search tool gets searches and cited answers back, but the gateway bills every search at $0, so tracked spend under-reports the real xAI invoice

1. They send POST https://litellm-domain/v1/responses with "model": "xai/grok-4.5", "tools": [{"type": "web_search"}], and a question about today's news
2. The response completes with web citations, and its usage block shows "server_side_tool_usage_details": {"web_search_calls": 2}
3. The x-litellm-response-cost header and https://litellm-domain/ui/?page=logs show token cost only, with nothing added for the 2 searches
4. Retrying with "stream": true ends the same way: the final response.completed event carries the same usage and the logged spend again omits the searches
5. At month end the xAI invoice includes $5 per 1,000 search calls that no key, team, or budget on the gateway ever metered

After: the same request bills each search call at xAI's tools rate, so tracked spend matches the xAI invoice

1. They send POST https://litellm-domain/v1/responses with "model": "xai/grok-4.5", "tools": [{"type": "web_search"}], and a question about today's news
2. The response completes with web citations, and its usage block shows "server_side_tool_usage_details": {"web_search_calls": 2} in the same OpenAI Responses shape as before
3. The x-litellm-response-cost header and https://litellm-domain/ui/?page=logs now show token cost plus $0.005 per search call, $0.01 for the 2 calls
4. Retrying with "stream": true bills identically off the final response.completed event
5. Key, team, and budget spend line up with the xAI invoice, and admins can pin a different rate by setting search_context_cost_per_query on the model

Relevant issues

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Live before/after QA against real xAI grok-4.5, one proxy booted per commit with its own fresh DB: the before leg at the merge base bea3187, the after leg at this PR's head d96f76c. On every call below, xAI's own usage.cost_in_usd_ticks equals token cost plus $0.005 per web_search_calls, so the provider itself corroborates the billed rate to the last digit

/v1/responses, non-streaming

curl -sD - http://localhost:$PORT/v1/responses -H "Authorization: Bearer $MASTER_KEY" -H "Content-Type: application/json" \
  -d '{"model": "grok-4.5", "input": "What are the top world news headlines today? Use web search.", "tools": [{"type": "web_search"}]}'

Before at bea3187: usage reported "server_side_tool_usage_details": {"web_search_calls": 4} yet x-litellm-response-cost: 0.040014 and the spend log matched token-only cost exactly, while xAI's ticks said the call really cost $0.060014, so all 4 searches went unbilled

After at d96f76c: usage reported web_search_calls: 1, and x-litellm-response-cost: 0.022532 = $0.017532 tokens + 1 x $0.005, spend log identical and equal to xAI's own ticks, with the usage block still in the OpenAI Responses schema (input_tokens/output_tokens, details carried alongside)

/v1/responses, streaming

Same request body plus "stream": true, billed off the final response.completed event (no cost header exists on a stream, so the spend log is the evidence)

Before at bea3187: final event usage showed web_search_calls: 4, spend log 0.0483732 = token-only exactly, versus $0.0683732 per xAI's ticks

After at d96f76c: final event usage showed web_search_calls: 1 in the unchanged Responses schema, spend log 0.022794 = $0.017794 tokens + 1 x $0.005, equal to xAI's own ticks

/v1/chat/completions

curl -sD - http://localhost:$PORT/v1/chat/completions -H "Authorization: Bearer $MASTER_KEY" -H "Content-Type: application/json" \
  -d '{"model": "grok-4.5", "messages": [{"role": "user", "content": "What are the top world news headlines today? Use web search."}], "web_search_options": {}}'

Before at bea3187: the client-facing chat usage carried no web-search fields at all even though the debug log showed xAI reporting web_search_calls: 1 upstream, and x-litellm-response-cost: 0.0167504 was token-only, versus $0.0217504 per xAI's ticks

After at d96f76c: chat usage carried web_search_calls: 7 and x-litellm-response-cost: 0.095734 = $0.060734 tokens + 7 x $0.005, spend log identical and equal to xAI's own ticks

/v1/messages

curl -sD - http://localhost:$PORT/v1/messages -H "Authorization: Bearer $MASTER_KEY" -H "Content-Type: application/json" \
  -d '{"model": "grok-4.5", "max_tokens": 3000, "messages": [{"role": "user", "content": "What are the top world news headlines today? Use web search."}], "tools": [{"type": "web_search_20250305", "name": "web_search", "max_uses": 5}]}'

Before at bea3187: the searches ran (xAI reported web_search_calls: 7 upstream) and both x-litellm-response-cost: 0.0831932 and the spend log matched token-only cost exactly, while xAI's ticks said the call really cost $0.1181932, so all 7 searches went unbilled

After at d96f76c: upstream reported web_search_calls: 2 and the spend log billed 0.033998 = $0.023998 tokens + 2 x $0.005, matching xAI's own ticks to the last digit, though this endpoint's x-litellm-response-cost header still shows the token-only $0.023998 (see observations and caveats)

Observations from the QA runs

• Chat and messages ride xAI's Responses API bridge upstream
• This PR's bridge also surfaces xAI usage extras to clients
• /v1/messages returns chatcmpl-prefixed ids, pre-existing, left alone
• xAI always reports num_sources_used: 0, the retired billing field
• Spend-log id lookup non-streaming only, pre-existing, left alone
• /v1/messages header stays token-only, logged spend correct, pre-existing path
• /v1/messages usage hides search counts from clients, pre-existing, left alone
• xAI ignored the max_uses cap, ran extra searches, pre-existing

Type

🐛 Bug Fix

Caveats (if any)

• Falls back to a hardcoded $5/1k rate when the model map has no search pricing
• On /v1/messages the cost header stays token-only, only the logged spend is correct

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 2 committers have signed the CLA.

✅ mateo-berri
❌ geraint0923
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[codecov]

Codecov Report

❌ Patch coverage is 97.26027% with 2 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
...ore_utils/llm_cost_calc/tool_call_cost_tracking.py	91.66%	1 Missing ⚠️
litellm/llms/xai/chat/transformation.py	88.88%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!

[greptile-apps]

Greptile Summary

The PR updates xAI web-search billing to use server-side tool call counts while preserving the Responses API usage schema

• Carries provider usage extras through the shared logging bridge
• Uses model-configured search pricing when available and otherwise applies xAI’s current per-call rate
• Handles web-search output items represented as dictionaries
• Adds regression coverage across chat, Responses, streaming, and cost-calculation paths

Confidence Score: 5/5

The PR appears safe to merge

No blocking failure remains

Important Files Changed

Filename	Overview
litellm/llms/xai/cost_calculator.py	Bills reported web-search calls using model pricing when configured and the current xAI fallback otherwise
litellm/llms/xai/chat/transformation.py	Preserves xAI server-side tool usage details on normalized chat usage for downstream billing
litellm/llms/xai/responses/transformation.py	Removes the schema-breaking usage replacement so Responses clients retain input/output token fields
litellm/responses/utils.py	Generically carries provider usage extras into a logging-only chat Usage object without mutating client-visible responses
litellm/litellm_core_utils/llm_cost_calc/tool_call_cost_tracking.py	Detects reported server-side web-search calls and recognizes dictionary-shaped Responses output items

_{Reviews (5): Last reviewed commit: "fix(cost-tracking): bill web searches re..." | Re-trigger Greptile}

[veria-ai]

PR overview

All previously flagged issues have been addressed. No open security concerns remain on this pull request.

Security review

No open security issues remain on this pull request.

Fixed/addressed: 2 · PR risk: 0/10

[codspeed-hq]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing geraint0923:litellm_fix_xai_web_search_cost_billing (d96f76c) with litellm_internal_staging (06943b6)¹}

Footnotes

1. No successful run was found on litellm_internal_staging (d96f76c) during the generation of this report, so 06943b6 was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

[mateo-berri]

@greptileai

[greptile-apps]

Responses usage schema is replaced

When an xAI Responses API result includes server_side_tool_usage_details, this assignment replaces its ResponseAPIUsage with a chat Usage object, causing SDK and proxy clients to receive prompt_tokens and completion_tokens instead of the contractually expected input_tokens and output_tokens.

Knowledge Base Used: LLM Provider Adapters

[mateo-berri]

@greptileai

[mateo-berri]

@greptileai

[mateo-berri]

bugbot run

[mateo-berri]

@greptileai

[mateo-berri]

bugbot run

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit d96f76c. Configure here.}

[mateo-berri]

LGTM. Thank you for the contribution!