fix(xai): bill from the cost xAI reports instead of recomputing it

[Acacian]

TLDR

Problem this solves:

• xAI reports what it charged, LiteLLM ignores it
• LiteLLM re-derives the number from tokens and list rates
• Only web_search_calls is priced, other tool calls cost nothing
• Cached prompt discounts are missing from the estimate
• Streamed requests skip the tool surcharge entirely

How it solves it:

• Bill the amount xAI reports, when it sends one
• Fall back to today's token and per-call math otherwise
• Skip the search surcharge that amount already covers
• Carry it on usage.cost, the channel every route reads

User Flow

Before: a developer running Grok with server side web search sees spend that does not match their xAI invoice

1. They send POST https://litellm-domain/v1/chat/completions with "model": "xai/grok-4-latest" and "tools": [{"type": "web_search"}]
2. The answer comes back with the citations they asked for
3. They open https://litellm-domain/ui/?page=logs and the request is logged at a cost worked out from token counts, with a flat $0.005 added for every search the model ran
4. Their xAI console shows a smaller amount for that same request, and the gap grows with every search
5. They resend the request with "stream": true and get a third number in the logs, because the streamed route adds nothing for the searches at all

After: the same request is logged at the amount xAI says it charged

1. They send the same POST https://litellm-domain/v1/chat/completions with "model": "xai/grok-4-latest" and "tools": [{"type": "web_search"}]
2. The answer is unchanged
3. https://litellm-domain/ui/?page=logs shows the request at the figure xAI reported, tokens and every server side tool call together
4. That figure matches what the xAI console shows for the same request
5. Resending with "stream": true logs that same figure, and so does the same request sent to https://litellm-domain/v1/responses

Relevant issues

Relates to #35829, which reports that a provider-reported usage.cost is trusted verbatim as USD and records phantom spend when the upstream denominates that field in something else. This PR does not widen that path, and it is worth being explicit about why

• xAI does not report a bare cost float. It reports usage.cost_in_usd_ticks, an integer denominated by a documented constant of 10,000,000,000 ticks to the dollar (https://docs.x.ai/developers/cost-tracking)
• Both the conversion and the validation happen inside the xAI adapter, before anything reaches usage.cost. xai_reported_cost_in_usd() accepts only a non-negative integer, refusing bool as an int subclass, and returns USD. A malformed, negative or absent value yields None and the request falls back to the existing pricing with the search surcharge left intact
• So what lands on usage.cost here is an already validated USD amount derived from a field with a defined unit, not an arbitrary number of unknown denomination. Provider-reported usage.cost is trusted verbatim as USD spend with no unit validation — causes astronomical phantom spend #35829's failure mode is the generic path accepting the latter, and nothing in this PR loosens that

One consequence is worth stating rather than leaving to be discovered: on the streaming path streaming_handler.py::_propagate_usage_cost_to_hidden_params() copies usage.cost into _hidden_params, and response_cost_calculator() returns that before completion_cost() runs, so llms/xai/cost_calculator.py is not consulted for streamed requests. That is safe here precisely because the conversion and the validation already happened in the adapter. The guard in the cost calculator covers the non-streaming path, where usage reaches it directly

Nothing in the tracker reports the remaining xAI mismatch itself. I searched issues for xai, grok, live search, web search cost and cost_in_usd_ticks, and the only other xAI cost bug on file is #15338, which is about tiered token pricing and is already closed. So there is no issue to close here

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• The handful of test files covering my change pass locally, e.g. uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
• My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

To be clear about what this is: not a live capture, but a documented xAI usage payload replayed through the transformations and then through the cost path spend tracking calls, at both commits. I do not have an xAI key with server side search billing enabled to capture a real invoice against. The curl commands a reviewer can run against a live proxy are at the end, and I am happy to add that run if you would rather have it

import httpx
from unittest.mock import Mock

import litellm
from litellm.litellm_core_utils.streaming_handler import CustomStreamWrapper
from litellm.llms.xai.chat.transformation import XAIChatCompletionStreamingHandler, XAIChatConfig
from litellm.llms.xai.responses.transformation import XAIResponsesAPIConfig
from litellm.responses.utils import ResponseAPILoggingUtils
from litellm.types.utils import Choices, Message, ModelResponse

TICKS = 137560000
TOOL_DETAILS = {"web_search_calls": 12, "x_search_calls": 0, "code_execution_calls": 0}
CHAT_USAGE = {"prompt_tokens": 6153, "completion_tokens": 412, "total_tokens": 6565,
              "server_side_tool_usage_details": TOOL_DETAILS, "cost_in_usd_ticks": TICKS}

def _chat_body(usage):
    return {"id": "chatcmpl-xai", "object": "chat.completion", "created": 0, "model": "grok-4-latest",
            "choices": [{"index": 0, "message": {"role": "assistant", "content": "x"}, "finish_reason": "stop"}],
            "usage": usage}

def _billed(usage):
    r = ModelResponse(id="p", model="grok-4-latest", object="chat.completion", created=0,
                      choices=[Choices(index=0, message=Message(role="assistant", content="x"),
                                       finish_reason="stop")])
    r.usage = usage
    return litellm.completion_cost(completion_response=r, model="xai/grok-4-latest",
                                   custom_llm_provider="xai", call_type="completion")

chat = XAIChatConfig().transform_response(
    model="grok-4-latest", raw_response=httpx.Response(status_code=200, json=_chat_body(CHAT_USAGE)),
    model_response=ModelResponse(), logging_obj=Mock(), request_data={},
    messages=[{"role": "user", "content": "x"}], optional_params={}, litellm_params={}, encoding=None)

parsed = XAIChatCompletionStreamingHandler(streaming_response=iter([]), sync_stream=True).chunk_parser(
    {"id": "chatcmpl-xai", "object": "chat.completion.chunk", "created": 0, "model": "grok-4-latest",
     "choices": [], "usage": CHAT_USAGE})
assembled = litellm.stream_chunk_builder(chunks=[parsed])
CustomStreamWrapper._propagate_usage_cost_to_hidden_params(assembled)
streamed = litellm.cost_calculator.response_cost_calculator(
    response_object=assembled, model="grok-4-latest", custom_llm_provider="xai",
    call_type="completion", optional_params={}, cache_hit=None, base_model=None)

responses = XAIResponsesAPIConfig().transform_response_api_response(
    model="grok-4-latest",
    raw_response=httpx.Response(status_code=200, json={
        "id": "resp_xai", "object": "response", "created_at": 0, "model": "grok-4-latest",
        "status": "completed", "output": [], "parallel_tool_calls": False, "tool_choice": "auto",
        "tools": [], "usage": {"input_tokens": 6153, "output_tokens": 412, "total_tokens": 6565,
                               "server_side_tool_usage_details": TOOL_DETAILS,
                               "cost_in_usd_ticks": TICKS}}),
    logging_obj=Mock())
responses_usage = ResponseAPILoggingUtils._transform_response_api_usage_to_chat_usage(responses.usage)

print(f"xAI reported for this request: ${TICKS / 1e10:.6f}")
print(f"  /v1/chat/completions          billed: ${_billed(chat.usage):.6f}")
print(f"  /v1/chat/completions (stream) billed: ${streamed:.6f}")
print(f"  /v1/responses                 billed: ${_billed(responses_usage):.6f}")

Before (40423e6)

1. Check out the merge base and run the script above with PYTHONPATH=. uv run python pof.py
2. All three routes price from tokens, and the two non streamed ones add the search surcharge on top

xAI reported for this request: $0.013756
  /v1/chat/completions          billed: $0.084639
  /v1/chat/completions (stream) billed: $0.024639
  /v1/responses                 billed: $0.084639

After (ddcbba0)

1. Check out the PR tip and run the same script with PYTHONPATH=. uv run python pof.py
2. All three routes now bill the amount xAI reported

xAI reported for this request: $0.013756
  /v1/chat/completions          billed: $0.013756
  /v1/chat/completions (stream) billed: $0.013756
  /v1/responses                 billed: $0.013756

On the two non streamed routes the 12 searches were charged at $0.005 each, $0.06 on top of the $0.0246 token estimate, against the $0.0138 xAI actually charged. The streamed route dropped the searches entirely and priced from tokens alone, which is why it landed somewhere else again. All three now agree with the provider

To run it against a live proxy with a real xAI key instead, with XAI_API_KEY set and the proxy started as in CLAUDE.md:

curl -s http://localhost:4000/v1/chat/completions -H "Authorization: Bearer sk-1234" \
  -H 'Content-Type: application/json' \
  -d '{"model":"xai/grok-4-latest","tools":[{"type":"web_search"}],
       "messages":[{"role":"user","content":"What did xAI ship this week? cite sources"}]}' \
  | python -c 'import json,sys; d=json.load(sys.stdin); print("ticks:", d["usage"].get("cost_in_usd_ticks"), "-> $", d["usage"].get("cost_in_usd_ticks",0)/1e10)'

curl -s http://localhost:4000/v1/responses -H "Authorization: Bearer sk-1234" \
  -H 'Content-Type: application/json' \
  -d '{"model":"xai/grok-4-latest","tools":[{"type":"web_search"}],
       "input":"What did xAI ship this week? cite sources"}' | python -m json.tool | grep -i cost

then open http://localhost:4000/ui/?page=logs and compare the logged spend for those two requests against https://console.x.ai usage for the same minute

Type

🐛 Bug Fix

Changes

Since #30817 the xAI cost calculator counts server side searches from usage.server_side_tool_usage_details.web_search_calls and prices them at xAI's published $5 per 1,000 calls, falling back to that list rate because no xai entry in the price map carries search_context_cost_per_query. That is the right rate for that one tool, and LiteLLM is still doing xAI's arithmetic for it

Every xAI response already states the amount charged, in usage.cost_in_usd_ticks, at 10^10 ticks to the dollar. Per xAI's cost tracking docs that single figure is "the actual amount billed, after all applicable discounts (including prompt caching reductions) have been applied, and inclusive of all token costs and server-side tool invocation costs". This PR reads it and prices the request from it, which is the shape llms/perplexity/cost_calculator.py already uses for a provider that reports its own cost. get_cost_for_web_search_request needed no change, because the xAI calculator now returns 0 for the search surcharge once the reported total has been applied, the same way the perplexity branch two lines above it does

Three things the re-derived number cannot reach, all of which the reported one already accounts for. xAI's pricing page lists Web Search, X Search and Code Execution at $5 per 1,000 calls, Collections Search at $2.50 and File Attachments at $10, but only web_search_calls is priced today, so a request whose server_side_tool_usage_details reports x_search_calls or code_execution_calls is billed nothing for them. Prompt caching is the second: xai/grok-4 and xai/grok-4-latest carry no cache_read_input_token_cost, so cached input is priced at the full $3 per 1M while xAI has already discounted it. The third is drift, since each of those rates needs a matching LiteLLM change every time xAI revises one

The conversion lives in the xAI chat and responses transformations, which restate the ticks in USD on usage.cost. That is the field litellm already carries a provider stated cost in, and using it is what keeps this change inside the provider adapter: ResponseAPILoggingUtils already copies usage.cost onto the chat Usage for /v1/responses, and the streaming chunk assembler already carries cost through aggregation, so neither had to learn an xAI wire field. An earlier revision of this PR instead invented a parallel channel and taught litellm/responses/utils.py to preserve unmodelled provider fields, which changed /v1/responses for every provider to serve one. That is gone; the diff is now entirely under litellm/llms/xai/ and its mirrored tests

Without a reported figure nothing changes: the existing token math and the existing per-call search surcharge both run exactly as before

Only the documented shape of that field is trusted, a non-negative integer, with bool refused since it is an int subclass. A caller who can set api_base controls the response body, so a negative amount would otherwise be billed as negative and subtracted from that caller's own recorded spend, slipping past a budget. Anything that is not a non-negative integer falls back to token pricing with the search surcharge still applied

What that check cannot do is make a self-hosted api_base trustworthy, and neither can token pricing: the token counts come from the same response body, so an endpoint reporting "prompt_tokens": 0, "completion_tokens": 0 already logs $0 today. Trusting a reported total hands such a caller nothing they did not already have, and it is the exposure the perplexity calculator has carried for a while

On #35829, which reports a nano denominated usage.cost recorded verbatim as $3,144,000 of spend: this PR is the narrow per provider opt in that issue asks for as its first suggested fix, "only trust usage.cost for providers where the calling code explicitly opts in (e.g. Perplexity's own cost_calculator.py already does this narrowly and correctly)". The unit is converted from xAI's documented tick unit before it ever reaches usage.cost rather than guessed, so the confusion in that report cannot arise on this path. The blanket trust in response_cost_calculator that the issue is actually about is untouched and still wants a decision above this file

On why not put the price in model_info, as #24372 did for gemini: that was right for gemini, since Google does not tell you what it charged and a configured price is the only option there. xAI does tell you. Going that way means adding tool prices to all 44 xai entries and keeping every one of them current through each xAI revision, and it still cannot express what xAI bills, since the server side tools are charged per invocation across several distinct tools at different rates and the model decides how many of each to call

Not touched: the model price maps, and any provider neutral file

Caveats

Medium

• Proof of fix replays a documented payload, not a live invoice

Low

• Streamed requests bill from _hidden_params, bypassing the xAI calculator
• Trusting the reported total stops litellm cross checking xAI arithmetic

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

[greptile-apps]

Greptile Summary

The PR teaches the xAI adapters to normalize provider-reported billing ticks into USD and prefer that total over local token and search-cost reconstruction.

• Propagates reported costs through chat, streaming, and Responses API transformations.
• Falls back to existing token and server-side search pricing when the reported amount is absent or malformed.
• Adds focused coverage for conversion, propagation, fallback, and surcharge suppression.

Confidence Score: 5/5

The PR appears safe to merge because no blocking failure remains.

No blocking failure remains.

Important Files Changed

Filename	Overview
litellm/llms/xai/chat/transformation.py	Normalizes xAI-reported cost for both non-streaming responses and streaming usage chunks.
litellm/llms/xai/common_utils.py	Adds centralized validation and conversion from xAI billing ticks to USD.
litellm/llms/xai/cost_calculator.py	Prefers validated provider-reported totals and suppresses duplicate web-search charges.
litellm/llms/xai/responses/transformation.py	Carries normalized provider-reported costs through the Responses API adapter.
tests/test_litellm/llms/xai/test_xai_cost_calculator.py	Covers reported-cost preference, fallback behavior, validation, and tool-surcharge suppression.

_{Reviews (7): Last reviewed commit: "fix(xai): bill from the cost xAI reports..." | Re-trigger Greptile}

[codecov]

Codecov Report

❌ Patch coverage is 95.74468% with 2 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
litellm/llms/xai/chat/transformation.py	92.85%	1 Missing ⚠️
litellm/llms/xai/responses/transformation.py	92.85%	1 Missing ⚠️

📢 Thoughts on this report? Let us know!

[Acacian]

@greptileai both findings addressed in the two follow-up commits, please re-review

[veria-ai]

PR overview

This pull request changes xAI billing to use the cost reported by xAI rather than recomputing it locally in the cost calculator.

One security issue remains open after two others were addressed. A non-finite provider-reported cost can bypass budget enforcement and corrupt spend counters when an attacker can control the xAI-compatible endpoint; validation should reject NaN and infinite values.

Open issues (1)

• Medium: Non-finite costs bypass budget enforcement — litellm/llms/xai/cost_calculator.py:57

Fixed/addressed: 2 · PR risk: 6/10

[Acacian]

@greptileai security finding fixed too, please re-review

[codspeed-hq]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing Acacian:fix-xai-provider-reported-tool-cost (ddcbba0) with litellm_internal_staging (40423e6)}

[Acacian]

@greptileai reworked to use usage.cost instead of a parallel field, so responses/utils.py is untouched now. Please re-review

[Acacian]

@mateo-berri you have recent history on both litellm/llms/xai/ and litellm/cost_calculator.py, so tagging you rather than pinging the bots again. Apologies for the noise if this is not your area

This is ready for a human look: CI is green, the branch is clean against litellm_internal_staging, and both bot reviewers are satisfied, with greptile at 5/5 and veria-ai reporting no open security issues. The one point veria-ai deferred to a maintainer rather than dismissing itself is answered in the cost_calculator.py thread just above. The change is confined to litellm/llms/xai/ and its mirrored tests

The short version: xAI reports what it charged on usage.cost_in_usd_ticks, litellm ignores it and reprices the request, and the web search surcharge it adds is $25 per 1,000 sources against xAI's published $5 per 1,000 calls, so recorded spend drifts from the xAI console and the gap widens with every cited source. This bills the reported amount when xAI sends one and falls back to today's token math when it does not

[Acacian]

@greptileai rebased onto current staging and reworked on top of merged #30817, which changed the fallback to per-call. Please re-review

[Acacian]

@greptileai rebased onto current staging, resolved two test import conflicts, and refreshed the proof of fix hashes. Please re-review

[veria-ai]

Medium: Non-finite costs bypass budget enforcement

This check accepts NaN because NaN < 0 is false, and usage.cost can come directly from the provider response rather than the integer-tick conversion. An attacker allowed to select an xAI-compatible endpoint can return cost: NaN; that value reaches key, user, and team spend counters, after which comparisons such as spend >= max_budget remain false. Reject values for which math.isfinite(reported_cost) is false before returning the reported cost.

[Acacian]

@greptileai pushed ddcbba0 after the last ping, moving one helper to module scope to clear the basedpyright budget. Please re-review