feat(xai): add native xAI batches and files support

[devin-ai-integration]

TLDR

Problem this solves:

• xAI has a Batch API but LiteLLM rejected provider=xai on /v1/files and /v1/batches
• xAI's batch API is not OpenAI compatible, so the generic OpenAI path cannot be reused

How it solves it:

• Adds a native xAI files config (upload, retrieve, list, delete, content), walking every page of xAI's file list by its pagination_token
• Adds a native xAI batches handler (create, retrieve, list, cancel)
• Maps xAI state counters (num_pending, num_success, ...) to OpenAI batch status and request_counts
• Serves paginated xAI results as an OpenAI style JSONL file at /v1/files/{batch_id}/content
• Bills xAI batches from the per-line usage in the results, folding xAI's reasoning tokens into completion tokens so the spend row matches xAI's own cost ticks
• Batch health checks pass the deployment's provider through, so a mode: batch xAI deployment no longer reports an OpenAI 401

Decisions made here rather than by accident: output_file_id is the batch id because xAI has no output file object (the alternatives were a synthetic file_ id that nothing could retrieve, or leaving it null and breaking every OpenAI client that reads results by it); created_at and cancelled_at are passed through at xAI's day resolution instead of being replaced by the proxy's clock; the caller's endpoint is echoed back but not sent to xAI, which routes each JSONL line by its own url

User Flow

Before: a developer who wants to run a batch of chat completions on xAI through the proxy is rejected at the first step

1. They send POST https://litellm-domain/v1/files?provider=xai with purpose=batch and a JSONL file
2. They get a 400 with LiteLLM doesn't support xai for 'create_file'. Only ['openai', 'azure', 'vertex_ai', 'manus', 'anthropic'] are supported.
3. They send POST https://litellm-domain/v1/batches?provider=xai with an input_file_id and get a 400 with LiteLLM doesn't support custom_llm_provider=xai for 'create_batch'

After: the same developer runs the whole OpenAI style batch lifecycle against xAI

1. They send POST https://litellm-domain/v1/files?provider=xai with purpose=batch and a JSONL file and get back a file object with an id like file_eb1ce109-a8d8-43a8-8831-d7afa27b2017
2. They send POST https://litellm-domain/v1/batches?provider=xai with that input_file_id and get back a batch object with an id like batch_fdc779da-139c-4390-b086-b9715a6b2f38 and status: validating
3. They poll GET https://litellm-domain/v1/batches/{batch_id}?provider=xai and watch status: in_progress with request_counts.completed climbing until status: completed, request_counts.completed equal to the number of lines and output_file_id set
4. They fetch GET https://litellm-domain/v1/files/{output_file_id}/content?provider=xai and get one OpenAI style JSONL line per request with custom_id and the chat completion body
5. GET https://litellm-domain/spend/logs?request_id={call id of the completed read} shows one spend row for the batch whose spend equals the sum of xAI's per-line cost_in_usd_ticks
6. GET https://litellm-domain/v1/batches?limit=2 lists the xAI batches with OpenAI style statuses, POST https://litellm-domain/v1/batches/{batch_id}/cancel?provider=xai returns status: cancelled, DELETE https://litellm-domain/v1/files/{file_id}?provider=xai returns deleted: true and a later GET of that file is a 404

Relevant issues

Fixes #20424

Linear ticket

Resolves LIT-8499

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• The handful of test files covering my change pass locally, e.g. uv run pytest tests/test_litellm/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
• My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Both legs ran the same flow through the same client: two proxy processes, each proxy_cli.py --config config.yaml --port <random free port> --num_workers 2, sharing one fresh Postgres database, with the steps alternating between the two processes so what one wrote is read back through the other. LITELLM_LOCAL_MODEL_COST_MAP=True so the branch's own price map is what bills the batch. The before leg booted from a worktree at the merge base 170eb7f (ports 45170 and 45527, database litellm_lit8499_qa_before); the after leg booted from the PR tip dc0e29c (ports 22959 and 24000, database litellm_lit8499_qa). XAI_API_KEY and LITELLM_MASTER_KEY come from the environment

config.yaml:

model_list:
  - model_name: grok-4.3
    litellm_params:
      model: xai/grok-4.3
      api_key: os.environ/XAI_API_KEY

files_settings:
  - custom_llm_provider: xai
    api_key: os.environ/XAI_API_KEY

general_settings:
  store_model_in_db: false

batch_input.jsonl:

{"custom_id": "req-1", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "grok-4.3", "messages": [{"role": "user", "content": "What is the capital of France? Answer in one word."}], "max_tokens": 40}}
{"custom_id": "req-2", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "grok-4.3", "messages": [{"role": "user", "content": "How many legs does a spider have? Answer with just the number."}], "max_tokens": 40}}
{"custom_id": "req-3", "method": "POST", "url": "/v1/chat/completions", "body": {"model": "grok-4.3", "messages": [{"role": "user", "content": "What color is the sky on a clear day? One word."}], "max_tokens": 40}}

Before (170eb7f)

Upload through proxy A:

$ curl -s -i -X POST 'http://127.0.0.1:45170/v1/files?provider=xai' -H "Authorization: Bearer $LITELLM_MASTER_KEY" -F purpose=batch -F file=@batch_input.jsonl
HTTP/1.1 400 Bad Request
content-type: application/json

{"error":{"message":"litellm.BadRequestError: LiteLLM doesn't support xai for 'create_file'. Only ['openai', 'azure', 'vertex_ai', 'manus', 'anthropic'] are supported.","type":"invalid_request_error","param":null,"code":"400"}}

Create a batch through proxy B:

$ curl -s -i -X POST 'http://127.0.0.1:45527/v1/batches?provider=xai' -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" -d '{"input_file_id":"file-placeholder","endpoint":"/v1/chat/completions","completion_window":"24h"}'
HTTP/1.1 400 Bad Request
x-litellm-call-id: 44619424-bc20-4476-941e-27a2fce7b9e3
content-type: application/json

{"error":{"message":"litellm.BadRequestError: LiteLLM doesn't support custom_llm_provider=xai for 'create_batch'","type":"internal_server_error","param":null,"code":"400"}}

The issue reporter's client, the litellm Python SDK, with litellm imported from the merge-base worktree:

$ python sdk_flow_before.py
litellm loaded from: .../LIT-8499/wt_before/litellm
BadRequestError: litellm.BadRequestError: LiteLLM doesn't support xai for 'create_file'. Only ['openai', 'azure', 'vertex_ai', 'manus', 'anthropic'] are supported.

Nothing succeeded, so there was nothing further to chain: every xAI files and batches call is rejected before it reaches xAI, on both proxy processes and in the SDK

After (dc0e29c)

1. Upload through proxy A (22959):

   $ curl -s -X POST "http://127.0.0.1:22959/v1/files?provider=xai" -H "Authorization: Bearer $LITELLM_MASTER_KEY" -F purpose=batch -F file=@batch_input.jsonl
   {"id":"file_eb1ce109-a8d8-43a8-8831-d7afa27b2017","bytes":663,"created_at":1790374175,"filename":"batch_input.jsonl","object":"file","purpose":"batch","status":"uploaded","expires_at":null,"status_details":null}
   

2. Create the batch through proxy B (24000) from the file proxy A uploaded:

   $ curl -s -X POST "http://127.0.0.1:24000/v1/batches?provider=xai" -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" -d '{"input_file_id":"file_eb1ce109-a8d8-43a8-8831-d7afa27b2017","endpoint":"/v1/chat/completions","completion_window":"24h"}'
   {"id":"batch_fdc779da-139c-4390-b086-b9715a6b2f38","completion_window":"24h","created_at":1790294400,"endpoint":"/v1/chat/completions","input_file_id":"file_eb1ce109-a8d8-43a8-8831-d7afa27b2017","object":"batch","status":"validating","cancelled_at":null,"cancelling_at":null,"completed_at":null,"error_file_id":null,"errors":null,"expired_at":null,"expires_at":1792886400,"failed_at":null,"finalizing_at":null,"in_progress_at":null,"metadata":{"name":"litellm-batch"},"model":null,"output_file_id":null,"request_counts":{"completed":0,"failed":0,"total":0},"usage":null}
   

3. Poll every 15 s, alternating A and B (curl -s -i "http://127.0.0.1:$PORT/v1/batches/batch_fdc779da-139c-4390-b086-b9715a6b2f38?provider=xai" -H "Authorization: Bearer $LITELLM_MASTER_KEY"), printing status request_counts output_file_id per read. xAI's queue held the third line for about seven minutes and both processes reported its counters as they moved:

   22:09:36 iter=1  port=22959 in_progress {"completed": 0, "failed": 0, "total": 3} None
   22:09:52 iter=2  port=24000 in_progress {"completed": 1, "failed": 0, "total": 3} None
   ...
   22:13:44 iter=17 port=22959 in_progress {"completed": 2, "failed": 0, "total": 3} None
   ...
   22:17:04 iter=30 port=24000 completed {"completed": 3, "failed": 0, "total": 3} batch_fdc779da-139c-4390-b086-b9715a6b2f38
   

   The final read, on B:

   HTTP/1.1 200 OK
   x-litellm-call-id: d28c9cf8-e017-4dbb-b91a-b3c71e0875c9
   content-type: application/json
   
   {"id":"batch_fdc779da-139c-4390-b086-b9715a6b2f38","completion_window":"24h","created_at":1790294400,"endpoint":"/v1/chat/completions","input_file_id":"file_eb1ce109-a8d8-43a8-8831-d7afa27b2017","object":"batch","status":"completed","cancelled_at":null,"cancelling_at":null,"completed_at":null,"error_file_id":null,"errors":null,"expired_at":null,"expires_at":1792886400,"failed_at":null,"finalizing_at":null,"in_progress_at":null,"metadata":{"name":"litellm-batch"},"model":null,"output_file_id":"batch_fdc779da-139c-4390-b086-b9715a6b2f38","request_counts":{"completed":3,"failed":0,"total":3},"usage":null}
   

4. Results through B, one OpenAI style JSONL line per request (usage trimmed to what the billing below uses):

   $ curl -s "http://127.0.0.1:24000/v1/files/batch_fdc779da-139c-4390-b086-b9715a6b2f38/content?provider=xai" -H "Authorization: Bearer $LITELLM_MASTER_KEY"
   {"id": "batch_req_req-1", "custom_id": "req-1", "response": {"status_code": 200, "request_id": "b6aeb89c-...", "body": {"id": "b6aeb89c-...", "object": "chat.completion", "model": "grok-4.3", "choices": [{"index": 0, "message": {"role": "assistant", "content": "Paris", ...}, "finish_reason": "stop"}], "usage": {"prompt_tokens": 204, "completion_tokens": 1, "prompt_tokens_details": {"cached_tokens": 128, ...}, "completion_tokens_details": {"reasoning_tokens": 123, ...}, "cost_in_usd_ticks": 3444800}, ...}}, "error": null}
   {"id": "batch_req_req-2", "custom_id": "req-2", "response": {"status_code": 200, "request_id": "4141843e-...", "body": {... "content": "8" ..., "usage": {"prompt_tokens": 206, "completion_tokens": 1, "prompt_tokens_details": {"cached_tokens": 128, ...}, "completion_tokens_details": {"reasoning_tokens": 116, ...}, "cost_in_usd_ticks": 3324800}, ...}}, "error": null}
   {"id": "batch_req_req-3", "custom_id": "req-3", "response": {"status_code": 200, "request_id": "0714c5ae-...", "body": {... "content": "Blue" ..., "usage": {"prompt_tokens": 205, "completion_tokens": 1, "prompt_tokens_details": {"cached_tokens": 192, ...}, "completion_tokens_details": {"reasoning_tokens": 207, ...}, "cost_in_usd_ticks": 4597200}, ...}}, "error": null}
   
   SUM prompt_tokens=615 completion_tokens=3 cached_tokens=448 reasoning_tokens=446 xai_ticks=11366800 xai_usd=0.00113668
   

5. Spend for that completed read, on A, by the call id from step 3:

   $ curl -s "http://127.0.0.1:22959/spend/logs?request_id=d28c9cf8-e017-4dbb-b91a-b3c71e0875c9" -H "Authorization: Bearer $LITELLM_MASTER_KEY" | jq -c '.[0] | {request_id, call_type, spend, prompt_tokens, completion_tokens, custom_llm_provider, batch_models: .metadata.batch_models, cached: .metadata.usage_object.prompt_tokens_details.cached_tokens, reasoning: .metadata.usage_object.completion_tokens_details.reasoning_tokens, cost_breakdown: {input_cost: .metadata.cost_breakdown.input_cost, output_cost: .metadata.cost_breakdown.output_cost}, batch_successful_requests: .metadata.batch_successful_requests, batch_failed_requests: .metadata.batch_failed_requests}'
   {"request_id":"batch_fdc779da-139c-4390-b086-b9715a6b2f38_batch_cost","call_type":"aretrieve_batch","spend":0.00113668,"prompt_tokens":615,"completion_tokens":449,"custom_llm_provider":"xai","batch_models":["grok-4.3","grok-4.3","grok-4.3"],"cached":448,"reasoning":446,"cost_breakdown":{"input_cost":0.00023868,"output_cost":0.000898},"batch_successful_requests":3,"batch_failed_requests":0}
   

   The row is what the branch's batch prices for xai/grok-4.3 give: input (615 - 448) x 1e-06 + 448 x 1.6e-07 = 0.00023868, output (3 + 446) x 2e-06 = 0.000898, total 0.00113668, which is exactly xAI's own 11366800 cost ticks summed over the three lines

6. List through A with no provider filter:

   $ curl -s "http://127.0.0.1:22959/v1/batches?limit=2" -H "Authorization: Bearer $LITELLM_MASTER_KEY" | jq -c '{object, has_more, first_id, last_id, data: [.data[] | {id, status, request_counts, output_file_id}]}'
   {"object":"list","has_more":true,"first_id":"batch_fdc779da-139c-4390-b086-b9715a6b2f38","last_id":"batch_85758d37-3ca7-42af-8ac4-cb6c1797e725","data":[{"id":"batch_fdc779da-139c-4390-b086-b9715a6b2f38","status":"completed","request_counts":{"completed":3,"failed":0,"total":3},"output_file_id":"batch_fdc779da-139c-4390-b086-b9715a6b2f38"},{"id":"batch_85758d37-3ca7-42af-8ac4-cb6c1797e725","status":"cancelled","request_counts":{"completed":0,"failed":1,"total":3},"output_file_id":null}]}
   

7. A file that does not exist, through B:

   $ curl -s -i "http://127.0.0.1:24000/v1/files/file_does_not_exist?provider=xai" -H "Authorization: Bearer $LITELLM_MASTER_KEY"
   HTTP/1.1 404 Not Found
   content-type: application/json
   
   {"error":{"message":"{\"code\":\"not-found\",\"error\":\"File not found\"}","type":"invalid_request_error","param":null,"code":"404"}}
   

8. Second batch from the same file: created on A, cancelled on B, re-read on A:

   $ curl -s -X POST "http://127.0.0.1:22959/v1/batches?provider=xai" -H "Authorization: Bearer $LITELLM_MASTER_KEY" -H "Content-Type: application/json" -d '{"input_file_id":"file_eb1ce109-a8d8-43a8-8831-d7afa27b2017","endpoint":"/v1/chat/completions","completion_window":"24h"}' | jq -c '{id, status, request_counts}'
   {"id":"batch_ec9b424b-ac6e-4e63-a664-da85b0edfbf8","status":"validating","request_counts":{"completed":0,"failed":0,"total":0}}
   $ curl -s -X POST "http://127.0.0.1:24000/v1/batches/batch_ec9b424b-ac6e-4e63-a664-da85b0edfbf8/cancel?provider=xai" -H "Authorization: Bearer $LITELLM_MASTER_KEY" | jq -c '{id, status, cancelled_at, request_counts}'
   {"id":"batch_ec9b424b-ac6e-4e63-a664-da85b0edfbf8","status":"cancelled","cancelled_at":1790294400,"request_counts":{"completed":0,"failed":2,"total":3}}
   $ curl -s "http://127.0.0.1:22959/v1/batches/batch_ec9b424b-ac6e-4e63-a664-da85b0edfbf8?provider=xai" -H "Authorization: Bearer $LITELLM_MASTER_KEY" | jq -c '{id, status, cancelled_at, request_counts}'
   {"id":"batch_ec9b424b-ac6e-4e63-a664-da85b0edfbf8","status":"cancelled","cancelled_at":1790294400,"request_counts":{"completed":0,"failed":3,"total":3}}
   

9. Delete the input file on A, re-read it on B:

   $ curl -s -X DELETE "http://127.0.0.1:22959/v1/files/file_eb1ce109-a8d8-43a8-8831-d7afa27b2017?provider=xai" -H "Authorization: Bearer $LITELLM_MASTER_KEY"
   {"id":"file_eb1ce109-a8d8-43a8-8831-d7afa27b2017","deleted":true,"object":"file"}
   $ curl -s -i "http://127.0.0.1:24000/v1/files/file_eb1ce109-a8d8-43a8-8831-d7afa27b2017?provider=xai" -H "Authorization: Bearer $LITELLM_MASTER_KEY"
   HTTP/1.1 404 Not Found
   content-type: application/json
   
   {"error":{"message":"{\"code\":\"not-found\",\"error\":\"File not found\"}","type":"invalid_request_error","param":null,"code":"404"}}
   

10. Files list across xAI's page boundary (xAI pages at 100 by pagination_token). With 9 files on the account, A, B and xAI direct all list 9. Then 100 one-line probe files were uploaded through A until xAI's first page filled and carried a pagination_token, and the list was read back through both processes, through the SDK, and directly from xAI:

    $ curl -s "http://127.0.0.1:22959/v1/files?provider=xai" -H "Authorization: Bearer $LITELLM_MASTER_KEY" | python3 -c 'import json,sys; d=json.load(sys.stdin); ids=[f["id"] for f in d["data"]]; print(len(ids), "files,", len(set(ids)), "unique, has_more", d["has_more"])'
    109 files, 109 unique, has_more False
    $ curl -s "http://127.0.0.1:24000/v1/files?provider=xai" -H "Authorization: Bearer $LITELLM_MASTER_KEY" | python3 -c '...same...'
    109 files, 109 unique, has_more False
    $ python sdk_sync_list_probe.py
    litellm.file_list(custom_llm_provider='xai') -> 109 files, 109 unique, in 0.4s
    xAI direct walk right after -> 109 files across 2 pages
    MATCH
    

    The 100 probe files were then deleted alternating A and B (deleted 100 of 100) and A listed 9 files again

11. The reporter's client, the litellm Python SDK (litellm imported from the tip worktree), running the same lifecycle:

    $ python sdk_flow.py
    create_file -> {"id": "file_3b7e1d2d-925b-4819-9dbf-14e769b21047", "purpose": "batch", "bytes": 663, "filename": "batch_input.jsonl"}
    create_batch -> {"id": "batch_bff26cf3-a3c7-46f0-8560-8f1c2e68147e", "status": "validating", "request_counts": {"completed": 0, "failed": 0, "total": 0}, "output_file_id": null, ...}
    retrieve_batch [15s] -> {"id": "batch_bff26cf3-a3c7-46f0-8560-8f1c2e68147e", "status": "completed", "request_counts": {"completed": 3, "failed": 0, "total": 3}, "output_file_id": "batch_bff26cf3-a3c7-46f0-8560-8f1c2e68147e", ...}
    file_content ->
    {"id": "batch_req_req-1", "custom_id": "req-1", "response": {"status_code": 200, ... "content": "Paris" ...}, "error": null}
    {"id": "batch_req_req-2", "custom_id": "req-2", "response": {"status_code": 200, ... "content": "8" ...}, "error": null}
    {"id": "batch_req_req-3", "custom_id": "req-3", "response": {"status_code": 200, ... "content": "Blue\n\\confidence{100}" ...}, "error": null}
    list_batches -> [{"id": "batch_bff26cf3-...", "status": "completed", "request_counts": {"completed": 3, "failed": 0, "total": 3}, ...}, {"id": "batch_fdc779da-...", "status": "in_progress", "request_counts": {"completed": 1, "failed": 0, "total": 3}, ...}]
    list_batches has_more/last_id -> True batch_fdc779da-139c-4390-b086-b9715a6b2f38
    create_batch (third) -> {"id": "batch_9dd23db7-2daf-4104-b709-0dcd5568facc", "status": "validating", ...}
    cancel_batch -> {"id": "batch_9dd23db7-2daf-4104-b709-0dcd5568facc", "status": "cancelled", "request_counts": {"completed": 0, "failed": 0, "total": 1}, ...}
    

12. Health check of a mode: batch xAI deployment (grok-4.3-batch = xai/grok-4.3 with model_info.mode: batch), which an earlier commit of this branch had turned into an OpenAI 401 No API key provided and c29a1ff fixed:

    $ curl -s -i "http://127.0.0.1:32834/health" -H "Authorization: Bearer $LITELLM_MASTER_KEY"
    HTTP/1.1 200 OK
    {"healthy_endpoints":[{"model":"xai/grok-4.3","model_id":"f926fe08d2d586a5031ea3b648e40d8081e4bf1e74050e3dc136d71a2fce46a7"}],"unhealthy_endpoints":[],"healthy_count":1,"unhealthy_count":0}
    

    The same check at the merge base was also healthy, so this is a regression guard rather than a before-fails leg

Mutation check for the tests: reverting bool(page.pagination_token) to is not None fails test_list_batches_treats_empty_pagination_token_as_last_page and test_file_content_stops_paging_on_empty_pagination_token; dropping the litellm.xai_key fallback fails test_retrieve_batch_falls_back_to_litellm_xai_key; making the files list stop after the first page fails test_list_files_walks_every_page_by_pagination_token

Observed next to the fix, none caused by this PR:

• Spend row model is empty; batch_models carries grok-4.3
• xAI's queue held one line about seven minutes
• completed_at stays null on a completed batch
• xAI ignores after on batches, honors it on files
• Cancelling reports still-queued lines as failed

Type

🆕 New Feature

Caveats (if any)

Medium

• output_file_id is the xAI batch id; xAI has no output file, so results are synthesized on read. Deliberate, see the decisions in the TLDR
• Result content is not streamable; the whole result set is paged into memory (1000 per page) before it is returned. Large batches need a streaming route, tracked as a follow-up rather than widened into this PR
• GET /v1/batches?provider=xai through managed batches rejects the provider filter with a 400; list without it. Pre-existing for every provider (same 400 at the merge base), tracked as a follow-up

Low

• Create keeps the caller endpoint on the returned batch and does not send it to xAI. xAI routes each JSONL line by its own url, so a file can mix chat, image, and video. Retrieve, list, and cancel report /v1/chat/completions because xAI stores no batch-level endpoint
• On download, the result unwraps the single key xAI puts on batch_result.response (chat_get_completion, responses, image_generation, video_generation, and so on) and puts that object in the output line's body. That is enough to return an image or video payload. xAI says those media URLs expire after 1 hour, and this PR does not download or persist the media
• xAI has no validating/finalizing states; status is derived from counters plus cancel_time, and a cancelled batch reports its still-queued lines as failed, mirroring xAI's counters
• xAI validation failures surface as status: failed with the xAI message in errors
• created_at, cancelled_at and expires_at come from xAI's date fields, which are day resolution, and completed_at stays null because xAI reports no completion time. Passed through as xAI states them rather than replaced by the proxy's clock
• Uploaded files always report purpose: batch and the list's purpose filter is ignored; xAI stores no purpose
• The files list walks every xAI page by pagination_token up to the shared 10,000-file cap and returns the whole set; the proxy's limit and after query params never reach a provider file list today for any provider, so xAI-specific handling of them in the shared handler would be scope creep here
• Walking the list while files are being deleted concurrently can repeat entries, since xAI's cursor drifts under deletes (a direct walk during 100 deletes returned 1700 rows over 17 pages before terminating). A stop-on-repeated-token guard would truncate a legitimate listing and dedupe belongs in the shared handler for every provider, so it is left as is

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

---

Note

Medium Risk
Touches batch/files routing, synthesized batch output downloads (in-memory pagination), and cost accounting for xAI batches; incorrect status mapping or usage folding could affect operations and billing but does not change core auth paths.

Overview
Adds native xAI support for the OpenAI-style files and batches lifecycle so custom_llm_provider=xai works end-to-end instead of being rejected.

New XAIBatchesHandler and transformation layer call xAI’s batch APIs (create, retrieve, list, cancel), derive OpenAI-like status and request_counts from xAI counters, and set output_file_id to the batch id when complete. Batch results are fetched by paging GET …/batches/{id}/results, converted to OpenAI JSONL (including chat, responses, image, and video result shapes), and exposed via file_content when the id is a batch_* id. Files use XAIFilesConfig for upload, list (with pagination), retrieve, delete, and raw /content download.

Wiring updates route xAI through these handlers in batches/main and files/main, register xAI in file provider literals and ProviderConfigManager, extend allowed batch endpoints (image/video routes in types), and fix batch health checks to pass custom_llm_provider into alist_batches. Billing adds xAI batch token rates (including above-200k batch tiers), folds xAI reasoning tokens into completion tokens when aggregating batch usage, and exposes the new pricing keys in logging/types/schema.

^{Reviewed by Cursor Bugbot for commit dc0e29c. Bugbot is set up for automated code reviews on this repo. Configure here.}

Link to Devin session: https://app.devin.ai/sessions/419a69dd7d2948c38360ef2dacaa592a
Open in Devin Desktop: https://app.devin.ai/desktop/session/419a69dd7d2948c38360ef2dacaa592a?variant=devin
Requested by: @mubashir1osmani

• dc0e29c passes /live-pr-risk

[devin-ai-integration]

I'll fix CI failures and address comments from users with write access. I'll skip comments containing "(aside)".

• Disable automatic comment, CI, and merge conflict monitoring

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
2 out of 3 committers have signed the CLA.

✅ mateo-berri
✅ mubashir1osmani
❌ Mubashir Osmani

---

Mubashir Osmani seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[mubashir1osmani]

bugbot run

[greptile-apps]

Confidence Score: 5/5

[Medium risk] Adds xAI support for batches and files APIs.

The PR appears safe to merge based on the reviewed changes and the current state of the previous findings

Findings

1.  Routine source prose ▶

Summary

This PR adds native xAI file and batch operations, converts paginated batch results to OpenAI-style JSONL, and adds batch pricing and usage handling. Since the previous review, it also adds pagination to xAI file listing and a test for that path

_{Reviews (10) · Last reviewed commit: "fix(xai): walk every page of the files l..."}

[codspeed]

Merging this PR will not alter performance

✅ 31 untouched benchmarks

---

_{Comparing litellm_xai_batches (dc0e29c) with main (a09f8b8)}

[cursor]

Stale Bugbot comment from a previous run.

[codecov]

Codecov Report

❌ Patch coverage is 98.11828% with 7 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
litellm/llms/xai/files/transformation.py	94.56%	5 Missing ⚠️
litellm/llms/xai/batches/transformation.py	98.66%	2 Missing ⚠️

📢 Thoughts on this report? Let us know!

[devin-ai-integration]

@greptileai

[mubashir1osmani]

bugbot run

[cursor]

Stale Bugbot comment from a previous run.

[devin-ai-integration]

Live QA at c7cd26e against real xAI, viewed in the Admin UI at http://localhost:4000/ui/logs/. xAI file upload, batch create, retrieve, results, list and cancel all show up as request logs with the batch cost tracked

Request Logs, xAI Batch and file rows

Batch detail: provider xai, 2 successful and 0 failed requests, cost 0.00010998

File upload detail hitting https://api.x.ai/v1/files

[mateo-berri]

@greptileai

[mateo-berri]

@greptileai

[greptile-apps]

Routine source prose
This module comment describes ordinary HTTP-client structure, which the repository's source-comment rule disallows. Similar prose in both transformation modules also needs removal before merging

Context Used: AGENTS.md (source)

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

[mateo-berri]

These docstrings explain xAI quirks the code cannot show (no status, batch id as output file, bare REST dates), which AGENTS.md's complex-logic exception allows

[greptile-apps]

You're right that the transformation docstrings explain provider-specific behavior. The handler docstring only describes implementation structure, so removing it remains appropriate.

[mateo-berri]

@greptileai

[mateo-berri]

@greptileai

[mateo-berri]

@greptileai

[mateo-berri]

@greptileai

[mateo-berri]

bugbot run

[cursor]

Stale Bugbot comment from a previous run.

[mateo-berri]

@greptileai

[mateo-berri]

@greptileai

[mateo-berri]

bugbot run

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit dc0e29c. Configure here.}

[mateo-berri]

LGTM