fix(batches): mark terminal batch with no output file as processed in CheckBatchCost

[devin-ai-integration]

TLDR

Problem this solves:

• Terminal completed batch with no output file re-polled forever
• batch_processed stuck false, so output/error files never deletable

How it solves it:

• Mark any terminal batch with nothing to bill as processed once
• Bill completed/expired batches that did produce an output file

User Flow

Before: a batch user who never re-checks their batch can never delete its input file once every request line fails; the gateway refuses cleanup forever

1. They upload all_failed.jsonl (every line invalid at execution time, e.g. temperature: 3) via POST https://litellm-domain/v1/files with purpose="batch" and target_model_names, getting back a unified file-... id
2. They create the batch via POST https://litellm-domain/v1/batches with that input file id; the response shows status: "validating" and their pipeline moves on without polling it
3. On the provider's own dashboard the batch soon ends as completed with an error_file_id and output_file_id: null; nothing was served, and correctly no spend appears at https://litellm-domain/ui/?page=logs
4. Hours later they try to clean up: DELETE https://litellm-domain/v1/files/{input_file_id} returns 400 "Cannot delete file ... referenced by 1 batch(es) in non-terminal state", telling them to wait for cost computation that never comes
5. They retry the DELETE for days; the same 400 comes back forever, no matter how long they wait
6. A different batch that expired after partially completing (the provider shows expired with a real output_file_id) also never shows any spend for its served lines at https://litellm-domain/ui/?page=logs

After: the same batch is finalized shortly after it ends, cleanup succeeds, and partially-served expired batches are billed

1. They upload all_failed.jsonl (every line invalid at execution time, e.g. temperature: 3) via POST https://litellm-domain/v1/files with purpose="batch" and target_model_names, getting back a unified file-... id
2. They create the batch via POST https://litellm-domain/v1/batches with that input file id; the response shows status: "validating" and their pipeline moves on without polling it
3. On the provider's own dashboard the batch soon ends as completed with an error_file_id and output_file_id: null; nothing was served, and correctly no spend appears at https://litellm-domain/ui/?page=logs
4. Within a couple of minutes of the batch ending, DELETE https://litellm-domain/v1/files/{input_file_id} returns 200 with "deleted": true
5. The expired batch with a real output_file_id now shows its served lines' spend at https://litellm-domain/ui/?page=logs

Relevant issues

Fixes #35354

Linear ticket

Resolves LIT-5193

Pre-Submission checklist

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review

Screenshots / Proof of Fix

Live two-leg QA against real OpenAI batches. Each leg boots a DB-backed proxy from a fresh worktree with managed files enabled and PROXY_BATCH_POLLING_INTERVAL=15, then drives the User Flow above with the OpenAI Python SDK pointed at the proxy. Re-checking a batch through the gateway finalizes it on both commits, so after creation neither leg ever touches the batch through the proxy again: batch state is confirmed against the provider directly, the same check an admin makes on their own OpenAI account

Before, proxy at merge base 9d069f21dc: the batch ends completed with only an error file and the input file can never be deleted (times PDT, 2026-08-14)

>>> client.files.create(file=open("all_failed.jsonl","rb"), purpose="batch",
...                     extra_body={"target_model_names": "my-gpt"})
file id: bGl0ZWxsbV9wcm94eTphcHBsaWNhdGlvbi9vY3RldC1zdHJlYW07dW5pZmllZF9pZCw1YzM1MGU2MS0xMmI1LTQ4NTYtYWMwMS0yY2VkMjkyNTQ5ZjM7...
>>> client.batches.create(input_file_id=<that file id>, endpoint="/v1/chat/completions",
...                       completion_window="24h")
batch id: bGl0ZWxsbV9wcm94eTttb2RlbF9pZDpteS1ncHQtdW5pZmllZC0xO2xsbV9iYXRjaF9pZDpiYXRjaF82YTdmOGVhMmI3ZTg4MTkwYWZiYmI1MDQ2ZmZjYTIxNQ
status: validating

The provider finishes the batch at 14:57 with nothing served (checked directly against OpenAI, not through the proxy):

$ curl -s https://api.openai.com/v1/batches/batch_6a7f8ea2b7e88190afbbb5046ffca215 \
    -H "Authorization: Bearer $OPENAI_API_KEY"
{"id": "batch_6a7f8ea2b7e88190afbbb5046ffca215", "status": "completed",
 "output_file_id": null, "error_file_id": "file-FGjQYjjE8GfvH2pDDcsXpo",
 "request_counts": {"total": 1, "completed": 0, "failed": 1}, ...}

Every input-file DELETE through the proxy is still refused hours later, identically at 17:19:47, 17:20:47, 17:21:47, and 17:25:44, roughly 2.5 hours and ~88 polling cycles after the batch went terminal:

$ curl -s -w "\nHTTP %{http_code}\n" -X DELETE http://localhost:41287/v1/files/<unified file id> \
    -H "Authorization: Bearer sk-1234"
{"error":{"message":"Cannot delete file bGl0ZWxsbV9wcm94eTphcHBsaWNhdGlvbi9vY3RldC1zdHJlYW07...
The file is referenced by 1 batch(es) in non-terminal state:
bGl0ZWxsbV9wcm94eTttb2RlbF9pZDpteS1ncHQtdW5pZmllZC0xO2xsbV9iYXRjaF9pZDpiYXRjaF82YTdmOGVhMmI3ZTg4MTkwYWZiYmI1MDQ2ZmZjYTIxNQ: validating.
To delete this file before complete cost tracking, please delete or cancel the referencing batch(es) first.
Alternatively, wait for all batches to complete and for cost to be computed (batch_processed=true).","type":"None","param":"None","code":"400"}}
HTTP 400

After, proxy at the PR tip eacea13a25: identical flow and the very first DELETE attempt succeeds

(same upload + create; batch batch_6a7f8a4b3ef081909f337e714fdb3629, status validating)
$ curl -s https://api.openai.com/v1/batches/batch_6a7f8a4b3ef081909f337e714fdb3629 \
    -H "Authorization: Bearer $OPENAI_API_KEY"
{"id": "batch_6a7f8a4b3ef081909f337e714fdb3629", "status": "completed",
 "output_file_id": null, "error_file_id": "file-XYQYeK4k9V7Qx3ExV1dvxv",
 "request_counts": {"total": 1, "completed": 0, "failed": 1}, ...}

$ curl -s -w "\nHTTP %{http_code}" -X DELETE http://localhost:43781/v1/files/<unified file id> \
    -H "Authorization: Bearer sk-1234"
{"id":"bGl0ZWxsbV9wcm94eTph...","bytes":181,"object":"file","status":"uploaded","purpose":"batch",
 "filename":"modified_file.jsonl","created_at":1786743348,"expires_at":1789335348,"status_details":null}
HTTP 200

The batch was finalized 5 seconds after the provider marked it terminal, well inside one 15s polling cycle, and the DELETE fired ~3.5 minutes later succeeded on the first attempt with no proxy-side batch check ever made

QA observations:

• Expired-billing leg not live-testable (24h expiry); regression tests cover
• File DELETE 200 returns file object, no deleted flag; pre-existing

Type

🐛 Bug Fix

Caveats (if any)

• Coexists with fix(batches): stop uncostable batches from starving the cost poll page #36714: that sweep retires aged uncostable rows, this marks terminal rows immediately
• Rebase note: expired-with-output moved from the persists test to the billing test

Changes

CheckBatchCost.check_batch_cost had two arms: completed with an output file (bill, then mark processed) and failed/expired/cancelled (mark processed without billing). A completed/complete batch with output_file_id=None matched neither, so batch_processed stayed false and the poller re-selected the same row on every cycle for the life of the deployment; output/error file deletion is gated on batch_processed, so those files could never be deleted through the managed API either.

The billing arm now triggers for completed/complete/expired when an output file is present, so an expired batch that really produced output is still billed (those request lines were served). It writes back the batch's real terminal status (normalizing completed to complete like the retrieve endpoint does) instead of hardcoding complete, so a billed expired batch stays expired in the DB, matching the stored batch JSON. The second arm now covers the full terminal set (completed/complete/failed/expired/cancelled) so any terminal batch with nothing to bill (failed/cancelled, or completed/expired with no output) is marked processed exactly once. Non-terminal statuses (validating/in_progress) still fall through both arms and are left for the next poll

The two report-flagged behaviours are covered as tests: a still-in-progress batch is not written back, and an expired batch with an output file is billed. The #35131-style case where the provider fetch itself raises is unchanged and out of scope here (that row is already left for retry by the existing except around aretrieve_batch)

On the rebase itself: staging's terminal arm now also mints unified managed file IDs before the write-back (ensure_batch_response_managed_file_ids), which this branch keeps; and since expired-with-output now takes the billing arm, test_terminal_status_persists_managed_output_file_ids parametrizes only failed/cancelled, with the expired case covered by test_expired_with_output_file_is_billed

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

---

Note

^{Cursor Bugbot is generating a summary for commit eacea13. Configure here.}

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 2 committers have signed the CLA.

✅ mateo-berri
❌ devin-ai-integration[bot]
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[greptile-apps]

Greptile Summary

The PR finalizes terminal batches that have no billable output and preserves billing for expired batches with output.

• Marks completed, complete, expired, failed, and cancelled batches without output as processed.
• Persists the actual expired status after billing while normalizing completed to complete.
• Adds regression coverage for no-output completion, non-terminal polling, and expired-batch billing.

Confidence Score: 5/5

The PR appears safe to merge.

No blocking failure remains; the only previous concern was acknowledged as a deliberate legacy-schema constraint rather than a defect introduced by this change.

Important Files Changed

Filename	Overview
enterprise/litellm_enterprise/proxy/common_utils/check_batch_cost.py	Expands terminal finalization and preserves the provider's expired status after successful cost tracking; no eligible follow-up defect remains.
tests/proxy_unit_tests/test_check_batch_cost.py	Updates terminal-status coverage and adds focused regressions for completed batches without output, non-terminal batches, and expired batches with output.

_{Reviews (3): Last reviewed commit: "fix(batches): persist real terminal stat..." | Re-trigger Greptile}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[mateo-berri]

@greptileai

[mateo-berri]

bugbot run

[cursor]

Expired rows still excluded from polling

Medium Severity

The new billing path runs for provider status expired with an output file, but the primary find_many still excludes DB status expired. After GET /v1/batches persists that status (without setting batch_processed), the poller never selects the row, so partial expired spend is still never recorded.

 

^{Reviewed by Cursor Bugbot for commit 1918469. Configure here.}

[mateo-berri]

Pre-existing exclusion, unchanged here: expired rows were never billed before either. Removing it would retro-bill every historical expired row; better as a follow-up

[mateo-berri]

@greptileai

[mateo-berri]

bugbot run

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

1 issue from previous review remains unresolved.

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit eacea13. Configure here.}

[mateo-berri]

LGTM