test(e2e): report batch cleanup leftovers as a plain UserWarning

[yuneng-berri]

TLDR

Problem this solves:

• A batch cleanup leftover warning crashes the whole Buildkite e2e run
• xdist cannot import the custom warning class on the controller

How it solves it:

• Warn with a plain UserWarning instead of BatchCleanupLeftover
• Tests and COVERAGE.md updated to match

User Flow

Before: a release qualifier running the litellm-e2e Buildkite pipeline gets a crashed run instead of test results

1. They trigger litellm-e2e with a litellm SHA and unblock the release candidate step
2. A batch test leaves a file in place because LiteLLM refuses to delete it while a batch is still cancelling
3. The full suite step dies with INTERNALERROR and ModuleNotFoundError: No module named 'batch_cleanup', and the retry step then fails with "no artifacts found"

After: the same leftover shows up as a warning and the run finishes with real results

1. They trigger litellm-e2e the same way
2. The same leftover happens
3. The run reports it as a UserWarning naming the file id and keeps going to a pass/fail verdict

Relevant issues

Same change as #43391, which already fixed this on rc/1.103.0. The rc/1.104.0 copy is its own PR

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• The handful of test files covering my change pass locally, e.g. uv run pytest tests/unit/<your_test_file>.py -v. Leave the suites (make test-unit-*, make test-unit) to CI: it finishes in ~15 minutes where a laptop takes an hour or more
• My PR passes all required CI/CD checks (e.g., lint, schema.d.ts sync check, etc.)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

The existing tests/e2e/batches/test_batch_cleanup.py covers both leftover paths and passes 50/50 on this branch

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Before (9ba552d)

1. Buildkite litellm-e2e build 313 ran the full suite at rc/1.104.0 6d672483c9, which has the same batch_cleanup.py as this base: https://buildkite.com/berriai-1/litellm-e2e/builds/313
2. The full suite step crashed on the warning:

INTERNALERROR>   File "/app/.venv/lib/python3.13/site-packages/xdist/workermanage.py", line 474, in unserialize_warning_message
INTERNALERROR>     mod = importlib.import_module(data["message_module"])
INTERNALERROR> ModuleNotFoundError: No module named 'batch_cleanup'
[gw4] node down:

After (f7e1c32)

1. Run pytest tests/e2e/batches/test_batch_cleanup.py -q
2. Output: 50 passed, including the two leftover cases that now assert a UserWarning whose message names the left file and batch
3. The warning category now lives in builtins, which the xdist controller can always import

Type

✅ Test

Caveats (if any)

Low

• Not re-run on Buildkite yet
  • The CI OpenAI account is currently out of credits, so a full e2e run fails for an unrelated reason
• The crash did not reproduce in a local xdist run
  • My local run had the batches directory importable on the controller; Buildkite does not

Final Attestation

• The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

[greptile-apps]

Confidence Score: 5/5

[Low risk] Test code replaces a custom warning class with the standard one.

The PR appears safe to merge based on the reviewed changes.

Summary

The PR replaces the custom batch-cleanup leftover warning with a plain UserWarning and updates the corresponding tests and coverage notes. No actionable issue was identified.

_{Reviews (1) · Last reviewed commit: "test(e2e): report batch cleanup leftover..."}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!