fix(files): raise resumable upload chunk size to cut GCS round-trips

[mubashir1osmani]

Relevant issues

Large vertex/Gemini batch file uploads through /v1/files return 499s (client closed connection) on multi-GB inputs.

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

Screenshots / Proof of Fix

Real-provider proof is pending; the Vertex project available for local verification has billing disabled, so the live /v1/files -> GCS path returns a 403 before the upload streams. The change is covered by a mutation-verified regression test (a deliberately blocking chunk generator must not starve a concurrent coroutine; it fails on a synchronous pull and passes with the offload) and a guard on the chunk-size floor. Once a Vertex project with active billing is available, the proof is a curl of a multi-GB batch upload against a live proxy showing the request completing well inside the client timeout.

Type

🐛 Bug Fix

Changes

The resumable upload to GCS was using an 8 MiB chunk size, so a 2 GB batch input made roughly 256 strictly-sequential PUTs whose combined round-trip time overran client and load-balancer timeouts and surfaced as a 499. This raises the chunk size to 32 MiB (still a 256 KiB multiple, which GCS requires for non-final chunks), cutting that to about 64 round-trips. Peak memory stays bounded at one chunk, and the request stays synchronous so the returned file object is real and POST /v1/batches keeps working immediately.

The companion event-loop offload (pulling each chunk via asyncio.to_thread so the per-chunk transform does not block the worker) already landed separately; this adds a mutation-verified regression test that guards it.

[greptile-apps]

Greptile Summary

This PR updates the Vertex/GCS resumable file upload path. The main changes are:

• Raises the default resumable upload chunk size from 8 MiB to 32 MiB
• Keeps the chunk size aligned to the GCS 256 KiB multiple requirement
• Adds tests for the chunk-size floor and async chunk generation offload
• Uses mock-based upload tests without adding real network calls

Confidence Score: 4/5

The change is narrowly scoped to the Vertex/GCS resumable upload path and is covered by targeted tests for chunk sizing and async chunk generation behavior.

The implementation keeps the GCS alignment requirement intact, bounds memory to one chunk, and adds regression coverage for the main upload behavior being changed. Live provider validation is still pending because the available project cannot complete the streaming path.

No files require follow-up beyond optional live Vertex/GCS validation when an enabled billing project is available.

T-Rex Logs

What T-Rex did

• Ran the chunk-size roundtrip tests to compare baseline and post-change head behavior, observing default_chunk_size_mib changing from 8 to 32 and corresponding shifts in observed_iterator_chunks and put counts.
• Performed a side-by-side timing check for base and head runs to assess performance and starvation, confirming both runs completed successfully with 200 OK and guard_ticks_ge_5 PASS, and that the head run shows non-starvation.

_{Ran code and verified through T-Rex}

_{Reviews (1): Last reviewed commit: "fix(files): raise resumable upload chunk..." | Re-trigger Greptile}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[mubashir1osmani]

Superseded by #31653, which fully reverts the upload mechanism to a single uploadType=media request (the regression was #31036 switching to chunked resumable), rather than just enlarging the resumable chunk size.