Managed batches fixes for vertex

[ephrimstanley]

Relevant issues

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have Added testing in the tests/litellm/ directory, Adding at least 1 test is a hard requirement - see details
• My PR passes all unit tests on make test-unit
• My PR's scope is as isolated as possible, it only solves 1 specific problem
• I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

CI (LiteLLM team)

CI status guideline:

• 50-55 passing tests: main is stable with minor issues.
• 45-49 passing tests: acceptable but needs attention
• <= 40 passing tests: unstable; be careful with your merges and assess the risk.

• Branch creation CI run
  Link:

• CI run for the last commit
  Link:

• Merge / cherry-pick CI run
  Links:

Type

🆕 New Feature
🐛 Bug Fix
🧹 Refactoring
📖 Documentation
🚄 Infrastructure
✅ Test

Changes

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
litellm	Error		Mar 1, 2026 1:46am

[ephrimstanley]

@greptileai

[greptile-apps]

Greptile Summary

This PR fixes several issues with Vertex AI managed batches and file operations: corrects the GcsSource.uris type from str to List[str], simplifies batch cost calculation by reading usageMetadata directly instead of transforming through VertexGeminiConfig, implements file retrieve/delete/content endpoints for Vertex AI GCS files, adds error body logging for batch creation failures, and fixes managed file ID resolution when model_info is nested under litellm_metadata.

• Type fix: GcsSource.uris corrected to List[str] with matching usage in VertexAIBatchTransformation
• Batch cost simplification: calculate_vertex_ai_batch_cost_and_usage now extracts token counts from usageMetadata directly and uses batch_cost_calculator, removing the complex VertexGeminiConfig transformation pipeline
• File operations: VertexAIFilesConfig now supports retrieve, delete, and content operations against GCS via the Storage JSON API
• Bug: transform_delete_file_response reconstructs a gs:// URI missing the bucket name — the object path is extracted from the URL but the bucket portion between /b/ and /o/ is dropped
• Bug: afile_retrieve accepts "vertex_ai" but delegates to sync file_retrieve which does not include "vertex_ai" in its Literal type, causing a mismatch at runtime
• Managed files: Fallback lookup for model_info under litellm_metadata correctly resolves file IDs in batch operations

Confidence Score: 3/5

• PR has two bugs that will cause runtime issues for Vertex AI file retrieval and delete ID reconstruction — should be fixed before merging.
• The batch cost simplification and type fix are solid improvements. However, the sync file_retrieve function was not updated to accept vertex_ai, so afile_retrieve will fail at runtime. Additionally, transform_delete_file_response produces incorrect GCS URIs missing the bucket name. Both are clear bugs introduced by this PR.
• Pay close attention to litellm/files/main.py (sync function missing vertex_ai provider) and litellm/llms/vertex_ai/files/transformation.py (delete response missing bucket in reconstructed URI).

Important Files Changed

Filename	Overview
litellm/files/main.py	Added vertex_ai to afile_retrieve Literal type, but the sync file_retrieve it delegates to was not updated — will cause runtime failures for vertex_ai file retrieval.
litellm/llms/vertex_ai/files/transformation.py	Implements GCS file retrieve/delete/content operations. Delete response reconstruction omits the bucket name from the returned file ID, producing an incorrect gs:// URI.
litellm/llms/vertex_ai/batches/transformation.py	Correctly fixes GcsSource(uris=...) to pass a list instead of a bare string, matching the updated type definition.
litellm/types/llms/vertex_ai.py	Type fix: GcsSource.uris changed from str to List[str] to match the Vertex AI API spec.
litellm/batches/batch_utils.py	Simplified batch cost calculation by extracting usageMetadata directly instead of using VertexGeminiConfig transformation. Clean, correct approach using batch_cost_calculator.
litellm/llms/vertex_ai/batches/handler.py	Adds try/except around HTTP POST to log error body on HTTPStatusError. Minor style issue with redundant hasattr check and f-string in logger.
enterprise/litellm_enterprise/proxy/hooks/managed_files.py	Adds fallback to check litellm_metadata.model_info when top-level model_info is absent — fixes batch file ID resolution for managed files.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[afile_retrieve vertex_ai] --> B[file_retrieve sync]
    B -->|"❌ Missing vertex_ai in Literal"| C[Routing Failure]
    
    D[Batch Create Request] --> E[VertexAIBatchTransformation]
    E --> F["GcsSource(uris=[file_id])"]
    F --> G[Vertex AI API POST]
    G -->|HTTPStatusError| H[Log error body + re-raise]
    G -->|200 OK| I[Transform to LiteLLMBatch]
    
    J[Batch Retrieve] --> K[Get output file content]
    K --> L[calculate_vertex_ai_batch_cost_and_usage]
    L --> M[Extract usageMetadata per line]
    M --> N[batch_cost_calculator]
    N --> O[Aggregate cost + Usage]
    
    P[File Delete Request] --> Q[_parse_gcs_uri]
    Q --> R["GCS Storage API DELETE /b/{bucket}/o/{object}"]
    R --> S[transform_delete_file_response]
    S -->|"❌ Missing bucket in gs:// URI"| T[Incorrect file ID returned]

Loading

_{Last reviewed commit: b16397a}

[greptile-apps]

_{11 files reviewed, 3 comments}

_{Edit Code Review Agent Settings | Greptile}

[greptile-apps]

Sync file_retrieve missing vertex_ai provider

afile_retrieve now accepts "vertex_ai" (and "gemini"), but it delegates to the synchronous file_retrieve at line 337 which still only accepts Literal["openai", "azure", "hosted_vllm", "manus"]. Calling afile_retrieve(file_id, custom_llm_provider="vertex_ai") will hit the sync function with a provider value it doesn't recognise, causing a type mismatch and likely a routing failure at runtime.

The Literal type on file_retrieve (line 339) needs to be updated to include "gemini" and "vertex_ai" as well.

[Sameerlite]

@ephrimstanley Can you fix this one

[greptile-apps]

Reconstructed delete file ID is missing the bucket name

The URL format is https://storage.googleapis.com/storage/v1/b/{bucket}/o/{encoded_object}. When this code does url.split("/o/")[-1], it extracts only the object path, omitting the bucket name. So for a file originally at gs://my-bucket/path/to/file.jsonl, the returned ID would be gs://path/to/file.jsonl instead of gs://my-bucket/path/to/file.jsonl.

The bucket name should also be extracted from the URL (between /b/ and /o/) and prepended. For example:

Suggested change

	file_id = "deleted"
	if hasattr(raw_response, "request") and raw_response.request:
	url = str(raw_response.request.url)
	if "/o/" in url:
	import urllib.parse
	encoded_name = url.split("/o/")[-1].split("?")[0]
	file_id = f"gs://{urllib.parse.unquote(encoded_name)}"
	file_id = "deleted"
	if hasattr(raw_response, "request") and raw_response.request:
	url = str(raw_response.request.url)
	if "/o/" in url:
	import urllib.parse
	bucket_part = url.split("/b/")[-1].split("/o/")[0]
	encoded_name = url.split("/o/")[-1].split("?")[0]
	file_id = f"gs://{bucket_part}/{urllib.parse.unquote(encoded_name)}"

[greptile-apps]

Redundant hasattr check and unsafe f-string in logger call

httpx.HTTPStatusError always has a .response attribute (it's a constructor parameter), so hasattr(e, 'response') is always True — this guard is misleading. More importantly, using an f-string in the logger.error() call means the string is always formatted, even when the error log level is disabled. Prefer %-style formatting:

Suggested change

	error_body = e.response.text if hasattr(e, 'response') else "N/A"
	litellm.verbose_logger.error(
	f"Vertex AI batch create failed: status={e.response.status_code}, body={error_body[:1000]}"
	)
	error_body = e.response.text
	litellm.verbose_logger.error(
	"Vertex AI batch create failed: status=%s, body=%s",
	e.response.status_code, error_body[:1000],
	)

[giulio-leone]

Automated patch bundle from next-100 unresolved backlog expansion.\nGenerated due limited direct branch-write access; please apply/cherry-pick minimal edits below.\n\n## PR #22464 — ephrimstanley/batch-fixes-feb27 (3 unresolved)

Unresolved thread summary

• T1 litellm/files/main.py:298 — Sync file_retrieve missing vertex_ai provider
• T2 litellm/llms/vertex_ai/files/transformation.py:414 — Reconstructed delete file ID is missing the bucket name
• T3 litellm/llms/vertex_ai/batches/handler.py:121 — Redundant hasattr check and unsafe f-string in logger call

Minimal patch proposals

• T1 litellm/files/main.py:298
  • Edit steps:
    1. Add vertex_ai provider handling to file_retrieve to match existing async/provider coverage.
    2. Add or update one focused regression test near this module for the corrected behavior.
• T2 litellm/llms/vertex_ai/files/transformation.py:414
  • Edit steps:
    1. Reconstruct gs:// file ids using both bucket and object path segments.
    2. Add or update one focused regression test near this module for the corrected behavior.
• T3 litellm/llms/vertex_ai/batches/handler.py:121
  • Edit steps:
    1. Switch to parameterized logging and remove redundant runtime attribute checks.
    2. Add or update one focused regression test near this module for the corrected behavior.

[Sameerlite]

LGTM, just make the small change i have asked

[Sameerlite]

@ephrimstanley Can you fix this one