Fix: Vertex AI Gemini labels field provider-aware filtering

[timelfrink]

Title

Fix: Vertex AI Gemini labels field provider-aware filtering

Relevant issues

Fixes #14556

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have Added testing in the tests/litellm/ directory, Adding at least 1 test is a hard requirement - see details
• I have added a screenshot of my new test passing locally
• My PR passes all unit tests on make test-unit
• My PR's scope is as isolated as possible, it only solves 1 specific problem

Type

🐛 Bug Fix

Changes

What this PR does

This PR fixes a critical issue where Google GenAI endpoints (generativelanguage.googleapis.com) failed when receiving requests with the labels field, which is only supported by Vertex AI endpoints (aiplatform.googleapis.com). The fix implements provider-aware filtering to include labels only for Vertex AI endpoints while excluding them for Google GenAI endpoints.

Problem being solved

Issue #14556 reported that when using LiteLLM with Google GenAI endpoints, requests would fail because the labels field was being included in the request payload. Google GenAI's API doesn't support the labels field (which is a Vertex AI specific feature), causing API errors and request failures.

Core Implementation Changes

• litellm/llms/vertex_ai/gemini/transformation.py:408-420: Added _is_google_genai_endpoint() helper function to detect Google GenAI vs Vertex AI endpoints based on the API base URL
• litellm/llms/vertex_ai/gemini/transformation.py:427: Updated _transform_request_body() to accept api_base parameter for provider detection
• litellm/llms/vertex_ai/gemini/transformation.py:508-509: Modified labels inclusion logic to only add labels for Vertex AI endpoints (not Google GenAI)
• litellm/llms/vertex_ai/gemini/transformation.py:561,609: Updated sync and async transform functions to pass api_base parameter through the transformation chain

Testing Changes

Added comprehensive test coverage in tests/test_litellm/llms/vertex_ai/gemini/test_vertex_ai_gemini_transformation.py:

• test_google_genai_excludes_labels(): Verifies Google GenAI endpoints exclude labels even when explicitly provided
• test_vertex_ai_includes_labels(): Confirms Vertex AI endpoints include labels when provided
• test_provider_detection(): Tests the endpoint detection logic for various URL patterns
• test_metadata_to_labels_vertex_only(): Ensures metadata-to-labels conversion only happens for Vertex AI

How to verify it

Automated Verification

• Tests pass: poetry run pytest tests/test_litellm/llms/vertex_ai/gemini/test_vertex_ai_gemini_transformation.py -v
• Linting passes: ruff check litellm/llms/vertex_ai/gemini/transformation.py
• Type checking passes: mypy litellm/llms/vertex_ai/gemini/transformation.py
• Integration tests pass: poetry run pytest tests/test_litellm/test_completion.py -k vertex_ai -v

Manual Verification

• Google GenAI requests with labels succeed (labels are filtered out)
• Vertex AI requests with labels continue to work (labels are included)
• No regression in existing Vertex AI functionality
• Endpoint detection correctly identifies provider type

Breaking changes

None - this is a backward-compatible bug fix that maintains existing functionality while fixing the Google GenAI compatibility issue.

Migration required

None - existing code will continue to work without changes.

Related issues/PRs

• Fixes [Bug]: requests to Google GenAI fail because of an extra "labels" field #14556 - Labels field causes Google GenAI requests to fail

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Preview	Comments	Updated (UTC)
litellm	Ready	Preview	Comment	Sep 15, 2025 7:21pm

[ahmadmucom]

This is really critical, any temp workaround for it ?

[ishaan-jaff]

[noahkiss]

Might now be a good time to include updated error messages, so that VertexAIException is not the hard-coded error for Gemini API calls (which confused many of us who were initially trying to figure out what broke over the past 24 hr)?

litellm/litellm/litellm_core_utils/exception_mapping_utils.py

Lines 1170 to 1363 in 30c3e7b

	elif (
	custom_llm_provider == "vertex_ai"
	or custom_llm_provider == "vertex_ai_beta"
	or custom_llm_provider == "gemini"
	):
	if (
	"Vertex AI API has not been used in project" in error_str
	or "Unable to find your project" in error_str
	):
	exception_mapping_worked = True
	raise BadRequestError(
	message=f"litellm.BadRequestError: VertexAIException - {error_str}",
	model=model,
	llm_provider="vertex_ai",
	response=httpx.Response(
	status_code=400,
	request=httpx.Request(
	method="POST",
	url=" https://cloud.google.com/vertex-ai/",
	),
	),
	litellm_debug_info=extra_information,
	)
	if "400 Request payload size exceeds" in error_str:
	exception_mapping_worked = True
	raise ContextWindowExceededError(
	message=f"VertexException - {error_str}",
	model=model,
	llm_provider=custom_llm_provider,
	)
	elif (
	"None Unknown Error." in error_str
	or "Content has no parts." in error_str
	):
	exception_mapping_worked = True
	raise litellm.InternalServerError(
	message=f"litellm.InternalServerError: VertexAIException - {error_str}",
	model=model,
	llm_provider="vertex_ai",
	response=httpx.Response(
	status_code=500,
	content=str(original_exception),
	request=httpx.Request(method="completion", url="https://github.com/BerriAI/litellm"), # type: ignore
	),
	litellm_debug_info=extra_information,
	)
	elif "API key not valid." in error_str:
	exception_mapping_worked = True
	raise AuthenticationError(
	message=f"{custom_llm_provider}Exception - {error_str}",
	model=model,
	llm_provider=custom_llm_provider,
	litellm_debug_info=extra_information,
	)
	elif "403" in error_str:
	exception_mapping_worked = True
	raise BadRequestError(
	message=f"VertexAIException BadRequestError - {error_str}",
	model=model,
	llm_provider="vertex_ai",
	response=httpx.Response(
	status_code=403,
	request=httpx.Request(
	method="POST",
	url=" https://cloud.google.com/vertex-ai/",
	),
	),
	litellm_debug_info=extra_information,
	)
	elif (
	"The response was blocked." in error_str
	or "Output blocked by content filtering policy"
	in error_str # anthropic on vertex ai
	):
	exception_mapping_worked = True
	raise ContentPolicyViolationError(
	message=f"VertexAIException ContentPolicyViolationError - {error_str}",
	model=model,
	llm_provider="vertex_ai",
	litellm_debug_info=extra_information,
	response=httpx.Response(
	status_code=400,
	request=httpx.Request(
	method="POST",
	url=" https://cloud.google.com/vertex-ai/",
	),
	),
	)
	elif (
	"429 Quota exceeded" in error_str
	or "Quota exceeded for" in error_str
	or "IndexError: list index out of range" in error_str
	or "429 Unable to submit request because the service is temporarily out of capacity."
	in error_str
	):
	exception_mapping_worked = True
	raise RateLimitError(
	message=f"litellm.RateLimitError: VertexAIException - {error_str}",
	model=model,
	llm_provider="vertex_ai",
	litellm_debug_info=extra_information,
	response=httpx.Response(
	status_code=429,
	request=httpx.Request(
	method="POST",
	url=" https://cloud.google.com/vertex-ai/",
	),
	),
	)
	elif (
	"500 Internal Server Error" in error_str
	or "The model is overloaded." in error_str
	):
	exception_mapping_worked = True
	raise litellm.InternalServerError(
	message=f"litellm.InternalServerError: VertexAIException - {error_str}",
	model=model,
	llm_provider="vertex_ai",
	litellm_debug_info=extra_information,
	)
	if hasattr(original_exception, "status_code"):
	if original_exception.status_code == 400:
	exception_mapping_worked = True
	raise BadRequestError(
	message=f"VertexAIException BadRequestError - {error_str}",
	model=model,
	llm_provider="vertex_ai",
	litellm_debug_info=extra_information,
	response=httpx.Response(
	status_code=400,
	request=httpx.Request(
	method="POST",
	url="https://cloud.google.com/vertex-ai/",
	),
	),
	)
	if original_exception.status_code == 401:
	exception_mapping_worked = True
	raise AuthenticationError(
	message=f"VertexAIException - {original_exception.message}",
	llm_provider=custom_llm_provider,
	model=model,
	)
	if original_exception.status_code == 404:
	exception_mapping_worked = True
	raise NotFoundError(
	message=f"VertexAIException - {original_exception.message}",
	llm_provider=custom_llm_provider,
	model=model,
	)
	if original_exception.status_code == 408:
	exception_mapping_worked = True
	raise Timeout(
	message=f"VertexAIException - {original_exception.message}",
	llm_provider=custom_llm_provider,
	model=model,
	)
	if original_exception.status_code == 429:
	exception_mapping_worked = True
	raise RateLimitError(
	message=f"litellm.RateLimitError: VertexAIException - {error_str}",
	model=model,
	llm_provider="vertex_ai",
	litellm_debug_info=extra_information,
	response=httpx.Response(
	status_code=429,
	request=httpx.Request(
	method="POST",
	url=" https://cloud.google.com/vertex-ai/",
	),
	),
	)
	if original_exception.status_code == 500:
	exception_mapping_worked = True
	raise litellm.InternalServerError(
	message=f"VertexAIException InternalServerError - {error_str}",
	model=model,
	llm_provider="vertex_ai",
	litellm_debug_info=extra_information,
	response=httpx.Response(
	status_code=500,
	content=str(original_exception),
	request=httpx.Request(method="completion", url="https://github.com/BerriAI/litellm"), # type: ignore
	),
	)
	if original_exception.status_code == 503:
	exception_mapping_worked = True
	raise ServiceUnavailableError(
	message=f"VertexAIException - {original_exception.message}",
	llm_provider=custom_llm_provider,
	model=model,
	)
	elif custom_llm_provider == "palm" or custom_llm_provider == "gemini":

[ishaan-jaff]

1 nit @timelfrink

[ishaan-jaff]

lgtm

[ghost]

@timelfrink we should not do this check in here

it should be in the gemini/chat/transformation -

litellm/litellm/llms/gemini/chat/transformation.py

Line 4 in f4e2870

convert_generic_image_chunk_to_openai_image_obj,

this is a cleaner separation