fix(vertex_ai): stabilize nullable tool schema request bodies and skip live test on 429

[mateo-berri]

Relevant issues

No GitHub issue; fixes the local_testing_part1 CI flake that turns unrelated PRs red, e.g. job 2010804 on #32390

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

Proofs for branch litellm_fix_vertex_429_flake_local_testing. Before runs captured at base commit 06a43d1, after runs at fix commit 8ac1b05

Before (CI): job 2010804 (local_testing_part1 on #32390, whose diff only touches unrelated realtime test harness code) fails on exactly one test, test_gemini_nullable_object_tool_schema_httpx, with litellm.RateLimitError ... 429 RESOURCE_EXHAUSTED from a live gemini-2.5-flash:generateContent call. The job's VCR verdict for the test is [VCR NOOP] played=0 entries=1 and its VCR diagnostic log shows why the cached episode was not replayed: the incoming and cached request bodies are both 807 bytes and first diverge at byte 404, where required and properties swap order inside customer_context.anyOf[0]

Before (local, at 06a43d1): the serialized request schema depends on the interpreter hash seed, producing two distinct bodies, which is exactly what makes the cassette miss and the call go live

$ git checkout 06a43d11c4 -- litellm/llms/vertex_ai/common_utils.py
$ for seed in 0 1 2 3 4 5 8 10; do PYTHONHASHSEED=$seed uv run python -c "
import json, hashlib
from litellm.llms.vertex_ai.common_utils import _build_vertex_schema
params = <the exact tool parameters from test_gemini_nullable_object_tool_schema_httpx>
print('PYTHONHASHSEED=$seed body_sha256_prefix', hashlib.sha256(json.dumps(_build_vertex_schema(params), separators=(',', ':')).encode()).hexdigest()[:12])"; done
PYTHONHASHSEED=0 body_sha256_prefix ad7bf2b3337b
PYTHONHASHSEED=1 body_sha256_prefix ad7bf2b3337b
PYTHONHASHSEED=2 body_sha256_prefix ad7bf2b3337b
PYTHONHASHSEED=3 body_sha256_prefix ad7bf2b3337b
PYTHONHASHSEED=4 body_sha256_prefix ad7bf2b3337b
PYTHONHASHSEED=5 body_sha256_prefix 182361427b0f
PYTHONHASHSEED=8 body_sha256_prefix 182361427b0f
PYTHONHASHSEED=10 body_sha256_prefix 182361427b0f

After (local, at 8ac1b05): every seed produces identical bytes, and the surviving ordering is the one the majority of seeds already recorded, so existing cassettes keep replaying

$ for seed in 0 1 2 3 4 5 8 10; do PYTHONHASHSEED=$seed uv run python -c "<same as above>"; done
PYTHONHASHSEED=0 body_sha256_prefix ad7bf2b3337b
PYTHONHASHSEED=1 body_sha256_prefix ad7bf2b3337b
PYTHONHASHSEED=2 body_sha256_prefix ad7bf2b3337b
PYTHONHASHSEED=3 body_sha256_prefix ad7bf2b3337b
PYTHONHASHSEED=4 body_sha256_prefix ad7bf2b3337b
PYTHONHASHSEED=5 body_sha256_prefix ad7bf2b3337b
PYTHONHASHSEED=8 body_sha256_prefix ad7bf2b3337b
PYTHONHASHSEED=10 body_sha256_prefix ad7bf2b3337b

After (live end-to-end, at 8ac1b05): real Vertex AI call, no mocks, real spend, litellm.completion(model="vertex_ai/gemini-2.5-flash", tools=<the exact flaking tool schema>, tool_choice="required")

finish_reason: tool_calls
tool_call fn: create_support_ticket
args: {"customer_context": null, "ticket_id": "some-ticket-id"}
response_cost: 0.00031560000000000003

Supplementary check of the new skip guard (quota exhaustion cannot be triggered on demand, so this one simulates the 429 at the HTTP transport; it verifies the test plumbing only). Running pytest tests/local_testing/test_amazing_vertex_completion.py::test_gemini_nullable_object_tool_schema_httpx with the same forced 429:

# after, at 8ac1b05670
SKIPPED [1] tests/local_testing/test_amazing_vertex_completion.py:3905: Rate limit error
1 skipped in 0.16s

# before, at 06a43d11c4
1 failed in 0.35s   (litellm.exceptions.RateLimitError)

New regression test fails on the pre-fix code and passes with the fix, and the touched suites are green at 8ac1b05:

$ pytest tests/test_litellm/llms/vertex_ai/test_vertex_ai_common_utils.py -q
74 passed in 7.18s
$ pytest tests/test_litellm/llms/vertex_ai/gemini/ -q
238 passed, 1 skipped in 5.89s
$ git stash push -- litellm/llms/vertex_ai/common_utils.py && pytest tests/test_litellm/llms/vertex_ai/test_vertex_ai_common_utils.py -q -k hash_seeds
FAILED tests/test_litellm/llms/vertex_ai/test_vertex_ai_common_utils.py::test_convert_schema_types_output_is_stable_across_hash_seeds
1 failed, 73 deselected in 8.08s

Type

🐛 Bug Fix

Changes

CircleCI job local_testing_part1 intermittently fails unrelated PRs with a Vertex AI 429 on test_gemini_nullable_object_tool_schema_httpx (observed on #32390, job 2010804; in a sample of 600 build_and_test workflows from 2026-07-03 to 2026-07-08 this was 1 of 16 local_testing_part1 failures)

Root cause: _convert_schema_types in litellm/llms/vertex_ai/common_utils.py converts "type": ["object", "null"] tool params to anyOf and moves properties, required etc. into the object branch by iterating a Python set. String set iteration order varies per process under hash randomization, so the JSON key order of the serialized request body differs from run to run; this test's schema produces exactly two orderings. The local_testing VCR harness matches requests as canonical bytes and deliberately does not normalize JSON key order, so any CI worker whose seed produces the ordering that is not in the Redis cassette misses and calls the live API. That call draws on the shared CI project quota (this job alone is 4 CircleCI nodes x pytest -n 4, the file has 20 gemini-2.5-flash call sites, and all concurrent PRs share the project), so it sometimes gets a 429. The test had no guard for that: the file-level litellm.num_retries = 3 is intentionally reset to the None default by the conftest isolation fixture, and unlike 22 sibling call sites in the same file there was no except litellm.RateLimitError: pytest.skip handler, so a single 429 turns an unrelated PR red. The failure then blocks the cassette save (the persister saves only on pass, and non-2xx responses are filtered from recording anyway), so nothing self-heals within the day, and the 24h cassette TTL restarts the record cycle daily

Fix, targeting two links of that chain. type_specific_fields becomes a tuple, so the anyOf branch is built in a fixed order and the request body is byte-identical across processes; after the first recording the cassette replays for every run regardless of seed, and only the intentional once-per-TTL re-record stays live. The test also adopts the file's standard skip-on-RateLimitError guard so that residual live call cannot fail an unrelated PR when quota happens to be exhausted. A new regression test builds the exact flaking schema through _build_vertex_schema in subprocesses under PYTHONHASHSEED 0-5 and asserts all serialized outputs are byte-identical; it fails deterministically on the pre-fix code and passes with the fix

[greptile-apps]

Greptile Summary

This PR fixes an intermittent CI failure caused by Python hash-seed non-determinism in the Vertex AI schema serialization path, and adds a skip guard so residual live-API rate-limit errors don't redden unrelated PRs.

• Core fix: type_specific_fields in _convert_schema_types is changed from a set to a tuple, which guarantees the same key-insertion order into the anyOf branch dict on every Python process, making the serialized request body byte-identical across all hash seeds and preventing VCR cassette misses that triggered live calls.
• Test guard: test_gemini_nullable_object_tool_schema_httpx now catches litellm.RateLimitError and calls pytest.skip, matching the pattern already used by 22 sibling tests in the same file.
• Regression test: A new subprocess-based test (test_convert_schema_types_output_is_stable_across_hash_seeds) deterministically validates the fix by running _build_vertex_schema under six distinct PYTHONHASHSEED values and asserting identical output — it fails on the pre-fix code and passes with it.

Confidence Score: 5/5

Safe to merge — a minimal, well-evidenced fix for a CI flake with no production behavior change for correct inputs.

The one-line set-to-tuple conversion is narrowly scoped to iteration order inside _convert_schema_types, preserves all existing semantics (membership checks and iteration are both valid on tuples), and is proven by subprocess regression tests and live Vertex AI end-to-end confirmation. The skip guard is already standard practice in the file. No unrelated files are touched.

No files require special attention.

Important Files Changed

Filename	Overview
litellm/llms/vertex_ai/common_utils.py	Changes type_specific_fields from a set literal to a tuple, ensuring deterministic insertion order when building the anyOf branch so JSON key order is stable across Python hash seeds.
tests/local_testing/test_amazing_vertex_completion.py	Wraps the live litellm.completion call in test_gemini_nullable_object_tool_schema_httpx with a try/except to skip on RateLimitError, matching the pattern used by 22 other call sites in the same file.
tests/test_litellm/llms/vertex_ai/test_vertex_ai_common_utils.py	Adds a regression test that spawns subprocesses under PYTHONHASHSEED 0–5 and asserts all serialized _build_vertex_schema outputs are byte-identical; confirms the fix deterministically and fails on the pre-fix code.

_{Reviews (1): Last reviewed commit: "fix(vertex_ai): stabilize nullable tool ..." | Re-trigger Greptile}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[codspeed-hq]

Merging this PR will not alter performance

✅ 30 untouched benchmarks

---

_{Comparing litellm_fix_vertex_429_flake_local_testing (8ac1b05) with litellm_internal_staging (d6cbf6e)}