Skip to content

[Fix] Update gemini-2.0-flash to gemini-2.5-flash in test_gemini - #24817

Merged
yuneng-berri merged 1 commit into
mainfrom
litellm_/strange-jackson
Mar 30, 2026
Merged

[Fix] Update gemini-2.0-flash to gemini-2.5-flash in test_gemini#24817
yuneng-berri merged 1 commit into
mainfrom
litellm_/strange-jackson

Conversation

@yuneng-berri

Copy link
Copy Markdown
Collaborator

Summary

Failure Path (Before Fix)

Integration tests in llm_translation_testing (test_gemini) using gemini/gemini-2.0-flash fail with a 404 error because gemini-2.0-flash is no longer available to new users.

Fix

Updated all 3 instances of gemini/gemini-2.0-flash to gemini/gemini-2.5-flash in tests/llm_translation/test_gemini.py:

  • test_gemini_with_grounding (streaming call)
  • test_gemini_with_empty_function_call_arguments
  • test_gemini_tool_use

Testing

  • llm_translation_testing with test_gemini

Type

🐛 Bug Fix
✅ Test

…gration tests

gemini-2.0-flash is no longer available to new users, causing 404 errors
in llm_translation_testing tests.
@CLAassistant

CLAassistant commented Mar 30, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@vercel

vercel Bot commented Mar 30, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
litellm Ready Ready Preview, Comment Mar 30, 2026 8:55pm

Request Review

@codspeed-hq

codspeed-hq Bot commented Mar 30, 2026

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 16 untouched benchmarks


Comparing litellm_/strange-jackson (b39748d) with main (1757e1d)

Open in CodSpeed

@greptile-apps

greptile-apps Bot commented Mar 30, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR updates three existing integration tests in tests/llm_translation/test_gemini.py from gemini/gemini-2.0-flash to gemini/gemini-2.5-flash to fix 404 errors caused by gemini-2.0-flash no longer being available to new users. The change is purely mechanical — no test logic, assertions, or structure is altered.

  • The model name swap is applied consistently across all three affected tests: test_gemini_with_grounding, test_gemini_with_empty_function_call_arguments, and test_gemini_tool_use.
  • The updated model is in the same Gemini family and the tests cover the same capabilities (grounding/search, empty function-call arguments, and streaming tool-use), so test coverage equivalence is maintained.
  • All three tests (test_gemini_with_grounding, test_gemini_with_empty_function_call_arguments, test_gemini_tool_use) make live network calls to the Gemini API rather than using mocks. This pre-existed this PR but is worth noting: the tests/llm_translation/ directory policy is to use only mock tests. The nearby test_claude_tool_use_with_gemini (added in the same file) demonstrates the preferred mocked approach.
  • The PR description does not include passing test output or CI run evidence to confirm the fix resolves the 404s, which is recommended per contribution guidelines.

Confidence Score: 5/5

  • Safe to merge — the change is a straightforward model-name update with no logic or assertion changes.
  • All findings are P2 (style/policy observations). The model-name swap is correct, no test coverage is weakened, and the tests still validate the same functionality. Pre-existing live-call issues are not introduced by this PR.
  • No files require special attention beyond the pre-existing real-network-call pattern in tests/llm_translation/test_gemini.py.

Important Files Changed

Filename Overview
tests/llm_translation/test_gemini.py Updates 3 occurrences of gemini/gemini-2.0-flashgemini/gemini-2.5-flash in existing integration tests; no logic changes.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[test_gemini_with_grounding] -->|model param| B["gemini/gemini-2.5-flash\n(was gemini-2.0-flash)"]
    C[test_gemini_with_empty_function_call_arguments] -->|model param| B
    D[test_gemini_tool_use] -->|model param| B
    B --> E[Gemini API Live Call]
    E --> F{Response}
    F -->|200 OK| G[Assertions pass]
    F -->|404 Not Found\ngemini-2.0-flash removed| H[Test fails ❌]
Loading

Comments Outside Diff (1)

  1. tests/llm_translation/test_gemini.py, line 534-574 (link)

    P2 Tests make real network calls

    test_gemini_with_grounding and test_gemini_with_empty_function_call_arguments (and test_gemini_tool_use at line 806) call litellm.completion() without any mocking, making live API requests to the Gemini network. The repo policy for tests/llm_translation/ is that only mock tests should live here, to ensure reliable execution in CI and for all developers locally.

    While these tests were not introduced by this PR, updating them perpetuates the issue. Consider converting them to use unittest.mock.patch or a similar approach (see test_claude_tool_use_with_gemini just above as a good example of a properly mocked test in the same file).

    Rule Used: What: prevent any tests from being added here that... (source)

Reviews (1): Last reviewed commit: "[Fix] Update gemini-2.0-flash to gemini-..." | Re-trigger Greptile

@ryan-crabbe-berri ryan-crabbe-berri left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@yuneng-berri
yuneng-berri merged commit bcfa535 into main Mar 30, 2026
82 of 117 checks passed
@yuneng-berri
yuneng-berri deleted the litellm_/strange-jackson branch March 30, 2026 21:05
fzowl pushed a commit to fzowl/litellm that referenced this pull request Jun 24, 2026
[Fix] Update gemini-2.0-flash to gemini-2.5-flash in test_gemini
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants