Skip to content

test fix us.anthropic.claude-haiku-4-5-20251001-v1:0 - #24931

Merged
ishaan-berri merged 2 commits into
mainfrom
litellm_fix_test_bedrock_model
Apr 1, 2026
Merged

test fix us.anthropic.claude-haiku-4-5-20251001-v1:0#24931
ishaan-berri merged 2 commits into
mainfrom
litellm_fix_test_bedrock_model

Conversation

@ishaan-berri

Copy link
Copy Markdown
Contributor

Relevant issues

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

  • I have Added testing in the tests/test_litellm/ directory, Adding at least 1 test is a hard requirement - see details
  • My PR passes all unit tests on make test-unit
  • My PR's scope is as isolated as possible, it only solves 1 specific problem
  • I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

CI (LiteLLM team)

CI status guideline:

  • 50-55 passing tests: main is stable with minor issues.
  • 45-49 passing tests: acceptable but needs attention
  • <= 40 passing tests: unstable; be careful with your merges and assess the risk.
  • Branch creation CI run
    Link:

  • CI run for the last commit
    Link:

  • Merge / cherry-pick CI run
    Links:

Type

🆕 New Feature
🐛 Bug Fix
🧹 Refactoring
📖 Documentation
🚄 Infrastructure
✅ Test

Changes

@vercel

vercel Bot commented Apr 1, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
litellm Ready Ready Preview, Comment Apr 1, 2026 5:59pm

Request Review

@codspeed-hq

codspeed-hq Bot commented Apr 1, 2026

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 16 untouched benchmarks


Comparing litellm_fix_test_bedrock_model (2ad0f42) with main (c987bdb)

Open in CodSpeed

@greptile-apps

greptile-apps Bot commented Apr 1, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR replaces the deprecated Bedrock model us.anthropic.claude-3-5-sonnet-20240620-v1:0 with us.anthropic.claude-haiku-4-5-20251001-v1:0 across 16 test files to unblock CI that was failing due to model unavailability. In addition to the model swap, several test files receive Black formatting cleanup (trailing commas, line-length normalization, whitespace), and .gitignore is improved by deduplicating .mypy_cache/* entries with a broader **/.mypy_cache/ glob.

Key changes:

  • Model replacement — all occurrences of the old Sonnet 3.5 cross-region inference profile ID are replaced with the new Haiku 4.5 ID in passthrough, batch, router, health-check, and translation tests
  • JSONL fixturesbedrock_batch_completions.jsonl and input_batch_completions.jsonl updated to use the new model in all 128 request records
  • Formattingtest_bedrock_passthrough_transformation.py, test_bedrock_files_transformation.py, and others reformatted to Black style (no logic changes)
  • .gitignore fix — duplicate .mypy_cache/* entry removed and pattern broadened to **/.mypy_cache/
  • No production code is changed; all changes are confined to the test layer

Confidence Score: 5/5

  • Safe to merge — all changes are test-only model string replacements and formatting cleanup with no production code impact.
  • Every changed file is a test file, JSONL fixture, or config/gitignore. The model swap is a like-for-like replacement (both are Anthropic cross-region inference profile IDs on Bedrock) in tests that exercise routing/passthrough infrastructure rather than model-specific capabilities, so test coverage is not weakened. No P0 or P1 issues were found.
  • No files require special attention.

Important Files Changed

Filename Overview
tests/local_testing/whitelisted_bedrock_models.txt Replaces deprecated us.anthropic.claude-3-5-sonnet-20240620-v1:0 with us.anthropic.claude-haiku-4-5-20251001-v1:0 in the whitelist.
tests/proxy_e2e_anthropic_messages_tests/test_config.yaml Underlying model updated to claude-haiku-4-5; model_name label still reads bedrock-claude-sonnet-3.5 (already flagged in prior review thread).
tests/pass_through_unit_tests/test_anthropic_messages_passthrough.py Model string replaced from claude-3-5-sonnet to claude-haiku-4-5 in all mock-based passthrough tests; no logic changes.
tests/llm_translation/test_bedrock_completion.py Model string swapped for passthrough route tests (invoke/converse-stream/invoke-with-response-stream); all assertions still valid with the new model identifier.
tests/test_litellm/test_router.py Model string replaced in router passthrough and credentials tests; assertions updated to match new model ID in endpoint paths.
tests/test_litellm/llms/bedrock/passthrough/test_bedrock_passthrough_transformation.py Non-ARN model_id in backward-compatibility test replaced with new model; also Black formatting applied throughout, no logic changes.
tests/batches_tests/test_bedrock_files_and_batches.py Model string swapped in batch create/retrieve calls; test_async_file_and_batch makes real AWS calls (pre-existing pattern, outside test_litellm/ scope).
.gitignore Deduplicates duplicate .mypy_cache/* entries and upgrades to **/.mypy_cache/ glob to cover all subdirectory caches.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[PR #24931 - Test Fix] --> B[Model String Replacement]
    A --> C[Code Formatting Cleanup]
    A --> D[.gitignore Fix]

    B --> B1["Old: us.anthropic.claude-3-5-sonnet-20240620-v1:0"]
    B --> B2["New: us.anthropic.claude-haiku-4-5-20251001-v1:0"]

    B2 --> T1[tests/pass_through_unit_tests/]
    B2 --> T2[tests/llm_translation/]
    B2 --> T3[tests/test_litellm/]
    B2 --> T4[tests/batches_tests/]
    B2 --> T5[tests/litellm_utils_tests/]
    B2 --> T6[tests/logging_callback_tests/]
    B2 --> T7[tests/local_testing/whitelisted_bedrock_models.txt]
    B2 --> T8[*.jsonl fixture files]

    C --> C1[Black formatting applied\nno logic changes]
    D --> D1["**.mypy_cache/* x2\n→ **/.mypy_cache/"]
Loading

Reviews (3): Last reviewed commit: "ignore mypy cache files" | Re-trigger Greptile

Comment on lines 8 to 11
litellm_params:
model: "bedrock/us.anthropic.claude-3-5-sonnet-20240620-v1:0"
model: "bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0"
aws_region_name: "us-east-1"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Stale model_name label after model switch

The model_name is still bedrock-claude-sonnet-3.5 but the underlying model is now bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0 (Haiku, not Sonnet). Any proxy routes or downstream tests that resolve this config by name will log/display the wrong model family, which could be confusing during debugging.

Consider renaming to match the new model, for example:

Suggested change
litellm_params:
model: "bedrock/us.anthropic.claude-3-5-sonnet-20240620-v1:0"
model: "bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0"
aws_region_name: "us-east-1"
- model_name: bedrock-claude-haiku-4-5
litellm_params:
model: "bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0"
aws_region_name: "us-east-1"

@yuneng-berri
yuneng-berri self-requested a review April 1, 2026 17:23
@yuneng-berri
yuneng-berri enabled auto-merge April 1, 2026 17:23
@clfhhc
clfhhc temporarily deployed to integration-postgres April 1, 2026 17:38 — with GitHub Actions Inactive
@clfhhc
clfhhc temporarily deployed to integration-postgres April 1, 2026 17:38 — with GitHub Actions Inactive
@clfhhc
clfhhc temporarily deployed to integration-postgres April 1, 2026 17:38 — with GitHub Actions Inactive
@clfhhc
clfhhc temporarily deployed to integration-postgres April 1, 2026 17:38 — with GitHub Actions Inactive
@clfhhc
clfhhc temporarily deployed to integration-redis-postgres April 1, 2026 17:38 — with GitHub Actions Inactive
@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you all sign our Contributor License Agreement before we can accept your contribution.
1 out of 2 committers have signed the CLA.

✅ ishaan-jaff
❌ clfhhc
You have signed the CLA already but the status is still pending? Let us recheck it.

@clfhhc
clfhhc temporarily deployed to integration-redis-postgres April 1, 2026 17:39 — with GitHub Actions Inactive
@clfhhc
clfhhc temporarily deployed to integration-postgres April 1, 2026 17:39 — with GitHub Actions Inactive
@clfhhc
clfhhc temporarily deployed to integration-postgres April 1, 2026 17:39 — with GitHub Actions Inactive
@clfhhc
clfhhc temporarily deployed to integration-postgres April 1, 2026 17:39 — with GitHub Actions Inactive
@clfhhc
clfhhc temporarily deployed to integration-postgres April 1, 2026 17:39 — with GitHub Actions Inactive
@clfhhc
clfhhc force-pushed the litellm_fix_test_bedrock_model branch from 7c02f9e to 2ad0f42 Compare April 1, 2026 17:57
@clfhhc
clfhhc temporarily deployed to integration-postgres April 1, 2026 17:57 — with GitHub Actions Inactive
@clfhhc
clfhhc temporarily deployed to integration-postgres April 1, 2026 17:57 — with GitHub Actions Inactive
@clfhhc
clfhhc temporarily deployed to integration-redis-postgres April 1, 2026 17:57 — with GitHub Actions Inactive
@clfhhc
clfhhc temporarily deployed to integration-postgres April 1, 2026 17:57 — with GitHub Actions Inactive
@clfhhc
clfhhc temporarily deployed to integration-postgres April 1, 2026 17:57 — with GitHub Actions Inactive
@ishaan-berri
ishaan-berri disabled auto-merge April 1, 2026 18:00
@ishaan-berri
ishaan-berri merged commit e4442a4 into main Apr 1, 2026
79 of 101 checks passed
@ishaan-berri
ishaan-berri deleted the litellm_fix_test_bedrock_model branch April 1, 2026 18:01
fzowl pushed a commit to fzowl/litellm that referenced this pull request Jun 24, 2026
* test fix us.anthropic.claude-haiku-4-5-20251001-v1:0

* ignore mypy cache files

---------

Co-authored-by: Ishaan Jaffer <ishaanjaffer0324@gmail.com>
Co-authored-by: David Chen <clfhhc@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants