test fix us.anthropic.claude-haiku-4-5-20251001-v1:0 - #24931
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Greptile SummaryThis PR replaces the deprecated Bedrock model Key changes:
Confidence Score: 5/5
|
| Filename | Overview |
|---|---|
| tests/local_testing/whitelisted_bedrock_models.txt | Replaces deprecated us.anthropic.claude-3-5-sonnet-20240620-v1:0 with us.anthropic.claude-haiku-4-5-20251001-v1:0 in the whitelist. |
| tests/proxy_e2e_anthropic_messages_tests/test_config.yaml | Underlying model updated to claude-haiku-4-5; model_name label still reads bedrock-claude-sonnet-3.5 (already flagged in prior review thread). |
| tests/pass_through_unit_tests/test_anthropic_messages_passthrough.py | Model string replaced from claude-3-5-sonnet to claude-haiku-4-5 in all mock-based passthrough tests; no logic changes. |
| tests/llm_translation/test_bedrock_completion.py | Model string swapped for passthrough route tests (invoke/converse-stream/invoke-with-response-stream); all assertions still valid with the new model identifier. |
| tests/test_litellm/test_router.py | Model string replaced in router passthrough and credentials tests; assertions updated to match new model ID in endpoint paths. |
| tests/test_litellm/llms/bedrock/passthrough/test_bedrock_passthrough_transformation.py | Non-ARN model_id in backward-compatibility test replaced with new model; also Black formatting applied throughout, no logic changes. |
| tests/batches_tests/test_bedrock_files_and_batches.py | Model string swapped in batch create/retrieve calls; test_async_file_and_batch makes real AWS calls (pre-existing pattern, outside test_litellm/ scope). |
| .gitignore | Deduplicates duplicate .mypy_cache/* entries and upgrades to **/.mypy_cache/ glob to cover all subdirectory caches. |
Flowchart
%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[PR #24931 - Test Fix] --> B[Model String Replacement]
A --> C[Code Formatting Cleanup]
A --> D[.gitignore Fix]
B --> B1["Old: us.anthropic.claude-3-5-sonnet-20240620-v1:0"]
B --> B2["New: us.anthropic.claude-haiku-4-5-20251001-v1:0"]
B2 --> T1[tests/pass_through_unit_tests/]
B2 --> T2[tests/llm_translation/]
B2 --> T3[tests/test_litellm/]
B2 --> T4[tests/batches_tests/]
B2 --> T5[tests/litellm_utils_tests/]
B2 --> T6[tests/logging_callback_tests/]
B2 --> T7[tests/local_testing/whitelisted_bedrock_models.txt]
B2 --> T8[*.jsonl fixture files]
C --> C1[Black formatting applied\nno logic changes]
D --> D1["**.mypy_cache/* x2\n→ **/.mypy_cache/"]
Reviews (3): Last reviewed commit: "ignore mypy cache files" | Re-trigger Greptile
| litellm_params: | ||
| model: "bedrock/us.anthropic.claude-3-5-sonnet-20240620-v1:0" | ||
| model: "bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0" | ||
| aws_region_name: "us-east-1" | ||
|
|
There was a problem hiding this comment.
Stale model_name label after model switch
The model_name is still bedrock-claude-sonnet-3.5 but the underlying model is now bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0 (Haiku, not Sonnet). Any proxy routes or downstream tests that resolve this config by name will log/display the wrong model family, which could be confusing during debugging.
Consider renaming to match the new model, for example:
| litellm_params: | |
| model: "bedrock/us.anthropic.claude-3-5-sonnet-20240620-v1:0" | |
| model: "bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0" | |
| aws_region_name: "us-east-1" | |
| - model_name: bedrock-claude-haiku-4-5 | |
| litellm_params: | |
| model: "bedrock/us.anthropic.claude-haiku-4-5-20251001-v1:0" | |
| aws_region_name: "us-east-1" |
|
|
7c02f9e to
2ad0f42
Compare
* test fix us.anthropic.claude-haiku-4-5-20251001-v1:0 * ignore mypy cache files --------- Co-authored-by: Ishaan Jaffer <ishaanjaffer0324@gmail.com> Co-authored-by: David Chen <clfhhc@gmail.com>
Relevant issues
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
tests/test_litellm/directory, Adding at least 1 test is a hard requirement - see detailsmake test-unit@greptileaiand received a Confidence Score of at least 4/5 before requesting a maintainer reviewDelays in PR merge?
If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).
CI (LiteLLM team)
Branch creation CI run
Link:
CI run for the last commit
Link:
Merge / cherry-pick CI run
Links:
Type
🆕 New Feature
🐛 Bug Fix
🧹 Refactoring
📖 Documentation
🚄 Infrastructure
✅ Test
Changes