Repository navigation
fix(bedrock): route cross-region openai inference profiles to correct endpoint - #40246
anton-codes-iac wants to merge 3 commits into
Conversation
Greptile SummaryThis PR classifies regional Bedrock OpenAI inference-profile identifiers as OpenAI routes so chat requests select the matching adapter instead of Converse
Confidence Score: 4/5The routing change appears sound, but the repository’s explicit test and comment requirements must be satisfied before merging No functional routing failure was established, but the new regression test checks implementation structure and contains prohibited narration comments Files Needing Attention: tests/test_litellm/llms/bedrock/test_bedrock_common_utils.py
|
| Filename | Overview |
|---|---|
| litellm/llms/bedrock/common_utils.py | Adds four regional inference-profile prefixes to the existing Bedrock OpenAI routing path |
| tests/test_litellm/llms/bedrock/test_bedrock_common_utils.py | Adds regional routing coverage, but tests internal classification and config identity instead of the resulting request behavior |
Reviews (1): Last reviewed commit: "fix(bedrock): route cross-region openai ..." | Re-trigger Greptile
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
… explicit converse overrides
|
Noticed a brief question earlier about ensuring users can still explicitly route to the Converse API if needed. Since "converse/" sits above the new mappings in route_mappings, that override works. Just pushed a few extra tests to exhaustively cover the terra/luna variants and separately test the Converse behavior for those who specifically need to use it. |
|
Closing unmerged: the #40080 image failure no longer reproduces on Converse at the current tip, and this Invoke route breaks max_tokens, streaming, and reasoning_effort live |
TLDR
Problem this solves:
global.openai.*) fail on image inputs.How it solves it:
global.openai.,eu.openai.,us.openai., andapac.openai.prefixes to the OpenAI route.User Flow
Before: an application sending an image payload to a cross-region Bedrock profile gets rejected by the provider.
/v1/chat/completionswith an image URL using the modelbedrock/global.openai.gpt-5.6-sol.400 Bad Requeststating the model doesn't support the image field for user messages.After: the exact same request routes to the correct provider endpoint.
/v1/chat/completionswith an image URL using the modelbedrock/global.openai.gpt-5.6-sol.200 OKwith the model's text response.Relevant issues
Fixes #40080
Linear ticket
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
Screenshots / Proof of Fix
Before
(Terminal output showing the
400 Bad RequestBedrockException for Converse API)After
(Terminal output showing your
pytestpassing successfully for the routing logic)Type
🐛 Bug Fix
Caveats (if any)
QA runbook
Final Attestation