Skip to content

fix(bedrock): route cross-region openai inference profiles to correct endpoint - #40246

Closed
anton-codes-iac wants to merge 3 commits into
BerriAI:litellm_internal_stagingfrom
anton-codes-iac:fix/bedrock-cross-region-openai-routing
Closed

anton-codes-iac wants to merge 3 commits into
BerriAI:litellm_internal_stagingfrom
anton-codes-iac:fix/bedrock-cross-region-openai-routing

Conversation

@anton-codes-iac

@anton-codes-iac anton-codes-iac commented Sep 8, 2026 •

Copy link
Copy Markdown

TLDR

Problem this solves:

  • Cross-region Bedrock profiles (global.openai.*) fail on image inputs.
  • They incorrectly route to the Bedrock Converse API instead of the OpenAI endpoint.

How it solves it:

  • Maps global.openai., eu.openai., us.openai., and apac.openai. prefixes to the OpenAI route.
  • Bypasses Converse transformation entirely for these specific inference profiles.

User Flow

Before: an application sending an image payload to a cross-region Bedrock profile gets rejected by the provider.

  1. The user sends POST /v1/chat/completions with an image URL using the model bedrock/global.openai.gpt-5.6-sol.
  2. The gateway returns a 400 Bad Request stating the model doesn't support the image field for user messages.

After: the exact same request routes to the correct provider endpoint.

  1. The user sends POST /v1/chat/completions with an image URL using the model bedrock/global.openai.gpt-5.6-sol.
  2. The gateway correctly maps the payload and returns a 200 OK with the model's text response.

Relevant issues

Fixes #40080

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

  • I have added meaningful tests
  • The handful of test files covering my change pass locally
  • My PR passes all required CI/CD checks
  • My PR's scope is as isolated as possible; it only solves 1 specific problem
  • I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review

Screenshots / Proof of Fix

Before

(Terminal output showing the 400 Bad Request BedrockException for Converse API)

After

(Terminal output showing your pytest passing successfully for the routing logic)

Type

🐛 Bug Fix

Caveats (if any)

QA runbook

Final Attestation

  • The tests check the right things, including the edge cases, and regressions in the respective real-world customer use-cases are not possible after this PR

@anton-codes-iac
anton-codes-iac requested a review from a team September 8, 2026 12:58
@codspeed

codspeed Bot commented Sep 8, 2026 •

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 31 untouched benchmarks


Comparing anton-codes-iac:fix/bedrock-cross-region-openai-routing (2a5e77d) with litellm_internal_staging (a73454b)1

Open in CodSpeed

Footnotes

  1. No successful run was found on litellm_internal_staging (559247f) during the generation of this report, so a73454b was used instead as the comparison base. There might be some changes unrelated to this pull request in this report. ↩

@greptile-apps

greptile-apps Bot commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR classifies regional Bedrock OpenAI inference-profile identifiers as OpenAI routes so chat requests select the matching adapter instead of Converse

  • Adds global, EU, APAC, and US OpenAI profile prefixes to Bedrock route selection
  • Extends explicit OpenAI-route recognition for the same identifiers
  • Adds parameterized routing tests, but those tests need to verify functional request behavior rather than implementation structure

Confidence Score: 4/5

The routing change appears sound, but the repository’s explicit test and comment requirements must be satisfied before merging

No functional routing failure was established, but the new regression test checks implementation structure and contains prohibited narration comments

Files Needing Attention: tests/test_litellm/llms/bedrock/test_bedrock_common_utils.py

Important Files Changed

Filename Overview
litellm/llms/bedrock/common_utils.py Adds four regional inference-profile prefixes to the existing Bedrock OpenAI routing path
tests/test_litellm/llms/bedrock/test_bedrock_common_utils.py Adds regional routing coverage, but tests internal classification and config identity instead of the resulting request behavior

Reviews (1): Last reviewed commit: "fix(bedrock): route cross-region openai ..." | Re-trigger Greptile

Comment thread tests/test_litellm/llms/bedrock/test_bedrock_common_utils.py Outdated
Comment thread tests/test_litellm/llms/bedrock/test_bedrock_common_utils.py
@codecov

codecov Bot commented Sep 8, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@anton-codes-iac

Copy link
Copy Markdown
Author

Noticed a brief question earlier about ensuring users can still explicitly route to the Converse API if needed. Since "converse/" sits above the new mappings in route_mappings, that override works. Just pushed a few extra tests to exhaustively cover the terra/luna variants and separately test the Converse behavior for those who specifically need to use it.

@yuneng-berri
yuneng-berri deleted the branch BerriAI:litellm_internal_staging September 13, 2026 04:50
@yuneng-berri yuneng-berri reopened this Sep 13, 2026
@mateo-berri

Copy link
Copy Markdown
Contributor

Closing unmerged: the #40080 image failure no longer reproduces on Converse at the current tip, and this Invoke route breaks max_tokens, streaming, and reasoning_effort live

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: GPT-5.6 cross-region inference profiles on Bedrock fail with image input (routed through Converse instead of OpenAI endpoint)

3 participants