Skip to content

fix(bedrock_mantle): forward AWS creds through the chat->responses bridge - #32338

Closed
Zerohertz wants to merge 3 commits into
BerriAI:litellm_internal_stagingfrom
Zerohertz:fix/mantle-chat-bridge-aws-creds
Closed

fix(bedrock_mantle): forward AWS creds through the chat->responses bridge#32338
Zerohertz wants to merge 3 commits into
BerriAI:litellm_internal_stagingfrom
Zerohertz:fix/mantle-chat-bridge-aws-creds

Conversation

@Zerohertz

@Zerohertz Zerohertz commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Relevant issues

Fixes #32336

Linear ticket

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

  • I have added meaningful tests
  • My PR passes all CI/CD checks (e.g., lint, format, unit tests)
  • My PR's scope is as isolated as possible; it only solves 1 specific problem
  • I have received a Greptile Confidence Score of at least 4/5 before requesting a maintainer review (Greptile reviews automatically once the PR is opened; only comment @greptileai to re-request a review after pushing changes)

Delays in PR merge?

If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).

Screenshots / Proof of Fix

bedrock_mantle/openai.gpt-5.* models are Responses-API-only, so every /chat/completions call to them is routed through the chat->responses bridge.
On that path the caller's AWS credentials never reached the SigV4 signer.

The signer below is monkeypatched only to print the region/creds it actually receives — the request never leaves the process, so no mock replaces the code under test (litellm.completion -> get_litellm_params -> bridge -> signer):

import litellm
from litellm.llms.bedrock.base_aws_llm import BaseAWSLLM

def spy(self, service_name, headers, optional_params, request_data, api_base, **kw):
    print("SIGN region =", optional_params.get("aws_region_name"),
          "| access_key =", optional_params.get("aws_access_key_id"))
    raise RuntimeError("stop")
BaseAWSLLM._sign_request = spy

creds = dict(aws_region_name="us-east-2",
             aws_access_key_id="AKIAEXAMPLE",
             aws_secret_access_key="secretexample")

# /responses — already correct
try:
    litellm.responses(model="bedrock_mantle/openai.gpt-5.4", input="hi",
                      api_base="https://bedrock-mantle.us-east-2.api.aws/v1", **creds)
except Exception: pass

# /chat/completions bridge — standard host and VPC endpoint host
for base in ("https://bedrock-mantle.us-east-2.api.aws/v1",
             "https://vpce-020c19ce7e8af3fac-3on5ps9r.bedrock-mantle.us-east-2.vpce.amazonaws.com/v1"):
    try:
        litellm.completion(model="bedrock_mantle/openai.gpt-5.4",
                           messages=[{"role": "user", "content": "hi"}], api_base=base, **creds)
    except Exception: pass

Before (creds dropped; region also lost when the host doesn't encode it):

responses            -> SIGN region = us-east-2 | access_key = AKIAEXAMPLE
chat / standard host -> SIGN region = us-east-2 | access_key = None
chat / VPC endpoint  -> SIGN region = us-east-1 | access_key = None

Against a real Mantle endpoint the chat calls fail with HTTP 500
invalid_api_key: "Credential should be scoped to a valid region.".

After (bridge signs with the caller's creds/region, matching /responses):

responses            -> SIGN region = us-east-2 | access_key = AKIAEXAMPLE
chat / standard host -> SIGN region = us-east-2 | access_key = AKIAEXAMPLE
chat / VPC endpoint  -> SIGN region = us-east-2 | access_key = AKIAEXAMPLE

Added regression test:

$ pytest tests/test_litellm/completion_extras/test_responses_bridge_provider_propagation.py -q
....                                                                     [100%]
4 passed

test_completion_forwards_aws_creds_into_bridge_litellm_params fails on main (asserts aws_region_name is None) and passes with this change.

Type

🐛 Bug Fix

Changes

litellm.completion() built litellm_params via get_litellm_params(...), but that call does not forward **kwargs. get_litellm_params() only pulls the OPTIONAL_KWARGS_KEYS (which include aws_region_name / aws_access_key_id / aws_secret_access_key) out of its own **kwargs, so with none forwarded the AWS credentials were dropped from litellm_params.
That aws-less litellm_params was then handed to responses_api_bridge.completion(...), and the downstream BedrockMantleResponsesAPIConfig.sign_request() signed with empty credentials.
For VPC-endpoint hosts, whose URL does not match bedrock-mantle.<region>.api.aws, _resolve_region() also fell back to the default region — so both the region and the credentials were wrong.

#30083 already added the AWS keys to OPTIONAL_KWARGS_KEYS and forwarded a _supplemental_provider_params GenericLiteLLMParams into get_llm_provider() to fix provider detection, but the litellm_params passed to the bridge was never updated, so signing stayed broken.

This PR reuses that already-computed _supplemental_provider_params and merges it back into litellm_params right after get_litellm_params(...), without clobbering values that are already set:

  • litellm/main.py — merge _supplemental_provider_params into litellm_params before dispatching the responses bridge.
  • tests/test_litellm/completion_extras/test_responses_bridge_provider_propagation.py — add a completion()-level regression test pinning that the AWS creds reach the bridge's litellm_params (the existing handler-level tests only cover forwarding once the params are already present).

The /responses path was already correct and is unchanged.

…idge

litellm.completion() dropped aws_region_name / aws_access_key_id /
aws_secret_access_key before dispatching the chat->responses bridge for
bedrock_mantle gpt-5.* (Responses-only) models. get_litellm_params() only
extracts OPTIONAL_KWARGS_KEYS from its own **kwargs, which the call site does
not forward, so the AWS params never reached litellm_params. The bridge then
signed the Bedrock Mantle request via SigV4 with empty credentials (and a
fallback region for VPC-endpoint hosts whose URL does not encode the region),
and Mantle rejected it with "Credential should be scoped to a valid region."

Reuse the already-computed _supplemental_provider_params and merge it into
litellm_params (without clobbering values already set) so the AWS credentials
survive into the bridge. /responses was unaffected.

Fixes BerriAI#32336

Signed-off-by: Zerohertz <ohg3417@gmail.com>
@greptile-apps

greptile-apps Bot commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR fixes a credential-forwarding bug in the bedrock_mantle chat→responses bridge: litellm.completion() called get_litellm_params() without forwarding **kwargs, so AWS SigV4 credentials were silently dropped from litellm_params before reaching BedrockMantleResponsesAPIConfig.sign_request().

  • litellm/main.py — after get_litellm_params(), merges the already-computed _supplemental_provider_params (all OPTIONAL_KWARGS_KEYS present in kwargs) back into litellm_params where keys are currently None, using a non-overwriting loop so explicitly-set values are preserved.
  • New regression test — patches responses_api_bridge.completion and asserts all three AWS credential keys survive into the litellm_params the bridge receives; relies entirely on mocks with no real network calls.

Confidence Score: 4/5

Safe to merge — the two-line production change is additive and non-overwriting, and the new test is fully mocked with no network calls.

The production change is a targeted, non-overwriting merge of already-computed provider params into litellm_params. The only notable aspect is that the merge runs for every completion() call rather than only when the bridge is active, which silently extends the fix to all providers that pass credentials through OPTIONAL_KWARGS_KEYS. This is functionally correct but undocumented scope. The regression test is solid but would give a confusing failure message if the bridge dispatch condition ever stops matching, since it has no guard asserting the mock was actually invoked.

The merge loop in litellm/main.py (lines 5287–5289) deserves a second glance to confirm the broader-than-bridge scope is intentional.

Important Files Changed

Filename Overview
litellm/main.py Merges _supplemental_provider_params into litellm_params after get_litellm_params() to forward AWS (and other provider) credentials into the chat->responses bridge. Fix is correct but applies to all completion paths, not just the bridge.
tests/test_litellm/completion_extras/test_responses_bridge_provider_propagation.py Adds a new regression test that patches responses_api_bridge.completion and asserts AWS credentials survive into litellm_params. No real network calls; correctly pins the bug condition.

Reviews (1): Last reviewed commit: "fix(bedrock_mantle): forward AWS creds t..." | Re-trigger Greptile

Comment thread litellm/main.py Outdated
@codecov

codecov Bot commented Jul 7, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

Zerohertz added 2 commits July 7, 2026 22:51
…ridge path

The _supplemental_provider_params merge ran after get_litellm_params() for
every completion() call, writing all OPTIONAL_KWARGS_KEYS (Azure, Vertex, GCS,
xAI, AWS) into litellm_params on non-bridge paths too. Move it into the
responses-bridge dispatch block, right before responses_api_bridge.completion(),
so the fix stays scoped to the bridge and does not change litellm_params for
other providers. Behavior on the bridge path is unchanged.

Signed-off-by: Zerohertz <ohg3417@gmail.com>
…s test

Add `assert captured` before the litellm_params assertions so the test fails
with a clear "responses_api_bridge.completion was never called" message if
bedrock_mantle/openai.gpt-5.4 stops being routed through the bridge, instead of
a confusing `None != "us-east-2"` failure.

Signed-off-by: Zerohertz <ohg3417@gmail.com>
@mateo-berri

Copy link
Copy Markdown
Contributor

Superseded by #32956, which fixes the credential drop at the get_litellm_params call so the aws_* kwargs reach every downstream path, not just the bridge dispatch. That change merged on 2026-07-13 and covers this case (see the bedrock_mantle responses-bridge regression test in tests/test_litellm/test_main.py). Thanks for the report and the patch; closing this one

@Zerohertz
Zerohertz deleted the fix/mantle-chat-bridge-aws-creds branch July 14, 2026 23:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: bedrock_mantle gpt-5.4/5.5 via /chat/completions drops AWS credentials

2 participants