fix(bedrock_mantle): forward AWS creds through the chat->responses bridge - #32338
fix(bedrock_mantle): forward AWS creds through the chat->responses bridge#32338Zerohertz wants to merge 3 commits into
Conversation
…idge litellm.completion() dropped aws_region_name / aws_access_key_id / aws_secret_access_key before dispatching the chat->responses bridge for bedrock_mantle gpt-5.* (Responses-only) models. get_litellm_params() only extracts OPTIONAL_KWARGS_KEYS from its own **kwargs, which the call site does not forward, so the AWS params never reached litellm_params. The bridge then signed the Bedrock Mantle request via SigV4 with empty credentials (and a fallback region for VPC-endpoint hosts whose URL does not encode the region), and Mantle rejected it with "Credential should be scoped to a valid region." Reuse the already-computed _supplemental_provider_params and merge it into litellm_params (without clobbering values already set) so the AWS credentials survive into the bridge. /responses was unaffected. Fixes BerriAI#32336 Signed-off-by: Zerohertz <ohg3417@gmail.com>
Greptile SummaryThis PR fixes a credential-forwarding bug in the
Confidence Score: 4/5Safe to merge — the two-line production change is additive and non-overwriting, and the new test is fully mocked with no network calls. The production change is a targeted, non-overwriting merge of already-computed provider params into litellm_params. The only notable aspect is that the merge runs for every completion() call rather than only when the bridge is active, which silently extends the fix to all providers that pass credentials through OPTIONAL_KWARGS_KEYS. This is functionally correct but undocumented scope. The regression test is solid but would give a confusing failure message if the bridge dispatch condition ever stops matching, since it has no guard asserting the mock was actually invoked. The merge loop in litellm/main.py (lines 5287–5289) deserves a second glance to confirm the broader-than-bridge scope is intentional.
|
| Filename | Overview |
|---|---|
| litellm/main.py | Merges _supplemental_provider_params into litellm_params after get_litellm_params() to forward AWS (and other provider) credentials into the chat->responses bridge. Fix is correct but applies to all completion paths, not just the bridge. |
| tests/test_litellm/completion_extras/test_responses_bridge_provider_propagation.py | Adds a new regression test that patches responses_api_bridge.completion and asserts AWS credentials survive into litellm_params. No real network calls; correctly pins the bug condition. |
Reviews (1): Last reviewed commit: "fix(bedrock_mantle): forward AWS creds t..." | Re-trigger Greptile
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
…ridge path The _supplemental_provider_params merge ran after get_litellm_params() for every completion() call, writing all OPTIONAL_KWARGS_KEYS (Azure, Vertex, GCS, xAI, AWS) into litellm_params on non-bridge paths too. Move it into the responses-bridge dispatch block, right before responses_api_bridge.completion(), so the fix stays scoped to the bridge and does not change litellm_params for other providers. Behavior on the bridge path is unchanged. Signed-off-by: Zerohertz <ohg3417@gmail.com>
…s test Add `assert captured` before the litellm_params assertions so the test fails with a clear "responses_api_bridge.completion was never called" message if bedrock_mantle/openai.gpt-5.4 stops being routed through the bridge, instead of a confusing `None != "us-east-2"` failure. Signed-off-by: Zerohertz <ohg3417@gmail.com>
|
Superseded by #32956, which fixes the credential drop at the get_litellm_params call so the aws_* kwargs reach every downstream path, not just the bridge dispatch. That change merged on 2026-07-13 and covers this case (see the bedrock_mantle responses-bridge regression test in tests/test_litellm/test_main.py). Thanks for the report and the patch; closing this one |
Relevant issues
Fixes #32336
Linear ticket
Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
@greptileaito re-request a review after pushing changes)Delays in PR merge?
If you're seeing a delay in your PR being merged, ping the LiteLLM Team on Slack (#pr-review).
Screenshots / Proof of Fix
bedrock_mantle/openai.gpt-5.*models are Responses-API-only, so every/chat/completionscall to them is routed through the chat->responses bridge.On that path the caller's AWS credentials never reached the SigV4 signer.
The signer below is monkeypatched only to print the region/creds it actually receives — the request never leaves the process, so no mock replaces the code under test (
litellm.completion->get_litellm_params-> bridge -> signer):Before (creds dropped; region also lost when the host doesn't encode it):
Against a real Mantle endpoint the chat calls fail with HTTP 500
invalid_api_key: "Credential should be scoped to a valid region.".After (bridge signs with the caller's creds/region, matching
/responses):Added regression test:
test_completion_forwards_aws_creds_into_bridge_litellm_paramsfails onmain(assertsaws_region_name is None) and passes with this change.Type
🐛 Bug Fix
Changes
litellm.completion()builtlitellm_paramsviaget_litellm_params(...), but that call does not forward**kwargs.get_litellm_params()only pulls theOPTIONAL_KWARGS_KEYS(which includeaws_region_name/aws_access_key_id/aws_secret_access_key) out of its own**kwargs, so with none forwarded the AWS credentials were dropped fromlitellm_params.That aws-less
litellm_paramswas then handed toresponses_api_bridge.completion(...), and the downstreamBedrockMantleResponsesAPIConfig.sign_request()signed with empty credentials.For VPC-endpoint hosts, whose URL does not match
bedrock-mantle.<region>.api.aws,_resolve_region()also fell back to the default region — so both the region and the credentials were wrong.#30083 already added the AWS keys to
OPTIONAL_KWARGS_KEYSand forwarded a_supplemental_provider_paramsGenericLiteLLMParamsintoget_llm_provider()to fix provider detection, but thelitellm_paramspassed to the bridge was never updated, so signing stayed broken.This PR reuses that already-computed
_supplemental_provider_paramsand merges it back intolitellm_paramsright afterget_litellm_params(...), without clobbering values that are already set:litellm/main.py— merge_supplemental_provider_paramsintolitellm_paramsbefore dispatching the responses bridge.tests/test_litellm/completion_extras/test_responses_bridge_provider_propagation.py— add acompletion()-level regression test pinning that the AWS creds reach the bridge'slitellm_params(the existing handler-level tests only cover forwarding once the params are already present).The
/responsespath was already correct and is unchanged.