Skip to content

fix(moa): route bedrock MoA slots through signed bedrock branch #54912

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
iizotov wants to merge 1 commit into from
Closed

fix(moa): route bedrock MoA slots through signed bedrock branch #54912

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 10 additions & 1 deletion agent/moa_loop.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -99,7 +99,16 @@ def _slot_runtime(slot: dict[str, str]) -> dict[str, Any]:
# provider-backed targets whose provider branch adds auth refresh,
# request metadata, or request-shape adapters. Keep those providers
# identified by name.
if resolved_provider in {"nous", "openai-codex", "xai-oauth"}:
# ``bedrock`` belongs here too: its provider branch builds an
# AWS-SigV4-signed client (or IAM-role-signed) against the
# bedrock-runtime endpoint. resolve_runtime_provider returns that
# endpoint's base_url plus a PLACEHOLDER api_key ("aws-sdk") — there is
# no real bearer token. Forwarding base_url+api_key makes call_llm treat
# it as a plain OpenAI-compatible endpoint and POST with an unsigned
# fake bearer, which Bedrock answers with an empty/malformed
# ChatCompletion (choices=None). Keeping it identified by name routes it
# through the real signed bedrock branch.
if resolved_provider in {"nous", "openai-codex", "xai-oauth", "bedrock"}:
return out
# Pass the resolved endpoint through so call_llm builds the request for
# the provider's actual API surface instead of auto-detecting. base_url
Expand Down