Skip to content

feat: add AWS Bedrock support for auxiliary LLM tasks - #12365

Closed
superyhee wants to merge 1 commit into
NousResearch:mainfrom
superyhee:feat/bedrock-auxiliary-support
Closed

superyhee wants to merge 1 commit into
NousResearch:mainfrom
superyhee:feat/bedrock-auxiliary-support

Conversation

@superyhee

Copy link
Copy Markdown

Problem

When the main inference provider is bedrock, auxiliary tasks (context compression, web summarization, memory flush, session search) fail with:

WARNING agent.auxiliary_client: resolve_provider_client: unhandled auth_type aws_sdk for bedrock
WARNING agent.auxiliary_client: Auxiliary auto-detect: no provider available (tried: openrouter, nous, local/custom, openai-codex, api-key).
Compression, summarization, and memory flush will not work.

This affects all Bedrock-only deployments (e.g. EC2 instances with IAM instance profiles) where no other LLM provider API key is configured.

Root Cause

resolve_provider_client() handles api_key, external_process, and oauth_* auth types, but has no handler for the aws_sdk auth type used by the Bedrock provider in PROVIDER_REGISTRY.

Solution

Add a BedrockAuxiliaryClient wrapper that translates the standard client.chat.completions.create() interface to Bedrock's Converse API via the existing bedrock_adapter.call_converse(), following the same pattern as CodexAuxiliaryClient and AnthropicAuxiliaryClient.

Changes

  • agent/auxiliary_client.py:

    • Add BedrockAuxiliaryClient / AsyncBedrockAuxiliaryClient wrapper classes (sync + async, same architecture as Codex/Anthropic wrappers)
    • Handle aws_sdk auth_type in resolve_provider_client() with region resolution priority: AWS_DEFAULT_REGION env → explicit base_urlus-east-1 default
    • Register BedrockAuxiliaryClient in _to_async_client()
  • tests/agent/test_bedrock_auxiliary.py (new):

    • 18 tests covering: client structure, adapter delegation, async wrapping, provider resolution, region priority, credential fallback, ImportError graceful handling

Design Decisions

  • Reuses bedrock_adapter.call_converse() — no new Bedrock API code; the adapter already handles OpenAI ↔ Converse format conversion
  • Deferred importsboto3 / bedrock_adapter only imported when actually needed, keeping the module lightweight for non-Bedrock users
  • Region from env var firstPROVIDER_REGISTRY hardcodes inference_base_url to us-east-1, so env var must take priority for cross-region deployments

Testing

  • 18/18 new tests pass
  • 173/173 existing test_auxiliary_client.py + test_bedrock_adapter.py tests pass (zero regressions)
  • Live-verified on EC2 c7g.xlarge (ap-southeast-1) with IAM instance profile + WeChat gateway

When the main inference provider is `bedrock`, auxiliary tasks (context
compression, web summarization, memory flush, session search) fail with
`unhandled auth_type aws_sdk` because resolve_provider_client() has no
handler for the aws_sdk auth type used by the Bedrock provider.

This adds a BedrockAuxiliaryClient wrapper that translates the standard
`client.chat.completions.create()` interface to Bedrock's Converse API
via the existing `bedrock_adapter.call_converse()`, following the same
pattern as CodexAuxiliaryClient and AnthropicAuxiliaryClient.

Changes:
- Add BedrockAuxiliaryClient / AsyncBedrockAuxiliaryClient wrapper classes
- Handle aws_sdk auth_type in resolve_provider_client() with region
  resolution priority: AWS_DEFAULT_REGION env > explicit base_url > us-east-1
- Register BedrockAuxiliaryClient in _to_async_client()
- 18 new tests covering sync/async/resolve/region/fallback paths

Fixes the 'No auxiliary LLM provider configured' warning for
Bedrock-only deployments (e.g. EC2 instances with IAM instance profiles).

Co-Authored-By: Craft Agent <agents-noreply@craft.do>
@teknium1

Copy link
Copy Markdown
Collaborator

Thanks for the thorough PR, @superyhee — the root cause you identified was real and well-diagnosed.

This is an automated hermes-sweeper review.

The specific gap you fixed (aws_sdk auth_type falling through to the "unhandled auth_type" warning in resolve_provider_client()) was independently addressed on main via commit 7dc6eb9fb (2026-04-23), which explicitly closes the cross-referenced issue #13919.

Evidence:

  • agent/auxiliary_client.py lines 2039–2068 now contain an elif pconfig.auth_type == "aws_sdk": branch that routes Bedrock through build_anthropic_bedrock_client() + AnthropicAuxiliaryClient
  • The exact warning message from your bug report (unhandled auth_type aws_sdk for bedrock) can no longer be triggered on current main
  • The landed approach differs slightly — it reuses AnthropicAuxiliaryClient rather than introducing a new BedrockAuxiliaryClient class — but covers the same IAM/boto3 credential chain and region resolution path

One nuance worth noting for follow-up: the landed fix routes through the Anthropic SDK Bedrock path, which means non-Anthropic models hosted on Bedrock (Llama, Mistral, etc.) may still not work for auxiliary tasks. If that's a use-case you care about, a follow-up issue scoped to call_converse()-backed auxiliary support for non-Anthropic Bedrock models would be a clean next step.

@teknium1 teknium1 closed this Apr 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P2 Medium — degraded but workaround exists provider/bedrock AWS Bedrock (boto3, IAM) type/feature New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants