Skip to content

feat(aux): add Bedrock Converse adapter for auxiliary LLM tasks - #15117

Closed
kobewang1234 wants to merge 1 commit into
NousResearch:mainfrom
kobewang1234:feat/bedrock-auxiliary-client
Closed

kobewang1234 wants to merge 1 commit into
NousResearch:mainfrom
kobewang1234:feat/bedrock-auxiliary-client

Conversation

@kobewang1234

Copy link
Copy Markdown

Summary

Add BedrockAuxiliaryClient that enables AWS Bedrock-hosted models (e.g. Claude Sonnet) to serve as auxiliary LLMs for tasks like context compression, web extraction summarization, session search, and vision.

Currently, users with provider: bedrock cannot use auxiliary LLM features because resolve_provider_client() has no Bedrock path. This PR adds the missing adapter.

Changes

  • _BedrockCompletionsAdapter: Translates chat.completions.create(**kwargs) into Bedrock Converse() API calls, reusing the existing bedrock_adapter.py helpers (convert_messages_to_converse, normalize_converse_response)
  • BedrockAuxiliaryClient (sync): OpenAI-client-compatible wrapper with .chat.completions interface
  • AsyncBedrockAuxiliaryClient: Async variant using asyncio.to_thread
  • _to_async_client(): Added Bedrock dispatch
  • resolve_provider_client(): Added if provider == "bedrock": branch that checks IAM credentials and creates the client

Configuration

Users can configure per-task auxiliary models in config.yaml:

auxiliary:
  compression:
    model: us.anthropic.claude-sonnet-4-6
  web_extract:
    model: us.anthropic.claude-sonnet-4-6

No provider key needed — when the main provider is bedrock, auxiliary resolution automatically uses the Bedrock path.

Testing

  • Verified resolve_provider_client('bedrock', 'us.anthropic.claude-sonnet-4-6') returns a valid BedrockAuxiliaryClient
  • Verified real API call through the adapter returns correct response
  • Verified _resolve_auto() picks up Bedrock client when main provider is bedrock

Add BedrockAuxiliaryClient that routes auxiliary tasks (compression,
web_extract, session_search, vision) through the AWS Bedrock Converse
API using IAM credentials.

Changes:
- Add _BedrockCompletionsAdapter: translates OpenAI chat.completions.create()
  into Bedrock Converse API calls, reusing existing bedrock_adapter.py helpers
- Add BedrockAuxiliaryClient (sync) and AsyncBedrockAuxiliaryClient (async)
- Register Bedrock type in _to_async_client() dispatch
- Add bedrock provider resolution in resolve_provider_client()

This enables users with provider: bedrock to use Bedrock-hosted models
(e.g. Claude Sonnet) for auxiliary tasks without needing separate API keys.
@alt-glitch alt-glitch added type/feature New feature or request P2 Medium — degraded but workaround exists comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint provider/bedrock AWS Bedrock (boto3, IAM) labels Apr 24, 2026
@alt-glitch

Copy link
Copy Markdown
Contributor

Supersedes #12365 — same feature (Bedrock auxiliary client). Also related to #11946 (feature request) and #12979 (full Bedrock Converse).

@teknium1

Copy link
Copy Markdown
Collaborator

Thanks for the contribution, @kobewang1234! This feature was implemented on main just before this PR was opened.

This is an automated hermes-sweeper review.

Evidence:

  • agent/auxiliary_client.py lines 2038–2069 already contain a Bedrock path in resolve_provider_client(): when pconfig.auth_type == "aws_sdk", it checks AWS credentials via has_aws_credentials(), builds a client via build_anthropic_bedrock_client(), and wraps it in AnthropicAuxiliaryClient with full async support.
  • This was added by commit 7dc6eb9fb (fix(agent): handle aws_sdk auth type in resolve_provider_client), merged 2026-04-24 — the same day this PR was opened.
  • Member @alt-glitch also noted this supersedes feat: add AWS Bedrock support for auxiliary LLM tasks #12365, which covers the same gap.

The implementation on main uses the Anthropic Bedrock SDK path rather than a hand-rolled Converse API shim, which is the preferred approach for the existing codebase. Closing as already implemented.

@teknium1 teknium1 closed this Apr 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P2 Medium — degraded but workaround exists provider/bedrock AWS Bedrock (boto3, IAM) type/feature New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants