feat(aux): add Bedrock Converse adapter for auxiliary LLM tasks - #15117
Closed
kobewang1234 wants to merge 1 commit into
Closed
kobewang1234 wants to merge 1 commit into
kobewang1234 wants to merge 1 commit into
Conversation
Add BedrockAuxiliaryClient that routes auxiliary tasks (compression, web_extract, session_search, vision) through the AWS Bedrock Converse API using IAM credentials. Changes: - Add _BedrockCompletionsAdapter: translates OpenAI chat.completions.create() into Bedrock Converse API calls, reusing existing bedrock_adapter.py helpers - Add BedrockAuxiliaryClient (sync) and AsyncBedrockAuxiliaryClient (async) - Register Bedrock type in _to_async_client() dispatch - Add bedrock provider resolution in resolve_provider_client() This enables users with provider: bedrock to use Bedrock-hosted models (e.g. Claude Sonnet) for auxiliary tasks without needing separate API keys.
Contributor
Collaborator
|
Thanks for the contribution, @kobewang1234! This feature was implemented on
Evidence:
The implementation on |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add
BedrockAuxiliaryClientthat enables AWS Bedrock-hosted models (e.g. Claude Sonnet) to serve as auxiliary LLMs for tasks like context compression, web extraction summarization, session search, and vision.Currently, users with
provider: bedrockcannot use auxiliary LLM features becauseresolve_provider_client()has no Bedrock path. This PR adds the missing adapter.Changes
_BedrockCompletionsAdapter: Translateschat.completions.create(**kwargs)into BedrockConverse()API calls, reusing the existingbedrock_adapter.pyhelpers (convert_messages_to_converse,normalize_converse_response)BedrockAuxiliaryClient(sync): OpenAI-client-compatible wrapper with.chat.completionsinterfaceAsyncBedrockAuxiliaryClient: Async variant usingasyncio.to_thread_to_async_client(): Added Bedrock dispatchresolve_provider_client(): Addedif provider == "bedrock":branch that checks IAM credentials and creates the clientConfiguration
Users can configure per-task auxiliary models in
config.yaml:No
providerkey needed — when the main provider isbedrock, auxiliary resolution automatically uses the Bedrock path.Testing
resolve_provider_client('bedrock', 'us.anthropic.claude-sonnet-4-6')returns a validBedrockAuxiliaryClient_resolve_auto()picks up Bedrock client when main provider is bedrock