feat: add Amazon Bedrock AgentCore Web Search as a native search provider - #34098
feat: add Amazon Bedrock AgentCore Web Search as a native search provider#34098CrypticDriver wants to merge 5 commits into
Conversation
|
|
Greptile SummaryThis PR adds Amazon Bedrock AgentCore as a native web search provider. The main changes are:
Confidence Score: 5/5This looks safe to merge.
|
| Filename | Overview |
|---|---|
| litellm/llms/bedrock/search/transformation.py | Adds AgentCore MCP transformation, authentication, signing-region resolution, and JSON or SSE response parsing. |
| litellm/llms/base_llm/search/transformation.py | Adds an optional no-op request-signing contract for search providers. |
| litellm/llms/custom_httpx/llm_http_handler.py | Calls the provider signing hook and preserves signed request bodies in sync and async search paths. |
| litellm/types/utils.py | Adds the AgentCore search provider enum value. |
| litellm/utils.py | Registers AgentCore with the existing search provider configuration manager. |
| tests/search_tests/test_agentcore_search.py | Covers request transformation, authentication boundaries, signing regions, signed transport, and MCP response parsing. |
Reviews (3): Last reviewed commit: "fix: honor AWS shared-config region for ..." | Re-trigger Greptile
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
![veria-ai[bot]](https://avatars.githubusercontent.com/in/1723776?s=60&v=4){kind=small}
PR overviewAll previously flagged issues have been addressed. No open security concerns remain on this pull request. Security reviewNo open security issues remain on this pull request. Fixed/addressed: 2 · PR risk: 0/10 |
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
|
|
||
| import httpx | ||
|
|
||
| from litellm.litellm_core_utils.litellm_logging import Logging as LiteLLMLoggingObj |
| import httpx | ||
|
|
||
| from litellm.litellm_core_utils.litellm_logging import Logging as LiteLLMLoggingObj | ||
| from litellm.llms.base_llm.chat.transformation import BaseLLMException |
| from litellm.llms.base_llm.search.transformation import ( | ||
| BaseSearchConfig, | ||
| SearchResponse, | ||
| SearchResult, | ||
| ) |
| SearchResponse, | ||
| SearchResult, | ||
| ) | ||
| from litellm.llms.bedrock.base_aws_llm import BaseAWSLLM |
| SearchResult, | ||
| ) | ||
| from litellm.llms.bedrock.base_aws_llm import BaseAWSLLM | ||
| from litellm.llms.bedrock.common_utils import BedrockError |
| ) | ||
| from litellm.llms.bedrock.base_aws_llm import BaseAWSLLM | ||
| from litellm.llms.bedrock.common_utils import BedrockError | ||
| from litellm.secret_managers.main import get_secret_str |
| from litellm.llms.apiserpent.search.transformation import ( | ||
| APISerpentSearchConfig, | ||
| ) | ||
| from litellm.llms.bedrock.search.transformation import AgentCoreSearchConfig |
6fef231 to
789e2ba
Compare
c5616c7 to
bf7e2c2
Compare
df3e1b0 to
563b392
Compare
…ider Adds 'agentcore' to SearchProviders, backed by an AgentCore Gateway web-search connector target (MCP tools/call over Streamable HTTP). Web Search on Amazon Bedrock AgentCore is an AWS-managed web index (GA June 2026). Exposing it as a native search provider lets Bedrock users enable Claude Code / Anthropic-native WebSearch through websearch_interception with a pure-YAML config and AWS-native auth, keeping the whole search path inside AWS. Implementation: - New AgentCoreSearchConfig (litellm/llms/bedrock/search/) reusing BaseAWSLLM credential resolution. Auth follows the gateway's inbound authorizer type: AWS_IAM gateways get a SigV4-signed request (explicit aws_access_key_id/aws_secret_access_key params or the default credential chain); CUSTOM_JWT gateways get an OAuth2 bearer token via api_key / AGENTCORE_GATEWAY_TOKEN - SigV4 signing region is derived from the gateway URL so callers don't need aws_region_name to match their default region - Adds an optional sign_request() hook to BaseSearchConfig (no-op by default) and teaches the search HTTP handler to send a signed body verbatim, mirroring the existing anthropic_messages/chat pattern - Handles both plain-JSON and SSE-framed MCP responses, propagates MCP errors, truncates queries to the 200-char gateway limit Tested: - 13 unit tests: payload/signing, explicit AKSK passthrough, bearer token via api_key and env, query truncation, SSE frames, MCP error propagation, region derivation - Verified end-to-end against real AWS_IAM and CUSTOM_JWT gateways, including full Claude Code CLI WebSearch round-trips through the proxy with websearch_interception
563b392 to
b61484e
Compare
- Refuse to send the server-managed AGENTCORE_GATEWAY_TOKEN to a caller-supplied api_base (reuses resolve_server_api_key's trusted-host guard) — closes the token-exfiltration path via /search_tools/test_connection - Disable BaseAWSLLM's AWS_BEARER_TOKEN_BEDROCK fallback when signing: that token is a Bedrock Runtime credential and must not reach an AgentCore gateway - Parse SSE responses per spec: join multi-line data fields, iterate events, and return the JSON-RPC response (result/error) instead of the first data line — progress notifications no longer shadow the result - Validate tool_name ends with ___WebSearch so a caller-supplied name cannot invoke unrelated tools on the same gateway with the proxy's credentials - Send the documented maxResults default (10) explicitly instead of leaving it to the gateway - Custom gateway hostnames: raise a clear error when no signing region can be derived and none is configured, instead of signing for a guessed region - 7 new unit tests covering each fix (20 total)
|
Pushed ebdad6e addressing the review findings:
7 new unit tests cover each fix (20 total in |
The previous check only consulted AWS_REGION* env vars before rejecting custom hostnames, breaking deployments that configure their region via the AWS shared config (profile). Resolve through boto3's session (env vars + shared config) and only error when that chain yields nothing — never sign with a silently guessed region.
|
Pushed 2f342dc addressing the follow-up P1 (Preserve shared-config regions): custom-hostname region resolution now goes through boto3's session (env vars and AWS shared config / profile region) and only errors when that chain yields nothing — a deployment with |
Coverage collection runs against the sharded tests/test_litellm tree, so the provider tests living only in tests/search_tests were invisible to codecov (patch coverage reported ~31% despite the suite). Mirror them as tests/test_litellm/llms/bedrock/search/test_agentcore_search_transformation.py and add edge-case tests (malformed MCP content blocks, SSE without a JSON frame, notification-only streams, list request body, error-class mapping). transformation.py line coverage: 99% (26 tests x2 trees).
…ock CI on stale daily branch
|
Pushed e1629b7 to get CI green: it cherry-picks the 3-line This commit is unrelated to the AgentCore provider itself and its diff will disappear automatically once #34318 lands on the daily branch (or if I rebase onto a newer one — happy to do either). All checks are green now except |
|
@tin-berri Would you mind taking a look when you get a chance? This adds a native search provider following the same pattern as the TinyFish one you merged (#30634) — BaseSearchConfig + provider enum — and it closes #31819 which was tagged enhancement. All bot findings resolved (Greptile two rounds, Veria "no open concerns"), CI green except the pre-existing osv-scan, and it's verified end-to-end with a real Claude Code CLI setup. Happy to address any feedback quickly. Thanks! |
|
Docs are up as well: BerriAI/litellm-docs#702 adds the provider page (SDK + proxy usage, IAM/JWT auth, the Claude Code interception setup, tool-name rules and limits) and registers it in the search section. |
|
@mateo-berri Could you take a look at this one? It plugs a native AgentCore search provider into the websearch interception flow you own (#33129) — that's what makes Claude Code web search work on Bedrock deployments with zero client-side changes (closes #31819, tagged enhancement + claude code). Everything's ready for review: bot findings all resolved (Greptile two rounds, Veria "no open security concerns"), CI green except the pre-existing osv-scan, patch coverage 98%, verified end-to-end with a real Claude Code CLI setup, and docs are up at BerriAI/litellm-docs#702. Happy to retarget to a newer daily branch whenever one is cut. Thanks! |
|
superceded by #36331 |
4 participants
Relevant issues
Fixes #31819
Enables Anthropic-native
web_search(and the Claude Code WebSearch tutorial flow) for Bedrock users with an all-AWS search backend — related to the provider matrix indocs/my-website/docs/search/index.mdanddocs/my-website/docs/tutorials/claude_code_websearch.md.Pre-Submission checklist
Please complete all items before asking a LiteLLM maintainer to review your PR
@greptileaito re-request a review after pushing changes)Screenshots / Proof of Fix
All proofs are fully e2e against a real AgentCore Gateway (
websearchgateway-*.gateway.bedrock-agentcore.us-east-1.amazonaws.com/mcpwith aweb-searchconnector target) and real Bedrock Claude calls. Captured at commit6fef231f.1. Direct
litellm.search()(real gateway, SigV4):2. Claude Code request shape through the proxy (
web_search_20250305+ forcedtool_choice,websearch_interception+ this provider, Bedrock Claude Sonnet):3. Real Claude Code CLI end-to-end (
ANTHROPIC_BASE_URL→ proxy → Bedrock, WebSearch executed by this provider; zero client-side config):Bearer-token mode was additionally verified against a real
CUSTOM_JWTgateway (Cognitoclient_credentialstoken) with AWS ambient credentials deliberately removed, including a full Claude Code CLI round-trip.Unit tests:
Type
🆕 New Feature
Changes
Adds Amazon Bedrock AgentCore Web Search (
search_provider: agentcore) as a native search provider. Web Search on AgentCore is an AWS-managed web index (GA June 2026) exposed as an MCP tool on an AgentCore Gateway — no third-party search API or key, and queries never leave AWS.Combined with
websearch_interception, this gives Bedrock users a pure-YAML way to make Claude Code / Anthropic-nativeweb_search_20250305work:AgentCoreSearchConfig(litellm/llms/bedrock/search/transformation.py):BaseSearchConfig + BaseAWSLLM. Transforms the unified search request into an MCPtools/call, parses plain-JSON and SSE-framed MCP responses intoSearchResponse, propagates MCP errors, truncates queries to the 200-char gateway limit. Auth follows the gateway's inbound authorizer: SigV4 (bedrock-agentcoreservice; signing region auto-derived from the gateway URL) or OAuth2 bearer token (api_key/AGENTCORE_GATEWAY_TOKEN).BaseSearchConfig.sign_request()(new optional hook, no-op by default): mirrors the existingsign_requestpattern inanthropic_messages/chat. The search HTTP handler (sync + async) sends a signed body verbatim when present — re-serializing would invalidate the signature. No behavior change for the 17 existing search providers (regression suite passing above).SearchProviders.AGENTCORE+ config-map entry.tests/search_tests/test_agentcore_search.py.Notes / known limitations
tool_name: the gateway exposes the connector as<target-name>___WebSearch, where<target-name>is chosen by the user at target creation. The default (web-search-tool___WebSearch) matches the AWS documentation's boto3/CLI setup examples; targets created with other names needtool_nameset explicitly (misconfiguration surfaces as a propagated MCP "tool not found" error, not a silent failure). Auto-discovery viatools/listwas considered and deliberately left out: a gateway can legitimately host multiple*___WebSearchtargets (e.g. with different domain-filter policies), and silently picking one would be worse than an explicit config error.api_basefield, so UI-driven setup requires theAGENTCORE_GATEWAY_URLenv var on the proxy; config.yaml setup (above) is unaffected. Adding the field to the form is a UI-side follow-up that would also benefit other self-hosted providers (e.g. searxng).