fix(bedrock): subagent transport routing + retry transient streaming faults - #43916
fix(bedrock): subagent transport routing + retry transient streaming faults#43916gutosantos82 wants to merge 2 commits into
Conversation
… bugs
The Anthropic SDK's Bedrock event-stream decoder
(anthropic/lib/bedrock/_stream_decoder.py) raises a bare
ValueError("Bad response code, expected 200: {...}") when a streamed
event frame carries a non-200 status. The embedded :exception-type is
frequently a TRANSIENT server fault (internalServerException,
modelStreamErrorException, throttlingException, serviceUnavailableException,
modelTimeoutException) that AWS explicitly tells callers to retry.
Because the SDK raises a bare ValueError, the HTTP status is lost (the agent
sees "HTTP None") and the conversation loop's is_local_validation_error
predicate treats every ValueError/TypeError as a local programming bug,
aborting the turn non-retryably. Observed in practice on
global.anthropic.claude-fable-5 where a single transient internalServerException
killed a delegated subagent task; the identical request succeeded on retry.
Add a narrow carve-out: a ValueError matching the decoder's message AND
carrying a known transient exception-type is excluded from the local-bug
classification so the existing retry/fallback path runs. A genuine
validationException (real client bug) is intentionally NOT excluded and still
aborts. Mirrors the existing JSONDecodeError (NousResearch#14782) and NoneType-not-iterable
(NousResearch#33136) carve-outs.
Adds a regression test covering both the transient types (retryable) and
validationException + bare ValueError (still non-retryable).
|
Verification review — Bedrock transient streaming fault carve-out is correct and well-tested. Reviewed the diff in
Clean merge candidate. No CI checks yet (just submitted), but the logic is straightforward and the test coverage is thorough. |
…ions A delegated subagent whose runtime resolves to a Bedrock base_url (bedrock-runtime.<region>.amazonaws.com) was sent through the chat_completions transport: _detect_api_mode_for_url() recognized OpenAI, xAI, and Anthropic endpoints but not Bedrock, so it fell through to the default. The subagent then POSTed an OpenAI-shaped body to bedrock-runtime/.../chat/completions with 'Bearer None' (no SigV4), which Bedrock cannot process — surfacing as a PERSISTENT internalServerException 400 that failed identically across all retries (unlike the transient faults addressed in the first commit). Observed on global.anthropic.claude-fable-5: the main agent worked (it uses native Converse + boto3 credentials) but every delegate_task subagent died. Fix: - runtime_provider._detect_api_mode_for_url() now returns 'bedrock_converse' for bedrock-runtime.<region>.amazonaws.com URLs, mirroring the main agent's auto-detection in agent/agent_init.py. Includes a suffix-spoof guard so a lookalike host (…amazonaws.com.evil.test) is NOT matched. - delegate_tool: add 'bedrock_converse' to the explicit api_mode override allowlist so users can force it for non-standard endpoints too. Adds 5 detection tests (region variants, uppercase, lookalike host, path-segment spoof).
|
Closing this PR. After deeper investigation, the root cause of the original symptom turned out to be AWS Bedrock-side streaming instability for newly-launched models, not a hermes bug. Findings:
Re the two commits:
Workaround for users hitting this: route subagents (or the session) to an established model until AWS scales streaming capacity for the newest releases — e.g. |
Closes #43915.
Two related Bedrock fixes surfaced while running
global.anthropic.claude-fable-5with delegated subagents. The main agent worked; everydelegate_tasksubagent died with a Bedrock 400.Fix 1 — Subagent transport routing (the actual cause of the failure)
A delegated subagent whose runtime resolves to a Bedrock base_url (
bedrock-runtime.<region>.amazonaws.com) was routed through thechat_completionstransport, because_detect_api_mode_for_url()recognized OpenAI/xAI/Anthropic endpoints but not Bedrock — so it fell through to the default.The subagent then POSTed an OpenAI-shaped body to
bedrock-runtime/.../chat/completionswithAuthorization: Bearer None(no SigV4). Bedrock can't process this and returns a persistentinternalServerException400 — failing identically across all retries.Fix:
runtime_provider._detect_api_mode_for_url()returnsbedrock_conversefor Bedrock runtime URLs, mirroring the main agent's auto-detection inagent/agent_init.py. Includes a suffix-spoof guard (…amazonaws.com.evil.test→ not matched).delegate_tool: addbedrock_converseto the explicit api_mode override allowlist.Fix 2 — Retry transient streaming faults
The Anthropic SDK's Bedrock event-stream decoder (
anthropic/lib/bedrock/_stream_decoder.py:58) raises a bareValueError("Bad response code, expected 200: {...}")for non-200 event frames, dropping the HTTP status (agent seesHTTP None). When the embedded:exception-typeis a transient fault (internalServerException,modelStreamErrorException,throttlingException,serviceUnavailableException,modelTimeoutException), theis_local_validation_errorpredicate misclassified it as a non-retryable local bug.Fix: narrow carve-out so those transient types are retryable, while a genuine
validationExceptionstill aborts. Mirrors the existingJSONDecodeError(#14782) and NoneType-not-iterable (#33136) carve-outs.Tests
test_jsondecodeerror_retryable.pyunchanged (9 passed)