Skip to content

fix(bedrock): fall back to non-streaming InvokeModel when IAM denies InvokeModelWithResponseStream - #44293

Merged
teknium1 merged 1 commit into
mainfrom
hermes/hermes-fb8b4d98
Jun 11, 2026
Merged

fix(bedrock): fall back to non-streaming InvokeModel when IAM denies InvokeModelWithResponseStream#44293
teknium1 merged 1 commit into
mainfrom
hermes/hermes-fb8b4d98

Conversation

@teknium1

Copy link
Copy Markdown
Contributor

Summary

AWS Bedrock now works with InvokeModel-only IAM policies: when IAM denies bedrock:InvokeModelWithResponseStream, the agent falls back to the non-streaming converse() path instead of looping on AccessDenied.

Root cause: the agent loop hard-prefers streaming (conversation_loop.py:977), and an AWS AccessDeniedException for the streaming action never matched the existing "stream not supported" auto-fallback — so least-privilege IAM users could not use Bedrock at all.

Changes

  • agent/bedrock_adapter.py: new is_streaming_access_denied_error() (ClientError code check + wrapped-SDK message match); call_converse_stream() falls back to converse() on denial.
  • agent/chat_completion_helpers.py: bedrock_converse streaming branch retries inline via converse() and sets _disable_streaming so subsequent turns skip the doomed stream attempt; the streaming retry block also recognizes the denial for the AnthropicBedrock SDK path (cheap message pre-check before importing bedrock_adapter, avoiding its lazy boto3 install for unrelated providers).
  • Both paths print a one-line notice naming the IAM action that restores streaming.

Validation

Before After
InvokeModel-only IAM AccessDenied retry loop, unusable inline fallback to converse(), session continues non-streaming
Full IAM streaming streaming (unchanged)

Targeted tests: tests/agent/test_bedrock_adapter.py + tests/run_agent/test_streaming.py — 162/162 pass, including 7 new tests (detector positives/negatives, inline fallback, ThrottlingException still propagates).

Infographic

bedrock-iam-streaming-fallback

…InvokeModelWithResponseStream

IAM policies scoped to bedrock:InvokeModel only (a common least-privilege
setup) reject converse_stream() with AccessDeniedException. The agent loop
hard-prefers streaming and the denial never matched the 'stream not
supported' auto-fallback, so InvokeModel-only users looped on AccessDenied
forever.

- agent/bedrock_adapter.py: new is_streaming_access_denied_error()
  detector (ClientError code check + wrapped-SDK message match);
  call_converse_stream() falls back to converse() on denial.
- agent/chat_completion_helpers.py: bedrock_converse streaming branch
  retries inline via converse() and sets _disable_streaming so later
  turns skip the doomed stream attempt; the chat-completions retry
  block also recognizes the denial for the AnthropicBedrock SDK path
  (message pre-check avoids importing bedrock_adapter — and its lazy
  boto3 install — for unrelated providers).

Both paths print a one-line notice telling the user which IAM action
restores streaming.
@github-actions

Copy link
Copy Markdown
Contributor

🔎 Lint report: hermes/hermes-fb8b4d98 vs origin/main

ruff

Total: 0 on HEAD, 0 on base (➖ 0)

🆕 New issues: none

✅ Fixed issues: none

Unchanged: 0 pre-existing issues carried over.

ty (type checker)

Total: 10715 on HEAD, 10712 on base (🆕 +3)

🆕 New issues (1):

Rule Count
invalid-assignment 1
First entries
agent/bedrock_adapter.py:237: [invalid-assignment] invalid-assignment: Object of type `None` is not assignable to `<class 'ClientError'>`

✅ Fixed issues: none

Unchanged: 5606 pre-existing issues carried over.

Diagnostics are surfaced as warnings — this check never fails the build.

@liuhao1024

Copy link
Copy Markdown
Contributor

Verified ✅ — thorough IAM fallback with good coverage across all three call sites.

What was checked:

  • Traced the 3 fallback paths: call_converse_stream() in bedrock_adapter.py, _bedrock_call() in chat_completion_helpers.py, and the AnthropicBedrock SDK _call() path. All three correctly detect InvokeModelWithResponseStream denial and fall back to the non-streaming equivalent.
  • Verified is_streaming_access_denied_error() detection is appropriately scoped: requires invokemodelwithresponsestream in message and an auth-related error code (AccessDeniedException / UnauthorizedException) or "not authorized" phrasing. A ValidationException mentioning the action name won't trigger the fallback (test confirms).
  • Checked the lazy-import guard in _call(): invokemodelwithresponsestream in the lowered error message is checked first, avoiding a boto3 import for unrelated providers' errors. Smart optimization.
  • The _disable_streaming = True flag persists for the session — once denied, subsequent turns skip streaming immediately instead of re-failing.
  • Confirmed test coverage: detection unit tests (positive, negative, wrapped SDK form), integration test with mock client, error propagation test ensuring non-IAM errors still raise.

@teknium1
teknium1 merged commit e24c935 into main Jun 11, 2026
28 checks passed
@teknium1
teknium1 deleted the hermes/hermes-fb8b4d98 branch June 11, 2026 14:16
alt-glitch pushed a commit that referenced this pull request Jun 14, 2026
…InvokeModelWithResponseStream (#44293)

IAM policies scoped to bedrock:InvokeModel only (a common least-privilege
setup) reject converse_stream() with AccessDeniedException. The agent loop
hard-prefers streaming and the denial never matched the 'stream not
supported' auto-fallback, so InvokeModel-only users looped on AccessDenied
forever.

- agent/bedrock_adapter.py: new is_streaming_access_denied_error()
  detector (ClientError code check + wrapped-SDK message match);
  call_converse_stream() falls back to converse() on denial.
- agent/chat_completion_helpers.py: bedrock_converse streaming branch
  retries inline via converse() and sets _disable_streaming so later
  turns skip the doomed stream attempt; the chat-completions retry
  block also recognizes the denial for the AnthropicBedrock SDK path
  (message pre-check avoids importing bedrock_adapter — and its lazy
  boto3 install — for unrelated providers).

Both paths print a one-line notice telling the user which IAM action
restores streaming.
AIalliAI pushed a commit to AIalliAI/Hermes that referenced this pull request Jun 14, 2026
…InvokeModelWithResponseStream (NousResearch#44293)

IAM policies scoped to bedrock:InvokeModel only (a common least-privilege
setup) reject converse_stream() with AccessDeniedException. The agent loop
hard-prefers streaming and the denial never matched the 'stream not
supported' auto-fallback, so InvokeModel-only users looped on AccessDenied
forever.

- agent/bedrock_adapter.py: new is_streaming_access_denied_error()
  detector (ClientError code check + wrapped-SDK message match);
  call_converse_stream() falls back to converse() on denial.
- agent/chat_completion_helpers.py: bedrock_converse streaming branch
  retries inline via converse() and sets _disable_streaming so later
  turns skip the doomed stream attempt; the chat-completions retry
  block also recognizes the denial for the AnthropicBedrock SDK path
  (message pre-check avoids importing bedrock_adapter — and its lazy
  boto3 install — for unrelated providers).

Both paths print a one-line notice telling the user which IAM action
restores streaming.
T02200059 pushed a commit to T02200059/hermes-agent that referenced this pull request Jun 18, 2026
…InvokeModelWithResponseStream (NousResearch#44293)

IAM policies scoped to bedrock:InvokeModel only (a common least-privilege
setup) reject converse_stream() with AccessDeniedException. The agent loop
hard-prefers streaming and the denial never matched the 'stream not
supported' auto-fallback, so InvokeModel-only users looped on AccessDenied
forever.

- agent/bedrock_adapter.py: new is_streaming_access_denied_error()
  detector (ClientError code check + wrapped-SDK message match);
  call_converse_stream() falls back to converse() on denial.
- agent/chat_completion_helpers.py: bedrock_converse streaming branch
  retries inline via converse() and sets _disable_streaming so later
  turns skip the doomed stream attempt; the chat-completions retry
  block also recognizes the denial for the AnthropicBedrock SDK path
  (message pre-check avoids importing bedrock_adapter — and its lazy
  boto3 install — for unrelated providers).

Both paths print a one-line notice telling the user which IAM action
restores streaming.
waefrebeorn pushed a commit to waefrebeorn/slermes that referenced this pull request Jul 2, 2026
…InvokeModelWithResponseStream (NousResearch#44293)

IAM policies scoped to bedrock:InvokeModel only (a common least-privilege
setup) reject converse_stream() with AccessDeniedException. The agent loop
hard-prefers streaming and the denial never matched the 'stream not
supported' auto-fallback, so InvokeModel-only users looped on AccessDenied
forever.

- agent/bedrock_adapter.py: new is_streaming_access_denied_error()
  detector (ClientError code check + wrapped-SDK message match);
  call_converse_stream() falls back to converse() on denial.
- agent/chat_completion_helpers.py: bedrock_converse streaming branch
  retries inline via converse() and sets _disable_streaming so later
  turns skip the doomed stream attempt; the chat-completions retry
  block also recognizes the denial for the AnthropicBedrock SDK path
  (message pre-check avoids importing bedrock_adapter — and its lazy
  boto3 install — for unrelated providers).

Both paths print a one-line notice telling the user which IAM action
restores streaming.
santhreal pushed a commit to santhreal/hermes-agent that referenced this pull request Jul 13, 2026
…InvokeModelWithResponseStream (NousResearch#44293)

IAM policies scoped to bedrock:InvokeModel only (a common least-privilege
setup) reject converse_stream() with AccessDeniedException. The agent loop
hard-prefers streaming and the denial never matched the 'stream not
supported' auto-fallback, so InvokeModel-only users looped on AccessDenied
forever.

- agent/bedrock_adapter.py: new is_streaming_access_denied_error()
  detector (ClientError code check + wrapped-SDK message match);
  call_converse_stream() falls back to converse() on denial.
- agent/chat_completion_helpers.py: bedrock_converse streaming branch
  retries inline via converse() and sets _disable_streaming so later
  turns skip the doomed stream attempt; the chat-completions retry
  block also recognizes the denial for the AnthropicBedrock SDK path
  (message pre-check avoids importing bedrock_adapter — and its lazy
  boto3 install — for unrelated providers).

Both paths print a one-line notice telling the user which IAM action
restores streaming.
donbowman pushed a commit to donbowman/hermes-agent that referenced this pull request Jul 13, 2026
…InvokeModelWithResponseStream (NousResearch#44293)

IAM policies scoped to bedrock:InvokeModel only (a common least-privilege
setup) reject converse_stream() with AccessDeniedException. The agent loop
hard-prefers streaming and the denial never matched the 'stream not
supported' auto-fallback, so InvokeModel-only users looped on AccessDenied
forever.

- agent/bedrock_adapter.py: new is_streaming_access_denied_error()
  detector (ClientError code check + wrapped-SDK message match);
  call_converse_stream() falls back to converse() on denial.
- agent/chat_completion_helpers.py: bedrock_converse streaming branch
  retries inline via converse() and sets _disable_streaming so later
  turns skip the doomed stream attempt; the chat-completions retry
  block also recognizes the denial for the AnthropicBedrock SDK path
  (message pre-check avoids importing bedrock_adapter — and its lazy
  boto3 install — for unrelated providers).

Both paths print a one-line notice telling the user which IAM action
restores streaming.
Gravezzz pushed a commit to Gravezzz/hermes-agent that referenced this pull request Jul 21, 2026
…InvokeModelWithResponseStream (NousResearch#44293)

IAM policies scoped to bedrock:InvokeModel only (a common least-privilege
setup) reject converse_stream() with AccessDeniedException. The agent loop
hard-prefers streaming and the denial never matched the 'stream not
supported' auto-fallback, so InvokeModel-only users looped on AccessDenied
forever.

- agent/bedrock_adapter.py: new is_streaming_access_denied_error()
  detector (ClientError code check + wrapped-SDK message match);
  call_converse_stream() falls back to converse() on denial.
- agent/chat_completion_helpers.py: bedrock_converse streaming branch
  retries inline via converse() and sets _disable_streaming so later
  turns skip the doomed stream attempt; the chat-completions retry
  block also recognizes the denial for the AnthropicBedrock SDK path
  (message pre-check avoids importing bedrock_adapter — and its lazy
  boto3 install — for unrelated providers).

Both paths print a one-line notice telling the user which IAM action
restores streaming.
leewenjie pushed a commit to leewenjie/hermes-agent that referenced this pull request Aug 7, 2026
…InvokeModelWithResponseStream (NousResearch#44293)

IAM policies scoped to bedrock:InvokeModel only (a common least-privilege
setup) reject converse_stream() with AccessDeniedException. The agent loop
hard-prefers streaming and the denial never matched the 'stream not
supported' auto-fallback, so InvokeModel-only users looped on AccessDenied
forever.

- agent/bedrock_adapter.py: new is_streaming_access_denied_error()
  detector (ClientError code check + wrapped-SDK message match);
  call_converse_stream() falls back to converse() on denial.
- agent/chat_completion_helpers.py: bedrock_converse streaming branch
  retries inline via converse() and sets _disable_streaming so later
  turns skip the doomed stream attempt; the chat-completions retry
  block also recognizes the denial for the AnthropicBedrock SDK path
  (message pre-check avoids importing bedrock_adapter — and its lazy
  boto3 install — for unrelated providers).

Both paths print a one-line notice telling the user which IAM action
restores streaming.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants