Skip to content

fix(bedrock): decode base64 data URL to bytes for Converse image source - #33536

Open
liuhao1024 wants to merge 2 commits into
NousResearch:mainfrom
liuhao1024:fix/bedrock-data-url-double-encoding
Open

fix(bedrock): decode base64 data URL to bytes for Converse image source#33536
liuhao1024 wants to merge 2 commits into
NousResearch:mainfrom
liuhao1024:fix/bedrock-data-url-double-encoding

Conversation

@liuhao1024

@liuhao1024 liuhao1024 commented May 27, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?

_convert_content_to_converse() in agent/bedrock_adapter.py passes the base64-encoded ASCII string from a data URL directly as ImageBlock.source.bytes. Bedrock's Converse API expects raw bytes — boto3 base64-encodes them at the wire layer. The result is double-encoding, and Bedrock rejects every image with:

ValidationException: Failed to sanitize image: Invalid or unsupported image format

Root Cause

Line 482 of agent/bedrock_adapter.py:

"source": {"bytes": data},   # BUG: 'data' is still base64-encoded string

data is the portion after the comma in data:image/png;base64,iVBORw0KGgo... — a base64 ASCII string. It needs to be decoded to raw bytes first.

Related Issue

Fixes #33317

Type of Change

  • 🐛 Bug fix (non-breaking change that fixes an issue)

Changes Made

  • See commit messages for detailed changes

How to Test

  1. Run pytest tests/ -q — all tests should pass
  2. Verify the specific scenario described above is resolved

Checklist

Code

  • I've read the Contributing Guide
  • My commit messages follow Conventional Commits (fix(scope):, feat(scope):, etc.)
  • I searched for existing PRs to make sure this isn't a duplicate
  • My PR contains only changes related to this fix/feature (no unrelated commits)
  • I've run pytest tests/ -q and all tests pass
  • I've added tests for my changes (required for bug fixes, strongly encouraged for features)
  • I've tested on my platform: macOS 26.4.1

Documentation & Housekeeping

  • I've updated relevant documentation (README, docs/, docstrings) — or N/A
  • I've updated cli-config.yaml.example if I added/changed config keys — or N/A
  • I've updated CONTRIBUTING.md or AGENTS.md if I changed architecture and workflows — or N/A
  • I've considered cross-platform impact (Windows, macOS) per the compatibility guide — or N/A

Code Intelligence

  • Analyzed: agent/bedrock_adapter.py:_convert_content_to_converse (called from convert_messages_to_converse, used by Bedrock transport for all multimodal messages)
  • Blast radius: LOW — single function, single code path (data URL images only), no impact on remote URL or text content
  • Related patterns: other adapters (anthropic, openai) handle data URLs differently — anthropic uses base64.b64decode() (correct), openai passes string (correct for its API). Bedrock is the only adapter that needs raw bytes.

Fixes #33317

The _convert_content_to_converse() function passed the base64-encoded
ASCII string directly as ImageBlock.source.bytes. Bedrock's Converse API
expects raw bytes (boto3 base64-encodes on the wire), so the image was
double-encoded and rejected with 'Invalid or unsupported image format'.

Fix: base64.b64decode(data) before passing to source.bytes.

Fixes NousResearch#33317
@alt-glitch alt-glitch added type/bug Something isn't working P3 Low — cosmetic, nice to have comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint labels May 28, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

Duplicate fix — PR #28085 already addresses the same bedrock image double-encoding bug (base64 string passed as source.bytes instead of decoded raw bytes). See also issue #33317 which was triaged as a duplicate of #28085.

@JiaDe-Wu

JiaDe-Wu commented Jun 1, 2026

Copy link
Copy Markdown
Contributor

Solid work. We hit the same double-encoding issue and fixed it in #34742. The root cause is identical — base64 string passed to source.bytes instead of decoded raw bytes. boto3 re-encodes at the wire layer. Your fix and ours are the same two-line change. Whichever lands first should close the other.

@liuhao1024

Copy link
Copy Markdown
Contributor Author

@alt-glitch @JiaDe-Wu Thanks for the context. I see #28085 and #34742 both address the same base64 double-encoding issue. Since all three PRs (#28085, #34742, #33536) are still open, I'll wait for the maintainer to decide which approach to merge. If either #28085 or #34742 merges first, I'm happy to close this PR as a duplicate.

@alt-glitch alt-glitch added provider/bedrock AWS Bedrock (boto3, IAM) duplicate This issue or pull request already exists labels Jun 30, 2026

@teknium1 teknium1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the focused Bedrock regression fix. The premise is confirmed on current main: agent/bedrock_adapter.py:525 extracts the data-URL payload and agent/bedrock_adapter.py:534 currently forwards that base64 string as image.source.bytes.

Problems

  • The proposed decode at agent/bedrock_adapter.py:535 is unguarded and non-strict. The converter accepts any data: URL, so malformed payloads may either be silently converted to junk bytes or raise while building the request. The related open implementation in #28085 uses base64.b64decode(..., validate=True), catches decode errors, and skips empty/malformed image blocks.

Suggested changes

  • Use strict decoding, catch binascii.Error/ValueError, and preserve adjacent text when discarding an invalid image block.
  • Add a malformed-data-URL regression case alongside the existing conversion test in tests/agent/test_bedrock_adapter.py:332.

This is an automated hermes-sweeper review.

Comment thread agent/bedrock_adapter.py
"image": {
"format": media_type.split("/")[-1] if "/" in media_type else "jpeg",
"source": {"bytes": data},
"source": {"bytes": base64.b64decode(data)},

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please use strict decoding and handle malformed payloads here. This branch accepts every data: URL, while plain b64decode() may accept junk characters or raise for invalid padding. Use validate=True, catch decode errors, and skip empty/malformed image blocks so an invalid attachment does not fail conversion of the whole message.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint duplicate This issue or pull request already exists P3 Low — cosmetic, nice to have provider/bedrock AWS Bedrock (boto3, IAM) sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bedrock_adapter: image data URLs sent as base64 string instead of raw bytes — all image uploads rejected by Bedrock 'Failed to sanitize image'

4 participants