fix(memory): flatten multimodal content before provider sync - #44738
Merged
kshitijk4poor merged 2 commits intoJun 12, 2026
Merged
Conversation
Multimodal turns carry message content as a list of typed parts
({type: "text"|"image_url", ...}). _sync_external_memory_for_turn
passed that list straight into MemoryManager.sync_all, and providers
feed it to regexes — Honcho's sync_turn calls sanitize_context, where
re.sub raised 'expected string or bytes-like object, got list'. Every
turn with an attached image silently never synced.
Flatten to plain text at the boundary: text parts joined, images noted
as an [N image(s)] marker so the attachment isn't erased from recall.
Fixing here covers all providers instead of patching each plugin.
(cherry picked from commit 705bdb6)
…rking it
The original fix added agent/memory_manager.py:flatten_message_content, but
that helper was a near-exact duplicate of
agent/codex_responses_adapter.py:_summarize_user_message_for_log — same
None/str/list dispatch, same {text,input_text,output_text}/{image_url,input_image}
part sets, the identical [N image(s)] marker, and the same str() fallback. The
only difference was the join separator (newline for memory vs space for the
log/trajectory previews the existing helper already serves), and that helper is
already imported into agent/turn_finalizer.py — the same file whose call site the
memory fix touches.
Parameterize the existing helper with sep=' ' (default preserves every current
logging/trajectory caller byte-for-byte) and call it with sep='\n' at the memory
boundary; drop the forked flatten_message_content. Repoints the unit tests to the
consolidated helper and adds a case locking the default space-join.
Single source of truth for multimodal-content flattening; no behavior change for
the fix or for existing callers.
AIalliAI
pushed a commit
to AIalliAI/Hermes
that referenced
this pull request
Jun 14, 2026
…ry-sync-multimodal-content fix(memory): flatten multimodal content before provider sync
T02200059
pushed a commit
to T02200059/hermes-agent
that referenced
this pull request
Jun 18, 2026
…ry-sync-multimodal-content fix(memory): flatten multimodal content before provider sync
waefrebeorn
pushed a commit
to waefrebeorn/slermes
that referenced
this pull request
Jul 2, 2026
…ry-sync-multimodal-content fix(memory): flatten multimodal content before provider sync
habarmc1223-sudo
pushed a commit
to habarmc1223-sudo/hermes-agent-fluxmem
that referenced
this pull request
Jul 8, 2026
…ry-sync-multimodal-content fix(memory): flatten multimodal content before provider sync
santhreal
pushed a commit
to santhreal/hermes-agent
that referenced
this pull request
Jul 13, 2026
…ry-sync-multimodal-content fix(memory): flatten multimodal content before provider sync
mgcstudios86
pushed a commit
to mgcstudios/hermes-agent
that referenced
this pull request
Jul 20, 2026
Per Bryntly's review (PR NousResearch#68072): - Use `Any` type hint instead of bare `text` (preserves type info) - Replace broad `except Exception` with `logger.warning(...)` so failures are observable instead of silently swallowed - Add 8 unit tests covering: string passthrough, empty, None, list of text parts, list with fence tags, list with image, scalar fallback, and a direct guard against the original TypeError-on-list regression Per alt-glitch comment (PR NousResearch#68072): - Docstring now references PR NousResearch#44738 (boundary normalization already merged in upstream) and frames this PR as a defensive guard for callers that forget the boundary normalization
Gravezzz
pushed a commit
to Gravezzz/hermes-agent
that referenced
this pull request
Jul 21, 2026
…ry-sync-multimodal-content fix(memory): flatten multimodal content before provider sync
leewenjie
pushed a commit
to leewenjie/hermes-agent
that referenced
this pull request
Aug 7, 2026
…ry-sync-multimodal-content fix(memory): flatten multimodal content before provider sync
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Salvage of #44517 by @erosika — original fix cherry-picked verbatim (authorship preserved), plus one follow-up commit that dedupes the helper. Based on current
main.Problem (commit 1, @erosika)
Turns with an attached image log this and silently never reach the memory backend:
Multimodal user messages carry content as a list of typed parts (
{type: "text"|"image_url", ...})._sync_external_memory_for_turnpassed that list straight intoMemoryManager.sync_all, and providers feed it to regexes — Honcho'ssync_turncallssanitize_contextin its synchronous prologue, wherere.subraises theTypeErrorbefore the sync thread spawns (caught bysync_all, surfaced as the warning above). mem0, supermemory, and the other providers expect a string the same way.The fix flattens content to plain text at the boundary in
_sync_external_memory_for_turn, beforesync_all/queue_prefetch_all— one fix covers every provider instead of patching each plugin:None→""[N image(s)]marker so the attachment isn't erased from recallBoth
original_user_messageandfinal_responseare guarded — both are typedAny.Follow-up (commit 2): reuse the existing helper instead of forking it
The original commit added
flatten_message_contenttoagent/memory_manager.py, but it was a near-exact duplicate ofagent/codex_responses_adapter.py:_summarize_user_message_for_log— sameNone/str/listdispatch, same{text,input_text,output_text}/{image_url,input_image}part sets, the identical[N image(s)]marker, and the same scalar fallback. The only difference was the join separator (newline for memory vs the space-join the existing helper already uses for log/trajectory previews), and that helper is already imported intoagent/turn_finalizer.py— the same file whose call site the memory fix touches.This commit parameterizes the existing helper with
sep=" "(default preserves every current logging/trajectory caller byte-for-byte) and calls it withsep="\n"at the memory boundary; the forkedflatten_message_contentis dropped. Single source of truth for multimodal-content flattening, no behavior change for the fix or for existing callers.Tests
tests/agent/test_memory_provider.py— unit coverage for the consolidated helper (newline-join for memory, plus a case locking the default space-join), including a regression case asserting the flattened output survivessanitize_contexttests/run_agent/test_memory_sync_interrupted.py— boundary coverage: multimodal user message, multimodal response, and no-recoverable-text skipAll touched suites pass locally (110 tests), ruff clean. Verified end-to-end on the branch: a multimodal list flattens to
[1 image] <text>and survivessanitize_context; a raw list still crashes it (the bug); logging callers keep the space-join.Salvage of #44517.