fix(memory): flatten multimodal content before provider sync - #44517
fix(memory): flatten multimodal content before provider sync#44517erosika wants to merge 1 commit into
Conversation
Multimodal turns carry message content as a list of typed parts
({type: "text"|"image_url", ...}). _sync_external_memory_for_turn
passed that list straight into MemoryManager.sync_all, and providers
feed it to regexes — Honcho's sync_turn calls sanitize_context, where
re.sub raised 'expected string or bytes-like object, got list'. Every
turn with an attached image silently never synced.
Flatten to plain text at the boundary: text parts joined, images noted
as an [N image(s)] marker so the attachment isn't erased from recall.
Fixing here covers all providers instead of patching each plugin.
|
Verified
Test suite covers all branches including the original crash case ( |
|
Thanks for this @erosika — the diagnosis is exactly right: multimodal content reaches the provider sync as a list and crashes I've opened #44738 as a salvage that cherry-picks your commit verbatim (authorship preserved) and adds one follow-up: the new Closing this in favor of #44738 — full credit to you for the fix. Happy to fold in any feedback there. |
Problem
Turns with an attached image log this and silently never reach the memory backend:
Multimodal user messages carry content as a list of typed parts (
{type: "text"|"image_url", ...})._sync_external_memory_for_turnpassed that list straight intoMemoryManager.sync_all, and providers feed it to regexes — Honcho'ssync_turncallssanitize_contextin its synchronous prologue, wherere.subraises theTypeErrorbefore the sync thread ever spawns.The chain:
agent/turn_context.py—original_user_messageis the raw content (Any), a list for multimodal turnsagent/turn_finalizer.py→run_agent.py: _sync_external_memory_for_turn— no string guard (other consumers oforiginal_user_messageinturn_context.pydo guard withisinstance(..., str))plugins/memory/honcho/__init__.py: sync_turn—sanitize_context(user_content or ""): a non-empty list is truthy, so the list hitsre.subOther plugins (mem0, hindsight, supermemory, …) would choke on a list the same way.
Fix
Flatten content to plain text at the boundary in
_sync_external_memory_for_turn, beforesync_all/queue_prefetch_all— one fix covers every provider instead of patching each plugin. New helperflatten_message_contentinagent/memory_manager.py(alongsidesanitize_context, its main consumer):None→""[N image(s)]marker so the attachment isn't erased from recallBoth
original_user_messageandfinal_responseare guarded — both are typedAny.Tests
tests/agent/test_memory_provider.py— unit coverage forflatten_message_content, including a regression case asserting the flattened output survivessanitize_contexttests/run_agent/test_memory_sync_interrupted.py— boundary coverage: multimodal user message, multimodal response, and no-recoverable-text skipAll 141 tests across the touched suites pass.