fix: handle multimodal content in context compression summarization - #917
Closed
teknium1 wants to merge 1 commit into
Closed
fix: handle multimodal content in context compression summarization#917teknium1 wants to merge 1 commit into
teknium1 wants to merge 1 commit into
Conversation
The _generate_summary() method assumed message content is always a
string (msg.get('content') or ''). When content is a multimodal list
(e.g. [{type: 'text', text: '...'}, {type: 'image_url', ...}]), this
produced mangled output: len() returned the list length instead of
character count, and slicing produced list items instead of substrings.
Add _content_to_text() helper that safely converts any content format
to plain text:
- str → returned as-is
- None → empty string
- list (multimodal) → text parts joined, images replaced with [image]
- dict/other → JSON serialization with str() fallback
This ensures multimodal conversations compress correctly instead of
producing garbled summaries.
Inspired by PR #776 by @kshitijk4poor.
Contributor
Author
|
Thanks for the PR. I looked into this, but it doesn’t correspond to a real Hermes message-flow path today. We don’t keep raw multimodal image blocks in the conversation history that context compression summarizes; image handling goes through the vision tool, and the resulting text description is what gets stored in chat history. Because of that, this edge case isn’t currently actionable in the live architecture, so I’m closing this for now. If we later add a path that stores raw multimodal content directly in chat history, we can revisit it then. |
kiddhu
added a commit
to kiddhu/hermes-agent
that referenced
this pull request
Aug 21, 2026
fix(kanban): pin PR NousResearch#917 proof-module authority / 固定 PR NousResearch#917 证明模块权威
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes a bug where multimodal user messages (containing images, audio, etc.) would produce garbled output during context compression summarization.
The bug
In
_generate_summary(), message content was accessed as:When
contentis a multimodal list like:[ {"type": "text", "text": "What is in this image?"}, {"type": "image_url", "image_url": {"url": "data:image/png;base64,..."}}, ]The `or """ fallback doesn't trigger (non-empty lists are truthy), and then:
len(content)returns 2 (list length) instead of character countcontent[:1000]slices list items instead of charactersThe fix
New
_content_to_text()static method onContextCompressorSafely converts any content format to plain text:
strNonelist(multimodal)image_url→[image]; other types →[type_name]dict/ otherstr()fallbackChanged line in
_generate_summary()Tests added
TestContentToTextclass with 6 test cases:_generate_summary()— verifies the summarization prompt contains"What is in this image?"and"[image]"instead of raw dictsWhat does NOT change
content=Nonehandling that already worked via the existingTestGenerateSummaryNoneContenttestsInspired by PR #776 by @kshitijk4poor.