fix(moa): preserve prompt text of content-block messages in reference view - #61426
fix(moa): preserve prompt text of content-block messages in reference view#61426jrfbch wants to merge 1 commit into
Conversation
… view `_reference_messages` flattened every message's `content` with `content if isinstance(content, str) else ""`, so a user turn whose content is a list of blocks (the OpenAI/Anthropic multimodal shape produced whenever an image/file is attached, or any multi-part turn) had its text silently dropped to an empty string. Lax providers (deepseek, MiniMax) tolerated the resulting empty turn but the reference still lost the actual instruction; strict providers (Z.AI GLM) reject it outright with HTTP 400. Either way this contradicts the function's own documented contract that 'no context is lost' and the reference 'still has the full picture'. Add `_flatten_content_text()` which extracts and joins the text blocks and drops non-text blocks (images, etc.), handling str / list-of-blocks / None. Apply it at the main render path and the degenerate-case fallback (same bug class). Mirrors the text-extraction pattern already used in context_compressor and the provider adapters. Adds regression tests for the content-block flattening and the helper's shapes.
teknium1
left a comment
There was a problem hiding this comment.
Thanks for tracing this to the advisory transcript. The premise is confirmed on current main: agent/moa_loop.py:473 drops all non-string content before agent/moa_loop.py:480 emits the user turn.
Problems
- The new helper duplicates
agent.message_content.flatten_message_text(agent/message_content.py:34-49), which already supports the common Chat/Responses text fields and object parts while excluding media blocks. Reusing it avoids two normalizers drifting apart. - The new image-only test locks in an empty result. That still reaches
_reference_messagesas an empty user turn (agent/moa_loop.py:477-480), so the strict-reference empty-prompt case remains for image-only requests.
Suggested changes
- Use
flatten_message_textin both extraction paths and keep the MoA regression coverage. - Make the image-only advisory representation non-empty (or omit the empty turn) and test that strict-safe transcript shape.
Automated hermes-sweeper review.
| return "\n".join(lines) | ||
|
|
||
|
|
||
| def _flatten_content_text(content: Any) -> str: |
There was a problem hiding this comment.
Please reuse agent.message_content.flatten_message_text here rather than adding a second content normalizer. The existing helper already covers Chat/Responses text fields, mapping/object parts, and media filtering; this narrower copy will drift from those supported shapes.
| parts.append(text) | ||
| elif isinstance(block, str) and block: | ||
| parts.append(block) | ||
| return "\n".join(parts) |
There was a problem hiding this comment.
For an image-only user message this returns "", and _reference_messages will still send an empty user turn to the reference. That preserves the strict-provider empty-prompt failure described by this PR for that valid input shape; use a non-empty advisory placeholder or omit the empty turn.
|
Thanks @jrfbch — you had the correct root-cause diagnosis and fix direction 5 days before anyone else, and your analysis (str-only read flattening content-block turns to "", strict providers 400ing while lax ones silently lose the instruction) was exactly right. This landed on main via #64319 (commit 8582f35), which carries you as Co-authored-by. That PR covers the same Closing as landed-with-credit. Appreciate the contribution. |
Problem
_reference_messages(agent/moa_loop.py) builds the advisory view MoA sends toeach reference model. It flattens every message's
contentwith:When a user turn's
contentis a list of content blocks — the standardOpenAI/Anthropic multimodal shape produced whenever an image/file is attached,
or any multi-part turn — this drops the prompt text to an empty string.
Consequences:
still loses the actual instruction, silently degrading its advice.
400 ... 未正常接收到prompt参数("prompt param not received"), so thatreference fails every turn.
Either way this contradicts the function's own documented contract that
"no context is lost" and the reference "still has the full picture".
Fix
Add
_flatten_content_text()which extracts and joins thetextof each blockand drops non-text blocks (images, etc.), handling
str/ list-of-blocks /None. Apply it at the main render path and at the degenerate-case fallback(same bug class). This mirrors the text-extraction pattern already used in
context_compressorand the provider adapters.Tests
Adds two regression tests to
tests/run_agent/test_moa_loop_mode.py:test_reference_messages_flattens_content_blocks_to_text— a content-blockuser turn preserves its text and drops the image block.
test_flatten_content_text_shapes— helper handles str / block list /image-only / None.
Full
tests/run_agent/test_moa_loop_mode.pypasses (29/29) against currentmain.Repro
With a MoA preset using a Z.AI GLM reference model, any first-turn CLI query
(
hermes chat -q '...', whose content is content-block form) makes the GLMreference fail with
400 未正常接收到prompt参数, while deepseek/MiniMaxreferences "succeed" on a blanked prompt. After the fix all references receive
the real prompt text.