fix: handle multimodal content in interim text and avoid retrying local processing errors

[nanami7777777]

Fixes #66267

Problem

After a vision turn or context compaction, the next assistant message could trigger a nearly-infinite retry loop that ends with:

Error during OpenAI-compatible API call #59: expected string or bytes-like object, got 'list'

The API call itself succeeded; the crash happened during local post-processing of the previous assistant message when assistant_msg["content"] was a multimodal parts list instead of a string. Because the exception fired before messages.append(assistant_msg), history was unchanged and every retry re-entered the same crash.

Changes

1. run_agent.py: In _interim_assistant_visible_text, flatten content with flatten_message_text before passing it to _strip_think_blocks. This handles both string and structured parts list content.

2. agent/agent_runtime_helpers.py: Harden strip_think_blocks at the entry point so it coerces list/dict/None content to a string, dropping non-text multimodal parts before running regexes. This protects all call sites of the helper.

3. agent/conversation_loop.py: Classify errors in the outer loop based on traceback module provenance. Deterministic local processing errors (traceback passes through local post-processing modules and never enters chat_completion_helpers) now stop immediately instead of retrying until the iteration budget is exhausted. Error messages distinguish between API-call failures and local processing failures.

4. agent/chat_completion_helpers.py (second call site): In build_assistant_message, the reasoning-extraction fallback runs flatten_message_text() on content before the inline <think> regex and the surrogate sanitizer. This is the non-streaming -z / gateway path (build_assistant_message) that hit the same TypeError with list content at "API call fix(cli): show correct config file path in /config command #89" (reported by @bemany). Using flatten_message_text (rather than skipping the regex for non-str content) preserves the visible text from list parts instead of producing an empty string.

Why no early role == "assistant" guard?

A reviewer suggested moving the existing role == "assistant" guard before the text computation. It is intentionally not added: once flatten_message_text normalizes at the helper entry (strip_think_blocks), the pre-role call can no longer raise, and tool messages now return a safe string and are excluded by the existing dedup condition (role == "assistant" and finish_reason == "incomplete"). An earlier guard would be redundant defensive duplication. Open to adding it if maintainers prefer belt-and-suspenders.

Related work

• PR fix(agent): tolerate content-parts lists in interim visible text #66235 normalizes content inside _interim_assistant_visible_text.
• PR fix(agent): stop strip_think_blocks from crashing on multimodal content #66218 coerces content inside strip_think_blocks.

This PR addresses the same root cause but takes a slightly broader approach: it reuses the existing flatten_message_text helper at the call site, hardens strip_think_blocks against non-string input, prevents deterministic local post-processing errors from being retried until the iteration budget is exhausted, and additionally normalizes the build_assistant_message (non-streaming) call site flagged in the issue thread.

Testing

• tests/agent/test_think_scrubber.py — passed
• tests/run_agent/test_run_agent.py — passed
• tests/run_agent/test_66267_multimodal_interim.py — new, pins the regression:
  • build_assistant_message with list content does not raise TypeError
  • inline <think> inside list content is extracted + stripped correctly
  • _interim_assistant_visible_text is safe for tool messages (list content)
  • duplicate_previous_interim dedup guards against tool messages
  • Verified the tests fail without the fix (TypeError: expected string... got 'list') and pass with it.
• Linter checks on all modified files — passed

[alt-glitch]

This was generated by AI during triage.

Related: #66218, #66235, and #66275 repair the same multimodal-content crash at shared-helper and call-site boundaries. This PR additionally changes retry classification; please consolidate the overlapping scope and remove the unrelated documentation tag-text edits.

[tonydwb]

Code Review Summary

Verdict: Approved

Clean, well-scoped change with good test coverage. No security concerns.

---

Reviewed by Hermes Agent

[kshitijk4poor]

Merged via #66945 — your commits cherry-picked with authorship preserved (rebase-merge).

Your fix addressed the same root cause (#66267) with a broader approach: flatten_message_text at the call sites, hardening in strip_think_blocks, and a traceback-based error classifier to stop retrying deterministic local processing errors. The salvage additionally fixed the error classifier's module set (removed conversation_loop and run_agent which were making _hit_local always true), restored a lost None guard, and fixed docstring corruption.

Thanks for the thorough fix and the error classifier concept!