Skip to content

fix: handle multimodal content in think scrubber - #67372

Closed
rafasouzaramos-a11y wants to merge 1 commit into
NousResearch:mainfrom
rafasouzaramos-a11y:fix/multimodal-strip-think-blocks
Closed

fix: handle multimodal content in think scrubber#67372
rafasouzaramos-a11y wants to merge 1 commit into
NousResearch:mainfrom
rafasouzaramos-a11y:fix/multimodal-strip-think-blocks

Conversation

@rafasouzaramos-a11y

Copy link
Copy Markdown

Summary

  • normalize OpenAI-style multimodal content arrays to visible text before regex scrubbing
  • ignore image/non-text blocks in the display-text helper
  • cover text/input_text/output_text, string parts, image-only content, and think tags

Reproduction

A Telegram user message containing native image content is stored as a list. _interim_assistant_visible_text() passes that list to strip_think_blocks(), which called re.sub() directly and raised TypeError: expected string or bytes-like object, got list. The outer conversation loop retried repeatedly.

Verification

  • pytest tests/run_agent/test_run_agent.py::TestStripThinkBlocks -q → 30 passed
  • ruff check agent/agent_runtime_helpers.py tests/run_agent/test_run_agent.py → passed
  • live Telegram image → follow-up tool call completed with no new TypeError

@alt-glitch alt-glitch added type/bug Something isn't working comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P2 Medium — degraded but workaround exists labels Jul 19, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

This was generated by AI during triage.

Related to closed-unmerged #66341, which included an interim-text coercion alongside separate no-retry and documentation work. This focused live helper repair remains an active implementation; #26504 is a distinct API-server content-normalization path.

@teknium1

Copy link
Copy Markdown
Contributor

Thanks for the focused reproduction and coverage.

Automated hermes-sweeper review found this behavior is already implemented on current main:

  • agent/agent_runtime_helpers.py:685-707 normalizes list and dict content before the regex scrubber; list handling excludes thinking/reasoning blocks and preserves visible text.
  • run_agent.py:4819-4831 routes interim stored-message content through that shared helper.
  • tests/run_agent/test_run_agent.py:448-480 already covers block-list, dict, thinking-only, and empty-list content.
  • Commit 296494db0ee99f1cd9e384b2083e7a60aeb833ef (fix: stop infinite loop when assistant content is a block list) is an ancestor of current main; the related broader repair was also salvaged in merged PR fix: handle multimodal content in interim text and avoid retrying local processing errors #66945.

Closing as implemented on main.

@teknium1 teknium1 closed this Jul 19, 2026
@teknium1 teknium1 added the sweeper:implemented-on-main Sweeper: behavior already present on current main label Jul 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P2 Medium — degraded but workaround exists sweeper:implemented-on-main Sweeper: behavior already present on current main type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants