Skip to content

fix(memory_manager): sanitize_context handles multimodal list content - #68072

Open
mgcstudios86 wants to merge 2 commits into
NousResearch:mainfrom
mgcstudios:fix/sanitize-context-multimodal-list
Open

fix(memory_manager): sanitize_context handles multimodal list content#68072
mgcstudios86 wants to merge 2 commits into
NousResearch:mainfrom
mgcstudios:fix/sanitize-context-multimodal-list

Conversation

@mgcstudios86

Copy link
Copy Markdown

Bug

sanitize_context(text) in agent/memory_manager.py does text = re.sub(...) which expects a string.

When called with multimodal content (a list of typed parts, e.g. [{type: text, text: ...}, {type: image_url, ...}]), it raises:

TypeError: expected string or bytes-like object, got 'list'

This bug kills long-running QA workers after ~89 API calls when the memory provider passes back multimodal content. The model itself is fine — the error is in the post-call handler.

Fix

Detect non-string input and flatten via _summarize_user_message_for_log before regex sanitization. The test class TestFlattenMessageContent already exists — the implementation just wasn't using it.

def sanitize_context(text) -> str:
    if not isinstance(text, str):
        from agent.codex_responses_adapter import _summarize_user_message_for_log
        text = _summarize_user_message_for_log(text, sep="\n")
    text = _INTERNAL_CONTEXT_RE.sub('', text)
    text = _INTERNAL_NOTE_RE.sub('', text)
    text = _FENCE_TAG_RE.sub('', text)
    return text

Tests

All 10 tests in TestFlattenMessageContent pass.

Repro

Run a long-running agent with multimodal content + memory provider enabled. After ~89 API calls the worker dies with the exact error above.

Reported by

@matiasgonzalocalvo (mgcstudios user)

…tent

The sanitize_context() function used re.sub which expects a string.
When called with multimodal content (a list of typed parts), it raised
TypeError: expected string or bytes-like object, got 'list'.

This bug killed long-running QA workers after ~89 API calls when the
memory provider passed back multimodal content.

The fix: detect non-string input and flatten via
_summarize_user_message_for_log before regex sanitization.

All 10 tests in TestFlattenMessageContent pass.
@alt-glitch alt-glitch added type/bug Something isn't working comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint tool/memory Memory tool and memory providers area/memory Memory subsystem: store, providers, sync, background reviews P3 Low — cosmetic, nice to have sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state labels Jul 20, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

This was generated by AI during triage.

Related: merged #44738 normalizes multimodal content at the external-memory sync boundary and covers the reported provider-sync path. This patch adds a defensive guard directly in sanitize_context().

@Bryntly Bryntly left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. The logic correctly handles multimodal list contents avoiding a regex TypeError crash.

A couple of suggestions for improvement:

  1. Since type hints are used throughout the file, changing text: str to just text loses typing information. Consider using text: Any or text: str | list (since Any is already imported).
  2. The nested except Exception: blocks are quite broad and silently swallow errors (like ImportError or potential bugs in _summarize_user_message_for_log). It might be better to catch specific exceptions or log a warning when falling back to str(text).
  3. Consider adding a quick unit test for sanitize_context in tests/agent/test_memory_provider.py to verify it correctly processes a list input to prevent future regressions.

Thanks for fixing this!

Per Bryntly's review (PR NousResearch#68072):
- Use `Any` type hint instead of bare `text` (preserves type info)
- Replace broad `except Exception` with `logger.warning(...)` so
  failures are observable instead of silently swallowed
- Add 8 unit tests covering: string passthrough, empty, None,
  list of text parts, list with fence tags, list with image,
  scalar fallback, and a direct guard against the original
  TypeError-on-list regression

Per alt-glitch comment (PR NousResearch#68072):
- Docstring now references PR NousResearch#44738 (boundary normalization
  already merged in upstream) and frames this PR as a defensive
  guard for callers that forget the boundary normalization
@mgcstudios86

Copy link
Copy Markdown
Author

Thanks for the review @Bryntly — addressed all three suggestions.

What changed

1. Type hinttexttext: Any

def sanitize_context(text: Any) -> str:

2. Replaced broad except Exception with logger.warning(...)

  • ImportError (helper not available) → log + fall back to str()
  • generic Exception from the flatten helper → log + exc_info=True + fall back to str()
  • generic Exception from str() coercion → log + return ""
  • All fallback paths are now observable instead of silently swallowed.

3. Added 8 unit tests (tests/agent/test_memory_provider.py::TestSanitizeContextMultimodalList):

  • test_string_passthrough — fence-tag strip still works
  • test_empty_string / test_none_returns_empty
  • test_list_of_text_parts — the original bug
  • test_list_with_fence_tags — flatten happens before regex
  • test_list_with_image_part — multimodal image parts
  • test_scalar_fallback — int / bool coerces via str()
  • test_does_not_call_re_sub_on_list — direct guard against the regression

All 109 tests in test_memory_provider.py pass.

On @alt-glitch's note

Good catch on #44738 — the boundary normalization there is the right primary fix. Updated the docstring to call this out:

PR #44738 normalizes content at the external-memory sync boundary, so this
code path is normally not reached for that case; this function is the last
line of defense if a caller forgets the boundary normalization.

If you'd prefer, I can:

Happy to go with whichever you prefer.

@teknium1

Copy link
Copy Markdown
Contributor

Thanks for adding defensive coverage around multimodal content.

Problems

  • The external-memory sync path described in the report is already normalized on current main: run_agent.py:3838-3839 flattens both message values before sync_all() at run_agent.py:3846. That behavior landed in merged fix(memory): flatten multimodal content before provider sync #44738 (046f444ddc5b7fd1479e503e0c87f6690c0d5277). The new direct sanitize_context(list) tests therefore do not reproduce a remaining provider-sync failure.

Suggested changes

  • Please identify a current direct caller that can legitimately provide typed content parts to sanitize_context(), and cover that path; otherwise this is an optional broader defensive contract rather than a fix for the reported provider-sync regression.

Automated hermes-sweeper review.

@teknium1 teknium1 added the sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform label Jul 30, 2026
@alt-glitch alt-glitch added the needs-repro Bug needs reproduction steps label Aug 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/memory Memory subsystem: store, providers, sync, background reviews comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint needs-repro Bug needs reproduction steps P3 Low — cosmetic, nice to have sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state tool/memory Memory tool and memory providers type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants