fix(agent): harden structured message text projection

[Mikoto9901]

What

Follow-up to #66267 / #66945. Those fixes stopped the immediate crashes from
multimodal list content, but three gaps remain reproducible on current main:

1. Split reasoning tags break the scrubber. flatten_message_text joins
   parts with "\n" by default. When a provider splits <think> across two
   parts ("<thi" + "nk>secret</think>visible"), the reassembled string
   contains a newline the original never had, the <think>...</think> regex
   no longer matches, and reasoning leaks into the visible reply.

2. Unknown shapes leak into visible text. Untyped Mappings are read for
   text/content regardless of extra fields
   ({"provider": "x", "content": "SECRET"} → "SECRET";
   [{"tool_call_id": "abc", "content": "SECRET"}] → "SECRET"); objects of
   unknown type contribute attribute values; top-level str(content)
   stringifies unknown objects; Mappings whose accessors raise propagate the
   exception instead of yielding "".

3. Interim dedup has no role gate. The assistant-visible-text extractor
   runs on any dict message — user, tool, system — before checking whether it
   is an assistant turn at all.

There are also multiple slightly different list-flattening implementations
across call sites with divergent contracts.

How

• agent/message_content.py becomes the single canonical projector with a
  strict allowlist: plain str; typed text parts
  (text/input_text/output_text); explicit summary_text; and pure
  legacy wrappers (untyped Mappings whose keys are a subset of
  {text, content}). Everything else — unknown typed parts, images, base64,
  audio, tool metadata, encrypted/redacted reasoning, unknown objects,
  hostile Mappings — yields "".
• No unrestricted text/content reads from untyped unknown objects. Only
  explicitly allowlisted shapes contribute text.
• SDK-style provider objects are read only after their type is an
  allowlisted textual type (.type ∈ text/input_text/output_text);
  non-textual or unknown types are never read.
• No str()/repr() fallback anywhere in the projection path; Mappings
  whose accessors raise yield "" without propagating.
• The scrub pipeline (strip_think_blocks, build_assistant_message,
  interim visible text, refusal/length normalization) joins parts with
  sep="" so provider-split tags reassemble byte-exactly before scrubbing.
• Interim dedup gates on role == "assistant" before text extraction.
• Structured reasoning is captured before projecting assistant content so
  thinking blocks are not silently dropped.

Tests

• New tests/run_agent/test_structured_content_projection.py: typed parts,
  multi-part order, empty text, non-string text fields, unknown part types,
  unknown objects (.text/.content/custom __str__/nested), image/base64/
  reasoning/metadata non-leakage, split <think> reassembly, tool-role
  guard, refusal/length normalization, pure legacy wrappers, and hostile
  Mappings (raising get/keys/iteration, top-level and in-list, never
  propagating).
• Red→green on main: the leakage/hostile/role cases fail before this change.
• Existing test_66267_multimodal_interim.py, test_message_content.py,
  think/streaming-context scrubber suites and related Codex/Kimi/DeepSeek
  suites continue to pass.

No behavior change for plain-string content; no prompt, config, or transport
changes.

Compatibility contract

Preserved:

• plain string content
• typed text / input_text / output_text parts
• explicit summary_text
• pure untyped legacy wrappers whose keys are limited to text/content

Intentionally rejected:

• untyped mappings carrying provider, tool, metadata, or unknown fields
• unknown SDK object types
• images, audio, base64, files, and tool-result shapes
• encrypted or redacted reasoning
• arbitrary str()/repr() fallback
• mappings whose accessors raise

This is a visible-text projection contract, not a general message
serialization or history-schema migration.

Related defense: #67411

#67380 removes the concrete structured-content projection failure.

#67411 independently prevents a provider request from being reissued when
a Hermes-local processing failure occurs after a terminal provider response.

The PRs share no commits, are independently mergeable, and may land in
either order.

Merge-order matrix:

Order	Outcome
#67380 first	The concrete projector failure is removed; the generic post-terminal lifecycle defense waits for #67411.
#67411 first	The concrete failure becomes one local_post_response_error without a repeated provider request; #67380 later removes the root cause.
Both	The concrete content-shape failure is removed AND the generic post-terminal lifecycle boundary is enforced.

Current-main validation

Rebased onto e702a45b5 (current main) and semantically revalidated:

• clean current main still reproduces the residual gaps (22 RED failures:
  unknown-object/hostile-mapping leaks, split <think> scrub miss,
  non-assistant interim extraction, structured reasoning loss);
• typed visible-text parts remain supported; unknown / metadata / tool /
  media / hostile shapes stay fail-closed;
• provider-split reasoning tags are reassembled with sep="" before scrubbing;
• structured reasoning is preserved before visible-text projection;
• interim dedup extracts text only from assistant-role messages;
• plain-string behavior is unchanged;
• Anthropic / Kimi / DeepSeek / Codex and adjacent continuation suites pass.

[alt-glitch]

This was generated by AI during triage.

Related to merged #66945: this is a broader residual structured-content projection repair, not a duplicate of the already-landed immediate-crash fix.

[teknium1]

Thanks for extending the residual structured-content repair. Static inspection confirms the premise still holds on current main: agent/message_content.py:11-30,47-49 accepts arbitrary fields and stringifies unknown values, while agent/conversation_loop.py:4387-4403 performs the same permissive coercion in the live response path. run_agent.py:4819-4831 also sends stored structured content through the interim scrubber.

The allowlisted projector preserves the established supported shapes in tests/agent/test_message_content.py:8-25 while covering the missing unknown-object, hostile-mapping, split-tag, and non-assistant interim cases. The member note correctly distinguishes this from the already-merged list-crash repair in 296494db0ee99f1cd9e384b2083e7a60aeb833ef.

Automated hermes-sweeper review.

[Mikoto9901]

Rebased onto current main and semantically revalidated the structured-content projection boundary.

• current main still reproduces the residual projection gaps
• typed visible-text parts remain supported
• unknown, metadata, tool, media, and hostile shapes remain fail-closed
• provider-split reasoning tags are reassembled with sep="" before scrubbing
• structured reasoning is preserved before visible-text projection
• interim dedup extracts text only from assistant-role messages
• plain-string behavior is unchanged
• Anthropic / Kimi / DeepSeek / Codex and adjacent continuation suites pass
• fix(agent): scope provider terminal lifecycle per attempt #67411 remains an independent, merge-order-neutral related defense

This PR is ready for direct merge.