feat: expose structured per-turn content metadata from run_conversation()

[zccyman]

Problem

run_conversation() returns a flat dict where final_response is overwritten by the last non-tool-call turn. When the model emits substantive content alongside tool calls, that content is stored internally in _last_content_with_tools but never exposed to downstream consumers. This is the root cause of at least 4 open bugs:

• -z/oneshot 模式下 final_response 丢弃带工具调用的消息正文 #28326: oneshot/chat() loses content when model appends short follow-up after tool calls
• [Bug]: post_llm_call response overrides are applied after persistence, causing final_response/history mismatch #14894: post_llm_call response overrides cause persistence/history mismatch
• [Bug]: _last_content_with_tools fallback bypasses empty-response retries, causing silent agent loop termination mid-task #7968: _last_content_with_tools fallback bypasses empty-response retries
• [Bug]: Text content before tool calls is not delivered on Telegram (only visible in CLI) #6067: Telegram does not show text content before tool calls

Closes #28431

Solution

Add a ContentSegment dataclass to agent/conversation_loop.py:

@dataclass
class ContentSegment:
    content: str           # Text content (stripped of think blocks)
    had_tool_calls: bool   # Whether this turn also had tool calls
    tool_call_count: int   # Number of tool calls
    tool_names: list[str]  # Names of tools called

The result dict from run_conversation() now includes:

"content_segments": [  # NEW — list of ContentSegment
    ContentSegment(content="Report...", had_tool_calls=True, tool_call_count=1, tool_names=["web_search"]),
    ContentSegment(content="Done!", had_tool_calls=False),
]

How this fixes the 4 bugs

1. -z/oneshot 模式下 final_response 丢弃带工具调用的消息正文 #28326: chat() can construct final_response from content_segments instead of relying on a single overwritten string.
2. [Bug]: post_llm_call response overrides are applied after persistence, causing final_response/history mismatch #14894: post_llm_call hooks can inspect content_segments for informed decisions.
3. [Bug]: _last_content_with_tools fallback bypasses empty-response retries, causing silent agent loop termination mid-task #7968: Empty-response retry logic can check content_segments[-1] instead of _last_content_with_tools.
4. [Bug]: Text content before tool calls is not delivered on Telegram (only visible in CLI) #6067: Platform adapters can iterate content_segments to deliver each piece of content.

Changes

File	Change
agent/conversation_loop.py	+47: ContentSegment dataclass, collection in 2 loop branches, result dict field
tests/run_agent/test_run_agent.py	+74: 5 new tests in TestContentSegments

Testing

343 passed, 0 failed (including 5 new tests)

Backward Compatibility

Fully backward compatible. All existing dict keys (final_response, messages, etc.) are unchanged. New content_segments key is additive — consumers that don't read it are unaffected.

Follow-up Opportunities

With content_segments available, these internal mechanisms can be simplified in future PRs:

• Remove _last_content_with_tools hack
• Simplify empty-response retry logic
• Add platform-level content delivery from segments

[teknium1]

Thanks for taking on the content-before-tool-calls class of bugs. The premise still matters: current main still captures content alongside tool calls in _last_content_with_tools at agent/conversation_loop.py:3839, and there is no content_segments result field on main.

Problems

• This is stale against current main. run_conversation() now delegates result assembly to agent.turn_finalizer.finalize_turn() at agent/conversation_loop.py:4402, and the result dict is built in agent/turn_finalizer.py:326. The PR adds the new key to the old in-file result assembly.
• The PR does not yet fix #14894. Current main persists at agent/turn_finalizer.py:143, then invokes post_llm_call at agent/turn_finalizer.py:287; this PR does not change that ordering or pass segment metadata to the hook.
• The PR does not yet fix #6067. Gateway delivery still reads final_response directly, e.g. gateway/run.py:8848 / gateway/run.py:15721; no platform consumer is updated to send intermediate segments.
• Segment content can diverge from returned text: final response is still changed later by truncation handling in agent/conversation_loop.py:4311 and by finalizer transforms/footers in agent/turn_finalizer.py:204-282.

Suggested changes

• Port the metadata through the current finalize_turn(...) seam and add tests at that seam.
• Either narrow the claims to metadata exposure or wire one real consumer, especially the gateway path for #6067.
• Document whether segments are raw model-turn content or post-processed user-visible content.

This is an automated hermes-sweeper review.

[teknium1]

This records the no-tool segment before later final-response mutations such as truncation-prefix concatenation, think-block stripping, finalizer footers, and transform_llm_output, so content_segments[-1].content can disagree with the returned final_response.

[teknium1]

On current main the result dict is no longer assembled in conversation_loop.py; run_conversation() calls agent.turn_finalizer.finalize_turn(), so this new key needs to be added through that seam or the cherry-pick will miss the actual return path.

[teknium1]

If these segments are intended to fix gateway delivery, collecting the metadata is only half the change; no gateway/platform consumer in this PR reads content_segments, so text before tool calls will still be dropped by callers that only send final_response.