fix(api_server): robust Responses turn-start detection

[heyf]

What does this PR do?

/v1/responses has two reported misbehaviours that share a single root cause in gateway/platforms/api_server.py::_response_messages_turn_start_index:

1. Output side (phantom tool calls): on a turn that called no tools, the output array replays the previous turn's function_call / function_call_output items.
2. Storage side (history doubling): chained Responses history is concatenated onto itself and doubles every request — reported in Exponential conversation history doubling in Responses API (_build_response_conversation_history) #68257.

The helper located the current turn's start by a strict byte-for-byte prefix equality (agent_messages[:len(prior)] == prior) and, on any mismatch, returned 0. A 0 start makes _extract_output_items(start_index=0) walk the entire transcript and re-emit every historical tool_calls + tool message as this-turn output (phantom function_calls), and makes _build_response_conversation_history fall through to prior + current_user + full_transcript (doubling).

The strict equality breaks under several non-exceptional conditions, so this is a bug class, not a single trigger:

• the agent core prepends a private system message to the transcript (the Exponential conversation history doubling in Responses API (_build_response_conversation_history) #68257 trigger);
• the core reshapes history in place during a turn — alternation repair and api_content sidecar stamping in conversation_loop.py, provider cache markers — none of which change meaning but all of which defeat ==;
• compression replaces the history prefix with a summary (already partially handled on the storage side via result['_compressed'] from fix(api): persist compressed transcripts in ResponseStore to stop re-compression loops #69306, but the output side had no such fallback).

The fix makes _response_messages_turn_start_index robust instead of all-or-nothing: skip leading private system messages → tolerant semantic prefix match (compare only meaningful fields, ignore sidecar stamps) → reverse-anchor on the last user message equal to this turn's input → last resort trust only the final assistant message and never replay unattributable history tool calls. The suffix-only path (genuine current-turn tool calls, no prior) still returns 0. Because every call site funnels through this one helper, the fix benefits output extraction, storage, and streaming (_turn_transcript_messages) at once.

Relationship to #68282: #68282 strips the leading system message on the storage path only and explicitly does not touch the output-side offset. This PR fixes the shared helper itself, covering all three mismatch triggers (system-prefix + in-place reshaping + compression) and also the output-side phantom function_calls that #68282 leaves in place. Non-competing (same function, complementary scope) — happy to defer to maintainers; #68282's system-stripping intent is subsumed here.

Invariants respected: no mid-conversation prompt/tool/system mutation (prompt caching preserved), strict role alternation, no synthetic user messages.

Related Issue

Fixes #68257

Type of Change

• 🐛 Bug fix (non-breaking change that fixes an issue)

Changes Made

• gateway/platforms/api_server.py: rewrote _response_messages_turn_start_index — skip leading private system messages; tolerant semantic prefix match against prior + current_user then prior; reverse-anchor fallback on the last matching user message; final-assistant last resort; never replay unattributable history tool calls.
• gateway/platforms/api_server.py: _build_response_conversation_history strips leading private system messages so stored history round-trips cleanly as prior on the next chained request.
• tests/gateway/test_api_server.py: added TestTurnStartRobustness (6 behavior-contract regressions).

How to Test

1. python -m pytest tests/gateway/test_api_server.py -o 'addopts=' -q → 232 passed (6 new + 226 existing, no regressions).
2. The 6 new tests fail on main (RED) and pass with this change (GREEN).
3. Repro of the output-side symptom (fails on main, passes here):

from gateway.platforms.api_server import APIServerAdapter
conversation_history = [
    {"role":"user","content":"search the web for X"},
    {"role":"assistant","content":"","tool_calls":[{"id":"call_1","function":{"name":"web_search","arguments":'{"query":"X"}'}}]},
    {"role":"tool","tool_call_id":"call_1","content":"results..."},
    {"role":"assistant","content":"Here is what I found about X."},
]
user_message = "thanks, now just say hi"          # this turn calls no tools
mutated = [dict(m) for m in conversation_history]
mutated[1]["api_content"] = "<stamped>"            # in-place stamp defeats == prefix match
result = {"messages": mutated + [{"role":"user","content":user_message},{"role":"assistant","content":"hi"}], "final_response":"hi"}
idx = APIServerAdapter._response_messages_turn_start_index(conversation_history, user_message, result)
items = APIServerAdapter._extract_output_items(result, start_index=idx)
assert [i["type"] for i in items] == ["message"]   # main: ['function_call','function_call_output','message']

Checklist

Code

• I've read the Contributing Guide
• My commit messages follow Conventional Commits (fix(scope):, feat(scope):, etc.)
• I searched for existing PRs to make sure this isn't a duplicate (found Exponential conversation history doubling in Responses API (_build_response_conversation_history) #68257 / fix(api): keep Responses history system-free 🩷 #68282 / fix(gateway): avoid duplicated Responses history (salvage #18995) #21185 / fix(api): persist compressed transcripts in ResponseStore to stop re-compression loops #69306 — relationship explained above)
• My PR contains only changes related to this fix/feature (no unrelated commits)
• I've run the relevant suite (pytest tests/gateway/test_api_server.py) and all tests pass (232 passed)
• I've added tests for my changes (6 new behavior-contract regressions)
• I've tested on my platform: Ubuntu (Linux 6.8)

Documentation & Housekeeping

• I've updated relevant documentation (README, docs/, docstrings) — N/A (internal helper, no public-facing config/doc surface)
• I've updated cli-config.yaml.example if I added/changed config keys — N/A (no config keys)
• I've updated CONTRIBUTING.md or AGENTS.md if I changed architecture or workflows — N/A (behavior fix within existing helper)

Related issues

• Fixes Exponential conversation history doubling in Responses API (_build_response_conversation_history) #68257 (history doubling on chained Responses requests).
• Supersedes the narrower storage-only strip in fix(api): keep Responses history system-free 🩷 #68282.
• Also addresses the phantom function_call replay in the area introduced by fix(gateway): avoid duplicated Responses history (salvage #18995) #21185.
• Builds on the compression-summary handling from fix(api): persist compressed transcripts in ResponseStore to stop re-compression loops #69306, extending the same fallback to the output side.

---

Generated with AI assistance.

[teknium1]

Thanks for targeting the shared turn-boundary helper. The current detector is all-or-nothing at gateway/platforms/api_server.py:5594-5599, but this needs rework before it is safe to salvage.

Problems

• The claimed normal leading-system transcript is not produced by the current path: agent/turn_context.py:505 builds messages from history, agent/conversation_loop.py:1556 prepends the system prompt only to wire api_messages, and agent/turn_finalizer.py:577 returns messages.
• The proposed storage loop removes every leading system item. That can drop client state: the Responses input parser preserves arbitrary roles at gateway/platforms/api_server.py:4911-4920, so a client-provided leading system message is valid history.

Suggested changes

• Do not strip by role alone; identify a verified private-message producer/marker, or omit that path.
• Add an end-to-end regression for a current non-compression mismatch producer and a chaining regression preserving client-supplied leading system history.

This is an automated hermes-sweeper review.

[heyf]

Addressed the sweeper review and rebased the PR onto current main (03fa32c92d). The updated head is 8f8482eb57.

Changes made:

• Removed the role-based leading-system stripping entirely. The implementation now preserves client-supplied leading system history and treats it as part of prior.
• Reworked turn-boundary detection around semantic prefix matching, a reverse current-user anchor, and a conservative fallback that never replays unattributable historical tool calls.
• Preserved the suffix-only compatibility path: when prior history exists but result["messages"] contains only this turn's assistant(tool_calls) -> tool -> assistant suffix, genuine current-turn function calls and outputs are retained and storage prepends prior + current_user exactly once.
• Added real POST /v1/responses HTTP regressions for:
  1. an in-place api_content history mutation (the non-compression mismatch producer), proving no phantom function-call replay and no history doubling;
  2. client-supplied leading system history, proving it round-trips exactly once;
  3. prior history plus a suffix-only current tool turn, proving genuine current-turn tool calls and prior history are not lost.

Verification on current main:

• New regression is RED on unpatched main: output incorrectly contains the previous turn's function_call / function_call_output.
• TestTurnStartRobustnessE2E: 3 passed.
• HERMES_PYTHON=... scripts/run_tests.sh tests/gateway/test_api_server.py: 102 passed, 0 failed.
• Full tests/gateway: 5,113 passed; 7 unrelated order/environment-sensitive failures, all 7 pass when rerun independently in both patched and baseline worktrees.
• git diff --check and compileall pass.

The PR now consists of one commit on current main and changes only gateway/platforms/api_server.py and tests/gateway/test_api_server.py.

[christiannadeau]

Field confirmation: the in-place-reshaping trigger is the one that fires in production (v0.20.3)

Independent confirmation from a self-hosted deployment, offered as evidence for the
needs-decision label. I hit this in production, root-caused it from scratch, and only
afterwards found this PR describing the same mechanism — so this is a genuinely independent
reproduction rather than a restatement.

Environment: hermes-agent v0.20.3 (336059011, git install), /v1/responses with the
named conversation parameter, provider DeepSeek (deepseek-v4-pro), a long-running unattended
agent driven by a task pipeline — one conversation per task, many chained turns each.

The second bullet of this PR is the trigger, and it reproduces on current main

Of the three mismatch triggers listed here, the one I observe is in-place reshaping — not the
system-prefix path. That distinction matters, because #68257 was closed cannot-reproduce on the
grounds that the agent core does not prepend a private system message. That refutation is
correct and narrow: it disposes of the trigger #68257 named, but not of the defect. The prefix
comparison is strict == over whole dicts, so it is defeated by any in-place edit to any
prefix message — alternation repair, api_content stamping, whitespace normalisation. The
transcript stays semantically identical; only the dict changes.

Minimal repro against the storage path, run against installed v0.20.3, no mocks:

from gateway.platforms.api_server import APIServerAdapter as A

prior = [
    {"role": "user", "content": "turn 1"},
    {"role": "assistant", "content": "", "tool_calls": [
        {"id": "call_1", "type": "function",
         "function": {"name": "bash", "arguments": "{}"}}]},
    {"role": "tool", "tool_call_id": "call_1", "content": "ok"},
    {"role": "assistant", "content": "done"},
]
user_message = "turn 2"

agent_messages = [dict(m) for m in prior]
agent_messages.append({"role": "user", "content": user_message})
agent_messages.append({"role": "assistant", "content": "done 2"})

# A. untouched -> correct
res = {"messages": [dict(m) for m in agent_messages]}
assert A._response_messages_turn_start_index(prior, user_message, res) == 5
assert len(A._build_response_conversation_history(list(prior), user_message, res, "done 2")) == 6

# B. ONE prefix message reshaped in place (here: trailing-space normalisation)
mutated = [dict(m) for m in agent_messages]
mutated[3] = {**mutated[3], "content": "done "}
res2 = {"messages": mutated}
assert A._response_messages_turn_start_index(prior, user_message, res2) == 0     # prefix miss
out = A._build_response_conversation_history(list(prior), user_message, res2, "done 2")
assert len(out) == 11                                                            # expected 6

Stored transcript in case B — the whole prefix is re-appended after the current user turn:

[0] user      'turn 1'
[1] assistant ''         tool_calls=['call_1']
[2] tool      'ok'
[3] assistant 'done'
[4] user      'turn 2'
[5] user      'turn 1'        <- prefix restarts here
[6] assistant ''         tool_calls=['call_1']    <- call_1 now appears twice
[7] tool      'ok'
[8] assistant 'done '
[9] user      'turn 2'
[10] assistant 'done 2'

N messages become 2N + 1, so it compounds turn over turn.

Production numbers (observed on 0.19.0, before I updated)

Turn-start log lines for one conversation, each entry one chained request:

17:20:43  history=0
17:39:51  history=45
17:41:56  history=102
17:48:36  history=203

Alongside, agent.conversation_loop: Repaired N message-alternation violations before request
fires on the very turns that miss the prefix — the in-place reshaping and the doubling are
visible in the same log, seconds apart.

Cross-checked against ground truth: for that session state.db held a 59-message transcript
while response_store.db had stored 403 messages for the same conversation. Across 24
conversations in one profile the stored histories reached 3518, 1246, 390 and 378 messages.

I repaired one snapshot by hand (403 → 57, rebuilt from state.db). It was back to 85 after one
turn and 143 within a few hours — a data-level repair does not hold while the concatenation
stands.

Downstream consequence worth noting: it manufactures duplicate tool_call_ids

The re-appended prefix replays every tool_call_id. sanitize_api_messages then dedups them,
and before #64335 that left tool_calls: [] on the surviving assistant message, which DeepSeek
rejects outright:

HTTP 400: Invalid 'messages[45].tool_calls': empty array.
Expected an array with minimum length 1, but got an empty array instead.

Index 45 in the failing payload was the first message of the duplicated copy. This was a hard
stop, not a slowdown: the conversation could not advance at all, and the pipeline retried it
into the same 400 indefinitely.

#64335 (drop tool_calls key when dedup removes all calls) resolves that crash, and I have
verified it does — the same failing 94-message payload now sanitizes to zero empty tool_calls
and zero empty-content messages. But it is a downstream guard: with it in place the request
stops failing while the history still doubles. What remains is silent — unbounded context
growth, prompt-cache invalidation every turn (the prefix changes), full-price tokens on every
request, and premature compression. In our case that surfaced as an unexplained rise in
per-task API cost long before it surfaced as an error.

Caveat on scope: my runtime observations above are from 0.19.0. Since updating to 0.20.3 the
deployment has not yet run a chained conversation, so I am reporting the code-level repro on
0.20.3 and the runtime evidence on 0.19.0 — the two functions are byte-identical across those
versions, which is why I expect the behaviour to be unchanged.

Why this seems worth deciding on

The failure is silent by construction: nothing logs, nothing errors, and the only visible
symptoms are cost and latency until a strict provider turns it into a 400. The reverse-anchor
approach in this PR handles the trigger I observe, which strict equality cannot. If the concern
is blast radius on the output-extraction side, the storage-side half alone would already stop the
unbounded growth — happy to test any variant against this deployment.