fix(api): keep Responses history system-free 🩷

[kingrubic]

Summary

Fixes #68257.

When /v1/responses receives the agent's full transcript with one or more leading private system messages, prefix detection previously missed and persisted prior + current_user + full_transcript. Chained Responses history could grow exponentially and leak the system prompt into response_store.db.

This patch:

• strips only contiguous leading system messages from the stored client transcript;
• recognizes full transcripts after that prefix without changing output-item offsets;
• preserves compatibility with legacy turn-only suffixes (assistant-only and user+assistant);
• covers both streaming and non-streaming persistence behavior.

Verification

• Failing-first regression reproduced duplicate history and persisted system content.
• Responses endpoint + streaming suites: 27 passed.
• Ruff: passed.
• git diff --check: passed.
• The full tests/gateway/test_api_server.py baseline has one unrelated pre-existing health-detail failure on origin/main (degraded vs ok); the same test fails on a clean baseline worktree.

Generated with AI assistance.

[teknium1]

Thanks for the focused storage and offset handling work.

Problems

• I could not verify the claimed production transcript shape on current main. agent/turn_context.py:505 initializes the returned transcript from conversation history, while agent/conversation_loop.py:1540 prepends the private system prompt only to provider-wire api_messages. agent/turn_finalizer.py:577 then returns the separate messages transcript. The new system-prefixed mocks therefore do not reproduce the current normal Responses path.

Suggested changes

• Please identify a current producer of a leading-system result["messages"] transcript and add an end-to-end regression through that path, or re-scope the change to a verified current behavior.

This is an automated hermes-sweeper review.

[GottZ]

This was generated by AI during triage.

Summary

Three PRs reference #68257. #68282 targets a leading-system transcript shape, #70695 broadens turn-start matching while also stripping leading system messages, and #75289 targets the current repair-induced prefix mismatch by marking authoritative full transcripts and recovering the output boundary.

Related pull requests

• fix(api): keep Responses history system-free 🩷 #68282 best fix — (+109/-9) — n/a: The diff strips contiguous leading system messages and preserves full- and turn-only transcript forms without duplicating the current user turn. Consistent with the maintainer-bot keep_open review, its tests mock a leading-system result shape that the cited current agent path does not produce, so the patch needs a verified producer or a narrower scope.
• fix(api_server): robust Responses turn-start detection #70695 best fix — (+367/-8) — n/a: The diff adds semantic prefix matching, reverse anchoring, and output-replay protection across broader mismatch cases, but also removes every leading system message. Consistent with the maintainer-bot keep_open review, role-only stripping can discard valid client-provided system history, so the independently useful mismatch and output-boundary work should be split from that path.
• fix(api_server): trust full transcripts via _transcript_mode to stop repair-induced duplication #75289 best fix — (+250/-0) — n/a: The diff marks AIAgent results as authoritative full transcripts, stores them verbatim after repair-induced prefix drift, and reverse-anchors output to avoid replaying historical tool items. The contributor follow-up addresses the maintainer-bot keep_open review's output-boundary concern, but the endpoint test still supplies an already-repaired mocked transcript rather than invoking the real repair helper.

Duplicates

#68282 and the leading-system portion of #70695 substantially duplicate the same system-prefix mitigation. #75289 overlaps on the duplication symptom but addresses the distinct repair-induced prefix mismatch through an explicit full-transcript marker.

Suggested consolidation

Keep #75289 open with a salvage path: retain the authoritative-transcript marker and storage/output-boundary handling, and strengthen the endpoint regression to exercise the real repair helper. For #68282, author action: rebase onto main, or split out a fix tied to a verified current producer; for #70695, author action: rebase onto main, or split out the semantic mismatch and output-boundary work while removing role-only system stripping. Do not close either as a duplicate of #75289, because their leading-system scope is distinct.

Complex graph

flowchart LR
    classDef open fill:#dbeafe,stroke:#1d4ed8,color:#1e3a8a
    classDef merged fill:#dcfce7,stroke:#15803d,color:#14532d
    classDef closed fill:#e5e7eb,stroke:#6b7280,color:#1f2937
    classDef unverified fill:#f3f4f6,stroke:#9ca3af,color:#374151
    classDef best stroke-width:3px,stroke:#b45309
    classDef target stroke-width:3px,stroke:#4338ca
    I68257(["issue #68257 (closed)"])
    subgraph Dup68282 ["PRs duplicating each other"]
        P68282["PR #68282 (open)"]
        P70695["PR #70695 (open)"]
    end
    P68282 -->|best fix| I68257
    class I68257 closed
    class P68282 open
    class P70695 open
    class P68282 best
    class P70695 best
    class P68282 target
    click I68257 "https://github.com/NousResearch/hermes-agent/issues/68257"
    click P68282 "https://github.com/NousResearch/hermes-agent/pull/68282"
    click P70695 "https://github.com/NousResearch/hermes-agent/pull/70695"

Loading

Graph: solid arrow = fixes / best fix, dashed arrow = partial or unverified (see edge label); boxed group = PRs duplicating each other; amber border = best fix; indigo border = target; gray node = closed (state tag in the node label).

Cross-PR triage: Reviewed 3 pull requests and 1 issue in this complex. Each diff was read against this issue; Assessment working set: 42 kB of PR diffs, 16 kB of issue/PR text, 1 kB of discussion (3 comments), 7 verify verdicts. verdicts reflect diff content, not PR titles. Part of an automated triage batch.