fix: persist [System:] markers with role=system, demote at API call time

[yingliang-zhang]

Problem

[System:] messages (model-switch markers, personality pivots, verification nudges, continuation prompts) are persisted as role=user to avoid HTTP 400 on strict OpenAI-compatible providers (#48338). This causes the Desktop to render them as prominent right-aligned user message bubbles, visually pushing assistant responses out of view — users report their "response was washed away by a system message."

Root Cause

PR #54210 (fixing #48338) changed model-switch markers from role=system to role=user to avoid strict providers rejecting mid-array system messages. This conflated the API compatibility layer with the persistence/representation layer: the wire-format constraint was pushed into the DB, where the Desktop reads it as a real user message.

The same role=user pattern was then copied to all 7 [System:] injection points across tui_gateway/server.py and agent/conversation_loop.py.

Fix

Separate the concerns: persist with the correct role=system, demote to role=user only at API call time.

1. Persistence layer — use role=system (4 files, 7 injection points)

File	Injection point	Change
tui_gateway/server.py:~2440	Model-switch marker (history + DB)	user → system
tui_gateway/server.py:~2448	Model-switch marker (agent DB)	user → system
tui_gateway/server.py:~2454	Model-switch marker (scoped DB)	user → system
tui_gateway/server.py:~4098	Personality change marker	user → system
agent/conversation_loop.py:~1942	Continuation prompt (timeout/truncation)	user → system
agent/conversation_loop.py:~5111	"Continue now" tool-call nudge	user → system
agent/conversation_loop.py:~5184	Verification-stop nudge	user → system
agent/conversation_loop.py:~5242	Pre-verify nudge	user → system

2. API compatibility layer — demote in sanitize_api_messages (1 file)

Added a new step in sanitize_api_messages (agent/agent_runtime_helpers.py) that demotes any system message at position > 0 to role=user on the per-call copy. The stored transcript and Desktop rendering keep the correct system role; only the wire payload is adjusted.

This is the same chokepoint already used for empty tool_calls cleanup and orphaned tool repair — it runs on every API call and only mutates the per-call copy, never the persisted trajectory.

3. Desktop rendering — zero changes needed

The Desktop already has a SystemMessage component (apps/desktop/src/components/assistant-ui/thread/system-message.tsx) that renders role=system messages as small, centered, gray text. The toRuntimeMessage function (chat-runtime.ts:355) already maps role=system to this component. No frontend changes required.

Related

• Closes fix(tui): keep model switch markers out of API history #48356 (alternative approach using session_meta rows — this PR is simpler)
• Supersedes the role=user workaround from fix(tui): use role=user for model switch marker to avoid HTTP 400 on strict providers (#48338) #54210 / [Bug]: _append_model_switch_marker injects role:"system" mid-conversation → HTTP 400 on strict providers (vLLM, Qwen) #48338
• Addresses the verification-nudge persistence leak (perf(cache): stabilize prefix cache during verification loops #55733) — leaked nudges now render as small system messages instead of large user bubbles

Test

• 358 tests passed, 0 failed across test_tui_gateway_server.py (316), test_agent_guardrails.py (39), test_verification_stop_caching.py (3)
• 2 new tests for sanitize_api_messages demotion logic
• 3 existing tests updated to assert role=system

[tonydwb]

Code Review Summary

Automated review by Hermes Agent: No obvious issues found.

---

Reviewed by Hermes Agent

[tonydwb]

Code Review Summary

Automated review by Hermes Agent: No obvious issues found.

---

Reviewed by Hermes Agent

[teknium1]

Thanks for separating persisted transcript roles from the strict-provider wire workaround. The current-main premise is real: tui_gateway/server.py:2433-2460 stores model-switch markers as user, and tui_gateway/server.py:4084 does the same for personality pivots.

Problems

• agent/agent_runtime_helpers.py:2492 demotes every system message after index 0. This also rewrites supported system-role prefill messages: agent/conversation_loop.py:878-881 inserts agent.prefill_messages after the primary system prompt and calls this sanitizer at :900; tests/cron/test_scheduler.py:1864 supplies a system-role prefill example. The new behavior should be limited to Hermes's internal marker records.

Suggested changes

• Mark internal pivot/nudge messages explicitly and demote only those records at the API boundary.
• Add a regression test proving a non-marker system prefill remains system while an internal marker is demoted.

Automated hermes-sweeper review.

[yingliang-zhang]

Closing: this branch predates upstream's mid-turn guidelines + display_kind redesign (#78113 / a871948). Against latest main the runner metadata + TUI scaffolding-layer work conflicts architecturally with upstream's rollback/display_kind design, and the abandoned base-insert-in-flight flush direction was validated as unworkable (#80832 replaces it). Not resubmitting in this form.