fix(tui): exclude timeline rows from the rewind truncation ordinal

[necoweb3]

Summary

prompt.submit resolves truncate_before_user_ordinal against every role == "user" row in session["history"]. No client counts them that way. Bookkeeping timeline rows — model_switch, async_delegation_complete, auto_continue, hidden — are stored as durable user rows, but the desktop demotes them to role: 'system' (or drops them) in toChatMessages before visibleUserOrdinal counts.

The ordinal therefore resolves N real turns too early, and the replace_messages() that follows hard-DELETEs those extra completed exchanges. Same class as #70895: an index into the frontend's message array treated as an index into the backend's history.

Problem

Producer — apps/desktop/src/app/session/hooks/use-prompt-actions/utils.ts:362:

export function visibleUserOrdinal(messages: readonly ChatMessage[], end: number): number {
  return messages.slice(0, end).filter(m => m.role === 'user' && !m.hidden).length
}

Its only filter is !m.hidden, a client-side branch flag applyBranchVisibility sets on assistant rows. It never needs to know about display_kind, because by the time it runs toChatMessages has already removed those rows from the user role — apps/desktop/src/lib/chat-messages.ts:949-955:

const displayRole =
  message.display_kind === 'model_switch' ||
  message.display_kind === 'async_delegation_complete' ||
  message.display_kind === 'auto_continue'
    ? 'system'
    : message.role

and display_kind: 'hidden' yields no parts at all, so the row is skipped entirely.

Consumer — tui_gateway/server.py:10790:

user_indices = [i for i, m in enumerate(history) if m.get("role") == "user"]
...
truncated = history[: user_indices[ordinal]]
...
db.replace_messages(session["session_key"], truncated)

No display_kind exclusion. The marker rows are in session["history"] with role == "user" — server.py:11828 stamps message["display_kind"] on the in-memory row after the run, and hermes_state.py:7283 rehydrates it on resume.

Twin-guard asymmetry. The CLI performs the same logical operation — count user turns in a stored history — and does exclude them, with exactly this predicate:

• hermes_cli/cli_agent_setup_mixin.py:518 — if m.get("role") == "user" and not m.get("display_kind")
• hermes_cli/cli_commands_mixin.py:854 — same expression

Only the destructive gateway path omits it.

Neither existing guard fires. The skewed ordinal is always smaller than the true one, so it stays in range for the 4018 bounds check at server.py:10796 — whose own comment already names this hazard class ("silently truncating history to everything before it and persisting that loss via replace_messages — an unrecoverable overwrite of the session DB"). And the truncation is non-empty, so the 4028 confirm_empty_truncate guard (added for #70895/#70516) does not apply either.

Reproduced

Running the real client rule and the real gateway expressions over one history:

client ChatMessage roles : ['user','assistant','system','assistant','user','assistant','user','assistant']
client counts as 'user'  : ['Q0', 'Q1', 'Q2']
ordinal sent to gateway  : 2

gateway user_indices     : [0, 2, 4, 6]
gateway counts as 'user' : ['Q0', 'background agent work finished', 'Q1', 'Q2']
cut index                : 4

DELETED rows:
   user      Q1
   assistant A1
   user      Q2
   assistant A2

user intended to delete  : Q2 / A2
EXTRA rows destroyed     : ['Q1', 'A1']

A completed question and its answer, permanently deleted, with no warning.

Default config. _AUTO_CONTINUE_ENABLED_DEFAULT = True (server.py:6421); async_delegation_complete is emitted unconditionally by the notification poller (server.py:11379, :11457). No flag or setting is involved on either side.

Deterministic. Pure integer arithmetic over two lists — the skew is exactly the number of timeline rows before the target, on the first attempt, every time.

Frequent. Edit-a-message, Regenerate and Restore-checkpoint are always-visible controls and all funnel through this ordinal. Once one timeline row exists it is permanent for the session, so every subsequent rewind in that chat is misaligned, and the skew accumulates with each additional marker.

Unrecoverable. replace_messages takes its default active_only=False path — DELETE FROM messages WHERE session_id = ?, whose docstring says "DESTRUCTIVE by default: every row for the session is DELETEd (and drops out of the FTS index)". Nothing holds a copy: archive_and_compact's soft-archive is a different method and is not used here, so include_inactive=True returns nothing; there is no JSONL mirror on this path; and the desktop already sliced its own state.messages optimistically, then repaints from the truncated DB on the next resume.

Fix

Count the same rows the client counted:

user_indices = [
    i
    for i, m in enumerate(history)
    if m.get("role") == "user" and not m.get("display_kind")
]

The predicate is copied verbatim from the CLI siblings. It is exactly right for every display_kind value that exists — model_switch, async_delegation_complete and auto_continue are demoted to system client-side, and hidden is dropped — so no client ever counts any of them.

Scope

• tui_gateway/server.py — one list comprehension in the prompt.submit truncation branch, plus a comment. No change to the 4018/4028 guards, to replace_messages, or to any other RPC.
• No desktop change required: the client is already consistent with itself and with the CLI. Making the gateway agree with both is the smaller, safer half of the fix.

Related but distinct: #41275 (open) touches the same visibleUserOrdinal helper in the opposite direction — optimistic failed user bubbles make the client ordinal overshoot, producing a visible 4018 rejection. Its fix skips failed turns and cannot correct an undercount, and it describes no silent deletion.

Testing

New test_prompt_submit_ordinal_skips_display_kind_timeline_rows in tests/test_tui_gateway_server.py, built in the same idiom as the existing test_prompt_submit_can_truncate_before_user_ordinal directly above it: a history where a delegation-complete row sits between the first and second real exchanges, then an edit of the third user turn (ordinal 2).

It asserts the cut lands at "third" and that "second"/"second reply" survive. Fails on main — the stored transcript comes back as original_history[:4], having destroyed the second exchange — and passes here.

python -m pytest tests/test_tui_gateway_server.py -o addopts= -q
468 passed

The pre-existing test_prompt_submit_can_truncate_before_user_ordinal passes unmodified, so ordinary rewind behaviour is unchanged.

Note on tests/tui_gateway/: that directory has order-dependent failures unrelated to this change — running the full directory on unmodified main gives 6 failures (test_compute_host*, test_projects_rpc*, test_subagent_child_mirror), and 3 on this branch. Each of those tests passes in isolation on both.

[teknium1]

Thanks for the focused regression report and test case. This is an automated hermes-sweeper review; the requested behavior is already on current main.

• tui_gateway/methods_prompt.py:166-169 now resolves truncate_before_user_ordinal using only role == "user" rows without display_kind.
• The equivalent fix landed in cf258b6ae7933c70b568f0f9ab14c6a76b01aaaf (fix(session): widen display_kind filter to prompt.submit ordinal + rollback.restore), which is contained in current main.
• The later TUI handler split moved the implementation from tui_gateway/server.py into tui_gateway/methods_prompt.py; the behavioral guarantee remains intact.