fix(gateway): atomic JSONL transcript rewrite

[simbam99]

Summary

SessionStore.rewrite_transcript overwrites the session's JSONL transcript in place:

with open(transcript_path, "w", encoding="utf-8") as f:
    for msg in messages:
        f.write(json.dumps(msg, ensure_ascii=False) + "\n")

A crash between the truncate and the final flush — SIGKILL, OOM, power loss, a failing disk, or an exception raised during JSON serialization — leaves the transcript empty or partially written. This is the exact failure mode already acknowledged on the read side by TestLoadTranscriptCorruptLines (GH-1193), which learned to skip corrupt lines after the fact. This PR closes the write-side hole so the corrupt state is never reachable in the first place.

Why this matters

rewrite_transcript sits on several hot gateway paths:

• /retry — strips and replays the last assistant turn (gateway/run.py:5930)
• /undo — rewrites history back to a prior user turn (gateway/run.py:5896)
• /compress — persists the compressed history into a new session (gateway/run.py:7085)
• Tool-result truncation (gateway/run.py:4425)

For sessions predating the SQLite layer, the JSONL is the only source of history — the existing comment at gateway/session.py:1207 explicitly says so: "legacy JSONL transcript (may contain more history than SQLite for sessions created before the DB layer was introduced)." A bad shutdown during any of the flows above silently wipes that session. Even on post-DB sessions, a corrupted JSONL defeats the replay fallback in load_transcript.

Changes

1. utils.py — new atomic_jsonl_write(path, items, **dump_kwargs) helper. Mirrors the existing atomic_json_write / atomic_yaml_write pattern: tempfile + fsync + os.replace, with _preserve_file_mode / _restore_file_mode so Docker/NAS permission layouts from fix: preserve file permissions on atomic writes (Docker/NAS fix) #10618 are preserved, and BaseException cleanup so stray .tmp files are never left behind on KeyboardInterrupt / SystemExit.
2. gateway/session.py — SessionStore.rewrite_transcript now routes through atomic_jsonl_write. No behavior change on the happy path.

No API surface changes, no callers touched beyond the one in rewrite_transcript.

Alignment with recent merges

Same crash-safety pattern already shipped across the gateway:

• fix: preserve file permissions on atomic writes (Docker/NAS fix) #10618 — preserve file permissions on atomic writes (Docker/NAS fix)
• fix(gateway): make Telegram DM topic config writes atomic
• fix(gateway/weixin): ensure atomic persistence for critical session state
• fix(tui): atomic config persistence
• ACP history persistence atomicity (acp_adapter/session.py + hermes_state.py)

rewrite_transcript was the obvious remaining gateway write that still truncated-in-place.

Reproduction

Before the fix:

from unittest.mock import patch

# Populate a transcript, then crash during rewrite.
store.rewrite_transcript(sid, [
    {"role": "user", "content": "hello"},
    {"role": "assistant", "content": "hi"},
])

with patch("utils.json.dumps", side_effect=IOError("disk full")):
    try:
        store.rewrite_transcript(sid, [{"role": "user", "content": "replace"}])
    except IOError:
        pass

# Without the fix: transcript is empty / truncated.
# With the fix:    original two messages are still intact.
assert store.load_transcript(sid) == [
    {"role": "user", "content": "hello"},
    {"role": "assistant", "content": "hi"},
]

Tests

• New — tests/hermes_cli/test_atomic_jsonl_write.py: 14 cases covering crash safety (BaseException + mid-write IOError), tempfile cleanup on success and failure, unicode, generators, dump_kwargs forwarding, empty input, string path support.
• New — tests/gateway/test_session.py::TestSessionStoreRewriteTranscriptAtomicity: 4 cases verifying that a crash during rewrite_transcript preserves the prior transcript, that no .tmp files are left behind on success or failure, and that unicode survives the round trip.
• Preserved — TestSessionStoreRewriteTranscript and TestLoadTranscriptCorruptLines remain green.

Local run:

py -3.11 -m pytest tests/gateway/test_session.py tests/hermes_cli/test_atomic_jsonl_write.py -q
→ 37 passed

Risk

Very low. atomic_jsonl_write is a drop-in for the old open("w") + write loop, and the helper matches the existing atomic_json_write / atomic_yaml_write style line-for-line. Behavior on the happy path is unchanged; only the failure window is closed.

Checklist

• Bug fix with deterministic repro
• No new dependencies
• No API or config changes
• Unit + regression tests covering both the helper and the caller
• Matches existing atomic-write conventions in utils.py
• Preserves Docker/NAS permission behavior from fix: preserve file permissions on atomic writes (Docker/NAS fix) #10618
• No Windows-specific behavior; cross-platform by construction (uses the same primitives as atomic_json_write)

[alt-glitch]

Competing PRs for the same atomic transcript rewrite: #8065, #4985, #8077. Maintainers should pick one.

[alt-glitch]

Competing PRs for the same atomic transcript rewrite: #8065, #4985, #8077. Maintainers should pick one.

[teknium1]

Thanks for the careful crash-safety work. This is an automated hermes-sweeper review: current main has superseded the JSONL transcript path, so this PR's requested guarantee is already provided by the canonical SQLite implementation.

• 351fdcc6e6d763bd5d405d90d467a6d52eabf1f5 removed JSONL writes from SessionStore.append_to_transcript and rewrite_transcript; current gateway/session.py:2307 rewrites only through state.db.
• gateway/session.py:2329 is DB-only; the legacy JSONL fallback was removed by 024a8e3ee90d79df5b6e58d2b09976eec0e12a89.
• hermes_state.py:3638 performs the replacement as one SQLite write transaction, and tests/gateway/test_session.py:1536 verifies that an insert failure preserves the prior transcript.
• The member note identifying competing PRs (fix: make rewrite_transcript atomic for both SQLite and JSONL #8065, fix(gateway): rewrite transcripts atomically #4985, fix: make transcript rewrite atomic to prevent data loss on crash #8077) was reviewed; the current main design independently retired the shared JSONL target.

This behavior shipped in v2026.5.28.