fix(session-db): persist post-flush /steer mutation to SQLite

[youngting520]

What does this PR do?

Fixes a durable-persistence bug where a mid-turn /steer is delivered to the live model but is not written back to SQLite SessionDB (state.db). A later resume, session API read, or search could therefore replay a transcript that differs from the content that actually produced the answer.

It also fixes the sibling post-flush mutation in aggregate tool-output budget enforcement: enforce_turn_budget can rewrite an already-flushed tool result in place, leaving the durable row with the pre-budget content while the model sees the replacement.

Root cause

AIAgent._flush_messages_to_session_db() marks message dicts with _db_persisted after writing them. Later flushes skip those dicts, and the bounded identity-prefix scan can skip the entire already-seen prefix. /steer and aggregate budget enforcement intentionally mutate tool-result content after that first write, so the durable row remained stale.

Approach on current main

This revision is rebased directly onto 9712b8f0c and follows the current batched persistence design:

1. Steer injection and aggregate budget rewrites stamp _db_content_update_pending on the intentionally mutated tool message.
2. The bounded identity-prefix scan stops before a pending mutation.
3. The flush performs a scoped UPDATE of the active tool row, using its exact _row_id and falling back only to a unique tool_call_id.
4. UPDATEs reuse the current transcript-write guards and patience budget, so compression leases and compression-closed sessions remain protected.
5. New turn rows still land through one append_messages_batch() transaction. The batch API now returns row IDs only when return_row_ids=True; the live flush uses that opt-in to stamp freshly inserted tool messages without changing the default inserted-count contract.
6. TUI branch seeding keeps current main's bounded chunked batch writes, preserves assistant/tool pairing metadata, and re-stamps copied messages with child-session row IDs.

The original revision also re-added per-tool steer drains. Current main deliberately delivers /steer at the batch-end drain after aggregate budget enforcement, so this revision preserves that ordering and does not restore the old drain scheme.

Changes made

• agent/persistence_markers.py centralizes _db_persisted and _db_content_update_pending.
• run_agent.py updates pending durable rows, integrates row-ID capture with batched inserts, stops the prefix fast path at pending mutations, and strips private persistence metadata from JSON snapshots.
• hermes_state.py adds fail-closed update_tool_message_content(...) and opt-in row-ID returns for batched inserts.
• agent/agent_runtime_helpers.py, agent/conversation_loop.py, and tools/tool_result_storage.py explicitly mark the three intentional mutation sites.
• CLI, gateway, and TUI live-resume paths opt into row IDs; inspection/export consumers retain the historical transcript shape.
• tui_gateway/methods_session.py preserves tool pairing and child-row identity without reverting current main's chunked batch-copy optimization.
• agent/context_compressor.py removes both persistence markers when re-baselining compacted messages.
• Regression coverage exercises real SQLite, FTS updates, compression leases, rewritten and duplicate row identities, TUI branching, concurrent/sequential steer drains, aggregate-budget rewrites, marker stripping, and the batched write contract.

Validation

6 focused files: 914 passed, 0 failed
ruff check: passed
git diff --check: passed

The focused run includes:

• tests/run_agent/test_steer_sessiondb_persistence.py
• tests/hermes_state/test_append_messages_batch.py
• tests/tools/test_tool_result_storage.py
• tests/agent/test_context_compressor.py
• tests/run_agent/test_run_agent.py
• tests/test_tui_gateway_server.py

Scope / impact

Affected: incremental SQLite persistence for normal CLI, gateway, and TUI turns, plus TUI branch copies.

No model-tool schema, config key, system-prompt content, or user-facing command surface changes. Full-transcript rewrite paths remain unchanged, and current main's batch-end-only steer delivery remains intact.

Checklist

• Bug fix with real-path regression coverage
• Single commit rebased directly onto latest main
• Conventional commit title
• Reuses the opt-in _row_id contract; no duplicate row-id metadata
• Preserves batched turn flushes and bounded branch-copy transactions
• No unrelated config, documentation, or tool-schema changes
• Tested on Ubuntu / Python 3.11

[youngting520]

Re-ran the regression module on Ubuntu (Linux 6.17, x86_64) via the canonical
runner — all green, including the FTS reindex assertion (sqlite built with FTS5):

scripts/run_tests.sh tests/run_agent/test_steer_sessiondb_persistence.py
→ 7 passed, 0 failed (13.1s)

CI hasn't been able to run on this PR yet — as a cross-repo first-time
contributor it needs a maintainer to approve workflow runs. Happy to address
any review feedback.

[teknium1]

Thanks for tracing this to the incremental-flush marker path. The persistence gap remains on current main: agent/tool_executor.py:996-1005 flushes a tool row before /steer mutates it, while run_agent.py:1850-1851 skips that marker-stamped message on the later flush.

Problems

• tests/run_agent/test_steer_sessiondb_persistence.py:56 uses tempfile.mkdtemp() for the patched Hermes home without cleanup. The test helper should use tmp_path or a TemporaryDirectory owner so test runs do not leave directories behind.

Suggested changes

• Preserve the current-main persistence behavior while salvaging the row-update logic; GitHub currently reports this branch as non-mergeable against main.
• Keep the SQLite/FTS regression coverage, including the direct post-flush mutation case.

This is an automated hermes-sweeper review.