fix(agent): terminal activity stamp to prevent stuck last_activity_at - #80832
fix(agent): terminal activity stamp to prevent stuck last_activity_at#80832yingliang-zhang wants to merge 1 commit into
Conversation
|
This was generated by AI during triage. Summary: Problems:
Solution: Checked against |
de26441 to
92ca79d
Compare
|
Thanks @spfcraze for the accurate triage — good catch. I've updated the PR description to reflect the actual code structure: |
f5cff89 to
4abeb30
Compare
|
Rebased onto latest main ( |
47862b2 to
441dccd
Compare
The turn finally block called _reset_activity_labels_after_turn() which cleared description/provenance but never force-persisted last_activity_at. When all intermediate heartbeats were rate-limited (60s cadence) or silently dropped (0.5s write patience under DB contention), the timestamp stuck at the last successful heartbeat — breaking Desktop UI freshness detection and hiding the last N messages from the user. Fix: replace _reset_activity_labels_after_turn() in the turn finally with _finalize_activity_after_turn(), which: - Calls new SessionDB.finalize_session_activity() — one atomic UPDATE that stamps last_activity_at AND clears labels in the same write - Uses elevated write patience (2.0s vs 0.5s) since the turn is over and there is no response-critical path to protect - Does NOT call _touch_activity (avoids kanban bridge side-effect that could trigger a spurious continuation turn on a finished turn) - Does NOT bump in-memory _last_activity_ts (preserves NousResearch#15654 interrupt-recursive watchdog continuity) _reset_activity_labels_after_turn() is kept unchanged — compression and other consumers still use it. Tri-model review: K3/GLM/DSF 3/3 NEEDS_FIXES on original proposal, converged on (b)+(c) combined approach (dedicated method + atomic write).
1286f49 to
14cac94
Compare
Problem
The Desktop UI depends on
last_activity_atin the session DB to detect session freshness and trigger message list reloads. When this timestamp gets stuck, the UI stops showing new messages — the user sees "output swallowed" even though the messages are correctly persisted in the DB.Root Cause
The turn
finallyblock called_reset_activity_labels_after_turn()which clearedlast_activity_descriptionandlast_activity_provenancebut never force-persistedlast_activity_at.Mid-turn heartbeats via
_touch_activity()are rate-limited to one write per 60s (SESSION_ACTIVITY_HEARTBEAT_MIN_INTERVAL_SECONDS), and each write has a 0.5s patience budget (_ACTIVITY_WRITE_PATIENCE_S) to avoid blocking the response-critical path. Under DB contention (large state.db, many read connections), the 0.5s budget is exhausted and the write is silently dropped (fail-open by design).When ALL intermediate heartbeats in a turn are either rate-limited or dropped, and the turn
finallydoesn't force-persist,last_activity_atstays at the last successful heartbeat — potentially many messages behind.Fix
Replace
_reset_activity_labels_after_turn()in the turnfinallyblock with the new_finalize_activity_after_turn():SessionDB.finalize_session_activity()(new,hermes_state.py)UPDATEthat stampslast_activity_atAND clears labels in the same writeWHERE last_activity_at < ?) prevents backwards movementAIAgent._finalize_activity_after_turn()(new,run_agent.py)_touch_activity— avoids the kanban worker bridge (inject_new_comments_from_env) which could trigger a spurious continuation turn on a finished turn_last_activity_ts— preserves Cached agent reuse silently disables the inactivity timeout, producing a Still working iteration 0/60 (cached) loop after user interrupt #15654 interrupt-recursive watchdog continuity_reset_activity_labels_after_turn()— retained without a production callerOnce this PR lands,
_reset_activity_labels_after_turn()has no production callers (compression's label handling goes through_touch_activity(..., force_persist=True)andclear_session_activity_labels, not through this function). It is retained for test-only usage and as a utility for future callers that need label-only clearing without timestamp stamping.Tests
New test file
tests/run_agent/test_finalize_activity_after_turn.py(8 tests):_last_activity_tsis NOT bumped (Cached agent reuse silently disables the inactivity timeout, producing a Still working iteration 0/60 (cached) loop after user interrupt #15654 watchdog continuity)_last_activity_tsvalue (not a freshtime.time())_reset_activity_labels_after_turnstill works unchangedAll 22 tests pass (8 new + 14 existing in
test_session_activity_persist.py).