Skip to content

fix(tui): reschedule WS orphan reap while a turn is still running - #85598

Open
686f6c61 wants to merge 1 commit into
NousResearch:mainfrom
686f6c61:fix/85578-ws-orphan-reschedule
Open

fix(tui): reschedule WS orphan reap while a turn is still running#85598
686f6c61 wants to merge 1 commit into
NousResearch:mainfrom
686f6c61:fix/85578-ws-orphan-reschedule

Conversation

@686f6c61

@686f6c61 686f6c61 commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

What breaks

Desktop / TUI: disconnect while a turn is still running. The session stays in _sessions forever (ended_at NULL, no error). The in-memory agent is never released. Seen on v0.20.0, Linux/Docker. Config timeouts (gateway_timeout, session_stall_timeout, max_live_sessions) do not help — every eviction path skips running.

This is an eighth leak next to the family in #84047 (closest is F, but distinct).

Why

handle_ws detach points the session at _detached_ws_transport and arms one threading.Timer (_schedule_ws_orphan_reap, default 20s).

_ws_session_is_orphaned() is:

if session.get("running"):
    return False
return session.get("transport") is _detached_ws_transport

If the timer fires mid-turn, _reap() hits not orphaned and returns. The active-delegation branch one line below correctly calls _schedule_ws_orphan_reap again; the running branch did not. After that, nothing re-arms the timer. The 6h TTL reaper also treats running as non-evictable. If the turn never clears from the leak’s point of view, the session is immortal.

What this changes

A detached session that is still running reschedules the same grace timer (same as delegations). When the turn later clears, the next fire sees a real orphan and tears down.

Live transports and finalized sessions still return immediately (no reschedule loop).

Verify

pytest tests/test_tui_gateway_server.py::test_ws_orphan_reap_reschedules_while_mid_turn_then_reaps
# 1 passed

Fake Timer, session starts running=True + detached transport: first fire keeps it and arms again; after running=False, second fire teardowns with ws_orphan_reap.

Fixes #85578

The grace timer treated mid-turn detached sessions as not-orphaned and
returned without arming another timer. Eviction paths also skip
running sessions, so the in-memory agent leaked until process restart.

Fixes NousResearch#85578
@alt-glitch alt-glitch added type/bug Something isn't working comp/tui Terminal UI (ui-tui/ + tui_gateway/) P2 Medium — degraded but workaround exists duplicate This issue or pull request already exists sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state area/sessions Session lifecycle, resume, persistence, history labels Aug 13, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

This was generated by AI during triage.

Duplicate of #44102: the same detached mid-turn WS orphan-reap timer is re-armed so the session can be reaped after the turn finishes.

@686f6c61

Copy link
Copy Markdown
Contributor Author

Same bug as open #44102 (fixes #44045) — mid-turn detached sessions spare the one-shot grace timer and never get a second look.

#44102 is the earlier write-up; it is now CONFLICTING. Current main _reap() already has the delegation reschedule + _pop_session_by_id / _teardown_popped_session split, which that patch does not have (it still calls _close_session_by_id under the old layout).

This tip is that re-arm, folded into the existing reschedule path so mid-turn and live delegations share one timer. Happy to close this as a dupe if maintainers prefer rebasing #44102 instead — credit for the diagnosis stays there.

@Enough1122

Copy link
Copy Markdown
Contributor

AI code review — automated review for reference, author can ignore or act on any point.

fix(tui): reschedule WS orphan reap while a turn is still running

  1. tui_gateway/server.py:1091-1099 — the mid-turn branch defers the reap while running is true. If a turn wedges (hung model call with no timeout), the session is never reaped: the 6h TTL also skips running sessions, so a permanently-running session becomes effectively immortal. Consider bounding consecutive mid-turn reschedules (e.g. reap after N deferrals regardless) or tying the deferral to a maximum turn duration.

  2. The branch reads current.get("running") as a coarse "mid-turn" proxy; async background work is handled separately via _session_has_active_delegations. Worth confirming running is cleared promptly when a turn ends on every exit path (including error paths), or the reap will keep rescheduling after the turn is actually done.

  3. The new branch sits before the orphan check — correct for the fix, and a session that reconnects mid-turn (transport re-bound, so is _detached_ws_transport is false) correctly falls through to the orphan check and returns without rescheduling. A test covering the reconnect-mid-turn path would lock that behavior in.

@Enough1122

Copy link
Copy Markdown
Contributor

@C:/Users/admin/AppData/Local/Temp/opencode/review-85598.md

@alt-glitch alt-glitch added duplicate This issue or pull request already exists and removed duplicate This issue or pull request already exists labels Aug 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/sessions Session lifecycle, resume, persistence, history comp/tui Terminal UI (ui-tui/ + tui_gateway/) duplicate This issue or pull request already exists P2 Medium — degraded but workaround exists sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

tui_gateway: WS-orphan reaper never reschedules for a mid-turn session, leaving it permanently unreapable (v0.20.0)

3 participants