fix: TOCTOU race in close_sessions_for_transport

[eaglezzz0522-cloud]

Summary

The transport-disconnect teardown in _close_sessions_for_transport had a TOCTOU (time-of-check-time-of-use) race: it took a snapshot of owned sessions inside _sessions_lock, then iterated over them without re-validating the transport. A concurrent session.resume reconnect could rebind a session to a new transport between the snapshot and the close/detach.

Changes

• tui_gateway/server.py: snapshot session IDs under _sessions_lock, then re-validate each sessions transport under _session_resume_lock before acting
• If a session was already torn down or reattached since the snapshot, skip it
• _session_resume_lock is the same lock used by session.resumes warm-reuse rebind, so the race is eliminated

Verification

• The lock ordering matches the existing _session_resume_lock usage in _reuse_live_payload
• _teardown_popped_session is used instead of the removed _close_session_by_id for the close path
• Grace-orchan reaper (_schedule_ws_orphan_reap) still fires for detached sessions

Closes #77192

[alt-glitch]

This was generated by AI during triage.

Duplicate of #77129 — it implements the same transport-ownership revalidation and adds focused regression coverage.

[GottZ]

This was generated by AI during triage.

Summary

Seven PRs address or reference the same WS disconnect/session.resume transport-ownership TOCTOU reported by the issue complex. #77129 carries the recorded best-fix verdict for the underlying issues and the corrected race-interleaving regression test; #53525 proposes the same core repair with broader transport-rebind coverage, while the remaining PRs are duplicates or stacked variants.

Related pull requests

• fix(gateway): preserve rebound ws sessions during teardown #53525 fixes — (+137/-23) — keep_open: fixes teardown revalidation and extends the shared ownership boundary to prompt.submit, live-session payload, resume, and queued-prompt rebinds, with a deterministic prompt.submit regression test. It remains relevant despite the automated keep_open verdict because its salvageable broader coverage addresses rebinding paths beyond the narrower fix(tui-gateway): close WS disconnect/reconnect session race #77129 repair.
• fix(tui-gateway): close WS disconnect/reconnect session race #77129 best fix — (+106/-14) — best_fix for the underlying complex: revalidates transport ownership under _session_resume_lock -> _sessions_lock and includes the corrected test that starts on old_transport and rebinds during the actual snapshot/claim window. It remains relevant as the consolidated implementation explicitly retained by the [contributor:50 commits] discussion, superseding the stacked alternatives.
• fix(tui-gateway): close WS disconnect/reconnect TOCTOU race (RAH-06) #77205 [closed] best fix — (+73/-14) — duplicate of fix(tui-gateway): close WS disconnect/reconnect session race #77129, closed: it carries the same two-file transport-ownership revalidation repair and was recombined with test(tui-gateway): reproduce actual WS reattach-race interleaving (RAH-05) #77206 into fix(tui-gateway): WS disconnect/reconnect TOCTOU fix + real regression test (RAH-05 + RAH-06) #77212, so it remains relevant only as the closed RAH-06 reference variant.
• test(tui-gateway): reproduce actual WS reattach-race interleaving (RAH-05) #77206 [closed] best fix — (+106/-14) — duplicate of fix(tui-gateway): close WS disconnect/reconnect session race #77129, closed: it supplies the real interleaving regression test on top of the same fix, but the [contributor:50 commits] discussion recombined it with fix(tui-gateway): close WS disconnect/reconnect TOCTOU race (RAH-06) #77205 into fix(tui-gateway): WS disconnect/reconnect TOCTOU fix + real regression test (RAH-05 + RAH-06) #77212 and then consolidated the result into fix(tui-gateway): close WS disconnect/reconnect session race #77129.
• fix(tui-gateway): WS disconnect/reconnect TOCTOU fix + real regression test (RAH-05 + RAH-06) #77212 [closed] best fix — (+106/-14) — duplicate of fix(tui-gateway): close WS disconnect/reconnect session race #77129, closed: it combines the RAH-06 repair and RAH-05 regression test, but the [contributor:50 commits] discussion explicitly closed it in favor of fix(tui-gateway): close WS disconnect/reconnect session race #77129, which carries the consolidated fix and test.
• fix(ws): TOCTOU race in close_on_disconnect #77535 duplicate — (+29/-12) — duplicate of fix(tui-gateway): close WS disconnect/reconnect session race #77129: it implements the same snapshot, lock-order, ownership-revalidation, and teardown changes without the regression coverage carried by fix(tui-gateway): close WS disconnect/reconnect session race #77129. Keep it only as a reference; its claims are subsumed by the earlier consolidated best-fix PR.
• fix: TOCTOU race in close_sessions_for_transport #77570 partial — (+29/-12) — duplicate of fix(tui-gateway): close WS disconnect/reconnect session race #77129: its diff is materially identical to fix(ws): TOCTOU race in close_on_disconnect #77535 and lacks the focused regression test present in fix(tui-gateway): close WS disconnect/reconnect session race #77129. The [contributor:184 commits] duplicate note supports closure, while fix(tui-gateway): close WS disconnect/reconnect session race #77129 remains the consolidated reference implementation.

Duplicates

#77205, #77535, and #77570 duplicate the core repair in #77129; #77206 and #77212 are stacked/recombined variants of that repair plus the RAH-05 test. #53525 overlaps on the core fix but retains a distinct broader transport-rebind salvage path.

Suggested consolidation

Keep open #77129 with a salvage path as the consolidated implementation: retain its fix plus the deterministic real-interleaving regression test, consistent with the recorded best-fix verdicts and the [contributor:50 commits] consolidation decision. Keep #53525 open with a salvage path for its broader prompt.submit/live-payload/queued-prompt ownership-boundary coverage; close #77570 and #77535 as duplicates of #77129, and retain #77205, #77206, and #77212 as closed duplicate/stacked references.

Complex graph

flowchart TD
    classDef open fill:#dbeafe,stroke:#1d4ed8,color:#1e3a8a
    classDef merged fill:#dcfce7,stroke:#15803d,color:#14532d
    classDef closed fill:#e5e7eb,stroke:#6b7280,color:#1f2937
    classDef unverified fill:#f3f4f6,stroke:#9ca3af,color:#374151
    classDef best stroke-width:3px,stroke:#b45309
    classDef target stroke-width:3px,stroke:#4338ca
    I77127(["issue #77127 (open)"])
    I77192(["issue #77192 (open)"])
    subgraph Dup53525 ["PRs duplicating each other"]
        P53525["PR #53525 (open)"]
        P77129["PR #77129 (open)"]
        P77205["PR #77205 (closed)"]
        P77206["PR #77206 (closed)"]
        P77212["PR #77212 (closed)"]
        P77535["PR #77535 (open)"]
        P77570["PR #77570 (open)"]
    end
    P77570 -.->|partial| I77127
    P77570 -.->|partial| I77192
    class I77127 open
    class I77192 open
    class P53525 open
    class P77129 open
    class P77205 closed
    class P77206 closed
    class P77212 closed
    class P77535 open
    class P77570 open
    class P77129 best
    class P77129 best
    class P77129 best
    class P77205 best
    class P77206 best
    class P77212 best
    class P77212 best
    class P77570 target
    click I77127 "https://github.com/NousResearch/hermes-agent/issues/77127"
    click I77192 "https://github.com/NousResearch/hermes-agent/issues/77192"
    click P53525 "https://github.com/NousResearch/hermes-agent/pull/53525"
    click P77129 "https://github.com/NousResearch/hermes-agent/pull/77129"
    click P77205 "https://github.com/NousResearch/hermes-agent/pull/77205"
    click P77206 "https://github.com/NousResearch/hermes-agent/pull/77206"
    click P77212 "https://github.com/NousResearch/hermes-agent/pull/77212"
    click P77535 "https://github.com/NousResearch/hermes-agent/pull/77535"
    click P77570 "https://github.com/NousResearch/hermes-agent/pull/77570"

Loading

Graph: solid arrow = fixes / best fix, dashed arrow = partial or unverified (see edge label); boxed group = PRs duplicating each other; amber border = best fix; indigo border = target; gray node = closed (state tag in the node label).

Cross-PR triage: Reviewed 7 pull requests and 3 issues in this complex. Each diff was read against this issue; Assessment working set: 49 kB of PR diffs, 27 kB of issue/PR text, 6 kB of discussion (15 comments), 27 verify verdicts. verdicts reflect diff content, not PR titles. Part of an automated triage batch.

[eaglezzz0522-cloud]

Closing as duplicate of the earlier first-round PR (#77534 / #77129 / #77536 / #77537 respectively). The original individual PRs remain open for review.