fix(ssh): reconnecting keeps the remote server and its agents running

[t3dotgg]

A desktop SSH reconnect (laptop sleep, Wi-Fi drop, app restart) could stop the managed T3 server on the remote host and end every agent running there. It had three causes:

1. Self-kill. The managed server runs on the default home, so it writes ~/.t3/userdata/server-runtime.json itself. The launch script took that record for another server and stopped its own managed PID. This happened on every reconnect.
2. Readiness kill. A live managed server got a 2-second reuse probe. A server that was only busy for longer was stopped and replaced.
3. Stale forward. After sleep or a network drop, the dead local forward was closed through the path that also stops the remote server.

Fix

• When server-runtime.json names the PID this launcher recorded as managed, the script reuses that server and takes its port from the record. (fix(ssh): reconnecting no longer restarts the managed remote server #17684)
• A live managed server gets the same 60-second budget as a cold start. If it still does not answer, the launch fails with a retryable error and the server keeps running. A changed runner (an app update) still replaces it. (fix(ssh): keep remote agents running across reconnects #17919)
• Replacing a stale forward closes only the local tunnel. Disconnect and Remove still stop the server. (fix(ssh): keep remote agents running across reconnects #17919)
• A managed PID that is a zombie (crashed but not reaped) counts as gone, so reconnect starts a new server instead of waiting on a dead one.

#17919 also changed the reuse order and made adoption of another server wait 60 seconds with no launch fallback. That part is not included: #17684's PID check already stops the self-kill, and keeping main's adoption fallback means a stale runtime record cannot block reconnects. With #16102, that fallback cannot start a second server on the same home, because the new server exits with code 78.

Verification

• New and imported tests run the real launch script against real processes. They fail on main and pass here: own server reused, busy server (stopped, resumed after 5 s) waited for, unresponsive server left running, zombie managed server replaced, stale forward replaced without a remote stop, adopted server not probed again. The update-path and no-marker cases pass on both.
• End to end with the real server (fix(server): enforce one database owner and guard update rollback #16102 lock, real runtime record) in an isolated home:
  • main: each reconnect started a new server (3 reconnects, 3 PIDs).
  • this branch: reconnects kept the same PID. A 6 s stall was waited out (6.0 s). A server that never answered failed the launch after 60.6 s and kept running. A changed runner replaced the server, and the replacement took the lock. A separately started server was adopted, and the stop script left it alone.
  • fix(ssh): reconnecting no longer restarts the managed remote server #17684 alone: a 6 s stall still killed the server, and its replacement exited 78.
• vp test run packages/ssh/src/tunnel.test.ts, lint, and @t3tools/ssh typecheck pass. "archive runner script > installs once…" flakes locally on clean main too. That installer-lock code is unchanged here.
• Not verified: a real SSH hop, a macOS remote, the archive-mode helper binary (it uses the same checks as the node path), and the desktop app's stale-forward path outside the unit test.
• Known limit: during a stall longer than the client's 15 s connection setup budget, the first attempt reports a timeout. A later retry reuses the tunnel the desktop finished building. Showing the launch's own message there is a follow-up.

Fixes #15986. Fixes #16477.

Combines #17684 by @puccez and #17919 by @luke2x. Their commits are kept with authorship. This supersedes both PRs and overlaps #13521.

Combined, tested, and reviewed by Claude Opus 5.5 in Claude Code (T3 Code). Reviewed with sol-loop: 2 rounds with GPT-6.1-Sol.

Co-authored-by: puccez 133161547+puccez@users.noreply.github.com
Co-authored-by: luke2x 97411838+luke2x@users.noreply.github.com

🤖 Generated with Claude Code

[macroscopeapp]

Approvability

Verdict: Approved at 4572c6b

Macroscope's review found this PR approvable — This is a localized SSH reconnect bug fix that prevents stale tunnel cleanup from stopping the remote server or its agents, while preserving explicit disconnect and update replacement behavior. The added tests cover the affected lifecycle scenarios, including busy, adopted, and zombie processes.

^{You can add or adjust custom eligibility rules. Learn more.}

[github-actions]

Thread transfer impact

✅ Thread transfer remains within every enforced ceiling.

Provider	Metric	Main baseline	This PR	Impact	PR ceiling
Codex	Total thread wire	5.0 KiB	5.0 KiB	+7 B (+0.1%)	6.8 KiB	✅
Codex	Thread snapshot wire	3.8 KiB	3.8 KiB	0 B (0.0%)	4.9 KiB	✅
Codex	Live turn WebSocket wire	1.2 KiB	1.2 KiB	+7 B (+0.6%)	2.0 KiB	✅
Codex	Live turn WebSocket decoded	20.9 KiB	20.9 KiB	0 B (0.0%)	29.3 KiB	✅
Codex	Live turn messages	2	2	0 (0.0%)	8	✅
Claude	Total thread wire	5.0 KiB	5.0 KiB	−7 B (−0.1%)	6.8 KiB	✅
Claude	Thread snapshot wire	3.8 KiB	3.8 KiB	0 B (0.0%)	4.9 KiB	✅
Claude	Live turn WebSocket wire	1.2 KiB	1.2 KiB	−7 B (−0.6%)	2.0 KiB	✅
Claude	Live turn WebSocket decoded	21.2 KiB	21.2 KiB	0 B (0.0%)	29.3 KiB	✅
Claude	Live turn messages	2	2	0 (0.0%)	8	✅

Baseline: 8414d16 · PR result: 4572c6b · Source CI: success

Scenario and decoded snapshot size

10 historical turns, 5 command tools per turn, 878.9 KiB retained MCP result per historical turn, and a 1.05 MiB retained result in the measured turn.

• Codex decoded thread snapshot: 108.5 KiB
• Claude decoded thread snapshot: 108.8 KiB

Updated in place by a trusted workflow. PR artifacts are strictly validated and never executed.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

• Configuration used: Path: .coderabbit.config.ts
• Review profile: CHILL
• Plan: Team
• Run ID: 3e78fa38-0088-437e-87d2-e2342152ab93

📥 Commits

Reviewing files that changed from the base of the PR and between 8414d16 and 4572c6b.

📒 Files selected for processing: 2

• packages/ssh/src/tunnel.test.ts
• packages/ssh/src/tunnel.ts

Limit details: You’ve used all 10 included reviews currently available.

📝 Walkthrough

The SSH launch script now distinguishes its managed server from other default-runtime servers, treats zombie PIDs as stopped, and retains a live managed server when readiness checks time out. Reconnect cleanup also removes stale tunnel entries before closing them. Tests exercise these cases with shell processes.

Changes: SSH reconnect lifecycle

Layer / File(s)	Summary
Remote server reuse and handover packages/ssh/src/tunnel.ts, packages/ssh/src/tunnel.test.ts	The launcher checks process state, preserves a managed server whose PID matches the default runtime record, and adopts other ready default-runtime servers. It retains a managed server when readiness checks time out, while runner changes still trigger replacement. Tests cover reuse, adoption, readiness failures, runner changes, and exited processes.
Stale local tunnel cleanup packages/ssh/src/tunnel.ts, packages/ssh/src/tunnel.test.ts	Reconnect removes a stale tunnel entry before closing the tunnel. A test checks that replacing the local forward does not stop the remote server, while explicit disconnect does.

Priority: ⬆️ High

Estimated code review effort: 4 (Complex) | ~45 minutes

Change: Bug fix · Severity of issue fixed: High

Sequence Diagram(s)

sequenceDiagram
  participant SSHClient
  participant REMOTE_LAUNCH_SCRIPT
  participant DEFAULT_RUNTIME_FILE
  participant RemoteServer
  SSHClient->>REMOTE_LAUNCH_SCRIPT: Start reconnect launch
  REMOTE_LAUNCH_SCRIPT->>DEFAULT_RUNTIME_FILE: Read runtime PID and port
  DEFAULT_RUNTIME_FILE-->>REMOTE_LAUNCH_SCRIPT: Return runtime record
  REMOTE_LAUNCH_SCRIPT->>RemoteServer: Check process liveness and readiness
  RemoteServer-->>REMOTE_LAUNCH_SCRIPT: Return liveness and readiness results
  REMOTE_LAUNCH_SCRIPT-->>SSHClient: Reuse, adopt, replace, or report launch failure

Loading

Suggested reviewers: juliusmarminge

Merge Risk: ⚪ Minimal · up to 4572c

The inspected reconnect paths preserve the remote server when it is slow to respond and replace a stale local forward without stopping it. No issue identified here needs resolution before merge.

Pre-merge checks: 3 · 1

Failed checks: 1 warning

Check name	Status	Explanation	Resolution
Linked Issues check		The changes satisfy #15986: when server-runtime.json reports the PID recorded by this launcher, the script reuses that server and its recorded port. server_running also treats zombie processes as …	Implement the #16477 policy for runner changes and conflicting default-runtime adoption so a live managed server with active work is not stopped during reconnect. Defer replacement, request confirmation, or use another issue-approved behavi…

Passed checks: 3 passed

Check name	Status	Explanation
Title check		The title is concise, specific, and accurately describes the primary change: preserving the remote server and its agents during SSH reconnects.
Description check		The description clearly explains the problem, the three causes, the implemented fixes, scope, verification results, limitations, and linked issues. It uses a "Fix" heading instead of the template's "C…
Out of Scope Changes check		The stale-forward change closes only the local tunnel and supports the linked reconnect objective of preserving the remote server. The launch-script tests exercise the linked PID, readiness, zombie, a…

Full details: Linked Issues check

Explanation

The changes satisfy #15986: when server-runtime.json reports the PID recorded by this launcher, the script reuses that server and its recorded port. server_running also treats zombie processes as stopped. The changes satisfy the slow-readiness part of #16477: a live managed server receives the full readiness timeout and remains running after an unsuccessful probe. However, #16477 also identifies runner changes and another live default-runtime server as cases that can stop a managed server with active work. The script still kills REMOTE_PID when RUNNER_CHANGED=1, and it still kills a managed server when it adopts a different ready default-runtime server. The PR description confirms that the defer-or-prompt behavior is not included.

Resolution

Implement the #16477 policy for runner changes and conflicting default-runtime adoption so a live managed server with active work is not stopped during reconnect. Defer replacement, request confirmation, or use another issue-approved behavior. Add automated coverage for these cases.

• Fix all pre-merge checks with AI

✨ Finishing Touches

📝 Generate docstrings

• Commit to this branch
• Create a new PR

🧪 Generate unit tests (beta)

• Commit to this branch
• Create a new PR

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Usage-based review receipt

• Mode: Continue automatically
• Reviewed files: 2
• Charged: $0.50
• View usage details

Note

This review was completed with usage-based billing: files reviewed beyond your plan's included limits are billed at $0.25/file. View usage-based billing.

_{Comment @coderabbitai help to get the list of available commands.}

[t3dotgg]

Note

🤖 Claude Opus 5.5 responding on behalf of Theo

The linked-issues warning is a deliberate scope limit, not a gap in the #16477 fix.

• RUNNER_CHANGED is the update path. The #16477 triage leaves deferring updates during active runs as a separate maintainer decision. Julius also asked #17919 to keep that restart. The test "still replaces its managed server when the runner changes" covers it.
• The adoption handover is unchanged from main. It stops the recorded managed PID only when a different live server already owns ~/.t3. Since fix(server): enforce one database owner and guard update rollback #16102, a current server cannot start on a home that our managed server owns. With current versions on both sides, a reconnect does not reach this branch while that server is running.

The case #16477 reports, a busy server killed by the 2 s probe, is fixed and covered by the busy-server and never-ready tests.