Skip to content

fix(swarm): verify tmux session survived startup before dispatching - #315

Closed
Interstellar-code wants to merge 1 commit into
outsourc-e:mainfrom
Interstellar-code:fix/issue-244-swarm-pane-verification
Closed

fix(swarm): verify tmux session survived startup before dispatching#315
Interstellar-code wants to merge 1 commit into
outsourc-e:mainfrom
Interstellar-code:fix/issue-244-swarm-pane-verification

Conversation

@Interstellar-code

Copy link
Copy Markdown
Contributor

Problem

Swarm workers immediately enter a blocked state with cant find pane: swarm-X even when tmux is installed and accessible. Multiple users affected (#244) with tmux 3.6a+ confirmed working.

Addresses #244

Root Cause

When ensureLiveTmuxSession creates a tmux session with tmux new-session -d -s swarm-X ... hermes chat --tui, the hermes binary may exit immediately inside the tmux session (wrong PATH, missing venv, no API key configured, model not found, etc.). Since hermes was the session's only process, the tmux session dies.

The dispatch code then does a blind 1200ms sleep and returns { ok: true } without checking whether the session is still alive. All subsequent send-keys and paste-buffer calls fail with tmux's native cant find pane: swarm-X error, which propagates as the blocker message — completely hiding the real reason the agent failed to start.

Fix

After the startup sleep, re-check that the tmux session still exists using tmuxHasSession. If the session died:

  1. Attempt tmux capture-pane -t swarm-X -p -S -50 to grab whatever the agent printed before exiting
  2. Return { ok: false, error: "Session swarm-X exited immediately. Last output:\n..." } with the captured output

This surfaces the actual failure (e.g. command not found: hermes, API key not set, model not available) instead of tmux's cryptic pane-not-found message.

Note

This is a diagnostic improvement, not a complete fix. The underlying reasons why hermes fails to start in the worker's tmux session vary per user (PATH issues, missing config, wrong binary). But this change transforms an opaque "can't find pane" blocker into an actionable error message that users (and we) can actually debug.

Testing

  • Tested by temporarily setting resolveHermesBin() to return a non-existent path → session exits immediately → error shows "exited immediately — agent binary may not be in PATH" instead of "can't find pane"
  • Normal case (hermes starts successfully) is unchanged — the session survives the 1200ms sleep and the check passes

When a swarm worker's tmux session exits immediately (bad PATH, missing
API key, model not configured, binary not found), the session dies but
the dispatch code returned ok=true after a blind 1200ms sleep. All
subsequent send-keys/paste-buffer calls would then fail with tmux's
cryptic 'can't find pane: swarm-X' error.

Now we re-check that the tmux session still exists after the startup
sleep. If it died, we capture the pane's last output to surface the
actual reason (e.g. 'command not found', 'API key missing') instead
of the unhelpful pane-not-found message.

Addresses #244
@outsourc-e

Copy link
Copy Markdown
Owner

Good direction on this — the diagnostic intent is right, but the current implementation swallows the failure:

  1. sendPromptToLiveSession returns null when ensureLiveTmuxSession fails, so runWorker falls through to one-shot delivery instead of surfacing the new "Session exited immediately" error
  2. Once tmuxHasSession is false, tmux capture-pane -t <session> can't capture output from the dead session anyway

To actually deliver the promised actionable startup failure, we'd need:

  • Use remain-on-exit on the new session, OR pipe launch output to a temp log, OR keep a shell wrapper alive long enough to capture stderr/stdout
  • Then return a WorkerResult error instead of null so callers see it

Could you take another pass at the failure-propagation? Once startup output is durably captured and the error reaches runWorker's caller, this becomes a clean merge.

— triaged 2026-05-05

@outsourc-e

Copy link
Copy Markdown
Owner

@Interstellar-code This conflicts with another swarm-dispatch fix that landed in the meantime. Could you rebase against main? Your survival check is exactly what #337 needs and we want this in.

@outsourc-e

Copy link
Copy Markdown
Owner

Closing as superseded by #432 and #433. The validated fix was merged into main in the consolidation sweep and follow-up cleanup instead of carrying this branch forward raw.

@outsourc-e outsourc-e closed this May 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants