fix(swarm): verify tmux session survived startup before dispatching - #315
Closed
Interstellar-code wants to merge 1 commit into
Closed
Conversation
When a swarm worker's tmux session exits immediately (bad PATH, missing API key, model not configured, binary not found), the session dies but the dispatch code returned ok=true after a blind 1200ms sleep. All subsequent send-keys/paste-buffer calls would then fail with tmux's cryptic 'can't find pane: swarm-X' error. Now we re-check that the tmux session still exists after the startup sleep. If it died, we capture the pane's last output to surface the actual reason (e.g. 'command not found', 'API key missing') instead of the unhelpful pane-not-found message. Addresses #244
Owner
|
Good direction on this — the diagnostic intent is right, but the current implementation swallows the failure:
To actually deliver the promised actionable startup failure, we'd need:
Could you take another pass at the failure-propagation? Once startup output is durably captured and the error reaches — triaged 2026-05-05 |
Owner
|
@Interstellar-code This conflicts with another swarm-dispatch fix that landed in the meantime. Could you rebase against |
Owner
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Swarm workers immediately enter a blocked state with
cant find pane: swarm-Xeven when tmux is installed and accessible. Multiple users affected (#244) with tmux 3.6a+ confirmed working.Addresses #244
Root Cause
When
ensureLiveTmuxSessioncreates a tmux session withtmux new-session -d -s swarm-X ... hermes chat --tui, the hermes binary may exit immediately inside the tmux session (wrong PATH, missing venv, no API key configured, model not found, etc.). Since hermes was the session's only process, the tmux session dies.The dispatch code then does a blind 1200ms sleep and returns
{ ok: true }without checking whether the session is still alive. All subsequentsend-keysandpaste-buffercalls fail with tmux's nativecant find pane: swarm-Xerror, which propagates as the blocker message — completely hiding the real reason the agent failed to start.Fix
After the startup sleep, re-check that the tmux session still exists using
tmuxHasSession. If the session died:tmux capture-pane -t swarm-X -p -S -50to grab whatever the agent printed before exiting{ ok: false, error: "Session swarm-X exited immediately. Last output:\n..." }with the captured outputThis surfaces the actual failure (e.g.
command not found: hermes,API key not set,model not available) instead of tmux's cryptic pane-not-found message.Note
This is a diagnostic improvement, not a complete fix. The underlying reasons why hermes fails to start in the worker's tmux session vary per user (PATH issues, missing config, wrong binary). But this change transforms an opaque "can't find pane" blocker into an actionable error message that users (and we) can actually debug.
Testing
resolveHermesBin()to return a non-existent path → session exits immediately → error shows "exited immediately — agent binary may not be in PATH" instead of "can't find pane"