Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .kilo_workflow/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,7 @@ A section routinely takes hours of unattended machine time. The human cost is mi
|---|---|
| [`WORKFLOW.md`](WORKFLOW.md) | The canonical spec — roles, loops, gates, dispatch commands |
| [`dispatch-role.sh`](dispatch-role.sh) | Launches a kilo role agent in tmux with a clean environment and logged exit code |
| [`steer.sh`](steer.sh) | Delivers a message to a running interactive session (starter, planner, orchestrator) and confirms it was submitted |
| [`e2e-slot.sh`](e2e-slot.sh) | Machine-global semaphore (default 3) capping concurrent device/stack E2E phases; agents are never capped |
| [`learnings/`](learnings/) | Environment blockers and fixes, one file each, committed via PRs so every future run inherits them; `learnings/system/` is gitignored machine-local state |

Expand Down
22 changes: 21 additions & 1 deletion .kilo_workflow/WORKFLOW.md
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,26 @@ Role boundaries — reviewers never modify the tree, the implementer never commi

While a role agent runs, its dispatcher checks on it about every 7 minutes and unsticks infrastructure failures only: a wedged or crashed kilo CLI, a dead tmux window, a hung service the agent cannot restart itself. Product, logic, or review problems are not stuck states — route those through the escalation ladder.

### Steering a Live Interactive Session

Worker roles are steered by re-dispatching them (see Escalation). The interactive sessions — starter, planner, orchestrator — are steered in place with [`steer.sh`](steer.sh), never with hand-assembled `tmux send-keys`:

```bash
.kilo_workflow/steer.sh <section>-orchestrator "Scope change: drop slice 4; commit what holds and open the PR."
printf '%s' "$AMENDMENT" | .kilo_workflow/steer.sh <section>-orchestrator - # long or multi-line text via stdin
```

It prints `running` when the session took the message immediately and `queued` when the message is waiting behind the active turn; either way it is delivered. A non-zero exit means it is **not** delivered — inspect the target rather than sending a second copy. What the script encodes (details in `learnings/steering-a-running-kilo-session.md`):

- Enter as its own keystroke after the text. A trailing `Enter` in the same `send-keys` call submits short messages but is swallowed by long ones, which then sit unsent in the composer — the "wedged" session that is really an undelivered message.
- Bracketed paste, so a multi-line message stays one prompt. An unbracketed paste submits at every newline, and the first fragment gets acted on before the rest arrives.
- A refusal to paste into a pane that is not running the kilo CLI — a mistargeted steer executes in a shell.
- Delivery confirmed from the pane, never assumed.

**`N queued` in the footer is delivery working, not a wedge.** Queued messages land one at a time at turn boundaries, in order, and a session that chains tool calls for tens of minutes holds the whole queue that entire time. Never kill, relaunch, or escalate on a queue count; a wedge needs its own evidence (frozen build timer, stream or api error, dead process — see Planner Monitor Mode). `Escape` does not flush the queue.

Because delivery is ordered and turn-paced, a correction cannot overtake what it corrects: send ONE consolidated, self-contained message per change, never a drip of add-then-retract. When a change must take effect before the current turn ends, kill the session and relaunch it fresh with an updated handoff — faster than waiting on the queue, and it cannot half-apply. Everything a session needs at launch belongs in its launch message and handoff; steering a live one is the exception, not the channel.

### Escalation

When a loop iteration fails, escalate in order:
Expand Down Expand Up @@ -180,7 +200,7 @@ A dead orchestrator window is not automatically a crash — check the scratch di
- Scratch present with `$SCRATCH/final-report.md` → BLOCKED; relay the report to the user and close yourself. Leave the scratch directory alone — it is the blocker's evidence, and its presence is what distinguishes BLOCKED from COMPLETE for anyone who looks later.
- Scratch present with no final report → a crash; relaunch with a continuation handoff.

A live session waiting on a hands-on user answer is not wedged — read the pane before declaring a wedge. Long kilo runs die on provider stream stalls, and `--interactive` sessions can wedge on provider errors. Relaunch a dead or wedged orchestrator as a **fresh session** (never `--continue`) with a continuation handoff. After three consecutive relaunches with no new progress, stop and write the BLOCKED report yourself — the same rule bounds the starter's planner relaunches: the original handoff plus everything observably done so far — commits, PR state, passed rounds, held resources — assembled from `git log`, the PR, and the dispatch logs, so the new session verifies rather than redoes. See `learnings/kilo-interactive-orchestrator-wedges-relaunch.md`.
A live session waiting on a hands-on user answer is not wedged, and neither is one holding queued steers (see Steering a Live Interactive Session) — read the pane before declaring a wedge. Long kilo runs die on provider stream stalls, and `--interactive` sessions can wedge on provider errors. Relaunch a dead or wedged orchestrator as a **fresh session** (never `--continue`) with a continuation handoff. After three consecutive relaunches with no new progress, stop and write the BLOCKED report yourself — the same rule bounds the starter's planner relaunches: the original handoff plus everything observably done so far — commits, PR state, passed rounds, held resources — assembled from `git log`, the PR, and the dispatch logs, so the new session verifies rather than redoes. See `learnings/kilo-interactive-orchestrator-wedges-relaunch.md`.

### 2.1 Plan Reviewer

Expand Down
12 changes: 8 additions & 4 deletions .kilo_workflow/learnings/steering-a-running-kilo-session.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,11 @@
# Steering a running interactive kilo session rarely works — deliver scope before launch
# Steering a running interactive kilo session — use steer.sh, and `N queued` is not a wedge

Symptom: a scope change or steering message sent to a running `kilo run --interactive` session (orchestrator) never takes effect: the pane shows the text but nothing happens, or it sits queued forever while the session loops.
Symptom: a steering message sent to a running `kilo run --interactive` session (planner, orchestrator) never takes effect. Either the pane shows the text but nothing happens, or the footer sits at `N queued` for a long time and the session gets declared wedged and relaunched, losing in-flight work.

Cause: two independent traps. (1) `tmux send-keys` with a long message delivers it as a bracketed paste that sits UNSENT in the input box — the trailing Enter is consumed by the paste, so the message is never submitted. (2) Even a submitted message only enters kilo's interactive queue, which flushes at a turn boundary; a session busy in a long loop may not reach one for a long time, and Escape does not force a flush.
Three separate causes, all reproduced against kilo 7.4.16:

Fix: put everything the session needs in its launch message and handoff — deliver scope changes before launch whenever possible. When steering a live session is unavoidable: send ONE consolidated, self-contained message (not a drip of corrections), send the Enter as a separate `tmux send-keys -t <win> Enter` after the text, then capture the pane and verify the message shows as queued/submitted. If it must take effect now and the session is mid-loop, kill and relaunch fresh with an updated handoff instead of waiting.
1. **A trailing `Enter` in the same `send-keys` call is swallowed by long messages.** `tmux send-keys -t <t> "$MSG" Enter` submitted a 240-character message fine, but a 2.6k one landed in the composer **unsent** — kilo's composer reads a large chunk as a paste and the Enter becomes part of it. A separate `tmux send-keys -t <t> Enter` afterwards submits it.
2. **An unbracketed paste submits at every newline.** `tmux paste-buffer` of a three-line message produced **three** prompts (queue `0 → 3`), so the first fragment ran as its own turn before the rest arrived. `paste-buffer -p` (bracketed) delivered the same text as one prompt (queue `0 → 1`).
3. **`N queued` means delivered, not stuck.** Queued prompts drain one per turn boundary, in order — verified draining `4 queued` once a long bash turn ended. They do not drain between tool calls, so a session chaining tool calls holds the queue for as long as that takes, and `Escape` does not flush it.

Fix: steer with `.kilo_workflow/steer.sh <tmux-target> <message|->`. It sends Enter separately, uses bracketed paste, refuses panes not running the kilo CLI (a mistargeted steer otherwise **executes in a shell** — verified), and confirms delivery from the pane, printing `queued` or `running`. Never treat a queue count as a wedge; a wedge needs a frozen build timer, a stream or api error, or a dead process. When a change must land before the current turn ends, kill and relaunch fresh with an updated handoff instead of waiting — and prefer putting scope in the launch message and handoff so live steering stays the exception.
84 changes: 84 additions & 0 deletions .kilo_workflow/steer.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
#!/usr/bin/env bash
# Deliver a message to a running interactive kilo session (starter, planner,
# orchestrator) and prove it landed.
#
# steer.sh <tmux-target> <message> # message text, or - to read stdin
#
# Traps this exists for (see learnings/steering-a-running-kilo-session.md):
#
# 1. `tmux send-keys -t <t> "$MSG" Enter` submits short messages but NOT long
# ones — kilo's composer reads a large chunk as a paste and swallows the
# trailing Enter, so the text sits unsent in the input box forever. That is
# the "wedged" session agents keep reporting. Enter must arrive as its own
# keystroke, after the text.
# 2. A delivered message only reaches the model at the next turn boundary. The
# footer shows `N queued` until then, which is delivery working, not a wedge.
#
# Exit 0 once the message is submitted; prints `queued` (waiting for the current
# turn to end) or `running` (the session picked it up immediately). Exit 1 with
# the pane tail if it could not be submitted — the message may be sitting in the
# composer, so fix the target rather than sending a second copy.
set -euo pipefail

TARGET=${1:?tmux target — session, window or pane}
MSG=${2:?message text, or - for stdin}
[ "$MSG" = "-" ] && MSG=$(cat)
[ -n "${MSG//[[:space:]]/}" ] || { echo "steer: empty message" >&2; exit 1; }

CMD=$(tmux display-message -p -t "$TARGET" '#{pane_current_command}' 2>/dev/null || true)
[ -n "$CMD" ] || { echo "steer: no such tmux target: $TARGET" >&2; exit 1; }
# A mistargeted steer pasted into a shell pane RUNS as a shell command. Only
# panes running the kilo CLI (node, or bun on a source checkout) are steerable.
case $CMD in
node | bun | kilo) ;;
*)
echo "steer: $TARGET is running '$CMD', not a kilo CLI — refusing to paste into it" >&2
exit 1
;;
esac

# -J joins wrapped lines, so a needle survives the pane's width.
pane() { tmux capture-pane -pJ -t "$TARGET"; }
# The footer renders `<n> queued`; absent means nothing is waiting.
queued() { pane | grep -oE '[0-9]+ queued' | tail -1 | cut -d' ' -f1 || true; }
# First line of the message, whitespace-collapsed, as the needle for the
# submitted `›`-prefixed echo in the scrollback.
needle=$(printf '%s' "${MSG%%$'\n'*}" | tr -s '[:space:]' ' ' | cut -c1-60)
# Still unsent: the needle is on screen on a line the scrollback could not have
# written — a submitted message is only ever echoed `›`-prefixed.
composed() { pane | grep -F "$needle" | grep -qv '^›'; }
# Per-invocation buffer: the tmux server's buffers are shared, so a fixed name
# lets two sections steering at once paste each other's message.
BUF=kilo-steer-$$

before=$(queued); before=${before:-0}

# load-buffer + paste-buffer, not send-keys: the message travels as data, so a
# literal `Enter`, `;` or `C-c` in the text cannot be read as a key name. `-p`
# wraps it in bracketed paste, which keeps a multi-line message as ONE prompt —
# an unbracketed paste submits at every newline, so a three-line steer arrives
# as three prompts and the first fragment gets acted on alone.
tmux load-buffer -b "$BUF" - <<<"$MSG"
tmux paste-buffer -d -p -b "$BUF" -t "$TARGET"

# Separate keystroke, after the composer has settled — trap 1.
for attempt in 1 2 3; do
# Re-press Enter only while the text is demonstrably still in the composer.
# Failing to *confirm* a submission is not evidence the Enter was lost, and a
# blind resend would submit whatever sits in the composer by then.
if [ "$attempt" -gt 1 ] && ! composed; then break; fi
sleep 1
tmux send-keys -t "$TARGET" Enter
for _ in 1 2 3 4; do
sleep 1
now=$(queued); now=${now:-0}
if [ "$now" -gt "$before" ]; then echo "queued"; exit 0; fi
# Submitted and started: the scrollback echoes it as a user message. Match
# only lines the composer cannot produce, so unsent text is never a pass.
if pane | grep -qF "› $needle"; then echo "running"; exit 0; fi
done
done

echo "steer: submission unconfirmed — inspect $TARGET before resending" >&2
pane | grep -vE '^ *$' | tail -15 >&2
exit 1
Loading