Skip to content

fix: recover Claude away-mode digest delivery on Herdr - #96

Merged
withally merged 4 commits into
mainfrom
fm/fm-afk-inject-wedge-w2
Aug 31, 2026
Merged

withally merged 4 commits into
mainfrom
fm/fm-afk-inject-wedge-w2

Conversation

@withally

Copy link
Copy Markdown
Owner

Intent

Fix away-mode digest injection wedging on a Claude primary running on Herdr. Reproduce live first in a named isolated Herdr lab through the real native /afk path; capture ANSI rows and daemon sub-causes; verify current main delivery and reconstruct Incident B to identify the exact pre-PR-94 matched row, treating Incident C as the same family unless evidence differs. Separate trigger, masking condition, and symptom, and prove the smallest row or footer-token counterfactual. Keep one fleet-wide Claude classifier/composer-boundary owner and preserve genuine-turn rendered-busy deferral plus bright human composer pending deferral; every deferral must log native-busy, rendered-busy with the matched row, or composer verdict. Add narrow fail-safe post-max-alarm recovery: only after a rendered-busy matched row is byte-identical for N polls while native state is not working or working only because of the tracked background shell, re-read the composer and proceed once only when affirmatively empty, never pending or unknown. Add portable regressions for exact Incident B bypass-permissions, agent-count, Update installed, dim suggestion, and dark truecolor ghost rows as idle while genuine spinner and esc-to-interrupt rows remain busy. Extend and run the live opt-in guard to prove exactly one idle post-/afk delivery, real foreground-turn deferral, and typed-text deferral against installed Claude and Herdr; update both verification docs with dated exact commands, versions, and outputs; run lint and tests. Keep one fix per PR. A separately observed prompt-submit-to-spinner delivery race is an independent follow-up recorded in the report and excluded from this PR. Cap review-gate rounds at two; residual findings become PR follow-ups.

What Changed

  • Scoped Claude busy detection to the current Herdr context with ANSI-aware composer classification, preserving genuine foreground-turn and human-text deferrals while logging exact matched rows.
  • Added fail-safe post-alarm recovery requiring a byte-identical elapsed busy row, safe native state, and an affirmatively empty composer before one submit.
  • Expanded portable and live Herdr/Claude regressions and updated dated verification documentation, including Incident B evidence and the excluded prompt-submit race follow-up.

Risk Assessment

🚨 High: Captain, the post-alarm recovery can send without the required raw-row stability proof, while the change also regresses generic ANSI matching and leaves a required verification record non-reproducible.

Testing

Fresh portable composer, daemon, and Herdr backend behavior suites passed. The real opt-in Herdr 0.8.2 + Claude Code 2.1.251 guard passed through native /afk, including exactly-once idle delivery, foreground rendered-busy deferral with native-state and matched-row evidence, and bright typed-text deferral. The 75-line ANSI transcript is stored at the evidence path above; the worktree is clean. Lint/static-analysis and the full repository suite were not run because this assigned phase explicitly forbids them.

Evidence: Herdr + Claude live away-mode guard

Source: Herdr + Claude live away-mode guard

Live guard exited 0 and exercised native Claude /afk in isolated Herdr session fm-lab-fm-afk-inject-we-86841-4718, proving idle delivery, rendered-busy deferral, matched ANSI rows, and pending human-text preservation.

verdict: idle-post-afk agent_status=working composer=empty pane_is_busy_rc=1 broad_match_rc=0 scoped_match_rc=1 subcause=idle native-state=working matched-row=  ⏵⏵ bypass permissions on · 1 shell · esc to interrupt · ← 1 agent · ↓ to manage       /rc
ansi-rows-begin: idle-post-afk
�[0m�[38;2;255;255;255m⏺�[0m Captain, entering away mode now.

�[0m�[38;2;78;186;101m⏺ �[0m�[1mBash�[0m(bin/fm-afk-launch.sh start-native 2>&1; echo "exit=$?")
�[0m�[38;2;153;153;153m  ⎿  �[0mexit=0
     
�[0m�[38;2;78;186;101m⏺ �[0m�[1mBash�[0m(FM_AFK_STATE_PREPARED=1 bin/fm-afk-start.sh)
�[0m�[38;2;153;153;153m  ⎿  Running in the background (↓ to manage)�[0m
  
�[0m�[38;2;215;119;87m✳ Schlepping… �[0m�[38;2;153;153;153m(11s · ↓�[0m �[0m�[38;2;153;153;153m443 tokens)�[0m
�[0m�[38;2;153;153;153m  ⎿  Tip: Running multiple Claude sessions? Use /color and /rename to tell them apart at a �[0m
     �[0m�[38;2;153;153;153mglance.�[0m

�[0m�[38;2;136;136;136m─────────────────────────────────────────────────────────────────────────────────────────────�[0m
�[0m�[38;2;153;153;153m❯ �[0m
�[0m�[38;2;136;136;136m─────────────────────────────────────────────────────────────────────────────────────────────�[0m
  �[0m�[38;2;255;107;128m⏵⏵ bypass permissions on�[0m�[38;2;153;153;153m · �[0m�[38;2;0;204;204m1 shell�[0m�[38;2;153;153;153m · esc to interrupt · ← 1 agent · ↓ to manage       �[0m�[38;2;78;186;101m/rc�[0mansi-rows-end: idle-post-afk
ok - real Herdr 0.8.2 + Claude 2.1.251 (Claude Code): native working with rendered-idle empty composer submits once
verdict: active-foreground-before-escalation agent_status=working composer=empty pane_is_busy_rc=0 broad_match_rc=0 scoped_match_rc=0 subcause=rendered-busy native-state=working matched-row=✻ Tempering… (7s · ↓ 128 tokens)
ansi-rows-begin: active-foreground-before-escalation

�[0m�[38;2;80;80;80m�[48;2;55;55;55m❯ �[0m�[38;2;255;255;255m�[48;2;55;55;55mUse Bash to run python3 -c 'import time; time.sleep(150)' and then reply exactly �[0m�[48;2;55;55;55m          �[0m
�[0m�[48;2;55;55;55m  �[0m�[38;2;255;255;255m�[48;2;55;55;55mFM_AFK_CLAUDE_GUARD_FOREGROUND_86734 and nothing else.�[0m�[48;2;55;55;55m                                     �[0m
  
�[0m�[38;2;153;153;153m �[0m �[0m�[1mBash�[0m(python3 -c 'import time; time.sleep(150)')
�[0m�[38;2;153;153;153m  ⎿  Running… (5s · timeout 3m 20s)�[0m
     �[0m�[38;2;153;153;153m(ctrl+b to run in background)�[0m

�[0m�[38;2;215;119;87m✢�[0m �[0m�[38;2;218;124;92mTempering…�[0m�[38;2;215;119;87m �[0m�[38;2;153;153;153m(7s · ↓�[0m �[0m�[38;2;153;153;153m128 tokens)�[0m
�[0m�[38;2;153;153;153m  ⎿  Tip: Dynamic workflows let Claude write a script that orchestrates many agents for you. �[0m
     �[0m�[38;2;153;153;153mMention the keyword �[0m�[38;2;177;185;249multracode�[0m�[38;2;153;153;153m or ask Claude to use a workflow directly.�[0m

�[0m�[38;2;136;136;136m─────────────────────────────────────────────────────────────────────────────────────────────�[0m
�[0m�[38;2;153;153;153m❯ �[0m
�[0m�[38;2;136;136;136m─────────────────────────────────────────────────────────────────────────────────────────────�[0m
  �[0m�[38;2;255;107;128m⏵⏵ bypass permissions on�[0m�[38;2;153;153;153m · �[0m�[38;2;0;204;204m1 shell�[0m�[38;2;153;153;153m · esc to interrupt · ← 1 agent · ↓ to manage       �[0m�[38;2;78;186;101m/rc�[0mansi-rows-end: active-foreground-before-escalation
verdict: active-foreground agent_status=working composer=empty pane_is_busy_rc=0 broad_match_rc=0 scoped_match_rc=0 subcause=rendered-busy native-state=working matched-row=✶ Tempering… (14s · ↓ 128 tokens)
ansi-rows-begin: active-foreground

�[0m�[38;2;80;80;80m�[48;2;55;55;55m❯ �[0m�[38;2;255;255;255m�[48;2;55;55;55mUse Bash to run python3 -c 'import time; time.sleep(150)' and then reply exactly �[0m�[48;2;55;55;55m          �[0m
�[0m�[48;2;55;55;55m  �[0m�[38;2;255;255;255m�[48;2;55;55;55mFM_AFK_CLAUDE_GUARD_FOREGROUND_86734 and nothing else.�[0m�[48;2;55;55;55m                                     �[0m
  
�[0m�[38;2;153;153;153m⏺�[0m �[0m�[1mBash�[0m(python3 -c 'import time; time.sleep(150)')
�[0m�[38;2;153;153;153m  ⎿  Running… (12s · timeout 3m 20s)�[0m
     �[0m�[38;2;153;153;153m(ctrl+b to run in background)�[0m

�[0m�[38;2;215;119;87m✽�[0m �[0m�[38;2;220;129;97mTempering…�[0m�[38;2;215;119;87m �[0m�[38;2;153;153;153m(14s · ↓�[0m �[0m�[38;2;153;153;153m128 tokens)�[0m
�[0m�[38;2;153;153;153m  ⎿  Tip: Dynamic workflows let Claude write a script that orchestrates many agents for you. �[0m
     �[0m�[38;2;153;153;153mMention the keyword �[0m�[38;2;177;185;249multracode�[0m�[38;2;153;153;153m or ask Claude to use a workflow directly.�[0m

�[0m�[38;2;136;136;136m─────────────────────────────────────────────────────────────────────────────────────────────�[0m
�[0m�[38;2;153;153;153m❯ �[0m
�[0m�[38;2;136;136;136m─────────────────────────────────────────────────────────────────────────────────────────────�[0m
  �[0m�[38;2;255;107;128m⏵⏵ bypass permissions on�[0m�[38;2;153;153;153m · �[0m�[38;2;0;204;204m1 shell�[0m�[38;2;153;153;153m · esc to interrupt · ← 1 agent · ↓ to manage       �[0m�[38;2;78;186;101m/rc�[0mansi-rows-end: active-foreground
verdict: pending-human-text agent_status=idle composer=pending pane_is_busy_rc=1 broad_match_rc=1 scoped_match_rc=1 subcause=idle native-state=idle matched-row=none
ansi-rows-begin: pending-human-text
�[0m�[48;2;55;55;55m  �[0m�[38;2;255;255;255m�[48;2;55;55;55m(pre-read; re-arm not needed — watcher daemon-managed)�[0m�[48;2;55;55;55m                                     �[0m

�[0m�[38;2;255;255;255m⏺ �[0mFM_AFK_CLAUDE_GUARD_ACK_86734

�[0m�[38;2;153;153;153m✻�[0m �[0m�[38;2;153;153;153mWorked for 19s · done 9:48 AM · 1 shell still running�[0m

�[0m�[38;2;80;80;80m�[48;2;55;55;55m❯ �[0m�[38;2;255;255;255m�[48;2;55;55;55mUse Bash to run python3 -c 'import time; time.sleep(150)' and then reply exactly �[0m�[48;2;55;55;55m          �[0m
�[0m�[48;2;55;55;55m  �[0m�[38;2;255;255;255m�[48;2;55;55;55mFM_AFK_CLAUDE_GUARD_FOREGROUND_86734 and nothing else.�[0m�[48;2;55;55;55m                                     �[0m
  
�[0m�[38;2;255;107;128m⏺�[0m �[0m�[1mBash�[0m(python3 -c 'import time; time.sleep(150)')
�[0m�[38;2;153;153;153m  ⎿  Interrupted · What should Claude do instead?�[0m

�[0m�[38;2;136;136;136m─────────────────────────────────────────────────────────────────────────────────────────────�[0m
❯ bright-human-draft-86734
�[0m�[38;2;136;136;136m─────────────────────────────────────────────────────────────────────────────────────────────�[0m
  �[0m�[38;2;255;107;128m⏵⏵ bypass permissions on�[0m�[38;2;153;153;153m · �[0m�[38;2;0;204;204m1 shell�[0m                                                    �[0m�[38;2;78;186;101m/rc�[0mansi-rows-end: pending-human-text
ok - real Herdr 0.8.2 + Claude 2.1.251 (Claude Code): rendered-busy and pending-composer deferrals preserve human text
evidence: session=fm-lab-fm-afk-inject-we-86841-4718 native=working rendered=idle composer=empty delivered_once=1 rendered-busy=1 native-state=working=1 composer=pending=1

Pipeline

Updates from git push no-mistakes

✅ **intent** - passed

✅ No issues found.

✅ **Rebase** - passed

✅ No issues found.

⚠️ **Review** - 4 issues (2 errors, 2 warnings)
  • 🚨 bin/fm-composer-lib.sh:382 - The active-context classifier hardcodes styled=1, but Herdr call sites pass the plain fm_backend_capture result. A real dim Press up to edit queued messages hint is then classified as pending; the early return skips the spinner check, pane_is_busy reports idle, and inject_msg can send during a genuine foreground turn. Use an ANSI capture or pass the actual capture capabilities.
  • 🚨 bin/fm-supervise-daemon.sh:688 - The intent permits recovery only while native state is not working or is working solely because of the tracked background shell. This code treats native unknown as eligible because it checks the terminal record only for the literal working; a transient agent.get failure plus a stable rendered row and empty composer can therefore send. Please confirm whether unknown is authorized, or keep it ineligible.
  • ⚠️ tests/fm-afk-herdr-claude-busy-guard-live-e2e.test.sh:227 - The broad matcher stores its row in FM_BUSY_MATCHED_ROW, but the evidence line prints FM_PANE_BUSY_MATCHED_ROW. For the idle Incident-B case that value is empty, so the exact pre-PR-94 matched row is not captured.
  • ⚠️ tests/fm-composer-lib.test.sh:709 - The Incident-B matrix accepts any nonzero return code, so unknown (rc=2) passes as idle. That can leave pane_is_busy deferring while the regression still passes; assert the required idle verdict rc=1.
  • 🚨 docs/verification/supervision.md:210 - The required report record is incomplete: the updated verification section does not identify the exact pre-PR-94 matched row, state whether Incident C is the same family, or record the separately observed prompt-submit-to-spinner race as an excluded follow-up. It only gives a generic footer fragment and one spinner example.
  • ⚠️ docs/verification/supervision.md:218 - The documented live command hardcodes /Users/ivan/Projects/firstmate/bin/fm-herdr-lab.sh, which can run a helper from another checkout while the test sources the current worktree. Use the current checkout helper or record the external helper's exact version and hash.

🔧 Fix: Harden Claude away-mode capture and recovery guards
4 issues (2 errors, 2 warnings) still open:

  • 🚨 bin/fm-composer-lib.sh:398 - FM_CLAUDE_BUSY_MATCHED_ROW is derived after ANSI stripping and normalization, then compared across polls by recovery. Two captures with identical visible text but different rendered bytes (for example bright busy versus dim ghost styling) can therefore unlock delivery. This contradicts the required criterion: “only after a rendered-busy matched row is byte-identical for N polls.” Propagate the raw matched row or fail closed when raw bytes are unavailable.
  • ⚠️ bin/backends/herdr.sh:2698 - The new ANSI-first Herdr path passes raw ANSI rows to the generic matcher. An anchored custom matcher such as ^⏵⏵ matched the previous plain capture but fails when the row begins with SGR bytes, causing a genuinely busy non-Claude pane to be reported idle. Strip ANSI before generic matching or keep ANSI capture scoped to Claude.
  • ⚠️ bin/fm-composer-lib.sh:394 - The active-composer exception matches Press up to edit queued messages without verifying that the row is dim or ghosted. A bright human draft such as Please explain Press up to edit queued messages, alongside a real tool/spinner row, is reported as rendered-busy instead of composer-pending, contrary to preserving bright human composer pending deferral. Require the hint's rendered styling or otherwise distinguish the system hint from typed text.
  • 🚨 docs/verification/runtime-backends.md:644 - The second verification record still invokes /Users/ivan/Projects/firstmate/bin/fm-herdr-lab.sh from another checkout and records only generic proof prose rather than the exact observed output. This contradicts the required criterion to “update both verification docs with dated exact commands, versions, and outputs” and is not reproducible from this worktree. Use $(git rev-parse --show-toplevel)/bin/fm-herdr-lab.sh and include the exact captured output here.
✅ **Test** - passed

✅ No issues found.

  • herdr --version and claude --version
  • bash tests/fm-composer-lib.test.sh
  • bash tests/fm-daemon.test.sh
  • bash tests/fm-backend-herdr.test.sh
  • HERDR_LAB_HELPER="$(git rev-parse --show-toplevel)/bin/fm-herdr-lab.sh" FM_AFK_HERDR_CLAUDE_LIVE=1 tests/fm-afk-herdr-claude-busy-guard-live-e2e.test.sh
  • Evidence existence/tail check and git status --short --untracked-files=all
✅ **Document** - passed

✅ No issues found.

🔧 **Lint** - 1 issue found → auto-fixed ✅
  • ⚠️ linter found issues (exit code 1)

🔧 Fix: Quote native-unknown recovery subcause; fm-lint passes
✅ Re-checked - no issues remain.

✅ **Push** - passed

✅ No issues found.

@withally
withally merged commit 07fd20f into main Aug 31, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant