test(fm-branch-mod): add live e2e cases for branch persistence and child-session rotation - #31
Merged
Merged
Conversation
The live test gains a second lab and two RCA cases: a minutes-later wake resumes the same persisted agent when transcript saving is on (send succeeds, no rotation), while an inherited CLAUDE_CODE_CHILD_SESSION makes every resume unresumable, so the wake rotates to a fresh agent instead of being passed to main or dropped. Count assertions settle across append lag and the rotation chain is asserted by invariant rather than a frozen rotation count.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Intent
Add two live e2e cases to tests/fm-branch-claude-mod-live-e2e.test.sh covering the RCA blind spots: (1) with CLAUDE_CODE_CHILD_SESSION=1 inherited by the branch agent, a minutes-later wake must rotate with why=unresumable to a fresh agent (second agent.spawn) and the wake must still be delivered to that fresh agent, never passed to main and never dropped; (2) with persistence on (the default scrubbed launch), the same minutes-later wake must resume the SAME agent: the send succeeds, no rotation, no new spawn. Reuse the existing harness and keep every existing assertion; use two fully separate labs (fresh tmux socket and scratch home per case) so stale counters or locks cannot leak across cases. The test pins the module's Claude Code version, so run it end to end on the pinned version and quote the ok lines and duration as evidence. Then bin/fm-lint.sh, a one-line docs mention in docs/claude-supervision-branch.md Verification, commit on fm/fm-branch-live-gaps, rebase onto the default branch, and validate through this pipeline to a PR. Design decisions made during the work, all deliberate: count assertions settle across append lag because module event lines are appended by fire-and-forget shell spawns and can become visible to the test late under load (a settle helper re-reads a failing count up to ten times before failing); the rotation chain is asserted by invariant (every rotation why=unresumable and ok:true, agent.spawn >= 2, wake.passed=0, handback.dropped=0) rather than a frozen rotation count because every fresh agent inherits the broken launch shape; the redundant pre-gap-send success equality assertion was dropped in favor of the rotation event's own sendDetail as proof of the failed resume. The live test ran green end to end twice: on the 2.1.277 PATH shim pre-rebase (6/6 ok, 379s) and post-rebase on the shipped 2.1.278 pin (6/6 ok, 410s).
What Changed
tests/fm-branch-claude-mod-live-e2e.test.shnow runs two fully separate labs (own tmux socket and scratch home each) and adds two minutes-later-wake cases: with transcript persistence on the wake resumes the same agent with a successful send and no rotation; withCLAUDE_CODE_CHILD_SESSION=1inherited by the branch agent the wake rotates withwhy=unresumableto a fresh agent (a secondagent.spawn) and is delivered there, never passed to main or dropped.make_lab,start_claude_session,teardown_lab), a pause flag for the stand-in crewmate so no wake flows during the gap, asettlehelper that re-reads a failing count up to ten times to absorb append lag, amin-count argument onwait_event, and a scrubbed tmux server launch (with an intentionalSC2046disable) so an ambient child-session marker cannot mask the defect; all four existing assertions are retained.docs/claude-supervision-branch.mdVerification mentions the two-lab persistence and rotation proof, anddocs/verification/runtime-backends.mdrecords the six-case live run on the 2.1.278 pin (6/6 ok, exit 0) with per-lab event counts.🤖 Generated with Claude Code
Risk Assessment
✅ Low: Test-only and one-line docs change; both new labs assert exactly the intent's required outcomes (same-agent resume with no rotation/spawn, and unresumable rotation with delivery to the fresh agent and wake.passed=0/handback.dropped=0), every prior assertion survives in settled form, lab isolation (fresh socket, scratch home, per-lab shim/dummy paths, idempotent teardown) traces correctly, and the only finding is an optional tail loop that exceeds the stated scope.
Testing
Drove tests/fm-branch-claude-mod-live-e2e.test.sh live against Claude Code 2.1.278 (the module pin) with FM_BRANCH_MOD_LIVE=1 FM_BRANCH_MOD_LIVE_KEEP=1 from inside a Claude session whose environment carries CLAUDE_CODE_CHILD_SESSION=1, so the new tmux-server scrub was exercised for real. All six ok lines printed, exit 0, 474 s. Kept lab logs confirm the intent: lab 1 shows one agent.spawn, two agent.send both success:true resuming the same agent id a9d3ae4abc23edf8c, zero agent.rotated, and a genuine 100 s pause in dummy.status; lab 2 shows two agent.rotated why=unresumable ok:true with sendDetail "No transcript found for agent ID", three agent.spawn, three wake.delivered via spawn, wake.passed=0, handback.dropped=0, and a 95 s pause. A concurrent monitor read each lab's tmux server:
show-environment -gcarried no CLAUDE_CODE_CHILD_SESSION or CLAUDECODE on either socket, while the lab 2 pane child process did carry CLAUDE_CODE_CHILD_SESSION=1, proving the marker reaches only Claude's process env, not the ambient server. No UI surface; artifacts are CLI transcript and persisted event logs. Lab directories removed; worktree clean.ls /tmp | grep fm-branch-claude-liveempty after the runEvidence: Live e2e run on Claude Code 2.1.278 (6/6 ok, 474 s)
Source: Live e2e run on Claude Code 2.1.278 (6/6 ok, 474 s)
ok - Claude Code 2.1.278 loads the supervision-branch mod enabled and main holds the session lock ok - the first routine wake passes the classifier and spawns the branch agent, which reports it routine ok - the captain-class wake is passed to main by the classifier with a covering outcome row, and the next routine wake reaches the same agent through SendMessage ok - a wake after the in-memory window reaches the same persisted agent: the send succeeds, no rotation ok - one spawn, every send successful, no dropped hand-back, no backstop delivery across the run ok - a minutes-later wake on an inherited CLAUDE_CODE_CHILD_SESSION rotates to a fresh agent (why=unresumable) and keeps the wake EXIT=0 DURATION=474sEvidence: Lab 1 (persistence on) module events, dummy status, Claude debug log
Source: Lab 1 (persistence on) module events, dummy status, Claude debug log
Evidence: Lab 2 (inherited CLAUDE_CODE_CHILD_SESSION) module events: two why=unresumable rotations
Source: Lab 2 (inherited CLAUDE_CODE_CHILD_SESSION) module events: two why=unresumable rotations
Evidence: tmux server global-env scrub check during the run
Source: tmux server global-env scrub check during the run
fm-branch-claude-2268063 server-global-env: none | pane-child(pid 2268661) CLAUDE_CODE_CHILD_SESSION: none fm-branch-claude-child-2268063 server-global-env: none | pane-child(pid 2324241) CLAUDE_CODE_CHILD_SESSION: CLAUDE_CODE_CHILD_SESSION=1Pipeline
Updates from git push no-mistakes
✅ **intent** - passed
✅ No issues found.
✅ **Rebase** - passed
✅ No issues found.
tests/fm-branch-claude-mod-live-e2e.test.sh:319- wait_branch_quiet declares the branch idle after turn.complete.branch is stable for five 2 s reads (~10 s), but a Sonnet turn dispatched just before pause_dummy can run longer than that. Sequence: send logged at t-1, pause at t, five stable reads by t+8, GAP_BASELINE/SENDS_BEFORE_GAP captured, turn completes at t+15 during sleep 75. After the gap, wait_branch_turn "$GAP_BASELINE" then passes on the pre-gap turn, so the 'turn after the gap' evidence in both labs can be vacuous. Not a wrong pass: lab 1 still fails on the send-equals-success check if the resume failed, and lab 2 is gated by wait_event agent.rotated and wake.delivered via:spawn count 2. Only the post-gap turn assertion is weakened. Remedy if wanted: also require count wake.delivered unchanged across the stability window, or lengthen the stable window past a turn.tests/fm-branch-claude-mod-live-e2e.test.sh:462- The lab-2 tail loop compares successful sends (count agent.send '&feat(bin): record shadow advisory facts and add the candidate gates scorer #34;success&feat(bin): record shadow advisory facts and add the candidate gates scorer #34;:true') against SENDS_BEFORE_GAP, which is the count of ALL agent.send lines (line 350). Any pre-gap failed attempt (e.g. a ref-retry attempt 0) makes the success count start below the baseline, so a post-gap successful send would not exit the loop and the test would rely solely on the third spawn. In a fresh lab no pre-gap failure is expected, so no wrong result today. One-token fix: capture SUCCESS_BEFORE_GAP=$(success_sends) in production_gap and compare against that.tests/fm-branch-claude-mod-live-e2e.test.sh:400- settle 10 '[ count agent.spawn = 1 ]' (lines 400, 406) cannot lag into failure: the single spawn line was already awaited in scenario 2, and a late second spawn line makes the condition false only after settle has already returned true. The settle wrapper is dead here; a plain equality (as before) is equivalent. Not required by intent; removal is the simplification.🔧 No changes applied.
1 warning still open:
tests/fm-branch-claude-mod-live-e2e.test.sh:453- The lab-2 tail loop (lines 453-471) is a component beyond the stated intent. Intent case (1) requires: rotation why=unresumable to a fresh agent (second agent.spawn), wake delivered to that agent, never passed to main, never dropped. Every one of those is already asserted by lines 434-452, and the stated invariant (every rotation unresumable and ok:true, agent.spawn >= 2, wake.passed=0, handback.dropped=0) is fully checked there too. The tail loop then waits up to 240 s more for either a THIRD spawn or a successful send, an acceptance path ("on a version that keeps agents warm, the send succeeds") the intent does not name, and adds run time to a test the intent measures at 379-410 s. No intent requirement needs it; the wake.passed=0 / handback.dropped=0 re-checks at lines 469-470 are equivalent to those at 449-450 for the wakes the intent covers. Remedy: remove lines 453-470 (keep the final pass line), or if the author wants the chain proven, say so and keep it as is.🔧 **Test** - 3 issues found → auto-fixed ✅
tests/fm-branch-claude-mod-live-e2e.test.sh:429- Live lab 2 (CLAUDE_CODE_CHILD_SESSION=1 inherited) fails on the pinned Claude Code 2.1.278 in two consecutive runs:not ok - Claude Code 2.1.278 never reached the unresumable rotation. With a real 92 s gap (flag-file pause), the post-gap send still succeeded ("success":true, "message":"Resuming agent fm-branch") and the subagent transcript was written to ~/.claude/projects/-tmp-fm-branch-claude-live-bKrEZ6-home/7ecbb7af-…/subagents/agent-a50513d76756d901c.jsonl, so the marker does not switch transcript persistence off here and the rotation the test asserts never happens. Either the premise (child-session marker breaks resume on 2.1.278) no longer holds on this host, or the lab needs a different way to make the agent unresumable. Author decision needed.tests/fm-branch-claude-mod-live-e2e.test.sh:200- pause_dummy/resume_dummy usedpkill -STOP/-CONT -f "$LAB/dummy.sh"; tmux 3.4's server SIGCONTs a stopped pane process at once (reproduced: STAT stays Ss+ 0.3 s after kill -STOP, appends continue), so no gap ever elapsed and lab 1's minutes-later assertion passed vacuously (sends every ~43 s, no >10 s gap in dummy.status). Fixed in the worktree: the dummy loop now idles whilestate/dummy.pauseexists; run 2 shows 102 s / 92 s gaps. Uncommitted diff in the worktree and at evidence/test-fix-flag-file-pause.diff.not ok - Claude Code 2.1.278 never reached the unresumable rotation; run2-lab2-child-session-events.jsonl shows post-gap agent.send success=true resuming the same agent, no agent.r…pkill -STOPpause; probe shows tmux 3.4 SIGCONTs the pane. Fixed in worktree (flag file); run2 dummy.status shows 92–…FM_BRANCH_MOD_LIVE=1 FM_BRANCH_MOD_LIVE_KEEP=1 bash tests/fm-branch-claude-mod-live-e2e.test.sh(run 1, committed code): 5 ok, 1 not ok, 618 sprobe:kill -STOPon a tmux new-window pane process,ps -o stat0.3 s and 6 s later (stays Ss+, appends continue) — tmux 3.4 SIGCONTs the paneedited pause_dummy/resume_dummy to astate/dummy.pauseflag file read by the dummy loopFM_BRANCH_MOD_LIVE=1 FM_BRANCH_MOD_LIVE_KEEP=1 bash tests/fm-branch-claude-mod-live-e2e.test.sh(run 2, flag-file pause): 5 ok, 1 not ok, 693 sevent-timeline extraction from both labs' branch-mod-events.jsonl and dummy.status gap analysis (102 s lab 1, 92 s lab 2 in run 2)probe of the scrubbed launch env: CLAUDE_CODE_CHILD_SESSION=1 present in the pane environmentinspected ~/.claude/projects/-tmp-fm-branch-claude-live-bKrEZ6-home/*/subagents for lab 2 transcript files (present)🔧 Fix applied.
✅ Re-checked - no issues remain.
ls /tmp | grep fm-branch-claude-liveempty after the runFM_BRANCH_MOD_LIVE=1 FM_BRANCH_MOD_LIVE_KEEP=1 tests/fm-branch-claude-mod-live-e2e.test.shon Claude Code 2.1.278 (exit 0, 474 s, 6/6 ok)claude --version= 2.1.278 = CLAUDE_CODE_PIN in .claude/mods/fm-branch-mod/hooks/branch.tsConcurrent monitor:tmux -L <lab socket> show-environment -g | grep CLAUDE_CODE_CHILD_SESSIONand/proc/<claude pid>/environfor both labsPost-run inspection of kept branch-mod-events.jsonl per lab: counts of agent.spawn, agent.send, agent.rotated, wake.delivered, wake.passed, handback.dropped, backstop.deliveredPost-run gap check over dummy.status timestamps (>10 s gaps) for both labs🔧 **Document** - 1 issue found → auto-fixed ✅
tests/fm-branch-claude-mod-live-e2e.test.sh:224- bin/fm-lint.sh exits 1 on the target commit: ShellCheck SC2046 onenv $(unset_inherited) "$REAL_TMUX" ...(word splitting of the unset_inherited expansion is intentional). Not a documentation change; needs an inline# shellcheck disable=SC2046with the reason, or an array, in the test itself.🔧 Fix applied.
✅ Re-checked - no issues remain.
✅ **Lint** - passed
✅ No issues found.
✅ **Push** - passed
✅ No issues found.