fix(bin): retire task-keyed watcher markers and orphan journals at teardown - #5584
Closed
karotkriss wants to merge 3 commits into
Closed
karotkriss wants to merge 3 commits into
karotkriss wants to merge 3 commits into
Conversation
…ardown Teardown removed a task's metadata, turn-end and inbox files but left the watcher's .seen-<id>_turn-ended signature behind, and a Herdr presentation journal whose pane the close path proved gone but could not match to a live workspace was never retired, so a long-lived home accumulated dead markers. Retire the turn-ended seen marker at every teardown site alongside the status and heartbeat markers the presentation retire already owns, retire an orphaned journal (a version 1 attempt or one bound to the closed pane) once the Herdr presence gate has proven that pane gone while keeping a journal bound to any other pane for the session-start sweep, and have each locked wake drain rotate away the scratch files an interrupted drain left under the queue lock. Refs kunchenguid#5252
…space confirmed gone
…ition in teardown comment and docs
karotkriss
force-pushed
the
fm/fm-up-5252-teardown-markers
branch
from
September 25, 2026 18:44
f874756 to
4bd4c90
Compare
Contributor
Author
kunchenguid
pushed a commit
that referenced
this pull request
Sep 28, 2026
…ardown (#5997) * WIP: retire task-keyed watcher markers and orphan journals at teardown Re-applies old PR #5584 on current main: teardown retires the turn-ended .seen-* signature and an orphaned Herdr presentation journal whose workspace is already gone, and the wake-drain rotates its own dead scratch files. Not yet validated through no-mistakes. * no-mistakes(ci): Fixed the Greptile P1 finding in bin/backends/herdr.sh. fm_backend_herdr_projection_token_workspace_gone used `! ... jq -e ... 2>&1`, which swallowed a jq runtime error (thrown when a non-object workspace entry, e.g. a number before a live token-bearing workspace, hits `.label`) and flipped it to a "gone" verdict, causing teardown to delete a still-live v1 presentation journal. Invariant: a workspace-query error/ambiguity must never be read as token absence; only a cleanly-parsed list with no token-bearing label is "gone". Replaced the body with a single jq verdict (unknown/present/gone): a non-array list or any non-object/non-string-label entry yields "unknown", jq errors/empty output fall through `|| return 1` to unknown, and only "gone" returns 0. Sibling fm_backend_herdr_projection_endpoint_matches_journal already fails safe on jq error (empty match -> journal kept), so it needed no change, matching the author's scoping. Added test_teardown_retains_v1_journal_when_workspace_query_ambiguous driving real teardown with a malformed workspace-list entry, proving the journal is kept and no workspace close occurs. Verified the old logic returns GONE on that input (test fails before, passes after); full tests/fm-teardown.test.sh suite passes (exit 0) and shellcheck is clean. Marker-naming finding left untouched per explicit out-of-scope instruction
knowttl
pushed a commit
to knowttl/firstmate
that referenced
this pull request
Sep 29, 2026
…ardown (kunchenguid#5997) * WIP: retire task-keyed watcher markers and orphan journals at teardown Re-applies old PR kunchenguid#5584 on current main: teardown retires the turn-ended .seen-* signature and an orphaned Herdr presentation journal whose workspace is already gone, and the wake-drain rotates its own dead scratch files. Not yet validated through no-mistakes. * no-mistakes(ci): Fixed the Greptile P1 finding in bin/backends/herdr.sh. fm_backend_herdr_projection_token_workspace_gone used `! ... jq -e ... 2>&1`, which swallowed a jq runtime error (thrown when a non-object workspace entry, e.g. a number before a live token-bearing workspace, hits `.label`) and flipped it to a "gone" verdict, causing teardown to delete a still-live v1 presentation journal. Invariant: a workspace-query error/ambiguity must never be read as token absence; only a cleanly-parsed list with no token-bearing label is "gone". Replaced the body with a single jq verdict (unknown/present/gone): a non-array list or any non-object/non-string-label entry yields "unknown", jq errors/empty output fall through `|| return 1` to unknown, and only "gone" returns 0. Sibling fm_backend_herdr_projection_endpoint_matches_journal already fails safe on jq error (empty match -> journal kept), so it needed no change, matching the author's scoping. Added test_teardown_retains_v1_journal_when_workspace_query_ambiguous driving real teardown with a malformed workspace-list entry, proving the journal is kept and no workspace close occurs. Verified the old logic returns GONE on that input (test fails before, passes after); full tests/fm-teardown.test.sh suite passes (exit 0) and shellcheck is clean. Marker-naming finding left untouched per explicit out-of-scope instruction
RooseveltAdvisors
pushed a commit
to RooseveltAdvisors/firstmate
that referenced
this pull request
Sep 29, 2026
…ardown (kunchenguid#5997) * WIP: retire task-keyed watcher markers and orphan journals at teardown Re-applies old PR kunchenguid#5584 on current main: teardown retires the turn-ended .seen-* signature and an orphaned Herdr presentation journal whose workspace is already gone, and the wake-drain rotates its own dead scratch files. Not yet validated through no-mistakes. * no-mistakes(ci): Fixed the Greptile P1 finding in bin/backends/herdr.sh. fm_backend_herdr_projection_token_workspace_gone used `! ... jq -e ... 2>&1`, which swallowed a jq runtime error (thrown when a non-object workspace entry, e.g. a number before a live token-bearing workspace, hits `.label`) and flipped it to a "gone" verdict, causing teardown to delete a still-live v1 presentation journal. Invariant: a workspace-query error/ambiguity must never be read as token absence; only a cleanly-parsed list with no token-bearing label is "gone". Replaced the body with a single jq verdict (unknown/present/gone): a non-array list or any non-object/non-string-label entry yields "unknown", jq errors/empty output fall through `|| return 1` to unknown, and only "gone" returns 0. Sibling fm_backend_herdr_projection_endpoint_matches_journal already fails safe on jq error (empty match -> journal kept), so it needed no change, matching the author's scoping. Added test_teardown_retains_v1_journal_when_workspace_query_ambiguous driving real teardown with a malformed workspace-list entry, proving the journal is kept and no workspace close occurs. Verified the old logic returns GONE on that input (test fails before, passes after); full tests/fm-teardown.test.sh suite passes (exit 0) and shellcheck is clean. Marker-naming finding left untouched per explicit out-of-scope instruction
andrewesweet
pushed a commit
to andrewesweet/firstmate
that referenced
this pull request
Sep 30, 2026
…ardown (kunchenguid#5997) * WIP: retire task-keyed watcher markers and orphan journals at teardown Re-applies old PR kunchenguid#5584 on current main: teardown retires the turn-ended .seen-* signature and an orphaned Herdr presentation journal whose workspace is already gone, and the wake-drain rotates its own dead scratch files. Not yet validated through no-mistakes. * no-mistakes(ci): Fixed the Greptile P1 finding in bin/backends/herdr.sh. fm_backend_herdr_projection_token_workspace_gone used `! ... jq -e ... 2>&1`, which swallowed a jq runtime error (thrown when a non-object workspace entry, e.g. a number before a live token-bearing workspace, hits `.label`) and flipped it to a "gone" verdict, causing teardown to delete a still-live v1 presentation journal. Invariant: a workspace-query error/ambiguity must never be read as token absence; only a cleanly-parsed list with no token-bearing label is "gone". Replaced the body with a single jq verdict (unknown/present/gone): a non-array list or any non-object/non-string-label entry yields "unknown", jq errors/empty output fall through `|| return 1` to unknown, and only "gone" returns 0. Sibling fm_backend_herdr_projection_endpoint_matches_journal already fails safe on jq error (empty match -> journal kept), so it needed no change, matching the author's scoping. Added test_teardown_retains_v1_journal_when_workspace_query_ambiguous driving real teardown with a malformed workspace-list entry, proving the journal is kept and no workspace close occurs. Verified the old logic returns GONE on that input (test fails before, passes after); full tests/fm-teardown.test.sh suite passes (exit 0) and shellcheck is clean. Marker-naming finding left untouched per explicit out-of-scope instruction
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Intent
Refs #5252
After a task closes, teardown removes its metadata, turn-end and inbox files but leaves the per-task
.seen-*and.hb-surfaced-*watcher markers and any orphan Herdr presentation journal behind, so a home accumulates thousands of dead markers that slow every session start and wake drain.Retire those task-keyed marker families at teardown and rotate the wake-drain scratch files, while leaving the task's
data/<id>/directory in place.This addresses the state-marker part of upstream issue #5252.
What Changed
.seen-*signatures for its status and turn-ended files - in all three cleanup paths (main, remote secondmate, and home-children), and removes the task's orphaned Herdr presentation journal once the recorded pane is proven gone, while leavingdata/<id>/in place.teardown_herdr_journal_orphanedplus a newfm_backend_herdr_projection_token_workspace_gonehelper inherdr.shso a journal is retired only when it binds the exact closed pane, or is a version 1 attempt whose token-bearing projected workspace is confirmed gone; otherwise-bound, still-present, or unreadable journals are retained for the session-start sweep, with the stale-quarantine warning reworded accordingly.fm-wake-drain.shnow rotates away scratch files (.main-eligible-rows.tmp.*,.wake-rows.consume.*,.wake-queue.retire.*,.wake-queue.ack.*,.wake-queue.actor-view.*) left by a drain that died mid-write, under the queue lock before draining; docs and tests updated to cover the marker retirement and scratch rotation.Risk Assessment
✅ Low: Well-bounded teardown cleanup that mirrors existing Herdr correlation helpers, resolves the prior v1-journal-strands-workspace finding with a fail-safe workspace-gone check, uses exact (non-glob) marker paths matching the watcher's minting, rotates only self-minted scratch under the queue lock, and is covered by behavior-based tests.
Testing
Baseline: ran the full tests/fm-teardown.test.sh suite (101 ok, 0 failures) driving the real teardown script, which includes all four new marker/journal scenarios and confirms no regression in existing teardown paths. The wake-queue suite was run whole but hit its 550s timeout on an unrelated slow secondmate-liveness test before reaching the drain test, so the drain scratch-rotation scenario was driven in isolation against the real bin/fm-wake-drain.sh (pass) and against the pre-fix base script (fail), proving it is a real regression test. No visual surface is involved - this is CLI/shell tooling, so evidence is CLI transcripts of observable filesystem state after driving the real scripts. Worktree left clean; no live fleet watchers were touched.
bash tests/fm-teardown.test.shline 29 ok - test_teardown_retires_task_watcher_markers_and_orphan_journalEvidence: Teardown marker/journal scenarios (real fm-teardown.sh)
Source: Teardown marker/journal scenarios (real fm-teardown.sh)
Evidence: Drain scratch rotation: pass on fix, fail on base
Source: Drain scratch rotation: pass on fix, fail on base
Pipeline
Updates from git push no-mistakes
✅ **intent** - passed
✅ No issues found.
✅ **Rebase** - passed
✅ No issues found.
✅ **Review** - passed
✅ No issues found.
✅ No issues found.
✅ **Test** - passed
✅ No issues found.
bash tests/_focus_teardown.test.sh(filtered driver running only the 4 new fm-teardown tests via the real bin/fm-teardown.sh through run_teardown)bash tests/_focus_wake.test.sh(filtered driver running test_drain_rotates_orphaned_scratch against the real bin/fm-wake-drain.sh)Reverted bin/fm-teardown.sh, bin/backends/herdr.sh, bin/fm-wake-drain.sh to base b42d4fa and re-ran both drivers to confirm fail-before/pass-afterRestored source to HEAD and removed the transient _focus_*.test.sh drivers; git status clean✅ No issues found.
bash tests/fm-teardown.test.shline 29 ok - test_teardown_retires_task_watcher_markers_and_orphan_journalbash tests/fm-teardown.test.sh(full suite: 101 ok, 0 not ok, exit 0), covering the 4 new tests: test_teardown_retires_task_watcher_markers_and_orphan_journal, test_teardown_retains_journal_bound_to_another_pane, test_teardown_retires_v1_journal_when_projected_workspace_gone, test_teardown_retains_v1_journal_when_projected_workspace_presentIsolated driver sourcing tests/wake-helpers.sh running test_drain_rotates_orphaned_scratch against the real bin/fm-wake-drain.sh (ok, exit 0)Same drain test against the base commit 8d2ee29 copy of bin/fm-wake-drain.sh (not ok, exit 1) - confirming the regression reproduces before the fix✅ **Document** - passed
✅ No issues found.
✅ No issues found.
✅ **Lint** - passed
✅ No issues found.
✅ No issues found.
✅ **Push** - passed
✅ No issues found.
✅ No issues found.