Skip to content

fix: integrate upstream reliability fixes into house - #187

Merged
jazz127 merged 8 commits into
housefrom
fm/firstmate-house-upstream-merge-r11
Oct 5, 2026
Merged

jazz127 merged 8 commits into
housefrom
fm/firstmate-house-upstream-merge-r11

Conversation

@jazz127

@jazz127 jazz127 commented Oct 5, 2026 •

Copy link
Copy Markdown
Owner

Merge the six upstream commits through f470a01c098c1536d83b802874bd954a2c04b506 into the fork's house, based on 80faf6ad184b94258117b1cd681210ec6214c7c5.
The fetched upstream tip matches the brief's pin.
No-mistakes added documentation follow-up 0b54a9ea80bcb8bbb82cf3179ec39a7e54e0d2eb, correcting stale quota failure-wake wording, daemon and script-index descriptions, and the duplicated LaunchAgent wait bound. Runtime behavior is unchanged by that follow-up.
Commit 1ee36a03a8332dc7d31be28b552abe9cdba3f10b is a true merge with those two parents.

House conflict disposition

  • Stable remote-job identity: upstream does not cover the complete feature.
    Retain Linux start ticks and boot identity, legacy-owner recognition, C-locale macOS start reads, and house identity checks in fm_remote_job_recorded_owner_alive.
    The independent upstream heartbeat records the serving owner's PID and start identity and replaces stale-incarnation contents before refreshing readiness.
  • Duplicate-worker prevention and Linux supervision: upstream does not cover the complete feature.
    Retain house's Linux start guard, duplicate-tree cleanup, supervisor, and pidfd-bound signaling while adopting upstream's serialized Darwin repair and launchd ownership checks.
  • LaunchAgent asynchronous bootout: upstream covers waiting for removal before bootstrap, demonstrated by its reload implementation and the asynchronous bootout regression.
    Retain house-only bootout-refusal diagnostics and tolerance of an already-absent service through the shared bootout helper; use upstream's ten-second unload wait.
  • MacOS owner replacement: house's blanket refusal of live non-Linux PID signals conflicts with upstream untracked-owner recovery.
    Prefer upstream only inside Darwin LaunchAgent owner replacement, with PID/start/command checks before TERM and PID escalation.
    This relaxes that house refusal for this repair path; Linux pidfd protection and generic house tree cleanup remain intact.
  • Promoted worker delivery authority: upstream scratch guidance does not cover house's same-turn action and handoff-qualified validation authority.
    Keep both sets of generated-interface assertions.

The declared-wait classifier, branch-report implementation, host-backed supervision, and Claude context refresh remain intact.
Source comparisons and the final conflict resolutions are in the evidence artifact below.

Validation

Synthetic/offline executable regression coverage: 17 selected suites, 16 passing after the final remote-job reruns, and one pre-existing failure.
The initial merge exposed stale heartbeat identity recovery and untracked macOS owner replacement failures; both final remote-job suites pass after integration fixes.
bash bin/fm-test-run.sh tests/fm-calm-pi-extension.test.sh fails at Pi follow-up loaded-on case (loaded_on) did not reach the ready composer on the merged head, pinned upstream alone, and pinned house alone.
The three logs also report the expected installed Pi package unavailable.
This failure is left unchanged under the baseline-failure rule.
Canonical lint, explicit final remote-script lint, and documentation audience checks passed.
Linux-only lifecycle/signaling cases reported platform skips on this macOS host; no deployed remote-host observation is claimed.

No-mistakes Review found no defects. Its Test phase completed focused executable checks without a product failure, then returned an inconclusive acceptance finding. Firstmate explicitly selected skip for test-2; the Test step is skipped, with these scenarios not exercised against deployed services before merge:

  • Recover from transient quota failures and reset the failure streak after a successful reading.
  • Ensure a Darwin worker without unnecessary turnover, including concurrent repairs replacing stale or untracked owners.
  • Deliver stuck-busy and bookkeeping-failure escalations to the supervisor in away and quiet modes.

Synthetic/offline checks cover related paths, but do not establish those deployed-service outcomes. No deployed-service coverage is claimed for them.

House CI is green on 0b54a9ea80bcb8bbb82cf3179ec39a7e54e0d2eb; the PR is ready for review.
The background regression follow-up is running on 0b54a9ea80bcb8bbb82cf3179ec39a7e54e0d2eb: all 243 remaining suites, excluding 22 real Herdr lifecycle suites as firstmate directed. The runner uses its proven parallel phases and serial fallback, with a 25-minute per-script limit. It will report passed/failed counts, guard skips, exclusions, and established upstream-only failure attribution in a PR comment. No follow-up changes will be pushed to this branch.

22 excluded real Herdr lifecycle suites
  • tests/fm-afk-inject-herdr-e2e.test.sh
  • tests/fm-afk-launch.test.sh
  • tests/fm-afk-pi-herdr-return-e2e.test.sh
  • tests/fm-backend-autodetect-smoke.test.sh
  • tests/fm-backend-herdr-agent-exit-shell-e2e.test.sh
  • tests/fm-backend-herdr-eventwait-smoke.test.sh
  • tests/fm-backend-herdr-focus-flash-e2e.test.sh
  • tests/fm-backend-herdr-launcher-workspace-e2e.test.sh
  • tests/fm-backend-herdr-presentation-e2e.test.sh
  • tests/fm-backend-herdr-project-spaces-e2e.test.sh
  • tests/fm-backend-herdr-prune-safety-e2e.test.sh
  • tests/fm-backend-herdr-respawn-idem-e2e.test.sh
  • tests/fm-backend-herdr-smoke.test.sh
  • tests/fm-backend-herdr-stale-active-tab-e2e.test.sh
  • tests/fm-backend-herdr-workspace-per-home-e2e.test.sh
  • tests/fm-control-herdr-smoke.test.sh
  • tests/fm-herdr-attached-viewer-live-e2e.test.sh
  • tests/fm-herdr-pi-stale-registration-live-e2e.test.sh
  • tests/fm-herdr-session-cleanup-e2e.test.sh
  • tests/fm-herdr-submit-confirm-live-e2e.test.sh
  • tests/fm-herdr-version-floor-live-e2e.test.sh
  • tests/fm-send-secondmate-marker-herdr-e2e.test.sh

evidence-artifact: /tmp/fm-firstmate-house-upstream-merge-r11/final-pipeline-evidence.txt
evidence-command: python3 /tmp/fm-firstmate-house-upstream-merge-r11/capture-evidence.py
evidence-captured: 2026-10-05T05:35:19Z

Pipeline

kunchenguid and others added 8 commits October 3, 2026 21:06
…henguid#6530)

Pi 1.0.1's createToolHtmlRenderer reads getToolRenderers and ignores
getToolDefinition. Calm /export still includes stock grep HTML; the
fixture has to pass the lookup key the installed Pi actually reads.
* fix(procevent-quota): tolerate consecutive slow quota-axi reads

The quota allowance poll treated any quota_json failure as terminal, so one
slow quota-axi --json (measured max ~29s under a 48s derived bound) shut the
watch down until someone re-armed it, and the detail always said
"missing/incompatible". Tolerate three consecutive failed or timed-out reads
before going terminal, reset the streak on any good read, and report a
timeout distinctly from a missing or incompatible tool. Each timed poll runs
exactly one bounded --version and one bounded --json: validate the captured
version text through fm_quota_axi_version_compatible rather than launching a
second probe, and describe a mixed failure streak by count plus last cause.

* no-mistakes(document): Document quota polling failure tolerance

* no-mistakes(ci): Fixed ci-2 and ci-3. Permanent quota read failures (rc 2 missing, rc 3 incompatible) now report on the first poll, while transient rc 1/4 failures retain the existing three-failure retry behavior. The missing-binary test now uses an isolated PATH without quota-axi and asserts both permanent failures stop at condition_polls: 1. Verification passed: tests/fm-procevent-quota.test.sh, canonical fast lint for both changed files, bash syntax checks, and git diff --check

* no-mistakes(review): Classify untimed quota version failures as transient

* no-mistakes(document): Clarify quota polling failure budget
…enguid#6505)

* fix(teardown): clarify scratch guidance and dirty worktree refusals

Keep ship proof material outside the task worktree and distinguish untracked-only leftovers from tracked edits without changing cleanup guards.

Fixes kunchenguid#6319

* fix(ci): Fixed ci-3 only. Both promotion outputs now replace the scout restriction and require external scratch storage and a clean worktree before done. Verification: 27 delivery tests passed, five mutations caught, restored test passed, pinned ShellCheck and syntax/whitespace checks passed. ci-1, ci-2, and ci-4 remain untouched
…nguid#6518)

* Escalate inbox instructions stuck behind a busy worker

Count consecutive busy-deferred due doorbells durably and escalate at the configured bound without typing into the worker pane.

Fixes kunchenguid#6445

* fix(review): Fix inbox escalation deduplication and busy streak resets

* fix(review): Preserve busy inbox escalations through daemon supervision

* fix(document): Correct busy-inbox escalation documentation

* fix(ci): Fixed SC2034 in tests/fm-task-inbox.test.sh by including the loop counter in the failure diagnostic. Source-aware lint, all 34 inbox tests, and git diff --check pass. Behavior portable serial 6 reproduces identically on base 1f3e769 and target 78156b8 with Pi 1.0.1: an unrelated renderer API change breaks the unchanged Calm test. No Calm changes made; that failure is addressed separately by kunchenguid#6516. Logs retained in scratchpad-ci/

* fix(ci): Fixed ci-2, ci-3, and ci-4: successor failures surface, reset alerts deduplicate, and oversized busy limits fall back to two. Passed 47 inbox tests, 7 focused daemon checks, all 13 mutation checks, lint, documentation checks, and diff checks. Evidence: scratchpad-ci-selected/summary.json. ci-1 remains unchanged and unwaived. Fresh live Herdr proof remains with the outer driver
* fix: prevent healthy remote job worker turnover

* no-mistakes(review): Serialize full LaunchAgent repair and verify launchd-tracked owners

* no-mistakes(review): Let launchd-tracked unpublished spawns start before reloading

* no-mistakes(document): Document remote worker heartbeat and serialized LaunchAgent recovery

* no-mistakes(ci): Full CI log showed the idle-worker regression exceeded its outdated command budget (82 versus 80) after independent heartbeat ownership checks were added. Raised the budget to 120 while retaining the separate busy-poll sleep limit. Remote-job and LaunchAgent executable tests passed, as did bash syntax validation and git diff --check. No production behavior changed

* no-mistakes(ci): Fixed missing readiness recovery under verified live ownership, preserving the serving PID and private file mode. Added executable regressions for deletion during a blocked sweep and stale readiness diagnostics without LaunchAgent reload. Deletion regression failed before the fix. Both remote-job test suites, bash syntax validation, and git diff --check passed

* no-mistakes(ci): Full CI log identified a flaky ownership-loss test racing an already-authorized heartbeat refresh. Replaced backdating and a fixed sleep with bounded observation of readiness expiry through the public probe. Production behavior unchanged. Remote-job and LaunchAgent executable suites passed; bash syntax validation and git diff --check passed

* fix: wait for launchd bootout cleanup

* no-mistakes(document): Document remote worker recovery and read-only turnover verification

* no-mistakes(review): Publish worker identity before lock owner records

* no-mistakes(document): Document worker identity publication safety invariant

* no-mistakes(ci): Fixed ci-1: replacement workers discard predecessor readiness before publishing identity and roll back identity if lock-owner recording fails. Added executable regressions reproducing both defects. LaunchAgent and orphan-reap suites, shell syntax checks, and git diff --check passed. No live service state was modified

* no-mistakes(ci): Restored lock-owner-before-identity publication and removed the identity rollback and reordering-only tests. Retained an executable regression proving predecessor readiness is rejected until replacement startup completes. LaunchAgent and orphan-reap suites, shell syntax checks, and git diff --check passed. Other changes remain intact; the retained-identity interrupted-repair edge remains out of scope. No live service state was modified
* fix(remote-job): reap expired seq claims with one directory walk

The hourly claim sweep forked uname+stat per .seq-claims entry and blocked
serving for ~85s at ~17k dirs. Delete expired empty claim dirs with a single
find -exec rmdir batch and cache the host uname for remaining mtime reads.

* no-mistakes(review): Restore original path mtime helper and drop uname cache

* no-mistakes(test): Restore claim retention eligibility; focused retention and serving tests pass

* no-mistakes(document): Document single-walk claim cleanup and regression entrypoints

* no-mistakes(ci): Fixed Lint 2’s reproduced SC1091 by adding the tests/lib.sh ShellCheck source directive to the retention test. Runtime behavior is unchanged. ShellCheck passed for both new claim tests; Bash syntax, the retention behavior test, and git diff --check passed

* fix(tests): keep fixture registries out of git worktree roots

A TMPDIR pointed at a repository root placed live .fm-test-* registries
beside tracked files, and a concurrent git add during the claim-walk CI
fix round committed three of them. Route registries and fixture roots
through a TMPDIR that refuses git worktree roots, remove the stray files,
and pin the escape with a behavioral cleanup test.

* no-mistakes(review): Preserve whole-second claim expiry in single-walk sweep

* no-mistakes(document): Correct temporary-directory resolution documentation

* no-mistakes(ci): Fixed ci-3 by changing only the stale, fresh, and read-only orphan fixture paths in tests/fm-test-fixture-cleanup.test.sh to use $FM_TEST_TMPDIR. All seven tests passed both normally and with TMPDIR set to the worktree root. Shell syntax and git diff --check passed
Merge f470a01 while retaining the house identity, readiness, and delivery contracts. Combine independent heartbeat recovery with stable owner records, and use the upstream Darwin stop path for LaunchAgent owner replacement while retaining Linux pidfd cleanup.
@jazz127
jazz127 merged commit 914d4bb into house Oct 5, 2026
1 check passed
@jazz127
jazz127 deleted the fm/firstmate-house-upstream-merge-r11 branch October 5, 2026 05:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants