Skip to content

feat(bin): surface documentation-vault drift during bootstrap - #22

Merged
HelloWorldSungin merged 6 commits into
mainfrom
fm/fm-vault-drift-check
Aug 4, 2026
Merged

HelloWorldSungin merged 6 commits into
mainfrom
fm/fm-vault-drift-check

Conversation

@HelloWorldSungin

Copy link
Copy Markdown
Owner

Re-landed on the fork from the already-validated branch (previously opened as kunchenguid#1634, which this account cannot merge upstream). Code unchanged.


Intent

Build a cheap, deterministic check that surfaces documentation-vault drift for every registered project, so a vault can never again silently fall dozens of commits behind (an ArkNode-AI vault fell 52 commits / 7 days behind on 2026-07-27 and nothing surfaced it - detection depended entirely on a human remembering to look).

Settled design constraints that were given up front, so they are deliberate choices rather than oversights:

  • DETECTION ONLY, never auto-write a vault. Vault content is curated knowledge needing judgment; an automated writer would manufacture exactly the stale-but-confident prose that caused the original harm. So there is intentionally no remediation, no sync, no scheduling.
  • Handle BOTH vault shapes because they behave completely differently and need different remedies: an in-repo vault directory, which a crewmate can and should update inside its own worktree; and an external symlinked vault pointing at a separate git repo, which a worktree-isolated crewmate structurally CANNOT write and must never be told to.
  • No false alarms on a project with no vault. A tracked directory merely named vault/ without the OKF bundle marker 00-Home.md (e.g. ArkNode-AI's scripts/testing/vault fixture) is deliberately ignored, or the check would alarm forever on a test fixture.
  • An ABSENT or BROKEN external symlink must report distinctly from a STALE vault, because 'drift cannot be measured at all' is the failure that HIDES staleness, not a form of it, and its remedy differs. ArkNode-AI has both problems at once.
  • Report the commit count behind and the drift window, not a bare boolean. Both endpoints are commit timestamps rather than the wall clock, deliberately, so runs are deterministic and repeatable.
  • Verify each project's real vault shape by INSPECTION rather than trusting the registry prose in data/projects.md.
  • Read-only against project clones: never write under projects/, never touch the captain's live Obsidian copy under ~/.superset/vaults/. A test pins this.
  • Keep it proportionate: a small detector, not a sync engine or control plane. Resist added configuration surface beyond the two thresholds detection needs.

Integration was specified to follow the EXISTING detect-only bootstrap convention rather than inventing a new alarm channel, so it emits a VAULT_DRIFT: diagnostic line from bin/fm-bootstrap.sh's detect path (running in detect-only/lock-refused sessions too, since it is read-only), with the handling playbook added to the bootstrap-diagnostics skill and the load trigger added to AGENTS.md section 13.

Two pre-existing, unrelated problems were also found while validating this work, and the standing instruction is to fix test failures encountered along the way. One of them - the session-start suite forcing its MISSING diagnostic by deleting fake node, which a host-installed /usr/bin/node silently satisfied again - was deliberately REVERTED in a follow-up commit on this branch because the same fix already exists on another branch with an open PR, and carrying two copies would guarantee a merge conflict. That revert commit is intentional, not an accidental regression. The other one is kept: bin/fm-test-run.sh's coverage guard compared LC_ALL=C-sorted files using an ambient-locale comm, so --check-coverage failed outright in any non-C locale; nothing else is addressing it.

What Changed

  • Add a read-only detector that inspects project clones for stale in-repo and external documentation vaults, reporting commit and day drift while distinguishing absent, broken, and invalid external targets.
  • Surface vault diagnostics after fleet sync and during detect-only bootstrap sessions, with configurable thresholds, handling guidance, and documentation.
  • Add deterministic detector and bootstrap coverage, and make the test coverage guard locale-independent.

Risk Assessment

✅ Low: The detector is read-only and well bounded, and the prior post-sync, marker-validation, and same-repository false-negative paths are now addressed at their shared boundaries with focused regression coverage.

Testing

After branch-diff inspection, all three targeted behavior scripts and the non-C-locale coverage check passed; fixed-timestamp direct and detect-only bootstrap CLI runs produced attached transcripts proving correct classifications, deterministic counts and windows, marker suppression, and read-only behavior, with a clean worktree afterward.

Evidence: Direct vault-drift CLI transcript
$ FM_HOME=<fixture-home> bin/fm-vault-drift.sh
VAULT_DRIFT: ArkNode-AI: external vault link absent at vault - the project declares an external vault there but this clone has none, so vault drift cannot be measured here at all; restore the link in the clone, or track the vault through its own registered repo
VAULT_DRIFT: ArkNode-AI: external vault stale at projects/trading-signal-ai/vault -> /tmp/no-mistakes-evidence/01KYZECJV2WRBS5KZ71QEJY85W/vault-drift-e2e/external-vaults/ArkNode-AI - vault last updated 2023-11-14, 52 project commits landed since, drift window 7d; the vault is a separate repo, which an isolated project worktree cannot write, so dispatch the update against that repo's own clone
VAULT_DRIFT: Hermes: in-repo vault stale at vault/ - vault last updated 2023-11-14, 20 project commits landed since, drift window 1d; a crewmate can refresh it in its own worktree

$ repeat the same command and compare byte-for-byte
deterministic-repeat: identical

$ compare project and vault Git HEAD/tree/status signatures before and after both runs
read-only-signatures: unchanged

$ verify the tracked scripts/testing/vault directory without 00-Home.md produced no diagnostic
markerless-vault-fixture: ignored
Evidence: Detect-only bootstrap relay transcript
$ FM_BOOTSTRAP_DETECT_ONLY=1 FM_HOME=<fixture-home> bin/fm-bootstrap.sh | grep ^VAULT_DRIFT:
VAULT_DRIFT: ArkNode-AI: external vault link absent at vault - the project declares an external vault there but this clone has none, so vault drift cannot be measured here at all; restore the link in the clone, or track the vault through its own registered repo
VAULT_DRIFT: ArkNode-AI: external vault stale at projects/trading-signal-ai/vault -> /tmp/no-mistakes-evidence/01KYZECJV2WRBS5KZ71QEJY85W/vault-drift-e2e/external-vaults/ArkNode-AI - vault last updated 2023-11-14, 52 project commits landed since, drift window 7d; the vault is a separate repo, which an isolated project worktree cannot write, so dispatch the update against that repo's own clone
VAULT_DRIFT: Hermes: in-repo vault stale at vault/ - vault last updated 2023-11-14, 20 project commits landed since, drift window 1d; a crewmate can refresh it in its own worktree

Pipeline

Updates from git push no-mistakes

✅ **intent** - passed

✅ No issues found.

✅ **Rebase** - passed

✅ No issues found.

🔧 **Review** - 2 issues found → auto-fixed (2) ✅
  • 🚨 bin/fm-bootstrap.sh:906 - The required goal says vault drift must not silently fall dozens of commits behind, but this check runs before normal-mode fleet_sync. A clone can initially appear current, then bootstrap fast-forwards it by 52 commits and exits without checking the resulting state. Run detection after fleet_sync in normal mode while retaining a read-only detect-only path.
  • 🚨 bin/fm-vault-drift.sh:141 - The required no-false-alarm criterion relies on 00-Home.md to distinguish a vault, but every root vault symlink is accepted and check_external never verifies that marker. A project with an unrelated vault symlink to any Git repository can therefore emit a stale-vault diagnostic. Validate the marker before stale classification, distinguishing declared-but-invalid targets from undeclared non-vault symlinks.

🔧 Fix: Fix post-sync vault drift and marker validation
1 error still open:

  • 🚨 bin/fm-vault-drift.sh:191 - The required external shape is “an external symlinked vault pointing at a separate git repo,” but rev-parse --show-toplevel accepts an ancestor repository. A declared ignored vault/ directory containing 00-Home.md but no separate .git resolves to the project repo, making vault_ts equal project HEAD and silently reporting no drift. Before measuring timestamps, require the resolved vault repository to differ from the project repository and report target invalid otherwise.

🔧 Fix: Reject project repository as external vault target
✅ Re-checked - no issues remain.

✅ **Test** - passed

✅ No issues found.

  • Inspected git diff --stat and changed files for 1e247571aa75e00b00c7a01c4830025ecd44dc61..838f0195e7c0a5bee94ae7c63c3e6a140665875e
  • bin/fm-test-run.sh tests/fm-vault-drift.test.sh tests/fm-bootstrap.test.sh tests/fm-test-run.test.sh
  • LC_ALL=en_US.utf8 bin/fm-test-run.sh --check-coverage
  • Ran bin/fm-vault-drift.sh twice against fixed-timestamp external, in-repo, absent-link, and markerless fixtures; compared output byte-for-byte and verified unchanged Git signatures
  • Ran FM_BOOTSTRAP_DETECT_ONLY=1 bin/fm-bootstrap.sh against the fixture and captured the relayed diagnostics
  • git status --porcelain confirmed no testing artifacts remained in the worktree
✅ **Document** - passed

✅ No issues found.

✅ **Lint** - passed

✅ No issues found.

✅ **Push** - passed

✅ No issues found.

@cursor

cursor Bot commented Aug 4, 2026

Copy link
Copy Markdown

Bugbot is not enabled for your account, so this pull request was not reviewed.

Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs.

kunchenguid and others added 5 commits August 4, 2026 03:54
* feat(bin): widen the remote runtime PATH and add a remote doctor preflight

The fixed remote entrypoint hard-coded a four-directory PATH, so a remote
account whose tools live under nix or a per-user profile could not run basic
Firstmate work without a login shell. The entrypoint now composes its child
PATH from the code root's bin, the account's ~/.local/bin, the common
package-manager directories that actually exist on the host, and the portable
system tail, deduplicated and in a fixed order, still under env -i with the
same variable allowlist and no shell command string.

fm-remote-doctor.sh reports that exact PATH by inheriting it from its own
entrypoint launch rather than recomposing it, so the ordering keeps one owner.
It is read-only, reports where each required and optional tool resolved, and
exits non-zero naming every required tool that did not. Remote seeding runs it
as a preflight before anything is created on the host and restores the registry
when it fails.

* no-mistakes(review): Harden remote git authorization and missing-tool diagnostics

* no-mistakes(document): Document remote PATH doctor and safe shims

* no-mistakes(lint): Fix ShellCheck findings in remote path tests

* no-mistakes(lint): Suppress exported fixture's false-positive ShellCheck warning
A project's knowledge vault could fall dozens of commits behind with nothing
surfacing it, because noticing depended entirely on someone remembering to look.
fm-vault-drift.sh closes that gap as a cheap, read-only inspection of every
registered project clone, relayed by bootstrap as a VAULT_DRIFT diagnostic line
in both normal and detect-only sessions.

It is detection only: a vault is curated knowledge, so an automated writer would
manufacture exactly the stale-but-confident prose the check exists to catch.

The two vault shapes are told apart by inspecting the clone, never by trusting
registry prose, because they need different remedies: an in-repo vault a
crewmate can refresh in its own worktree, and an external symlinked vault living
in a separate repo that an isolated project worktree structurally cannot write.
An absent or broken link reports distinctly from staleness, since "drift cannot
be measured at all" is the failure that hides staleness rather than a form of it.
Staleness carries the commit count behind and the drift window, both derived
from commit timestamps so a run is deterministic.

A directory that merely shares the name vault/ without the OKF bundle marker is
never reported, so test fixtures and sample trees raise no false alarm.

Also fixes two pre-existing test problems found while validating: the
session-start ordering and composition cases forced a MISSING diagnostic by
removing node, which a host-installed /usr/bin/node silently satisfied again,
and the test-runner coverage guard compared LC_ALL=C-sorted files with an
ambient-locale comm, so it failed outright in any non-C locale.
@HelloWorldSungin
HelloWorldSungin force-pushed the fm/fm-vault-drift-check branch from bd374bd to 1a541eb Compare August 4, 2026 04:05
CI runs on Git 2.54, which enables automatic maintenance by default. Against
this file's rapid fixture commits that maintenance races the index and deletes
loose objects the pending commit still references, so the fixture dies with
"invalid object ... Error building trees" partway through the 52-commit case
and the stale external vault is never measured.

Reproduced at 8/8 under Git 2.54 and 0/8 after disabling maintenance on the
fixture repos; Git 2.51 and 2.43 never trip it, which is why it passed locally
and only surfaced in CI. gc.auto=0 does not cover it - this is the maintenance
path, not the gc one.

The detector is unchanged: this only makes fixture history deterministic.
@HelloWorldSungin
HelloWorldSungin merged commit b45b811 into main Aug 4, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants