Scheduled audits: run them on a timer, configure them from the CLI, and email what they find

CHANGELOG.md

@@ -1,9 +1,15 @@
 # Changelog
 
-## 1.0.1-beta.0 — 2026-08-12
+## 1.0.1-beta.0 — 2026-08-14
 
 ### Features
 
+- Gather everything the audit owns under `audit/`, as layout 4. `auth.json` becomes `audit/session.json`, `next-audit.json` becomes `audit/reminder.json`, and `state/audit-schedule.json` becomes `audit/schedule.json`, so one directory answers "what does the audit know about this machine" the way `policies/` answers it for enforcement. Two new paths join them, and the split between them is the design rather than tidiness: `session.json` holds the tokens and is `user-typed`, while `machine.json` holds this machine's report id and its digest watermark and is `identity`. Both fields have to outlive a sign-out — regenerate the id and the server sees a brand-new machine on every logout, reset the watermark and the next digest re-reports months of history as though it just happened — so they cannot live in the file a sign-out deletes. **`auditDir` is now deliberately absent from `HOME_CLASSES`.** It was classified `derived` wholesale, which was correct for a directory holding two caches and became a trap the moment a credential moved in: `resettablePaths()` is a filter over that table, so a reset and every future migration would have deleted the user's tokens. It is MIXED now and classified per-file, exactly like `state/` already is, and the `COVERED_BY_PARENT` guard records it as the second entry mapping to itself. The migration is three moves and no deletions, each a rename with a copy fallback because `audit/` and the home root land on different filesystems once `$HOME` is a network mount and `rename(2)` returns `EXDEV` there. A missing source is success — most homes never signed in, so two of the three files are absent on the majority of machines — and an existing destination wins, because re-running the step is exactly what happens when a later step in the same chain throws and the user retries; the stale original is dropped rather than left at the root, since a second copy of a bearer credential is a liability. `session.json`'s mode is reasserted to `0600` afterwards rather than assumed, because a rename preserves it and the copy fallback inherits the umask. All three are backed up first: `auth.json` is a live credential that, unlike every other file in that list, was never on a delete list and so has never had a copy taken before a migration touched it. `next-audit.json` is MOVED rather than retired even though the scheduled-audit work replaces reminders, because a migration that deleted it before that work landed would drop a cadence a person chose with no way back if the follow-up slipped. (#695)
+
+### Fixes
+
+- Stop `detectLayout()` deriving a landmark's layout from whatever this build speaks. `config.toml` with no `config.json` returned `LAYOUT_VERSION - 1`, which read correctly while current was 3 and became silent data loss at 4: a genuine layout-2 home was reported as layout 3, so `planMigration` ran only the 3 → 4 step — which finds none of layout 3's files, moves nothing, and stamps the home as current. `config.toml` and `credentials.toml` would never be carried into JSON, orphaning the cloud token and `daemon.configured` on a machine that then reads as fully migrated. A landmark identifies ONE layout and is never relative. The `config.json` branch above it had the same shape with a different ending: that file proves "layout 3 or later" and cannot separate the two, so a layout-3 home that lost its `VERSION` was called current, the 3 → 4 move never ran, and the user was silently signed out with `auth.json` still sitting on disk. What actually separates 3 from 4 is where the audit's files sit, so it now asks that directly — any of the three still at the root means stale — and when none are present the two layouts are identical on disk, the step would move nothing, and current is the correct non-destructive answer. Found by the layout-4 bump: the assertion that caught it was pinned to `2` and started failing the moment the constant moved, which is the whole reason it was written that way. (#695)
+
 - Move the nightly doc translation onto the canary box too, so one machine and one installer carry both scheduled jobs. Runner minutes were the entire cost of both crons; the LLM spend is identical wherever they run. The runner image already knew how to lock, check out a ref and hand off to a script from that checkout, so `$CANARY_JOB` now selects WHICH script — `jobs/canary.sh` (the integration suite, 11:00 local) or `jobs/translate.sh` (the translation, 02:00 local) — resolved to a path rather than through a case statement, so a third job is a new file in the repo and never an image rebuild. Everything per-run is keyed by job: the **lock** above all, because one shared lock lets a canary wedged on a vendor CLI swallow the night's translation and the swallow is a clean `exit 0` that reports nowhere; also the clone, since translate commits and switches branches inside its checkout, and the log. `install.sh` grew `--jobs`, per-job `--at-*` flags and one cron line per job, each behind its own marker so installing one never strips the other's; it validates credentials **per job**, so installing only the canary never demands a translation PAT, and it prints the timezone cron resolved, because "02:00" read as UTC on an IST box is 07:30 and the person reading the output is the one who would be surprised. Three things collapse in the move and are why the job is shorter than the workflow it replaces: the 14-way matrix was runner parallelism, not translation structure (cli.ts already fans out over pages x languages under one limit, so one process at `TRANSLATE_MAX_CONCURRENT=16` reproduces CI's exact peak of `max-parallel: 4` x 4 — which deletes the artifact round-trip, the per-language cache fragments and the ~35-line script that merged them); the Actions cache layer becomes a 13 KB file symlinked into the checkout from the work dir; and `consolidate`'s re-checkout-and-overlay existed only because its siblings ran on other machines. The one genuinely new credential is a push token — Actions minted a repo-scoped `GITHUB_TOKEN` that died with the job, and a box needs a long-lived fine-grained PAT, which is why it goes in a git credential helper rather than the remote URL: git echoes the remote back on a push error and the Slack crash-note carries the log tail. The translate job posts **nothing** to Slack — its output is the pull request it opens, which the PR list already says; its failures land in the run log and the exit code. The canary keeps reporting on every run including the quiet ones, so silence from it means the box did not run rather than that all was well. (#694)
 
 - Audit the documentation weekly, on the same box. `mintlify validate` and `validate:mdx` answer "does this build", per PR, on the pages a PR touches — and pass happily on a corpus that builds perfectly and is quietly wrong: a page nobody has edited since the CLI it documents was rewritten, a page in the nav that is gone, a page in **no** nav and so unreachable by any reader, an in-body link to something renamed, a translation still describing last quarter's behaviour. None of that fails a build, which is precisely the shape a periodic sweep catches and a per-PR gate structurally cannot. `docs-audit` runs Mondays at 04:00 and posts what it found. It is the cheapest job on the box — **no gateway key, no push token, no sibling containers**, so it installs on a machine holding no credentials at all beyond the webhook — and that is deliberate: an audit that could also FIX what it finds would need write access and a much longer argument about what it may change unattended. It **reports and exits 0 by design**; `--fail-on-findings` exists for a future caller that wants a gate and is off by default, because a docs audit that turns the build red the day a page crosses an age threshold gets switched off within a week, and then there is neither a gate nor a report. It reports two ways: the weekly Slack post, and one `[auto] docs audit` tracking ISSUE kept current on GitHub — opened when there is something to do, its body refreshed each week, and closed when a week comes back clean, so an open issue always means "there is something to do" rather than "this ran once, months ago". An issue and not a PR, deliberately: a report is not a change, so a weekly PR would either sit open forever or auto-merge a file nobody reads, and an audit opening a FIXING PR would have almost nothing safe to put in it — a dangling nav entry might mean "delete the entry" or "restore the page", an orphan page might be deliberately unlisted, a broken link has no inferable target, and each is a judgement this job cannot make. Its token is correspondingly weak, `Issues: read+write` and nothing else, since it never changes a file; leave it empty and the job degrades to Slack alone. `countActionable` decides open-vs-closed and deliberately EXCLUDES stale and never-translated pages, because the nightly translation closes both by itself and counting them would hold the issue open forever — the only way a tracking issue can actually fail. The judgement lives in `scripts/docs-audit.ts` — pure functions taking the git log, the file list and the cache as arguments, so every detector is unit-tested in **both** directions (it fires on the bad case, and stays silent on the good one) without a repo, a docs tree or a clock; the shell job is only box wiring around `bun run docs:audit`, which anyone can run by hand. Two details worth knowing: it reads the same translation cache the nightly job writes, or every page would report as never-translated every week — a 672-line finding that is an artefact of where a file lives rather than a fact about the docs; and it skips link forms it cannot resolve (external, anchors, relative) rather than guessing, because the first finding nobody can reproduce is what gets the whole weekly post ignored. It also hardened the ref check. Matching the NAME against one known-stale branch (`origin/failproofaid`) only ever caught that one branch — a merged-and-deleted feature branch sailed straight through, which is exactly what was sitting in a real `secrets.env`: `CANARY_REF=origin/feat/canary-local-runner`, so the box would have tested a frozen tree forever and never said so. The installer now asks the REMOTE whether the branch still exists, which catches every deleted branch without naming any, and warns (without refusing) on anything that is not `origin/main` — legitimate for a one-off, rarely right for a cron line. Scheduling it also taught the installer to say weekly at all: a spec is now `"M H"` or a full five-field cron expression, and a job name may carry a dash (`docs-audit` is a valid path component and an invalid shell variable name), so every per-job lookup goes through one conversion rather than each site remembering. (#694)

__tests__/hooks/fp-home.test.ts

@@ -66,7 +66,7 @@ describe("fp-home layout", () => {
     // writes — and an absent file is indistinguishable from a lane that has
     // never run. Kept next to the Rust literal so the pair has to be changed
     // together.
-    expect(H.auditScheduleFile()).toBe(resolve(home, "state", "audit-schedule.json"));
+    expect(H.auditScheduleFile()).toBe(resolve(home, "audit", "schedule.json"));
   });
 
   it("keeps run/ shallow — sockets must fit in SUN_LEN", () => {
@@ -188,8 +188,13 @@ describe("HOME_CLASSES", () => {
     customPoliciesDir: "policiesDir",
     customAgentsEventsDir: "customAgentsDir",
     customAgentsFailedDir: "customAgentsDir",
-    auditDashboardFile: "auditDir",
-    auditCacheDir: "auditDir",
+    // `auditDir` maps to ITSELF, the second entry to do so after `stateDir` and
+    // for the same reason: layout 4 made it MIXED. It holds `session.json` (a
+    // credential) and `machine.json` (an identity) alongside three derived
+    // caches, so it is classified per-file and the parent is deliberately absent
+    // from `HOME_CLASSES`. Its children are therefore classified directly and no
+    // longer appear here.
+    auditDir: "auditDir",
     daemonSocket: "runDir",
     workerSocket: "runDir",
     daemonLock: "runDir",
@@ -235,10 +240,12 @@ describe("HOME_CLASSES", () => {
     const classified = new Set(H.HOME_CLASSES.map((e) => e.path()));
     for (const [child, parent] of Object.entries(COVERED_BY_PARENT)) {
       const parentFn = H[parent] as (h?: string) => string;
-      // `stateDir` is the one entry that maps to itself: it is deliberately NOT
-      // classified, because it is MIXED — `spool/` and `telemetry-id` must never
-      // be dropped while a dozen scratch files under it should be. Listing the
-      // parent is exactly how a reset came to delete undelivered events.
+      // `stateDir` and `auditDir` map to themselves: both are deliberately NOT
+      // classified, because both are MIXED — `spool/` and `telemetry-id` must
+      // never be dropped while a dozen scratch files under `state/` should be,
+      // and `audit/` holds a session token and a machine identity next to two
+      // caches. Listing the parent is exactly how a reset came to delete
+      // undelivered events, and is what would have deleted the token here.
       if (child === parent) {
         expect(classified.has(parentFn())).toBe(false);
         continue;
@@ -365,6 +372,41 @@ describe("detectLayout", () => {
     if (state.kind === "stale") expect(state.found).toBe(2);
   });
 
+  it("reports a layout-2 home as 2, never as 'one behind whatever this build is'", () => {
+    // The landmark identifies ONE layout. `found: LAYOUT_VERSION - 1` read
+    // correctly while current was 3 and silently became data loss at 4: a real
+    // layout-2 home was reported as 3, so only the 3 → 4 step ran — which finds
+    // none of layout 3's files, moves nothing, and stamps the home current.
+    // config.toml and credentials.toml would never be carried into JSON, leaving
+    // the cloud token and `daemon.configured` orphaned on a machine that now
+    // reads as fully migrated.
+    writeFileSync(H.legacy.configToml(), 'mode = "oss"\n');
+    const state = detectLayout();
+    expect(state.kind).toBe("stale");
+    if (state.kind === "stale") expect(state.found).toBe(2);
+  });
+
+  it("calls a config.json home with layout-3 audit files still at the root stale, not current", () => {
+    // `config.json` proves "3 or later" and cannot separate them, so the audit
+    // files' POSITION is the discriminator. Getting this wrong skips the 3 → 4
+    // move: auth.json stays at the root, `audit/session.json` never appears, and
+    // the user is silently signed out with the file still sitting on disk.
+    writeFileSync(H.configFile(), "{}");
+    writeFileSync(H.legacy.authJson(), "{}");
+    const state = detectLayout();
+    expect(state.kind).toBe("stale");
+    if (state.kind === "stale") expect(state.found).toBe(3);
+  });
+
+  it("calls a config.json home with no layout-3 audit files current", () => {
+    // The other direction: with none of those three present the two layouts are
+    // identical on disk — the step would move nothing — so reporting stale would
+    // run a migration to achieve exactly nothing, on the commonest home there is
+    // (one that has never signed in).
+    writeFileSync(H.configFile(), "{}");
+    expect(detectLayout().kind).toBe("current");
+  });
+
   it("distinguishes a FUTURE layout from a stale one", () => {
     // Telling someone to reset a home written by a newer CLI would delete data
     // a simple upgrade would have read fine.