fix(widgets): re-arm a dev session on a changed folder id and bound persistent stat errors

[mrgoonie]

Fixes #611. Follows #610 (#546). Refs #496. Consistent with #612 (#538): a swapped link never widens what a session may use. Official docs: digitopvn/clarkcant-web#137.

What changes

A folder that is still there under a new id is watched again, not stopped. packages/core/src/widget-dev-engine.ts
tells a replaced folder apart from a gone one. devRootState returns a fourth state, replaced: the path is still a
directory, but its device or file id differs from the one recorded when watching started. On replaced the engine:

1. logs the old and new dev:ino to stderr;
2. lets go of the old watcher (and, on Linux and macOS, the handle it held open);
3. records the new identity and watches the folder now at that path;
4. schedules a change build through the same debounce as a saved change, reported through onBuild;
5. schedules the same catch-up build the start uses (DEV_ENGINE_WATCH_CATCH_UP_MS), so a save made before the new
   watcher is live (FSEvents starts asynchronously) is still built. It reports only news.

A folder a build deletes and makes again keeps the session live. The old watcher reports the deletion straight
away, before the build tool has made the folder again, so a missing folder (ENOENT, ENOTDIR) counts as gone only
after DEV_ENGINE_ROOT_MISSING_GRACE_MS (2 s, rootMissingGraceMs).

• Until then no build starts, and the path is looked at again on the 1 s check and on every watcher event.
• A real folder (not a link) back at the same canonical path within the grace re-arms as replaced and is built.
• Anything else that is gone stops at once: something other than a folder, or a path reached through a link.
• While watching, rootGone() honours the same grace (both share one missingSince). The runtime asks it before
  following every build, so a build already running when the folder went, or a rebuild asked for during the grace,
  fails on the missing files and leaves the session live. The folder is built again once it is back. Without
  watching (watch: false), rootGone() stays strict.

This covers rm -rf out && build and rmdir /s /q out && xcopy src out when the folder is back within 2 s, plus FUSE
and network drives that give a folder that still exists a new id. A build that takes longer than 2 s to make the folder
again still stops the session as folder-gone, and the docs say so.

A link or junction swap is not a replaced folder.

• The engine resolves its root once, when it starts, to the real path (realpathSync.native), and watches and builds
  that path. A folder given through a link or junction is not taken for a swap: for example, clark widget dev on a
  symlinked ~/Projects or a junctioned workspace. engine.root is that real path.
• A replaced look counts as gone when lstat shows the path is a symlink or junction, or when its realpath no
  longer equals the recorded one. That covers the dev folder itself and any parent swapped for a link.
• On Windows and macOS, a realpath that differs only in case is the same place only when no folder on the path is a
  link (lstat on each folder up to the root). So a folder made again as Out for out re-arms. A parent swapped
  for a link to a sibling named in another case, on a case-sensitive volume, does not.
• build() checks the realpath just before reading the folder and again after copying the files. It fails with
  FILES_LINK_REFUSED if the path now leads elsewhere, which narrows the window for a swap to the copy itself.
  Closing that window fully would need reads relative to a held folder handle.

Re-arms are bounded.

• More than DEV_ENGINE_REARM_MAX (30) replaced looks in a row stop watching through onWatchError, with a
  message naming the last old and new ids. The runtime turns that into watch-failed.
• A run counts only back-to-back looks: any look that finds the folder unchanged resets it.
• So a folder made again once per build never trips the cap, while a filesystem that gives a new id on every look
  does.

No duplicate build. A debounce or catch-up callback whose rootStillThere() look re-armed the folder returns early,
because the re-arm already scheduled its own build.

A persistent non-ENOENT stat error is bounded. After DEV_ENGINE_ROOT_UNREADABLE_MS (30 s) of continuous failures,
measured with performance.now(), the engine stops watching and calls onWatchError with a message that names the
error code. The runtime turns that into stopReason: "watch-failed".

Runtime (apps/runtime/src/application/widget-dev-sessions.ts):

• checkRoot refuses a folder with 403 ROOT_UNREADABLE, "… cannot be read on this node (EPERM)", instead of "does
  not exist", when its stat or its realpath fails with anything other than ENOENT/ENOTDIR.
• A resume refused that way stops as watch-failed, not folder-gone or root-refused. The resume log line names the
  refusal message.
• A rootUnreadableMs pass-through option for tests.

Person-visible copy. shell.dev.stopReason.watch-failed (EN and VI) now says why watching failed: the system
stopped reporting changes, or the folder could not be read for 30 seconds. The status line already starts with
"No longer watching the folder · build N keeps running". Clark's develop_widget text says the same.

Contracts: the watch-failed and folder-gone JSDoc in packages/contracts/src/widget-dev-session.ts is
updated. The enum is unchanged.

Docs: docs/open-interfaces.md and docs/open-interfaces.vi.md document:

• the 2 s grace, what happens to a build that overlaps it, and that a slower rebuild stops the session;
• that a folder chosen through a link is watched at its real path, the link-swap rule, and the case rule;
• the realpath check around the copy;
• the back-to-back re-arm cap and its log;
• 403 ROOT_UNREADABLE.

The stray blank line in the refusal list is gone, and the odd line wraps in both files are fixed.

Official docs: 403 ROOT_UNREADABLE is added to the widget dev session refusals in docs/api.html and
vi/docs/api.html in digitopvn/clarkcant-web#137, which is open and not merged.

Review items

• Round 3, fixed:
  • R1, a folder given through a link failing every build: the root is resolved once at start.
  • R2, rootGone() ignoring the grace: it now honours it while watching. This was the cause of the local pnpm verify
    failures of the cross-process runtime test.
  • The case-only nit: such a match now needs a path with no link on it.
  • The doc wraps.
• Round 2, fixed: M1 (grace and re-arm), M2 (clarkcant-web#137), m1 (realpath check just before and after the copy),
  m2 (back-to-back re-arm cap), n1 (case-insensitive realpath compare on Windows and macOS), n2 (blank line in the
  refusal list, EN and VI), n3 (ROOT_UNREADABLE when realpath fails with EPERM).
• Skipped:
  • m3, the extra unchanged build after a re-arm on Windows. It is probably last-access notifications from the
    new watcher, it costs one digest, and it is not a cheap fix.
  • A dedicated test for n3, because the runtime spec's node:fs mock covers statSync only.

Tests

All tests use real fs.watch watchers on real temp folders. The link tests use real junctions on Windows and symlinks
elsewhere.

• core: a folder deleted and made again in the same tick is built as generation 2, and a save in the new folder
  builds generation 3.
• core (new): another process deletes the folder and copies it back 400 ms later → no onRootGone, generation 2
  from the new files, still watching, and a later save builds generation 3.
• core: a folder still there under a new file id is re-armed once, gives exactly one change:unchanged build, and
  later saves still build.
• core: the dev folder, or a parent of it, swapped for a link to another tree → onRootGone once, nothing built,
  rootGone() true, the latest generation is still the chosen package.
• core (new): a parent swapped for a link after the last look, then rebuild() → fails with FILES_LINK_REFUSED, and
  nothing from the other tree is built.
• core: a folder whose id changes on every look → stops after DEV_ENGINE_REARM_MAX looks in a row, with a message.
• core (new): a folder given 40 new ids in turn, each followed by unchanged looks → still watched.
• core (new, Windows and macOS): a folder made again as Out for out → re-armed and built, not gone.
• core (new): a folder given through a junction or symlink, with watch: false and no cache (the clark widget dev
  shape) → generation 1, engine.root is the real path, and a save builds generation 2.
• core (new): the same folder watched → a save through the link builds generation 2, nothing is gone.
• core (new): a watched folder deleted, then rebuild() → the build fails, rootGone() is false and it is still
  watched; once the grace is over, onRootGone fires once and rootGone() is true.
• core: a persistent EPERM stops watching within the bound; shorter failures restart the bound.
• runtime (new): another process deletes the session's folder and copies it back 400 ms later → the session stays
  live, and generation 2 is installed.
• runtime (new): another process deletes the session's folder and copies it back 800 ms later, and a rebuild runs
  while no folder is at the path → the rebuild answers live with a failed lastBuild, and generation 2 is installed
  once the folder is back.
• runtime: a parent swapped for a link → the session stops as folder-gone, and generation 1 of the chosen package
  stays active.
• runtime: an EPERM folder → a start is refused 403 ROOT_UNREADABLE "cannot be read on this node (EPERM)", and a
  resume stops as watch-failed.
• runtime: the two EPERM watch tests set the failing path to realpathSync.native(root), the canonical path the engine
  stats (the macOS fix).
• conversation-client: the watch-failed line says why it failed and that build N keeps running.

Results on Windows 11, Node 24, after merging main (which now includes #612, #622, #624 and #631):

Check	Result
Round-3 tests against the round-2 engine (f2e3d9ee; engine reverted, specs kept)	4 fail: both linked-root tests, the core grace test for rootGone(), and the runtime rebuild-during-recreate test
Round-2 tests against the round-1 engine (e1067a85)	5 fail: both cross-process recreate tests, the build-time link check, the back-to-back cap, and the cap message
widget-dev-engine.spec.ts + widget-dev-sessions.spec.ts + widget-dev-status.spec.ts, 3 runs	88/88 on each run
pnpm typecheck	exit 0
pnpm invariants	15/15 pass
eslint on the changed files	exit 0
pnpm verify, 2 runs	exit 0 on both runs: invariants, typecheck, eslint, and 573 test files passed (1 skipped); 7795 tests passed

Overlap

• feat(widgets): let the person choose a project folder for widget dev sessions #612, test: remove spec directories with the retrying async helper, since rmSync never retries on Windows #622, fix(desktop): keep a detached window's lease alive, close it with its conversation, and let isolated frames load #624 and fix(runtime): retry worker and widget-dev cleanup with the promise form of rm #631 are merged into main, and main is merged here.
  • The earlier spec conflicts were resolved by taking main's comments and import lines.
  • No rmSync in either spec passes maxRetries.
  • test: remove spec directories with the retrying async helper, since rmSync never retries on Windows #622's cleanup helper, removeTestDirectory, is kept.
  • fix(runtime): retry worker and widget-dev cleanup with the promise form of rm #631 merged without conflicts. prune is async, and both call sites in widget-dev-sessions.ts are
    await prune(sessionId).

[mrgoonie]

Review attestation: ready to merge at c42f015e0281550dd814ccbd01ecafb2bd16ff33, reviewed by agent:code-reviewer.

A push to this PR makes this attestation stale; the new head needs its own review.