Skip to content

watcher: derive OS watch masks from a per-context Op subscription - #36418

Draft
Properrr wants to merge 2 commits into
oven-sh:mainfrom
Properrr:claude/watcher-op-subscription
Draft

Properrr wants to merge 2 commits into
oven-sh:mainfrom
Properrr:claude/watcher-op-subscription

Conversation

@Properrr

Copy link
Copy Markdown
Contributor

Note

Depends on #36416 — this branch is stacked on it (it needs Op::MOVE_FROM and rewrites the kqueue fflags helper that PR introduces). Opened as a draft; I'll rebase and mark ready once #36416 lands. Review the diff against that branch to see only this change.

What does this PR do?

Every bun.Watcher requested the same fixed inotify mask / kqueue fflags / ReadDirectoryChangesW filter regardless of what its context actually read, so consumers paid to have events delivered, decoded and dispatched only to drop them in a contains() check.

Watcher::init now takes the set of Ops its context consumes, and each backend derives its registration from it: file_mask/dir_mask (inotify), vnode_fflags (kqueue), notify_filter (ReadDirectoryChangesW). An op outside the set is never requested, so a context must not rely on one it did not subscribe to — that contract is documented on init.

The dev server subscribes to everything except METADATA: it keys off DELETE/RENAME and re-resolves whatever a directory event names, but a permission or timestamp change cannot alter the bundle graph. --hot/--watch keeps METADATA, since that is what makes touch and chmod observable at all.

The Windows filter is coarser than the other two by nature — it selects which changes wake the call rather than which FILE_ACTION_* come back, and one filter bit can produce several actions. Metadata is deliberately absent there: Windows reports it as FILE_ACTION_MODIFIED, indistinguishable from a content write, so requesting it would mis-report chmod as Op::WRITE.

This is not a throughput optimization

I want to be straight about that, because it would be easy to assume otherwise. Measured on a realistic workload (100 files, no synthetic chmod pass):

full mask −IN_ATTRIB −dir IN_MODIFY
in-place save (editors, build writes) 400 events 400 (0%) 100 (−75%)
atomic save (write tmp + rename) 700 events 600 (−14%) 600 (−14%)

IN_ATTRIB never fires for an in-place write, and the drain path costs 0.24 µs/event against a ~211 ms reload — so event volume is not on the critical path. --watch reload latency is unchanged (211.7 ms vs 216.1 ms mean over 30 samples against a same-profile main build, i.e. within noise).

The value is that the registration now states what each consumer actually needs, and that narrowing a consumer is a one-line change. The one large lever — dir IN_MODIFY at −75% — is not taken here: it is how a file that is not individually watched gets noticed at all, so dropping it changes what the reloader sees and belongs in its own PR with its own measurement of per-event dispatch cost.

How did you verify your code works?

The dev server's resulting inotify file mask is byte-identical to the one requested before this change, verified with strace, so that narrowing is observably free:

--watch  (all)          IN_MODIFY|IN_ATTRIB|IN_MOVED_TO|IN_DELETE_SELF|IN_MOVE_SELF|IN_EXCL_UNLINK
DevServer (no METADATA) IN_MODIFY|IN_MOVED_TO|IN_DELETE_SELF|IN_MOVE_SELF|IN_EXCL_UNLINK

That also demonstrates the filter is actually filtering rather than merely plumbed — the two consumers register different masks.

Linux (x86_64): test/cli/watch, test/cli/hot, test/bake/dev/hot — 35 pass, 0 fail. The full test/bake/dev suite (26 files, 161 tests) also passes with the dev-server narrowing applied.

Windows is compile-checked only (cargo check --target x86_64-pc-windows-msvc); CI is the gate for the notify_filter hunk.

Properrr added 2 commits July 30, 2026 15:00
… overflow

`src/watcher` mapped several `Op` bits it never asked the OS for, so those bits
were unreachable and some filesystem changes produced no event at all.

- `Op::METADATA` was dead on *every* platform. kqueue read `NOTE_ATTRIB` in
  `watch_event_from_kevent`, but `add_file_descriptor_to_kqueue_without_checks`
  only requested `NOTE_WRITE|NOTE_RENAME|NOTE_DELETE`; inotify never had
  `IN_ATTRIB` in either mask. `touch` and `chmod` therefore produced no watcher
  event on Linux at all, while ReadDirectoryChangesW reported them as a write.
  Now requested for **file** watches on both backends. Deliberately kept off
  directory watches: on inotify, `IN_ATTRIB` on a directory reports metadata
  changes of every entry inside it *by name*, and consumers treat a named
  directory event as "re-resolve this entry", so a bare `touch` would reload the
  module. Threading `WatchItemKind` into the kqueue registration is what makes
  that split expressible.

- A rename inside a watched directory only reported its arrival half
  (`IN_MOVED_TO`). `IN_MOVED_FROM` is now requested and mapped to a new
  `Op::MOVE_FROM`, so the vacated name is invalidated too.

- On Windows, `FILE_ACTION_ADDED` and `FILE_ACTION_RENAMED_NEW_NAME` produced a
  `WatchEvent` with no op bits at all, making file creation invisible to every
  consumer matching on `WRITE|DELETE|RENAME`. They now map to `CREATE` and
  `MOVE_TO`. `RENAMED_OLD_NAME` keeps `RENAME` alongside the new `MOVE_FROM` so
  existing consumers are unaffected.

Dropped events were also ignored: `IN_Q_OVERFLOW` arrives with `wd == -1`,
matches no watchlist entry, and fell through the lookup in `watch_loop_cycle`, so
changes the kernel discarded were never noticed. `resync_after_overflow` now
replays a synthetic `WRITE` per watched path. `WRITE` rather than `DELETE` is
deliberate: it makes consumers re-check without driving the eviction path and its
fixed 8096-entry `evict_list`.

Two tests assert the new events through `BUN_WATCHER_TRACE`; both fail against a
build without this change (no event at all for `utimes`, and only `move_to` for a
rename). Platform skips are capability-based: ReadDirectoryChangesW cannot
distinguish metadata from a content write, and kqueue has no per-entry departure
event to map to `move_from`.

Verified on Linux (test/cli/watch, test/cli/hot: 24 pass, 0 fail) and on macOS
arm64, where `utimes on a watched file records a metadata event` passes — the
end-to-end confirmation that `Op::METADATA` is reachable on kqueue. The kqueue
semantics were checked against macOS 26.5.2 with a standalone probe: NOTE_ATTRIB
fires for utimes/chmod on a file vnode, and a directory vnode reports it only for
its own metadata, never its entries. Windows is compile-checked only.
Every `bun.Watcher` requested the same fixed inotify mask / kqueue `fflags` /
`ReadDirectoryChangesW` filter regardless of what its context actually read, so
consumers paid to have events delivered, decoded and dispatched only to drop them
in a `contains()` check.

`Watcher::init` now takes the set of `Op`s its context consumes, and each backend
derives its registration from that set: `file_mask`/`dir_mask` for inotify,
`vnode_fflags` for kqueue, `notify_filter` for ReadDirectoryChangesW. An op
outside the set is never requested, so a context must not rely on one it did not
subscribe to — that contract is documented on `init`.

The dev server subscribes to everything except `METADATA`: it keys off
DELETE/RENAME and re-resolves whatever a directory event names, but a permission
or timestamp change cannot alter the bundle graph. Its resulting inotify file mask
is byte-identical to the one requested before this change (verified with strace),
so the narrowing is observably free. `--hot`/`--watch` keeps `METADATA`, since
that is what makes `touch` and `chmod` observable at all.

The Windows filter is coarser than the other two by nature: it selects which
*changes* wake the call rather than which `FILE_ACTION_*` come back, and one
filter bit can produce several actions. Metadata is deliberately absent there —
Windows reports it as `FILE_ACTION_MODIFIED`, indistinguishable from a content
write, so requesting it would mis-report `chmod` as `Op::WRITE`.

This is not a throughput optimization and is not presented as one. Measured on a
realistic workload, dropping `METADATA` removes 0% of events for an in-place save
and 14% for an atomic save, and the drain path costs 0.24 us/event against a
~211 ms reload — so event volume is not on the critical path. The value is that
the registration now states what each consumer actually needs.

Linux: test/cli/watch, test/cli/hot, test/bake/dev/hot — 35 pass, 0 fail.
Windows is compile-checked only.
@Properrr
Properrr force-pushed the claude/watcher-op-subscription branch from 2c0956d to 487c244 Compare July 30, 2026 22:09
@Properrr

Copy link
Copy Markdown
Contributor Author

Rebased (was conflicting, now mergeable). Force-pushed 2c0956d32 → 487c244f9.

This is stacked on #36416, so it needed replaying onto that PR's rebased head as well as onto current main. Five files conflicted; four were the same mechanical shape as #36416 — main narrowed visibility (pub → pub(crate), pub(crate) → private) on items this PR extends. Kept upstream's visibility with this PR's additions in each case:

  • INotifyWatcher.rs — coalesce_interval, plus this PR's subscription field and file_mask
  • KEventWatcher.rs — fd and new(), plus subscription and vnode_fflags
  • WindowsWatcher.rs — the DirWatcher fields and next(), plus the filter field and notify_filter
  • Watcher.rs — a doc comment both sides had reformatted; merged, and corrected the example to the real three-argument Watcher::init signature

The fifth needed judgement rather than merging: hot_reloader.rs. main deleted the unsafe fn init this PR was modifying, so I took the deletion rather than resurrecting it. The subscription argument was already applied by auto-merge to the call site that replaced it (Watcher::init(reloader, ctx.watcher_top_level_dir(), HOT_RELOAD_SUBSCRIPTION)), so nothing was lost — but that one is worth a second look from a reviewer, since it is the only resolution where I dropped code rather than combining it.

Verified across targets, since most of this lives behind #[cfg]: cargo check -p bun_watcher on linux, aarch64-apple-darwin and aarch64-pc-windows-msvc all clean. watcher-trace.test.ts 6 pass, test/cli/hot/hot.test.ts 12 pass.

Still a draft — flagging the rebase so the branch is reviewable, not proposing it for merge yet.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant