worktree: Reload git state when a watcher rescan covers a repository - #61541
Merged
Conversation
When the OS file watcher loses sync (e.g. its event queue overflows), it drops pending events and reports a single Rescan event for the watched root. Two bugs caused git changes hidden behind such a rescan to be silently lost, leaving the git panel and branch indicator stale until something else touched .git: 1. A rescan only re-scanned worktree entries. Since the Rescan event's path is the worktree root and .git is excluded from entry scanning, no git reload was ever triggered, even though the dropped events may have included changes inside .git. Now a Rescan event schedules a git state reload for every repository whose git directory lies under the rescanned path. 2. Re-scanning re-inserts the repository entry with git_dir_scan_id: 0. The snapshot diff detects git changes by comparing scan ids, so the reset could wipe out a bump made earlier in the same scan cycle, swallowing the corresponding UpdatedGitRepositories signal. The scan id is now carried forward on re-insertion, and a debug assertion in changed_repos enforces that it never regresses. The two bugs masked each other in test_dot_git_dir_event_does_not_suppress_children: the root-rescan case only passed because the buggy scan id reset made the diff fire spuriously. Fixing either bug alone turns that test red, which is why both fixes land together. Adds FakeFs::simulate_watcher_overflow, which models the kernel's watch queue overflowing (buffered events are discarded and replaced by a Rescan for the given root), and uses it in: - test_watcher_overflow_rescan_reloads_git_state: a git change whose events are lost to an overflow must still be picked up via the rescan. - test_git_update_in_same_batch_as_rescan_is_not_lost: a git event processed in the same batch as a rescan must not lose its scan id bump to the repository re-insertion. - test_random_git_updates_with_watcher_overflows: randomized property test asserting every git state change is eventually signaled via UpdatedGitRepositories under random batching, delays, and overflows. - test_random_worktree_changes: now also injects watcher overflows, extending its convergence property to rescan reconciliation.
5 tasks
RemcoSmitsDev
approved these changes
Jul 24, 2026
RemcoSmitsDev
left a comment
Collaborator
There was a problem hiding this comment.
Hey eholk, thanks for making these 2 fixes. Sorry I haven't pushed this forward, but I couldn't consistently reproduce this issue anymore, sometime not at all. So was hard to say if it was really fixed except by the tests that I wrote similar to yours.
But these 2 fixes makes total sense to me, since I had the same fixes😀.
Anthony-Eid
approved these changes
Jul 24, 2026
Anthony-Eid
left a comment
Contributor
There was a problem hiding this comment.
Agree with Remco, awesome work
Contributor
Author
|
Thanks for the review and all the investigation, @RemcoSmitsDev. It definitely made it a lot easier to pick this up and push this forward. To be honest, I hadn't directly reproed these issues either but I've definitely seen git sync issues and the tests looked good so this should be a significant step forward. |
mdz-axo
added a commit
to mdz-axo/zed-kask
that referenced
this pull request
Jul 25, 2026
Upstream changes (zed-industries/zed main, 27 commits): - agent: Add agent.compaction_model setting for context compaction (zed-industries#60012) - agent: Show effort selector for anthropic compatible providers (zed-industries#61579) - acp: Update agent-client-protocol SDK to 2.0.0 (zed-industries#61570) - client: Extract proxy handshakes into new proxy_handshake crate (zed-industries#61427) - collab: Fix multiworkspace location out of sync bugs (zed-industries#61598) - editor: Fix sticky header drag cancels autoscroll (zed-industries#53592) - editor: Fix crash when copying and pasting using multiple cursors (zed-industries#61545) - editor: Skip untitled buffers when saving a multi-buffer (zed-industries#61380) - gpui: Fix images not being drawn with rounded corners with ObjectFit::Cover (zed-industries#61383) - gpui: Fix deadlock in performance profiler and reenable it (zed-industries#61584) - git_ui: Prevent Git panel bindings in repository selector (zed-industries#61282) - language_model: Add explicit OpenAI conversation compaction and fix Anthropic compaction (zed-industries#61370) - markdown: Fix squashed Mermaid diagrams in markdown preview (zed-industries#61260) - Opus 5 BYOK Support (zed-industries#61596) - repl: Show add-cell controls in empty notebooks (zed-industries#61329) - search: Escape seeded buffer search query in regex mode (zed-industries#57748) - settings: Fix VS Code import appending duplicate file associations (zed-industries#61355) - settings: Split VSCode and Zed keymap files (zed-industries#61532) - Treat blank spawn_agent session IDs as absent (zed-industries#60893) - worktree: Reload git state when a watcher rescan covers a repository (zed-industries#61541) - Plus 7 more minor fixes. Merge fixes: - crates/agent/src/thread.rs: replay_tool_call used 'message_ix' (undefined) after auto-merge; renamed to 'owning_message_ix' (the parameter name). - Cargo.toml: Removed stale workspace members hkask-wallet and hkask-git-cas (both directories deleted in prior commits but workspace entries remained). - kask/crates/hkask-regulation/src/wallet_manager.rs: Stubbed consume() and settle_rjoules() on WalletBudgetPort — these were API-key encumbrance operations from the deleted hkask-wallet crate; regulation tracks per-agent gas balances, not per-key encumbrances. - kask/crates/hkask-regulation/src/wallet_gas_calibrator.rs: Fixed test to use crate::agent_wallet_store::WalletStore instead of hkask_storage::WalletStore. - kask/crates/hkask-regulation/Cargo.toml: Added tokio macros feature to dev-dependencies for #[tokio::test]. - kask/crates/kask_bridge/Cargo.toml: Added futures dependency (needed by context_injector.rs for futures::executor::block_on). - kask/crates/kask_bridge/src/context_injector.rs: Fixed futures_util::executor to futures::executor (futures-util doesn't include executor module). Release Notes: - N/A
0arm
pushed a commit
to 0arm/zed
that referenced
this pull request
Jul 26, 2026
…ed-industries#61541) # Objective When the OS file watcher loses sync (e.g. its event queue overflows under heavy fs churn), it drops pending events and reports a single `Rescan` event for the watched root. Git changes hidden behind such a rescan were silently lost, leaving the git panel and branch indicator stale until something else touched `.git`. Contributes to zed-industries#13176. May fix zed-industries#60102, though that report predates zed-industries#60660 and may already be addressed by it on nightly (see Related PRs below). Two bugs caused this, and they masked each other: 1. **A rescan never triggered a git reload.** The `Rescan` event's path is the worktree root, not something inside `.git`, so it never populated `dot_git_abs_paths` in `process_events` — and since `.git` is excluded from entry scanning, the rescan produced no `.git` events either. The dropped git changes were simply never picked up. 2. **Re-scanning reset `git_dir_scan_id` to 0.** The snapshot diff detects git changes by comparing scan ids, so re-inserting the repository with a fresh id could wipe out a bump made earlier in the same scan cycle, swallowing the corresponding `UpdatedGitRepositories` signal. The masking is why these fixes land together rather than as two PRs: the root-rescan case in `test_dot_git_dir_event_does_not_suppress_children` only passed on `main` because the buggy scan-id reset made the snapshot diff fire spuriously. Fixing either bug alone turns that (currently green) test red. ## Solution - `process_events`: a `Rescan` event now schedules a git state reload for every repository whose git directory (`dot_git`, `common_dir`, or `repository_dir`) lies under the rescanned path, covering linked worktrees and gitfile repositories, including git dirs watched outside the worktree root. - `insert_git_repository_for_path`: carry the existing `git_dir_scan_id` forward when re-inserting a repository instead of resetting it to 0. Deliberately *not* bumped either: re-insertion is snapshot bookkeeping, not evidence of a git change — bumping would trigger spurious full reloads on non-lossy paths (explicit refreshes, path-prefix scans). Only `update_git_repositories` claims that git state changed. - `changed_repos`: a `debug_assert` enforcing that `git_dir_scan_id` never regresses, so future violations of this invariant fail loudly in tests instead of manifesting as a stale git panel. ## Testing New fault-injection infrastructure and tests: - `FakeFs::simulate_watcher_overflow` models the kernel's watch queue overflowing: buffered (undelivered) events are discarded and replaced by a single `Rescan` for the given root, mirroring FSEvents `kFSEventStreamEventFlagMustScanSubDirs`, inotify `IN_Q_OVERFLOW`, and Windows `ERROR_NOTIFY_ENUM_DIR`. - `test_watcher_overflow_rescan_reloads_git_state`: a git change whose events are lost to an overflow must still be picked up via the rescan (reproduces bug 1; fails on `main`). - `test_git_update_in_same_batch_as_rescan_is_not_lost`: a git event processed in the same batch as a rescan must not lose its scan-id bump to the repository re-insertion (reproduces bug 2; fails on `main`). - `test_random_git_updates_with_watcher_overflows`: randomized property test (100 iterations) asserting that every git state change is eventually signaled via `UpdatedGitRepositories` under random event batching, delays, and overflows. Fails on `main` within the first few seeds. - `test_random_worktree_changes` now also injects watcher overflows, extending its existing convergence property to rescan reconciliation of worktree entries (this already passed; the injection guards it going forward). Full `worktree` and `fs` suites pass. Verified the directed tests exercise the intended code paths via trace logging (the same-batch test hits `update_git_repositories` stamping followed by re-insertion, distinct from the overflow test where no `.git` event arrives at all). ## Related PRs The recent stale-git-state reports trace back to three distinct mechanisms that share one symptom. This PR addresses the third: - **Events never generated** — zed-industries#60660 (merged, in this PR's base): Linux's non-recursive watcher missed nested `refs/` directories, so external commits/fetches produced no events at all. That PR (together with zed-industries#60590, which explicitly rescans after Zed-initiated reset/fetch and touches only `git_store.rs`) fixed zed-industries#60348. No overlap with this PR; a merge against current `main` is clean, and the refs-watching tests are disjoint from the overflow/rescan tests added here. - **Events coalesced** — zed-industries#59876 (open, complementary): FSEvents can merge `.git` child events into a bare `.git` `Changed` event; the signal arrives, in a shape Zed ignores. @RemcoSmitsDev's review comment there describes this PR's failure mode and calls the two fixes complementary; this is effectively the follow-up promised in that comment. Both PRs touch the same region of `process_events`, so whichever lands second needs a trivial rebase, and zed-industries#59876 flips the bare-`.git` expectation in `test_dot_git_dir_event_does_not_suppress_children` Case 2, which this PR preserves. - **Events dropped** — this PR: the watcher generated events but lost them to a queue overflow, and the resulting `Rescan` did not reach the git reload path. zed-industries#59976 and zed-industries#60098 (merged) reduced how often this happens; this PR makes git state recover correctly when it does. ## Self-Review Checklist: - [x] I've reviewed my own diff for quality, security, and reliability - [x] Unsafe blocks (if any) have justifying comments - [x] The content adheres to Zed's UI standards ([UX/UI](https://github.com/zed-industries/zed/blob/main/CONTRIBUTING.md#uiux-checklist) and [icon](https://github.com/zed-industries/zed/blob/main/crates/icons/README.md) guidelines) - [x] Tests cover the new/changed behavior - [x] Performance impact has been considered and is acceptable Release Notes: - Fixed the git panel and branch indicator showing stale state after heavy file-system activity caused the file watcher to lose events
jolutz
pushed a commit
to jolutz/zed
that referenced
this pull request
Aug 8, 2026
…ed-industries#61541) # Objective When the OS file watcher loses sync (e.g. its event queue overflows under heavy fs churn), it drops pending events and reports a single `Rescan` event for the watched root. Git changes hidden behind such a rescan were silently lost, leaving the git panel and branch indicator stale until something else touched `.git`. Contributes to zed-industries#13176. May fix zed-industries#60102, though that report predates zed-industries#60660 and may already be addressed by it on nightly (see Related PRs below). Two bugs caused this, and they masked each other: 1. **A rescan never triggered a git reload.** The `Rescan` event's path is the worktree root, not something inside `.git`, so it never populated `dot_git_abs_paths` in `process_events` — and since `.git` is excluded from entry scanning, the rescan produced no `.git` events either. The dropped git changes were simply never picked up. 2. **Re-scanning reset `git_dir_scan_id` to 0.** The snapshot diff detects git changes by comparing scan ids, so re-inserting the repository with a fresh id could wipe out a bump made earlier in the same scan cycle, swallowing the corresponding `UpdatedGitRepositories` signal. The masking is why these fixes land together rather than as two PRs: the root-rescan case in `test_dot_git_dir_event_does_not_suppress_children` only passed on `main` because the buggy scan-id reset made the snapshot diff fire spuriously. Fixing either bug alone turns that (currently green) test red. ## Solution - `process_events`: a `Rescan` event now schedules a git state reload for every repository whose git directory (`dot_git`, `common_dir`, or `repository_dir`) lies under the rescanned path, covering linked worktrees and gitfile repositories, including git dirs watched outside the worktree root. - `insert_git_repository_for_path`: carry the existing `git_dir_scan_id` forward when re-inserting a repository instead of resetting it to 0. Deliberately *not* bumped either: re-insertion is snapshot bookkeeping, not evidence of a git change — bumping would trigger spurious full reloads on non-lossy paths (explicit refreshes, path-prefix scans). Only `update_git_repositories` claims that git state changed. - `changed_repos`: a `debug_assert` enforcing that `git_dir_scan_id` never regresses, so future violations of this invariant fail loudly in tests instead of manifesting as a stale git panel. ## Testing New fault-injection infrastructure and tests: - `FakeFs::simulate_watcher_overflow` models the kernel's watch queue overflowing: buffered (undelivered) events are discarded and replaced by a single `Rescan` for the given root, mirroring FSEvents `kFSEventStreamEventFlagMustScanSubDirs`, inotify `IN_Q_OVERFLOW`, and Windows `ERROR_NOTIFY_ENUM_DIR`. - `test_watcher_overflow_rescan_reloads_git_state`: a git change whose events are lost to an overflow must still be picked up via the rescan (reproduces bug 1; fails on `main`). - `test_git_update_in_same_batch_as_rescan_is_not_lost`: a git event processed in the same batch as a rescan must not lose its scan-id bump to the repository re-insertion (reproduces bug 2; fails on `main`). - `test_random_git_updates_with_watcher_overflows`: randomized property test (100 iterations) asserting that every git state change is eventually signaled via `UpdatedGitRepositories` under random event batching, delays, and overflows. Fails on `main` within the first few seeds. - `test_random_worktree_changes` now also injects watcher overflows, extending its existing convergence property to rescan reconciliation of worktree entries (this already passed; the injection guards it going forward). Full `worktree` and `fs` suites pass. Verified the directed tests exercise the intended code paths via trace logging (the same-batch test hits `update_git_repositories` stamping followed by re-insertion, distinct from the overflow test where no `.git` event arrives at all). ## Related PRs The recent stale-git-state reports trace back to three distinct mechanisms that share one symptom. This PR addresses the third: - **Events never generated** — zed-industries#60660 (merged, in this PR's base): Linux's non-recursive watcher missed nested `refs/` directories, so external commits/fetches produced no events at all. That PR (together with zed-industries#60590, which explicitly rescans after Zed-initiated reset/fetch and touches only `git_store.rs`) fixed zed-industries#60348. No overlap with this PR; a merge against current `main` is clean, and the refs-watching tests are disjoint from the overflow/rescan tests added here. - **Events coalesced** — zed-industries#59876 (open, complementary): FSEvents can merge `.git` child events into a bare `.git` `Changed` event; the signal arrives, in a shape Zed ignores. @RemcoSmitsDev's review comment there describes this PR's failure mode and calls the two fixes complementary; this is effectively the follow-up promised in that comment. Both PRs touch the same region of `process_events`, so whichever lands second needs a trivial rebase, and zed-industries#59876 flips the bare-`.git` expectation in `test_dot_git_dir_event_does_not_suppress_children` Case 2, which this PR preserves. - **Events dropped** — this PR: the watcher generated events but lost them to a queue overflow, and the resulting `Rescan` did not reach the git reload path. zed-industries#59976 and zed-industries#60098 (merged) reduced how often this happens; this PR makes git state recover correctly when it does. ## Self-Review Checklist: - [x] I've reviewed my own diff for quality, security, and reliability - [x] Unsafe blocks (if any) have justifying comments - [x] The content adheres to Zed's UI standards ([UX/UI](https://github.com/zed-industries/zed/blob/main/CONTRIBUTING.md#uiux-checklist) and [icon](https://github.com/zed-industries/zed/blob/main/crates/icons/README.md) guidelines) - [x] Tests cover the new/changed behavior - [x] Performance impact has been considered and is acceptable Release Notes: - Fixed the git panel and branch indicator showing stale state after heavy file-system activity caused the file watcher to lose events
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Objective
When the OS file watcher loses sync (e.g. its event queue overflows under heavy fs churn), it drops pending events and reports a single
Rescanevent for the watched root. Git changes hidden behind such a rescan were silently lost, leaving the git panel and branch indicator stale until something else touched.git.Contributes to #13176. May fix #60102, though that report predates #60660 and may already be addressed by it on nightly (see Related PRs below).
Two bugs caused this, and they masked each other:
Rescanevent's path is the worktree root, not something inside.git, so it never populateddot_git_abs_pathsinprocess_events— and since.gitis excluded from entry scanning, the rescan produced no.gitevents either. The dropped git changes were simply never picked up.git_dir_scan_idto 0. The snapshot diff detects git changes by comparing scan ids, so re-inserting the repository with a fresh id could wipe out a bump made earlier in the same scan cycle, swallowing the correspondingUpdatedGitRepositoriessignal.The masking is why these fixes land together rather than as two PRs: the root-rescan case in
test_dot_git_dir_event_does_not_suppress_childrenonly passed onmainbecause the buggy scan-id reset made the snapshot diff fire spuriously. Fixing either bug alone turns that (currently green) test red.Solution
process_events: aRescanevent now schedules a git state reload for every repository whose git directory (dot_git,common_dir, orrepository_dir) lies under the rescanned path, covering linked worktrees and gitfile repositories, including git dirs watched outside the worktree root.insert_git_repository_for_path: carry the existinggit_dir_scan_idforward when re-inserting a repository instead of resetting it to 0. Deliberately not bumped either: re-insertion is snapshot bookkeeping, not evidence of a git change — bumping would trigger spurious full reloads on non-lossy paths (explicit refreshes, path-prefix scans). Onlyupdate_git_repositoriesclaims that git state changed.changed_repos: adebug_assertenforcing thatgit_dir_scan_idnever regresses, so future violations of this invariant fail loudly in tests instead of manifesting as a stale git panel.Testing
New fault-injection infrastructure and tests:
FakeFs::simulate_watcher_overflowmodels the kernel's watch queue overflowing: buffered (undelivered) events are discarded and replaced by a singleRescanfor the given root, mirroring FSEventskFSEventStreamEventFlagMustScanSubDirs, inotifyIN_Q_OVERFLOW, and WindowsERROR_NOTIFY_ENUM_DIR.test_watcher_overflow_rescan_reloads_git_state: a git change whose events are lost to an overflow must still be picked up via the rescan (reproduces bug 1; fails onmain).test_git_update_in_same_batch_as_rescan_is_not_lost: a git event processed in the same batch as a rescan must not lose its scan-id bump to the repository re-insertion (reproduces bug 2; fails onmain).test_random_git_updates_with_watcher_overflows: randomized property test (100 iterations) asserting that every git state change is eventually signaled viaUpdatedGitRepositoriesunder random event batching, delays, and overflows. Fails onmainwithin the first few seeds.test_random_worktree_changesnow also injects watcher overflows, extending its existing convergence property to rescan reconciliation of worktree entries (this already passed; the injection guards it going forward).Full
worktreeandfssuites pass. Verified the directed tests exercise the intended code paths via trace logging (the same-batch test hitsupdate_git_repositoriesstamping followed by re-insertion, distinct from the overflow test where no.gitevent arrives at all).Related PRs
The recent stale-git-state reports trace back to three distinct mechanisms that share one symptom. This PR addresses the third:
refs/directories, so external commits/fetches produced no events at all. That PR (together with project: Explicitly scan on git reset and fetch #60590, which explicitly rescans after Zed-initiated reset/fetch and touches onlygit_store.rs) fixed Git Panel does not update after clicking the Uncommit button anymore #60348. No overlap with this PR; a merge against currentmainis clean, and the refs-watching tests are disjoint from the overflow/rescan tests added here..gitchild events into a bare.gitChangedevent; the signal arrives, in a shape Zed ignores. @RemcoSmitsDev's review comment there describes this PR's failure mode and calls the two fixes complementary; this is effectively the follow-up promised in that comment. Both PRs touch the same region ofprocess_events, so whichever lands second needs a trivial rebase, and Refresh git state on bare .git events #59876 flips the bare-.gitexpectation intest_dot_git_dir_event_does_not_suppress_childrenCase 2, which this PR preserves.Rescandid not reach the git reload path. worktree: Avoid dropping git repositories during watcher rescans #59976 and fs: Coalesce queued rescans after watcher overflow #60098 (merged) reduced how often this happens; this PR makes git state recover correctly when it does.Self-Review Checklist:
Release Notes: