Skip to content

fix(core): batch untracked file diffs instead of spawning git per file - #53449

Merged
nexxeln merged 2 commits into
anomalyco:v2from
Hona:batch-untracked-diff
Oct 8, 2026
Merged

nexxeln merged 2 commits into
anomalyco:v2from
Hona:batch-untracked-diff

Conversation

@Hona

@Hona Hona commented Oct 5, 2026 •

Copy link
Copy Markdown
Member

A working-tree diff ran 2 git processes per untracked file, one after another (statUntracked + patchUntracked). In a worktree with 257 untracked eval files, /api/vcs/diff ran longer than the app's 60 s request timeout. The app retried endlessly, page loads waited behind the stuck requests (blank tab), and the server eventually stalled.

This is a performance-only change: output is byte-identical to v2.

flowchart LR
  L[files v2 would diff one by one] --> E{regular file, not huge?}
  E -- no --> F["per-file v2 command (unchanged)"]
  E -- yes --> A["GIT_INDEX_FILE=$TMP (empty) git add -N --pathspec-from-file"]
  A --> D["git diff --numstat / --patch against $TMP"]
  D --> M{"chunk header == diff --git a/f b/f?"}
  M -- yes --> OK[use batched result]
  M -- no --> F
Loading
  • status() and files() keep the v2 flow. Before the loop, untrackedBatch prefetches results for exactly the files v2 would pass to statUntracked / patchUntracked. Each lookup falls back to the original per-file command.
  • The throwaway index starts empty and holds only those files. The user's index is never read or written. Tracked files, git rm --cached paths and nested repos keep their v2 handling.
  • git diff against an intent-to-add entry applies the same attributes as --no-index (text=auto, clean filters, -diff, binary, working-tree-encoding), so the patch text matches byte for byte.
  • Anything the batch can't reproduce exactly falls back: directories, symlinks, files over 5 MB, quoted paths, non-default diff prefixes, files git add rejects.
  • git add runs with --literal-pathspecs, --force, --sparse and -c core.splitIndex=false.

Benchmark (257 untracked files, Windows)

v2 this PR
git processes started 783 17
diff("working") 28.4–34.0 s 0.34–0.36 s
status() 14.1–14.3 s 0.24–0.26 s

Parity

400 randomized repos, comparing status() and diff() (working, branch, committed, context: 1) between v2's adapter and this one:

result
identical output 400 / 400
.git and index unchanged 400 / 400
git processes started (total) 26,877 → 17,984

The repos covered .gitattributes, git rm --cached, feature branches, nested repos, diff.mnemonicPrefix, repos without commits, staged-then-deleted files, intent-to-add-then-deleted files, core.splitIndex, symlinks, 6 MB and 11 MB files, sparse checkout, and subdirectory scopes.

Copilot AI balanced review requested due to automatic review settings October 5, 2026 23:34

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@Hona Hona changed the title fix(core): diff untracked files in one batched git call fix(core): batch untracked file diffs instead of spawning git per file Oct 6, 2026
@nexxeln
nexxeln merged commit a67cf4c into anomalyco:v2 Oct 8, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants