Skip to content

perf(server): overlap untracked discovery in detailed Git status - #28985

Closed
gchen0002 wants to merge 2 commits into
pingdotgg:mainfrom
gchen0002:codex/windows-git-status-overlap-20261101
Closed

gchen0002 wants to merge 2 commits into
pingdotgg:mainfrom
gchen0002:codex/windows-git-status-overlap-20261101

Conversation

@gchen0002

@gchen0002 gchen0002 commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

Detailed Git status currently resolves its comparison base before discovering untracked files. These independent reads add to the same response latency.

Overlap untracked-file discovery with base resolution at concurrency two. Keep index-path/deletion queries, temporary-index copying, and intent-to-add work after base resolution succeeds. This avoids speculative index preparation on base failure. The shared eight-process Git limit remains unchanged. The scan's existing pathspec bounds, hidden-untracked fallback, output/file limits, and returned status schema are preserved. Review previews use the same split helpers in their original sequence.

The existing real-Git success case now gates base verification on discovery and asserts that copying waits for a verified base. It also checks totals, literal/nested paths, source-index isolation, and successful cleanup. Three lifecycle cases cover interrupted discovery/base reads, base failure, and index failure with lock cleanup. No timing assertions, sleeps, polling, retries, or per-test timeouts were added. The reads have the same dependencies across operating systems, so scheduling stays shared; performance was measured on Windows.

Duplicate coverage was checked against open #21875, #28778, #26125, #24841, and merged #28374. Those address per-file preparation sharing, branch-config command count, branch-rename handling, review source selection, and untracked-root reuse. This changes scheduling within one detailed status read.

Windows measurements call the actual production statusDetailsLocal with includeDivergence:false and includeBranchChanges:true. The fixed main source is dd6c05ec313c6c870c7a7688e87143da5084031d; both variants share this checkout's dependencies and service layers. Host: win32 10.0.26300 x64, AMD Ryzen 5 5600, 32 GiB RAM, Node v24.18.0, git version 2.41.0.windows.1. Synthetic repositories contain 2,048 ignored build files and 43 one-line untracked files where specified; index stat data is refreshed before timing. Two warm-up pairs precede 20 alternating-order pairs per scenario, with verification commands stopped during measurement.

These are arithmetic means. Multi-project rows report batch completion and per-project response time from the common batch start, including queueing. Negative Mean change means less elapsed time. Small differences can reflect host noise; these fixture means are descriptive.

Scenario main this PR Mean change
501 tracked / 43 untracked 441.826 ms 413.525 ms -6.41%
30,721 tracked / 43 untracked 834.568 ms 760.474 ms -8.88%
Tiny clean (2 tracked) 274.514 ms 252.349 ms -8.07%
501 / 43, recorded branch base 414.436 ms 388.931 ms -6.15%
501 / 43, unrelated base 195.264 ms 194.259 ms -0.51%
30,721 / 43, unrelated base 526.115 ms 520.904 ms -0.99%
2 projects, batch completion 452.228 ms 430.543 ms -4.80%
2 projects, mean per-project latency 447.840 ms 426.627 ms -4.74%
4 projects, batch completion 561.394 ms 547.466 ms -2.48%
4 projects, mean per-project latency 548.368 ms 535.116 ms -2.42%
8 projects, batch completion 934.636 ms 953.164 ms +1.98%
8 projects, mean per-project latency 831.162 ms 874.794 ms +5.25%

The saturated eight-project fixture still regresses: +1.98% batch completion and +5.25% mean per-project response time. This remains an unresolved contention cost. The lower latency in the ordinary-read fixtures does not establish a general throughput improvement.

Successful calls keep the same command counts: 16 for unconfigured untracked fixtures, 13 for a recorded base, and 12 for clean. Single-project peak children remain two. Two-, four-, and eight-project batches keep 32, 64, and 128 commands, with observed peaks of four, eight, and eight for both variants. Index-copy bytes are unchanged on successful reads.

On base failure, this PR may still perform one additional read-only scan: 8 to 9 commands for the small unrelated-history fixture, and 11 to 12 for the large fixture. Both variants copy zero index bytes and create no temporary index in those error cases. This remains extra work even when elapsed time is similar. All 180 measured pairs return matching complete results; every sample preserves source-index bytes and leaves no recorded active children, temporary indexes, or locks. No CPU, whole-app/UI, or cross-platform performance gain is claimed.

Benchmark script, compact samples, full summary, and reproduction instructions. Place the script in a fresh directory outside a PR checkout with dependencies installed. Run node <directory>/status-benchmark.mjs 0 to prepare owned fixtures, then node <directory>/status-benchmark.mjs 20. It starts no server and uses no live T3 state.

Short runtime transcript from the script's checked results:

501 tracked / 43 untracked: branchChanges = { baseRef: "main", insertions: 43, deletions: 0 }
unrelated base: branchChanges = undefined, copiedIndexBytes = 0, tempIndexCount = 0
all 180 pairs: complete results match; source indexes unchanged; temporary indexes and locks removed

Verification:

  • vp test run apps/server/src/vcs/GitVcsDriverCore.test.ts -t 'reads Changes totals with untracked files when requested|cleans concurrent Changes preparation': four pass. The original overlap gate failed on main at the unchanged suite ceiling. The added copy-order assertion failed on the eager-preparation draft and passes with this split.
  • vp run check:changed fa31b66bbb2ae5234205a8628a78e038d917202f --concurrency 2: uses the two-file diff against the fixed merge-base to select affected checks. Format, lint, both knip passes, GraphQL, and scoped typechecking of three packages pass. Tests: 215 pass, 9 fail, 4 skip. The nine failures exactly match the previously reproduced main baseline on Windows: five SSH fixtures, root/shallow review, an invalid Windows filename, trailing-space cwd spawning, and a worktree separator assertion. The four changed-path cases pass.
  • Exactly one independent gpt-6.1-sol review was completed on initial revision 67785cbc6: no blocking correctness findings; it retained the four distinct cases and requested per-project saturation reporting. This later scan/index refinement was checked by the primary agent without a second independent review, preserving the requested one-review limit.

Model: gpt-6.1-sol. Harness: Codex in T3 Code.

@github-actions github-actions Bot added vouch:unvouched PR author is not yet trusted in the VOUCHED list. size:S 10-29 changed lines (additions + deletions). labels Nov 2, 2026
@github-actions github-actions Bot added size:M 30-99 changed lines (additions + deletions). and removed size:S 10-29 changed lines (additions + deletions). labels Nov 2, 2026
@gchen0002 gchen0002 changed the title perf(server): overlap Git preparation for Changes totals perf(server): overlap untracked discovery in detailed Git status Nov 2, 2026
@gchen0002

Copy link
Copy Markdown
Contributor Author

Closing this experiment: the measured latency gains do not justify proceeding with the remaining contention regressions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:M 30-99 changed lines (additions + deletions). vouch:unvouched PR author is not yet trusted in the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant