fix(isolation-core): bound stalled git commands - #8998
Closed
code-yeongyu wants to merge 5 commits into
Closed
code-yeongyu wants to merge 5 commits into
code-yeongyu wants to merge 5 commits into
Conversation
Owner
Author
|
Focused Windows soaks of the final head (windows-latest, packages/isolation-core/src/git, 20 iterations each): 36334212006 success, 36334218625 success, 36334225286 success. |
code-yeongyu
marked this pull request as ready for review
September 27, 2026 17:02
code-yeongyu
force-pushed
the
fix/win-ci-isolation-nested-hang
branch
from
September 27, 2026 17:28
c5b536c to
b6cfd61
Compare
… bounded git reads
…out instead of a real stalled git (#8997) The stand-in stalled process left a Windows grandchild holding the fixture directory past teardown (EBUSY, seen in integration run 36336999917 attempt 2). The deadline and tree teardown of a real stalled git stay covered in command.test.ts; this test now proves only the retry contract.
Owner
Author
|
Re-soak after the retry-test fix b76fdd0 (the earlier test's real stalled git left a Windows grandchild holding the fixture dir, EBUSY, in integration run 36336999917 attempt 2): windows-latest, packages/isolation-core/src/git, 20 iterations each: 36340075853 success, 36340077334 success, 36340078993 success. |
Owner
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #8997
Refs #8324
Summary
isolation-corecould wait forever when a local Git subprocess stopped making progress. The observed Windows failure leftgit ls-files --others --exclude-standard -zalive until Bun's 30-second test timeout killed it, after which fixture cleanup raced the still-held working directory and failed withEBUSY.This PR closes the loop at three layers:
runGitcall has a bounded lifetime and typed full-tree timeout teardown;Root cause
packages/isolation-core/src/git/baseline.ts:47-56runs the nested repository untracked-file scan throughrunGit.packages/isolation-core/src/git/command.ts:134-191previously had output limits and caller abort support, but no command deadline while the direct Git process remained alive.36329259514attempt 2, Bun reported one dangling process at the test timeout. The later child-process error identified the direct child asgit ls-files --others --exclude-standard -zin the copied nested repository.The log does not expose the internal Git-for-Windows wait that made this one invocation stop progressing. The original product defect was relying on every external Git process to exit eventually; the follow-up risk was that a deadline alone would turn the same stall into a deterministic red. Baseline reads therefore also remove Git's optional background/index acceleration paths and recover once from a transient external stall.
Fix
packages/isolation-core/src/git/command.ts:6-24GitCommandTimeoutErrorwith exit code 124 and the configured deadline.packages/isolation-core/src/git/command.ts:48-82taskkill /T /F.taskkillcannot start or exits nonzero.packages/isolation-core/src/git/command.ts:171-207packages/isolation-core/src/git/baseline.tsGIT_OPTIONAL_LOCKS=0, preventing optional lock-taking/index refresh side effects.-c core.fsmonitor=false -c core.untrackedCache=false, avoiding a Git-for-Windows fsmonitor daemon/hook and stale persistent untracked-cache state in freshly copied repositories.core.fscacheunchanged: Git for Windows documents it as per-process bulklstatcaching, not a daemon or persistent index feature.packages/isolation-core/src/git/command.test.ts:79packages/isolation-core/src/git/baseline.test.tsls-filesattempt to stall through a real Git alias..git/indexbyte-identical.Git documentation basis
GIT_OPTIONAL_LOCKS: false skips optional operations requiring locks, including index refresh side effects.core.fsmonitor: true enables the built-in monitor daemon on Windows/macOS; otherwise it may name a hook command.core.untrackedCacheandgit-update-index: the cache is persisted in the index and is added/removed when the index is read.core.fscache: bulk-reads and caches directorylstatdata inside Git; it does not create a background daemon or persistent cache, so this PR does not disable it.QA & Evidence
RED
bun test packages/isolation-core/src/git/command.test.ts7024.47ms; expectedGitCommandTimeoutError, receivedundefined..omo/evidence/20260927-win-ci-isolation-nested-hang/RED.md(local, intentionally untracked)24b6147ec696ffb6c4ea2ba3381ae7aca008874f1e5b9b2226d287a8b3bdd3e0Focused GREEN
bun test packages/isolation-core/src/git/command.test.ts packages/isolation-core/src/git/baseline.test.ts19 pass,0 fail; includes the command-deadline test and the stall-then-success baseline retry test.Package suite
bun test packages/isolation-core156 pass,0 fail,7 skip;163tests across24files.Static gates
bunx tsgo --noEmit -p packages/isolation-core/tsconfig.json— exit 0.bunx biome check <five changed files>— exit 0.git diff --check— exit 0.tsgoand has no TypeScript server installation; packagetsgois the authoritative type gate.Manual module exercise
A minimal Bun driver imported the real module and launched a live seven-second Git alias with a 50 ms deadline:
{"name":"GitCommandTimeoutError","exitCode":124,"timeoutMs":50,"elapsedMs":53}A second driver imported
captureRepoBaseline, forced the first untracked-file read to time out, used the production retry logger, and completed through the fresh process:Non-Windows control
bun test packages/isolation-core/src/git/baseline.test.ts --rerun-each 50550 pass,0 fail;50/50file runs.Local QA artifact
.omo/evidence/20260927-win-ci-isolation-nested-hang/LOCAL-QA.md(local, intentionally untracked)e400fe54024c986c20cfdab1002d6afabddd07525af746dd3fdf990f1f907982Windows proof
The first three runs were cancelled after lead review expanded the fix. Three replacement
windows-latestruns tested updated soak commit851930d7dc86a18344eb8cfdf4719cc119fa7e25, each repeatingpackages/isolation-core/src/git20 times:Total focused proof: 60 successful Windows iterations, with telemetry upload and job-summary steps also successful in every run.
Risks & residuals
runGitcaller without constraining normal merge operations to the shorter baseline budget.GitCommandTimeoutError, inside the read-only baseline path. No mutating Git command is retried.