ci: Retry Windows nextest aborts from transient 0xc0000142 spawn failures - #13240
Merged
Merged
Conversation
…ures Windows runners intermittently refuse to initialize new processes for a few seconds under nextest process churn, aborting every in-flight and subsequent test with 0xc0000142 (STATUS_DLL_INIT_FAILED). The previous mitigation capped concurrency for subprocess-heavy integration binaries, but the latest occurrence cascaded through plain unit-test crates, so scope retries with a delay to Windows instead.
Contributor
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
anthonyshew
pushed a commit
that referenced
this pull request
Jul 4, 2026
## Release v2.10.4-canary.1 > [!CAUTION] > Versioned docs aliasing FAILED. [View logs](https://github.com/vercel/turborepo/actions/runs/28719728846) ### Changes - release(turborepo): 2.10.3-canary.8 (#13200) (`5987ed1`) - fix(watch): recover from slow initial hash instead of timing out (#13159) (`05e26cb`) - chore: Update vitest example to vitest 4 and use native test merging (#13202) (`e2b61dc`) - feat: Toggle between the TUI and streamed logs (#13203) (`1325455`) - fix: Resolve Bun scoped package deps that share a name with a dependency (#13207) (`e4773ab`) - feat: Select tasks by clicking rows in the TUI task list (#13206) (`c568f67`) - perf: Avoid deep-cloning dependency maps in lockfile closure cache (#13209) (`abd84cf`) - perf: Cache root internal dependencies in package graph (#13211) (`7ec8c6d`) - feat: Automatically copy TUI selection to clipboard on mouse release (#13208) (`61d6013`) - perf: Cache env wildcard matches across tasks during hashing (#13210) (`148b1dd`) - perf: Replace per-package graph traversals in scope filtering (#13212) (`28d1871`) - perf: Derive dirty hash from repo index (#13213) (`b521d32`) - fix: Include untracked symlinks in repo-index dirty hash (#13218) (`c3c91ab`) - refactor: Use upstream libghostty-vt crates instead of vendored bindings (#13205) (`c2115dc`) - fix: Remove devtools feature flag (#13219) (`65efe27`) - fix: Harden TUI terminal restore during shutdown (#13220) (`dccab93`) - fix: Correct gitignore precedence in untracked walk and memoize matcher chains (#13221) (`bc32fcc`) - release(turborepo): 2.10.3 (#13222) (`02f4ed0`) - ci: Retry Windows nextest aborts from transient 0xc0000142 spawn failures (#13240) (`222ffff`) - perf: Use mimalloc as the global allocator (#13237) (`c42a8a9`) - perf: Parse yarn v1 lockfiles in a single pass (#13241) (`6207099`) - ci: Fix cache outputs for Eve app (#13243) (`d8b556d`) - perf: Stop materializing spans for the disabled daemon log layer (#13244) (`623aadc`) - fix: Keep ancestor-scoped bun dependencies resolvable after prune renames (#13236) (`8658b66`) - perf: Overlap external dependency hashing with package file hashing (#13234) (`445b0c7`) - perf: Build tracked repo index concurrently with package graph (#13232) (`f4699c6`) - perf: Release tokio runtime at exit so background DNS lookups never stall the user (#13231) (`ae635bd`) - perf: Overhaul pnpm lockfile parsing and dependency closure computation (#13228) (`669676d`) - feat: Add futureFlags.experimentalCargoWorkspaces flag (no-op) (#13227) (`f546930`) - feat: Introduce toolchain provider abstraction (#13235) (`f03addb`) - perf: Parse Berry lockfiles in a single pass (#13242) (`6d36125`) - fix: Apply input exclusion globs to the filesystem walk (#13224) (`d3f9dd3`) - fix: Stop root-directory packages from claiming every file in change mapping (#13225) (`f80203a`) - chore: Rename turborepo-repository napi package to @turbo/repository (#13226) (`c1972b3`) - perf: Hash git blobs with hardware-accelerated SHA-1 (#13245) (`17ffa0c`) Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
anthonyshew
pushed a commit
that referenced
this pull request
Jul 6, 2026
## Release v2.10.4 > [!CAUTION] > Versioned docs aliasing FAILED. [View logs](https://github.com/vercel/turborepo/actions/runs/28819266360) ### Changes - release(turborepo): 2.10.3 (#13222) (`02f4ed0`) - ci: Retry Windows nextest aborts from transient 0xc0000142 spawn failures (#13240) (`222ffff`) - perf: Use mimalloc as the global allocator (#13237) (`c42a8a9`) - perf: Parse yarn v1 lockfiles in a single pass (#13241) (`6207099`) - ci: Fix cache outputs for Eve app (#13243) (`d8b556d`) - perf: Stop materializing spans for the disabled daemon log layer (#13244) (`623aadc`) - fix: Keep ancestor-scoped bun dependencies resolvable after prune renames (#13236) (`8658b66`) - perf: Overlap external dependency hashing with package file hashing (#13234) (`445b0c7`) - perf: Build tracked repo index concurrently with package graph (#13232) (`f4699c6`) - perf: Release tokio runtime at exit so background DNS lookups never stall the user (#13231) (`ae635bd`) - perf: Overhaul pnpm lockfile parsing and dependency closure computation (#13228) (`669676d`) - feat: Add futureFlags.experimentalCargoWorkspaces flag (no-op) (#13227) (`f546930`) - feat: Introduce toolchain provider abstraction (#13235) (`f03addb`) - perf: Parse Berry lockfiles in a single pass (#13242) (`6d36125`) - fix: Apply input exclusion globs to the filesystem walk (#13224) (`d3f9dd3`) - fix: Stop root-directory packages from claiming every file in change mapping (#13225) (`f80203a`) - chore: Rename turborepo-repository napi package to @turbo/repository (#13226) (`c1972b3`) - perf: Hash git blobs with hardware-accelerated SHA-1 (#13245) (`17ffa0c`) - release(turborepo): 2.10.4-canary.1 (#13247) (`a31389e`) - feat: Discover Cargo crates as packages (#13248) (`d813d12`) - perf: Reuse per-package external dependency hashes in run summaries (#13249) (`ef7ef06`) - perf: Stat workspace turbo.json files concurrently during discovery (#13251) (`57be8e5`) - chore: Add tracing spans to workspace discovery and globwalk phases (#13252) (`bd76946`) - fix: Avoid non-reentrant libc calls in concurrent shutdown process scans (#13256) (`bf2d827`) - perf: Memoize resolved task definitions during engine construction (#13257) (`9085289`) - perf: Compute transitive closures and external dependency hashes concurrently with run setup (#13250) (`b305fe2`) - perf: Probe microfrontends configs in parallel (#13262) (`ea7c1f8`) - feat: Execute Cargo crate tasks via cargo (#13261) (`ee81349`) - perf: Pre-size engine task collections (#13265) (`17bc22e`) - feat: Expose resolved experimentalCI task configuration in `turbo query` (#13264) (`0f564e5`) - feat: Derive input and output globs for Cargo tasks (#13263) (`05b0e74`) - perf: Parse large pnpm lockfile sections in parallel (#13266) (`ac4155e`) - feat: Hash Cargo external dependencies per-crate (#13267) (`71300aa`) - ci: Install pnpm 10 in musl containers for Library Release (#13269) (`5ba9f66`) - ci: Force pnpm overwrite in Library Release musl containers (#13270) (`560ee2f`) - perf: Defer the untracked-scan barrier to first file-hash use (#13268) (`96b0f5a`) - ci: Fetch API-created commit before updating local ref (#13271) (`4cbaf48`) - release(library): 0.0.1-canary.22 (#13272) (`d2e0691`) - perf: Remove lock and dispatch overhead from task hash precomputation (#13273) (`dd6de07`) - fix: Fall back to lockfile detection in @turbo/repository when package manager is undeclared (#13275) (`8cf0e75`) - test: Add end-to-end coverage for Cargo workspaces (#13274) (`ce18f0a`) - release(library): 0.0.1-canary.23 (#13276) (`8e3a59f`) - release(turborepo): 2.10.4-canary.2 (#13278) (`7e02f94`) - fix: Collapse nested package-manager fallback conditional (#13279) (`af01fdf`) - feat: Make turbo watch Cargo-aware (#13280) (`39d623e`) - perf: Skip dependency-closure assembly for toolchains that derive nothing (#13277) (`ff0d508`) - feat: Make turbo prune Cargo-aware (#13281) (`ddc584d`) - fix: Raise the open-file soft limit at startup (#13282) (`947b478`) - fix: Stop flagging relative imports that resolve into node_modules in boundaries (#13284) (`11a68c7`) - perf: Evaluate simple include globs without wax compilation (#13285) (`189897a`) - chore: Dogfood native Cargo support in this repository (#13283) (`42f067b`) --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
This branch had an error being deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
The Windows Rust test job intermittently fails with a cascade of
0xc0000142(STATUS_DLL_INIT_FAILED, os error 1114) aborts, e.g. this run: 2,651 tests passed normally, then every remaining process spawn failed instantly for ~2.5 seconds, aborting ~880 tests. This is transient session resource exhaustion under nextest's process churn, not a test bug.We previously mitigated this class by capping concurrency for subprocess-heavy integration binaries (removed in #12999 after test trimming, see analysis in #12991). The latest occurrence cascaded through plain unit-test crates (turborepo-run-cache, turborepo-scm, turborepo-ui) that those caps never covered, so capping specific binaries is not sufficient.
What
Adds a Windows-only nextest override that retries aborted tests twice with a fixed 2s delay, keeping full concurrency.
How
Since the exhaustion clears within seconds, a delayed retry recovers regardless of which tests are in flight when the storm hits. The override is scoped with
platform = 'cfg(windows)', so Linux/macOS behavior is unchanged.Verified:
taplo format --check .config/nextest.toml,cargo nextest show-config version, and a Linuxcargo nextest run -p turborepo-microfrontendspass with the new config. Note for reviewers: retries also mask genuinely flaky Windows tests; flakes will show asTRY 2 PASSin logs rather than failing the job.