perf: Remove lock and dispatch overhead from task hash precomputation - #13273
Merged
Merged
Conversation
Task hash precomputation processes the task graph in topological waves. Three sources of overhead showed up on a 1191-package monorepo whose graph is 64 waves deep: - Every task locked a global results mutex to check whether any dependency was deferred, and every wave locked it again to merge. Waves are sequential, so the map needs no lock at all: workers read it immutably during a wave and the merge writes between waves. - The deferred-dependency scan walked each task dependency set even though deferral (JIT inputs, dependency-output inputs) is rare. Track deferred task IDs in a set and short-circuit when it is empty. - A deep chain means most waves hold one or two tasks; rayon dispatch costs more than the hashing itself there. Waves under four tasks now run inline.
Contributor
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
anthonyshew
pushed a commit
that referenced
this pull request
Jul 6, 2026
## Release v2.10.4-canary.2 > [!CAUTION] > Versioned docs aliasing FAILED. [View logs](https://github.com/vercel/turborepo/actions/runs/28806467849) ### Changes - release(turborepo): 2.10.4-canary.1 (#13247) (`a31389e`) - feat: Discover Cargo crates as packages (#13248) (`d813d12`) - perf: Reuse per-package external dependency hashes in run summaries (#13249) (`ef7ef06`) - perf: Stat workspace turbo.json files concurrently during discovery (#13251) (`57be8e5`) - chore: Add tracing spans to workspace discovery and globwalk phases (#13252) (`bd76946`) - fix: Avoid non-reentrant libc calls in concurrent shutdown process scans (#13256) (`bf2d827`) - perf: Memoize resolved task definitions during engine construction (#13257) (`9085289`) - perf: Compute transitive closures and external dependency hashes concurrently with run setup (#13250) (`b305fe2`) - perf: Probe microfrontends configs in parallel (#13262) (`ea7c1f8`) - feat: Execute Cargo crate tasks via cargo (#13261) (`ee81349`) - perf: Pre-size engine task collections (#13265) (`17bc22e`) - feat: Expose resolved experimentalCI task configuration in `turbo query` (#13264) (`0f564e5`) - feat: Derive input and output globs for Cargo tasks (#13263) (`05b0e74`) - perf: Parse large pnpm lockfile sections in parallel (#13266) (`ac4155e`) - feat: Hash Cargo external dependencies per-crate (#13267) (`71300aa`) - ci: Install pnpm 10 in musl containers for Library Release (#13269) (`5ba9f66`) - ci: Force pnpm overwrite in Library Release musl containers (#13270) (`560ee2f`) - perf: Defer the untracked-scan barrier to first file-hash use (#13268) (`96b0f5a`) - ci: Fetch API-created commit before updating local ref (#13271) (`4cbaf48`) - release(library): 0.0.1-canary.22 (#13272) (`d2e0691`) - perf: Remove lock and dispatch overhead from task hash precomputation (#13273) (`dd6de07`) - fix: Fall back to lockfile detection in @turbo/repository when package manager is undeclared (#13275) (`8cf0e75`) - test: Add end-to-end coverage for Cargo workspaces (#13274) (`ce18f0a`) - release(library): 0.0.1-canary.23 (#13276) (`8e3a59f`) Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
anthonyshew
pushed a commit
that referenced
this pull request
Jul 6, 2026
## Release v2.10.4 > [!CAUTION] > Versioned docs aliasing FAILED. [View logs](https://github.com/vercel/turborepo/actions/runs/28819266360) ### Changes - release(turborepo): 2.10.3 (#13222) (`02f4ed0`) - ci: Retry Windows nextest aborts from transient 0xc0000142 spawn failures (#13240) (`222ffff`) - perf: Use mimalloc as the global allocator (#13237) (`c42a8a9`) - perf: Parse yarn v1 lockfiles in a single pass (#13241) (`6207099`) - ci: Fix cache outputs for Eve app (#13243) (`d8b556d`) - perf: Stop materializing spans for the disabled daemon log layer (#13244) (`623aadc`) - fix: Keep ancestor-scoped bun dependencies resolvable after prune renames (#13236) (`8658b66`) - perf: Overlap external dependency hashing with package file hashing (#13234) (`445b0c7`) - perf: Build tracked repo index concurrently with package graph (#13232) (`f4699c6`) - perf: Release tokio runtime at exit so background DNS lookups never stall the user (#13231) (`ae635bd`) - perf: Overhaul pnpm lockfile parsing and dependency closure computation (#13228) (`669676d`) - feat: Add futureFlags.experimentalCargoWorkspaces flag (no-op) (#13227) (`f546930`) - feat: Introduce toolchain provider abstraction (#13235) (`f03addb`) - perf: Parse Berry lockfiles in a single pass (#13242) (`6d36125`) - fix: Apply input exclusion globs to the filesystem walk (#13224) (`d3f9dd3`) - fix: Stop root-directory packages from claiming every file in change mapping (#13225) (`f80203a`) - chore: Rename turborepo-repository napi package to @turbo/repository (#13226) (`c1972b3`) - perf: Hash git blobs with hardware-accelerated SHA-1 (#13245) (`17ffa0c`) - release(turborepo): 2.10.4-canary.1 (#13247) (`a31389e`) - feat: Discover Cargo crates as packages (#13248) (`d813d12`) - perf: Reuse per-package external dependency hashes in run summaries (#13249) (`ef7ef06`) - perf: Stat workspace turbo.json files concurrently during discovery (#13251) (`57be8e5`) - chore: Add tracing spans to workspace discovery and globwalk phases (#13252) (`bd76946`) - fix: Avoid non-reentrant libc calls in concurrent shutdown process scans (#13256) (`bf2d827`) - perf: Memoize resolved task definitions during engine construction (#13257) (`9085289`) - perf: Compute transitive closures and external dependency hashes concurrently with run setup (#13250) (`b305fe2`) - perf: Probe microfrontends configs in parallel (#13262) (`ea7c1f8`) - feat: Execute Cargo crate tasks via cargo (#13261) (`ee81349`) - perf: Pre-size engine task collections (#13265) (`17bc22e`) - feat: Expose resolved experimentalCI task configuration in `turbo query` (#13264) (`0f564e5`) - feat: Derive input and output globs for Cargo tasks (#13263) (`05b0e74`) - perf: Parse large pnpm lockfile sections in parallel (#13266) (`ac4155e`) - feat: Hash Cargo external dependencies per-crate (#13267) (`71300aa`) - ci: Install pnpm 10 in musl containers for Library Release (#13269) (`5ba9f66`) - ci: Force pnpm overwrite in Library Release musl containers (#13270) (`560ee2f`) - perf: Defer the untracked-scan barrier to first file-hash use (#13268) (`96b0f5a`) - ci: Fetch API-created commit before updating local ref (#13271) (`4cbaf48`) - release(library): 0.0.1-canary.22 (#13272) (`d2e0691`) - perf: Remove lock and dispatch overhead from task hash precomputation (#13273) (`dd6de07`) - fix: Fall back to lockfile detection in @turbo/repository when package manager is undeclared (#13275) (`8cf0e75`) - test: Add end-to-end coverage for Cargo workspaces (#13274) (`ce18f0a`) - release(library): 0.0.1-canary.23 (#13276) (`8e3a59f`) - release(turborepo): 2.10.4-canary.2 (#13278) (`7e02f94`) - fix: Collapse nested package-manager fallback conditional (#13279) (`af01fdf`) - feat: Make turbo watch Cargo-aware (#13280) (`39d623e`) - perf: Skip dependency-closure assembly for toolchains that derive nothing (#13277) (`ff0d508`) - feat: Make turbo prune Cargo-aware (#13281) (`ddc584d`) - fix: Raise the open-file soft limit at startup (#13282) (`947b478`) - fix: Stop flagging relative imports that resolve into node_modules in boundaries (#13284) (`11a68c7`) - perf: Evaluate simple include globs without wax compilation (#13285) (`189897a`) - chore: Dogfood native Cargo support in this repository (#13283) (`42f067b`) --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
anthonyshew
pushed a commit
that referenced
this pull request
Jul 7, 2026
## Release v2.10.5-canary.1 > [!CAUTION] > Versioned docs aliasing FAILED. [View logs](https://github.com/vercel/turborepo/actions/runs/28833246614) ### Changes - perf: Remove lock and dispatch overhead from task hash precomputation (#13273) (`dd6de07`) - fix: Fall back to lockfile detection in @turbo/repository when package manager is undeclared (#13275) (`8cf0e75`) - test: Add end-to-end coverage for Cargo workspaces (#13274) (`ce18f0a`) - release(library): 0.0.1-canary.23 (#13276) (`8e3a59f`) - release(turborepo): 2.10.4-canary.2 (#13278) (`7e02f94`) - fix: Collapse nested package-manager fallback conditional (#13279) (`af01fdf`) - feat: Make turbo watch Cargo-aware (#13280) (`39d623e`) - perf: Skip dependency-closure assembly for toolchains that derive nothing (#13277) (`ff0d508`) - feat: Make turbo prune Cargo-aware (#13281) (`ddc584d`) - fix: Raise the open-file soft limit at startup (#13282) (`947b478`) - fix: Stop flagging relative imports that resolve into node_modules in boundaries (#13284) (`11a68c7`) - perf: Evaluate simple include globs without wax compilation (#13285) (`189897a`) - chore: Dogfood native Cargo support in this repository (#13283) (`42f067b`) - release(turborepo): 2.10.4 (#13286) (`f2fce38`) - ci: Remove dead sccache configuration (#13289) (`558df3f`) - feat: Serve the Remote Cache as an sccache backend for Cargo tasks (#13288) (`0d9803f`) - fix: Reject output path traversal (#13290) (`733ccca`) - fix: Disable the sccache proxy when remote cache use is off (#13291) (`3249e22`) - feat: Embed sccache so the Cargo compile cache needs no installation (#13293) (`b6d0035`) Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
This was referenced Jul 27, 2026
Open
solsson
added a commit
to solsson/turbo
that referenced
this pull request
Jul 27, 2026
`--only` removes dependency edges pointing outside the filter set. Those tasks never run, never produce a task hash, and so contribute nothing to the downstream task's cache key. A change in an excluded package then produces a cache *hit* and turbo restores a stale artifact. Upstream treats the edge dropping as intended behavior for `--only`, and this change does not argue otherwise: the task graph is left exactly as upstream builds it. Instead the engine records each dropped edge, and the excluded package's source hashes are folded in where the missing task hash would have gone. Fresh start from upstream v2.10.6, squashing what remains relevant from #3 (previously released as v2.9.10-hashdepends.2). The dependsOn-output half of that PR is deliberately dropped: upstream solved it in v2.9.17-v2.10.6 via structured `inputs` modes (vercel#13043, vercel#13045, vercel#13125, vercel#13127, vercel#13129), closing vercel#8051. Express those cases with `mode: "jit"` or `mode: "dependencyOutputs"` instead. Notably vercel#13273 removed locking from task hash precomputation, which is the opposite of the `RwLock<HashMap>` the old branch carried. Changes from the previous implementation: - Fold the stand-in hashes into the task's dependency hash list rather than re-hashing the finished task hash. The tracker's copy is then compensated too, so dependents of an affected task also invalidate. This also covers all three hashing paths at once, including `calculate_task_hash_with_deferred_inputs`, which the old post-hoc approach would have missed for tasks that use `jit`/`dependencyOutputs` and have an edge dropped -- the exact combination in our bundle task. - Retain `dropped_dependencies` in `prune_to_reachable` so the compensation survives watch and subgraph runs. - Memoize per-package hashing; several tasks commonly drop the same package, and hashing walks the whole package tree. - Deduplicate dropped edges; a package can be both a topological and a direct dependency. - Pass `repo_index` so hashing uses the repo index fast path (vercel#13213). - Drop the `hash_string` helper, unnecessary under the new approach; turborepo-hash is now untouched by the fork. - Add positive tests that dropped edges are recorded on both the topological and direct-dependency paths, plus one asserting nothing is recorded without `--only`. The old branch tested none of this. - Reword the `#[should_panic]` test as a deliberate tripwire rather than a bug report, and pin its expected panic message. Refs: #1, #2, #3 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Task hash precomputation is the last gate before the first task starts. On a 1191-package monorepo the task graph is 64 topological waves deep, and instrumentation showed the wave loop paying for a global results mutex (locked once per task for a deferred-dependency check, again per wave to merge), a full dependency-set scan per task for deferral that almost never occurs, and rayon dispatch on ~60 waves holding one or two tasks each — where dispatch costs more than the hashing.
What
HashMapowned by the wave loop: waves are sequential, workers read it immutably during a wave, merges write between waves. No lock existed for a reason.par_iter.No semantic changes: same hashes, same wave ordering, same deferral behavior, same error surfacing.
How to verify
--dry=jsonbyte-identical;cargo test -p turborepo-lib432 passing.