Skip to content

Fewer false cache positives: hash --only inputs and dependsOn cache=false tasks - #3

Open
solsson wants to merge 16 commits into
mainfrom
cache-hash-dependson-and-only
Open

solsson wants to merge 16 commits into
mainfrom
cache-hash-dependson-and-only

Conversation

@solsson

@solsson solsson commented May 8, 2026

Copy link
Copy Markdown
Owner

Leksat and others added 14 commits March 30, 2025 17:35
When a task's inputs match outputs of its dependsOn tasks, defer file
hashing to dispatch time -- after dependencies have executed and their
output files exist on disk. This replaces the upfront hash computation
that saw stale or missing files.

Correctly handles:
- Deterministic deps: second run is FULL TURBO
- Non-deterministic deps (cache:false): dependent misses when output changes
- Transitive chains: prepare -> transform -> build

Implementation:
- Tasks with dep-output overlaps are skipped in precompute_task_hashes()
- At dispatch time (after deps executed), compute_deferred_hash() re-hashes
  the task's input files and computes the full task hash
- TaskHasher.hashes uses RwLock to allow updating through &self

4 integration tests: simple caching, transitive caching, config info
messages, and non-deterministic output detection.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Builds turbo binaries for 4 architectures (linux-amd64, linux-arm64,
darwin-amd64, darwin-arm64) and publishes them as GitHub release
assets with a sha256 checksums file.

Triggered by tag push. No npm publish.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When a non-deferred task depends on a deferred task, its precomputed
hash includes the deferred task's stale dependency hash. At dispatch
time, after the deferred task is re-hashed, propagate the rehash to
all downstream tasks by tracking which tasks changed.

Previously, deferred tasks were skipped entirely in precompute, causing
"Missing hash for dependent task" errors when non-deferred downstream
tasks tried to compute their dependency hashes.

Fix: all tasks precompute normally (with potentially stale file state).
At dispatch time, deferred tasks re-hash. Any task with a re-hashed
dependency also re-hashes. This cascade is safe because the engine
dispatches in dependency order.

Adds integration test for the pattern: prepare -> schemas (deferred)
-> checks (not deferred).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add config-only detection that identifies tasks whose inputs include
files produced by their dependencies' outputs. This pattern causes
unstable cache hashes because turbo computes file hashes before
dependencies execute.

Detection uses exact string matching, glob matching (via wax), and
directory containment (bare "dir" input matches "dir/**" output),
including through transitive dependency chains. Only same-package
dependencies are checked since cross-package tasks write to different
directories.

Results are deduplicated by task name and shown as info messages in
the run prelude, informing the config author regardless of --filter.

15 unit tests cover pattern matching (7), engine-level graph
traversal (5), and config-level deduplication (3).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Config-level flagging identifies turbo.json patterns where a task's
inputs match a dependency's outputs. At runtime, only defer hashing
for tasks where the dependency package actually has the script in its
package.json. If the dep has no script, the task is a no-op and won't
produce output files -- upstream behavior is preserved.

This eliminates unnecessary deferred rehashing for packages that
inherit root turbo.json task definitions but don't implement the
dependency task (e.g. eslint-config-y inheriting schemas/fetch
config but having no fetch script).

Also renames "detection" to "flagging" in comments and docs to
distinguish config-level analysis from runtime behavior.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Tasks whose inputs match a dependency's outputs have unreliable
precomputed hashes -- the real hash is only known after dependencies
execute. Mark these tasks with <DEFERRED> in the TaskHashTracker so
dry run output and summaries don't imply a stable cache prediction.

The real hash is still used internally for cache operations via
task_cache (correct because at dispatch time deps have executed).

Uses the same deferred_hash_tasks set for both dry and non-dry
modes -- no separate code paths.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The tracker override was applied unconditionally, causing build run
summaries to show <DEFERRED> instead of the real hash. Now only
applied in dry mode where deps haven't executed and the hash is
based on stale file state.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace generic <DEFERRED> with <DEPENDS_ON_OUTPUT: dep#task,...>
listing which dependency tasks produce the outputs that this task's
inputs match. This makes the dry run output actionable -- the user
can see exactly which dependency relationship causes deferred hashing.

Changes deferred_hash_tasks from HashSet to HashMap<TaskId, Vec<TaskId>>
to carry the triggering dependency information through to display.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace all "deferred"/"defer" naming introduced by this branch with
"depends-on-output" to clearly indicate the feature's purpose. The
term "deferred" was already used in the repo for telemetry, SCM, and
logging with generic "do later" semantics.

Renames: deferred_hash_tasks -> depends_on_output_tasks,
compute_deferred_hash -> compute_depends_on_output_hash,
rehashed_tasks -> output_changed_tasks. Debug log entries now use
"depends-on-output" prefix.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Documents a cache correctness issue where --only drops ^dependsOn
edges even for direct package.json dependencies. This causes stale
cache hits when upstream packages change.

The test asserts that `turbo bundle --only --filter=runtime` should
keep types#bundle in the graph (since runtime depends on types and
bundle declares ^bundle). Currently it doesn't — the edge is dropped
and changes in types are invisible to the cache key.

Marked #[should_panic] because the assertion currently fails.
When the engine is fixed to preserve these edges, remove should_panic.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
When --only filters out a task from the dependency graph, emit a
tracing::warn with the dropped edge, the affected task, and a
suggestion to add --filter for the missing package.

The warnings fire at the exact point where edges are dropped in
the engine builder, so they cannot drift from actual behavior.

Example output:
  --only: dropping topological dependency types#bundle from
  runtime#bundle. Changes in types won't affect the cache key
  for runtime#bundle. To include it, add --filter=types

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@solsson solsson changed the title Fewer false cache positives due to dependsOn rehash and --only full internal dryrun Fewer false cache positives: hash --only inputs and dependsOn cache=false tasks May 8, 2026
…ctness

When --only excludes a dependency task from execution, the downstream
task's cache key now includes file hashes from the excluded dependency's
package directory. This means source changes in dependency packages
correctly invalidate the cache even though those packages' tasks don't
run.

The implementation:
- Engine records dropped dependencies (from the warning commit)
- Visitor computes SCM file hashes for each dropped dep's package
- Hashes are combined with the original task hash via xxh64

This fixes cache staleness for patterns like:
  turbo bundle --only --filter=checkit-runtime
where ^bundle dependencies (e.g. yolean-op) are excluded from
execution but their source changes must still invalidate the cache.

Limitation: only works for packages that define the depended-on task
(e.g. yolean-op has a bundle script). Packages without the task
(e.g. yolean-types with no bundle script) have no ^bundle edge to
drop, so their files aren't tracked. Use $TURBO_ROOT$ inputs for
those cases.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@solsson
solsson force-pushed the cache-hash-dependson-and-only branch from 48cec6d to d57d360 Compare May 8, 2026 14:51
dtolnay/rust-toolchain installs targets for its configured toolchain,
but rust-toolchain.toml overrides to a different nightly. Add explicit
rustup target add to ensure the cross target is available for the
active toolchain.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@solsson

solsson commented Jul 27, 2026

Copy link
Copy Markdown
Owner Author

Research by Opus 5:

PR #3 — Fewer false cache positives: hash --only inputs and dependsOn cache=false tasks

Recommendation: partially close — split it. This PR bundles two independent fixes. Upstream has fully solved one and not started the other, so it should not be closed or rebased as a unit.

Half Upstream status Action
DependsOn output/input overlaps Solved in v2.9.17–v2.10.6, opt-in Drop from this PR — see #1
--only drops dependency edges Untouched Keep; this becomes the whole PR

Half 1: dependsOn overlaps — drop it

Upstream shipped structured inputs with startup / jit / dependencyOutputs modes (#13125), dependency-output hashing (#13129), and deferral cascade to dependents (#13045). Issue #8051 is closed.

Our bundle/fetch case — the motivating one, with a cache: false dependency — expresses directly. I verified against validate_dependency_outputs_inputs in v2.10.6 that the only constraint on a selected dependency is that it declares outputs; cache: false is not rejected:

{
  "tasks": {
    "fetch": { "cache": false, "outputs": ["target-fetched/**"] },
    "bundle": {
      "dependsOn": ["fetch", "^bundle"],
      "outputs": ["target/**"],
      "inputs": [
        "$TURBO_DEFAULT$",
        "!target-fetched/**",
        { "mode": "dependencyOutputs", "globs": ["target-fetched/**"], "from": ["fetch"] }
      ]
    }
  }
}

Concretely, these commits should come out of this PR:

  • 8612eb1ff5 defer file hashing for tasks with dependency output inputs
  • 8d09f6af36 rehash tasks whose dependencies were deferred
  • 9f343fbbaa only defer when dependency task has a script
  • cf9d0e2e08 / 86fcb58bd6 / 59d9f818f0 <DEFERRED> and <DEPENDS_ON_OUTPUT> dry-run display
  • 64274254aa deferred → depends-on-output rename
  • the turborepo-task-hash RwLock<HashMap> + update_file_hash change
  • TaskHashTracker::set_hash

The RwLock one matters beyond redundancy: upstream #13273 explicitly removed locking from the hash precompute path for performance. Keeping ours would reintroduce the contention they just eliminated, and it will conflict on every rebase.

dep_output_overlap.rs (569 lines, 15 unit tests) is the exception — it has no upstream counterpart and should survive, but re-targeted. Instead of implementing deferral itself, it can emit a synthesized dependencyOutputs input at engine-build time and let upstream's machinery take over. The glob intersection it already computes is precisely what vercel#13129's coverage validation demands. That turns ~800 lines of runtime machinery into a detection pass plus an injection point.

Half 2: --only — this is the real remaining content

Upstream v2.10.6's builder.rs still silently drops these edges, identical to 2.9.10 (details in the #2 comment). Keep:

  • Engine::dropped_dependencies / add_dropped_dependency and both recording sites in builder.rs
  • turborepo_hash::hash_string
  • the visitor fold-in of dropped-dependency SCM package hashes
  • c07e27839e the warning, and 2dbb4c5987 the #[should_panic] tripwire test

Rebase note: the fold-in currently patches one call site, calculate_task_hash. Upstream now has two — calculate_task_hash and calculate_task_hash_with_deferred_inputs (visitor lines ~763–800 at v2.10.6). Both need the dropped-dependency hash folded in, or a task that uses jit/dependencyOutputs and has an edge dropped by --only will silently skip the compensation. That combination is exactly our bundle task, so it is not hypothetical. Worth a test.

The documented limitation is still there

From d57d36034e:

only works for packages that define the depended-on task. Packages without the task have no ^bundle edge to drop, so their files aren't tracked.

Upstream has not changed anything that helps here — a package with no bundle script never produces a ^bundle edge, so there is nothing to record as dropped. $TURBO_ROOT$ inputs remain the workaround. This should stay in the PR description and ideally in the warning text, since it is the case most likely to bite: a types-only package with no build step is precisely the kind of dependency people expect to be tracked.

Suggested action

  1. Retitle to scope it to --only alone, e.g. "--only must not drop dependency packages from the cache key".
  2. Rebase onto v2.10.6 with only the Half-2 commits, plus dep_output_overlap.rs if we pursue auto-injection — otherwise defer that to its own PR.
  3. Add coverage for the deferred-inputs + --only combination noted above.
  4. Fold Document cache behavior with --only #2 in here.
  5. Tag the result v2.10.6-hashdepends.1. The release workflow needs its toolchain pin moved from nightly-2026-01-16 to nightly-2026-05-22 to match v2.10.6's rust-toolchain.toml; the release-turborepo profile and the cross-compilation-target step are unchanged.

solsson added a commit that referenced this pull request Jul 27, 2026
`--only` removes dependency edges pointing outside the filter set. Those
tasks never run, never produce a task hash, and so contribute nothing to
the downstream task's cache key. A change in an excluded package then
produces a cache *hit* and turbo restores a stale artifact.

Upstream treats the edge dropping as intended behavior for `--only`, and
this change does not argue otherwise: the task graph is left exactly as
upstream builds it. Instead the engine records each dropped edge, and the
excluded package's source hashes are folded in where the missing task
hash would have gone.

Fresh start from upstream v2.10.6, squashing what remains relevant from
#3 (previously released as v2.9.10-hashdepends.2).

The dependsOn-output half of that PR is deliberately dropped: upstream
solved it in v2.9.17-v2.10.6 via structured `inputs` modes (vercel#13043,
vercel#13045, vercel#13125, vercel#13127, vercel#13129), closing vercel#8051. Express
those cases with `mode: "jit"` or `mode: "dependencyOutputs"` instead.
Notably vercel#13273 removed locking from task hash precomputation, which is
the opposite of the `RwLock<HashMap>` the old branch carried.

Changes from the previous implementation:

- Fold the stand-in hashes into the task's dependency hash list rather
  than re-hashing the finished task hash. The tracker's copy is then
  compensated too, so dependents of an affected task also invalidate.
  This also covers all three hashing paths at once, including
  `calculate_task_hash_with_deferred_inputs`, which the old post-hoc
  approach would have missed for tasks that use `jit`/`dependencyOutputs`
  and have an edge dropped -- the exact combination in our bundle task.
- Retain `dropped_dependencies` in `prune_to_reachable` so the
  compensation survives watch and subgraph runs.
- Memoize per-package hashing; several tasks commonly drop the same
  package, and hashing walks the whole package tree.
- Deduplicate dropped edges; a package can be both a topological and a
  direct dependency.
- Pass `repo_index` so hashing uses the repo index fast path (vercel#13213).
- Drop the `hash_string` helper, unnecessary under the new approach;
  turborepo-hash is now untouched by the fork.
- Add positive tests that dropped edges are recorded on both the
  topological and direct-dependency paths, plus one asserting nothing is
  recorded without `--only`. The old branch tested none of this.
- Reword the `#[should_panic]` test as a deliberate tripwire rather than
  a bug report, and pin its expected panic message.

Refs: #1, #2, #3

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants