Include outputs from dependsOn tasks in same turbo run input hashes

[solsson]

This PR including the spec below is all LLM. I don't know Rust. But we depend heavily on Turborepo's caching and vercel#8051 unresolved means we get false positives and thus incorrect builds when task outputs match a dependent task's inputs. Such turbo.json config is valid and not warned against but the suggestion to add a test case was closed a year ago with no activity since.

Because I'm not qualified to review the PR I won't submit it upstream, but we've switched to running releases from here instead of official releases so the PR gets mileage.
If someone finds this PR and can give it a try I'd very much appreciate feedback as comments. See releases for binaries. There's no npm package for the fork.

This PR includes the test case from vercel#10253 by @Leksat

"same turbo run" in the title refers to the workaround: run every task as a new turbo process, in the order of dependsOn in tubo.json, with --only.

Claude Code session summary

Starting point: 22d12e8 (Leksat test case merge)

We need to fork turborepo to implement vercel#8051, while they closed vercel#10253. Learn the development tools needed for this repo and run all builds and tests with and without the merge in latest commit to confirm that we have both a working dev environment and a valid repro.

Set up toolchain (protoc, capnp, lld), built turbo, reproduced the bug (3 runs needed), confirmed tests pass.

Design

Yes. Three aspects of the design are key: 1) Detection based on config only... 2) Dry run is less important than avoiding false positives... 3) Actually pass the test case...

Entered plan mode, designed implementation.

First implementation (exclusion approach, later reverted)

(plan approved)

Commits on the old branch (superseded):

• bbc31de -- detection + warning
• b029382 -- engine-level tests
• 3a7f08d -- exclusion-based cache fix + integration tests
• e33f962 -- split warnings into config-level prelude and per-task execution

Build a binary and test in example-monorepo...

Tested in real project, confirmed caching worked (incorrectly, as we later discovered).

Warning refinement

do you agree with the detection warning in example-monorepo, for the other tasks that you did not test now?

Revealed false positives for packages without fetch scripts.

I have ideas but first, why wasn't for example contain's input "target-schemas" identified by the detection?

Found missing directory containment matching, added fix + tests.

The warnings printed on start should be on config only... the per-task detection would be the trigger to re-hash...

Redesigned warnings: config-level deduplicated in prelude, per-task at execution.

yes, still with test coverage for both kinds of detection

Commits on the old branch (superseded):

• 959a8e9 -- warning wording fix + filter per-task warnings to --filter scope

Course correction: exclusion was wrong

Warnings seem to work according to spec now. What were the original three solution areas?

I thought that the test case from Leksat/inputs-hash-issue-8051 was that the dependsOn with matching input should not be cached when the input changed in the same turbo run.

Realized exclusion approach was wrong: it dropped files entirely instead of deferring hashing.

In example-monorepo I have modified the example-monorepo-package1#fetch task to always write a new payload... Yes, revert "fix: exclude dependsOn outputs" and do the actual fix, including test coverage for a transitive case.

Clean branch

Now let's revisit the entire branch so I can make a maintainable and reviewable PR...

Created clean branch, then force-pushed. Current commits from this point:

1886cab -- ci: release workflow
687116e -- feat: detect overlaps in turbo.json
2a23641 -- fix: defer file hashing

Crash fix

I fail to get a build using turbo checks --filter=package2... Investigate, then create a unit test repro, then fix.

f4e7829 -- fix: rehash tasks whose dependencies were deferred (cascade fix)

Observability

seems to work. How can I get a debug log entry for when cache was recomputed?

Added debug log in compute_deferred_hash.

Seems to work. We can now change the warnings on config to info and the warning on each task with hash recompute we can use only the "deferred hash" debug...

Config overlap to info, per-task to debug only.

Dry run

I notice that --dry reports a hash for example-monorepo-package1#bundle although it depends on fetch output. Can we report "<DEFERRED>"?

Note that the logic must be the exact same code that builds use / We can not invent new code paths for dry-run

2a75009 -- show DEFERRED in dry run
f5c19ee -- fix to dry-only

Was there already a meaning of "deferred" in the repo?... Also did the change you made now affect build runs?

Found DEFERRED leaked into build summaries, fixed to dry-only.

<DEPENDS_ON_OUTPUT> is good. Or do we even have enough information to add a comma separated list?

93f3512 -- show <DEPENDS_ON_OUTPUT: example-monorepo-package1#fetch>

Naming

works. Can we use the "depends on output" terminology for log entries as well? Nothing with "defer".

7e12f70 -- rename all deferred terminology to depends-on-output

Performance

analyze why we seem to have deferred hash for all "#schemas" tasks in example-monorepo

Found 28 no-op rehashes for packages without fetch scripts.

Maybe we should not brand the config-only step as "detection" but rather as "flagging"...

045beae -- perf: only defer when dep has a script, rename detection to flagging

Spec

Problem

When a task's inputs include files produced by a dependsOn task's outputs, turbo computes file hashes before any task executes. This means:

• Run 1: The generated file doesn't exist yet -> hash X
• Run 2: The file now exists from run 1 -> hash Y (different) -> cache miss
• Run 3: Hash Y again -> cache hit

This requires 3 runs to reach stable caching. Additionally, when the dependency has cache: false and produces different output each run, the dependent task should miss -- but upstream turbo can't distinguish "file didn't exist yet" from "file content changed."

Upstream closed PR #10253 saying this is a breaking change for 3.0.

Solution

Two-layer approach: config-level flagging and depends-on-output hashing.

1. Config-level flagging (dep_output_overlap.rs)

Analyzes turbo.json task definitions to identify patterns where a task's inputs match a dependsOn task's outputs. Uses:

• Exact string matching
• Glob matching via wax (dist/** matches dist/bundle.js)
• Directory containment (bare target-schemas matches target-schemas/**)
• Transitive dependency traversal

Results are deduplicated by task name (package stripped) and shown as info messages in the run prelude:

   Task "schemas" inputs match "fetch" outputs: ["target-fetched/**"]

This fires regardless of --filter -- it informs the config author about the pattern.

2. Depends-on-output hashing (visitor dispatch)

At runtime, flagged tasks where the dependency package actually has a script get their file hash recomputed at dispatch time -- after dependencies have executed and output files exist on disk. Tasks where the dependency has no script (inherited root config, no-op) preserve upstream behavior.

The mechanism:

1. All tasks precompute hashes normally (with whatever files currently exist)
2. At dispatch time (after deps executed), depends-on-output tasks re-hash their input files
3. If the hash changed, downstream tasks also re-hash (cascade via output_changed_tasks tracking)
4. The engine dispatches in dependency order, so cascading is safe

3. Dry run

In --dry mode, dependencies don't execute, so the hash is based on stale file state. Instead of showing a misleading hash, depends-on-output tasks display:

  Hash = <DEPENDS_ON_OUTPUT: checkit-runtime#fetch>

This uses the same depends_on_output_tasks set -- no separate dry-run code path.

Behavior summary

Scenario	Upstream	This fork
Deterministic dep, run 2	Cache miss (needs run 3)	FULL TURBO
cache: false dep, output changes	Cache miss (unstable hash)	Cache miss (correct rehash)
cache: false dep, output identical	Cache miss (unstable hash)	Cache hit
Dep has no script (no-op)	N/A	Upstream behavior preserved
--dry	Shows stale hash	Shows <DEPENDS_ON_OUTPUT: dep#task>

Debug observability

With -vv:

depends-on-output checkit-runtime#bundle: hash changed abc123 -> def456
depends-on-output tsconfig-y#schemas: hash unchanged (abc123)

Grep: grep "depends-on-output\|inputs match.*outputs\|DEPENDS_ON_OUTPUT"

Test coverage

15 unit tests (dep_output_overlap.rs):

• Pattern matching: exact, glob-to-literal, literal-to-glob, directory containment, negation
• Engine graph: simple overlap, transitive chain, cross-package (no false positive), no-overlap, directory containment
• Config deduplication: across packages, distinct patterns, pattern merging

5 integration tests (dependent_task_hashing.rs):

• Simple: prepare -> build, FULL TURBO on run 2
• Transitive: prepare -> transform -> build, FULL TURBO on run 2
• Config info: verifies info message appears in output
• Changing output: cache: false dep with date output -> dependent correctly misses
• Depends-on-deferred: non-flagged task depending on flagged task (the crash case)

5 existing caching tests: all pass unchanged.

Files changed

File	Purpose
crates/turborepo-engine/src/dep_output_overlap.rs	Config-level flagging + unit tests
crates/turborepo-engine/src/lib.rs	Module registration
crates/turborepo-engine/Cargo.toml	wax dependency
crates/turborepo-lib/src/run/mod.rs	Flagging in prelude, depends-on-output set construction
crates/turborepo-lib/src/task_graph/visitor/mod.rs	Dispatch-time rehash, cascade, dry run display
crates/turborepo-task-hash/src/lib.rs	RwLock on hashes, update_file_hash, set_hash
crates/turborepo/tests/dependent_task_hashing.rs	Integration tests
turborepo-tests/integration/fixtures/dependent_task_hashing/	Test fixtures
.github/workflows/release-binary.yaml	Binary release workflow (4 arch)

[solsson]

Research by Opus 5:

PR #1 — Include outputs from dependsOn tasks in same turbo run input hashes

Recommendation: close as resolved upstream. Every mechanism this PR builds now exists in vercel/turborepo, and the upstream implementation is a strict superset of ours.

What upstream shipped

The issue this PR was chasing — vercel/turborepo#8051, "task must run twice before it caches" — is closed, resolved by a series of PRs across v2.9.17–v2.10.6:

Upstream PR	What it does	First stable release
#13043	JIT task input hashing	v2.9.17
#13045	Defers hashes for dependents of JIT tasks (our cascade)	v2.9.17
#13125	startup / jit / dependencyOutputs input modes	v2.10.0
#13129	Hashes selected dependency outputs rather than dep task hashes	v2.10.0
#13127	Deferred hash consumers in turbo watch	v2.10.0
#13273	Removes lock/dispatch overhead from hash precomputation	v2.10.4
#13426	Matches JIT inputs for affected tasks	v2.10.6

Upstream's visitor now has a first-class PrecomputedTask::Deferred state, a calculate_task_hash_with_deferred_inputs path, and dry-run reporting via hash: null + hashReason. It covers three paths this PR never touched: turbo watch, --affected, and the performance of the precompute phase.

Notably, vercel#13273 deliberately removed locking from TaskHasher, which is the opposite direction from this PR's RwLock<HashMap> + update_file_hash. Carrying our version forward would fight upstream rather than build on it.

Migrating our fixtures

The behavior is opt-in per task via structured inputs. Our three test fixtures map over cleanly.

packages/simple — build consumes prepare's output:

{
  "tasks": {
    "prepare": { "inputs": ["package.json"], "outputs": ["generated.txt"] },
    "build": {
      "dependsOn": ["prepare"],
      "inputs": [
        { "mode": "dependencyOutputs", "globs": ["generated.txt"], "from": ["prepare"] }
      ]
    }
  }
}

packages/transitive — each link declares its own dependency outputs:

{
  "tasks": {
    "prepare": { "inputs": ["package.json"], "outputs": ["generated.txt"] },
    "transform": {
      "dependsOn": ["prepare"],
      "outputs": ["transformed.txt"],
      "inputs": [
        { "mode": "dependencyOutputs", "globs": ["generated.txt"], "from": ["prepare"] }
      ]
    },
    "build": {
      "dependsOn": ["transform"],
      "inputs": [
        { "mode": "dependencyOutputs", "globs": ["transformed.txt"], "from": ["transform"] }
      ]
    }
  }
}

packages/depends-on-deferred — this is the cascade case, and it needs no config on the downstream task. Upstream defers checks automatically because its dependency schemas is deferred (vercel#13045):

{
  "tasks": {
    "prepare": { "inputs": ["package.json"], "outputs": ["generated.txt"] },
    "schemas": {
      "dependsOn": ["prepare"],
      "inputs": [
        { "mode": "dependencyOutputs", "globs": ["generated.txt"], "from": ["prepare"] }
      ]
    },
    "checks": { "dependsOn": ["schemas"], "inputs": ["package.json"] }
  }
}

Our real-world fetch → bundle shape works too. I checked validate_dependency_outputs_inputs in v2.10.6: the only requirement on a selected dependency is that it declares outputs — there is no restriction on cache: false, so this validates:

{
  "tasks": {
    "fetch": { "cache": false, "outputs": ["target-fetched/**"] },
    "bundle": {
      "dependsOn": ["fetch", "^bundle"],
      "outputs": ["target/**"],
      "inputs": [
        "$TURBO_DEFAULT$",
        "!target-fetched/**",
        { "mode": "dependencyOutputs", "globs": ["target-fetched/**"], "from": ["fetch"] }
      ]
    }
  }
}

The one thing that does not carry over

Upstream requires the overlap to be declared. There is no automatic detection — grep -i overlap across turborepo-engine, turborepo-lib/src/task_graph, and turborepo-task-hash at v2.10.6 returns nothing. vercel#13129 goes further and rejects dependencyOutputs.globs not covered by the selected dependency's declared outputs, so being explicit is a deliberate design choice, not an oversight.

So dep_output_overlap.rs and its 15 unit tests are the only part of this PR with no upstream counterpart. That capability is worth keeping, but it belongs in the follow-up work, not here — and in a much smaller form: rather than implementing deferral itself, it can synthesize a dependencyOutputs entry at engine-build time and let upstream's tested machinery do the rest. The glob intersection it already computes is exactly what satisfies vercel#13129's coverage rule.

Suggested action: close this PR, migrate our turbo.json files to mode: "dependencyOutputs", and track auto-detection separately if we still want zero-config correctness after seeing how the explicit config feels in practice.