perf(task): cache per-file content hashes for source_freshness_hash_contents

[jdx]

Summary

• When task.source_freshness_hash_contents = true, every freshness check currently re-reads and re-blake3-hashes every source file (discussion #9802).
• This caches each file's content hash keyed by (size, mtime_secs, mtime_nanos) — git's stat-info trick — in a per-task file under `STATE/task-sources/-content-cache`. Unchanged files are skipped on subsequent runs; the file is only re-hashed when its size or mtime moves.
• The cache is rebuilt from scratch each run, so entries for files no longer in `sources` are pruned and the cache file stays bounded by the current source set.

Design notes

• Serialization: `rmp_serde + zlib`, matching src/cache.rs and src/env_diff.rs. Considered `rkyv` for zero-copy reads, but the bottleneck this PR addresses is blake3 hashing of file contents (ms–s); parsing a `BTreeMap` of ≤1000s of entries with rmp_serde is microseconds and not on the critical path. Adding rkyv would mean a new heavy dep and a brittle byte format inconsistent with the rest of the codebase.
• Per-task scoping: cache key reuses `sources_hash_path`'s identity hash (task + config_source + root), so any change to the task definition invalidates the cache automatically.
• Atomic writes: `*.part-XXXX` + rename, same pattern as src/cache.rs:262. Two concurrent runs may lose each other's writes but cannot corrupt the cache.
• Failure handling: corrupt/unreadable cache → silently fall back to empty (`unwrap_or_default`); failed write → trace-level log. Correctness is preserved either way; only speed is at stake.
• Cache hit predicate: `size + mtime_secs + mtime_nanos`. False positives only if a file is rewritten with identical size and mtime preserved to the nanosecond — vanishingly rare and the same trick git uses safely.

Test plan

• New unit tests in src/task/task_source_checker.rs:
  • `content_hash_cache_reuses_unchanged_files` — second call reuses cached hashes; mutating a file invalidates.
  • `content_hash_cache_prunes_dropped_files` — files removed from `sources` drop out of the cache.
  • `content_hash_cache_round_trips_through_disk` — save/load round-trips; corrupt file falls back to empty.
• Existing e2e tests still pass: `test_task_source_freshness`, `test_task_source_freshness_with_cwd`, `test_task_run_sources`, `test_task_run_sources_negation`, `test_task_dep_invalidates_sources`.

🤖 Generated with Claude Code

---

Note

Medium Risk
Touches task invalidation/freshness logic by persisting and reusing per-file content hashes; bugs could cause tasks to incorrectly be treated as fresh or stale, though it falls back safely on cache errors.

Overview
When task.source_freshness_hash_contents is enabled, freshness checks now persist a per-task, per-file blake3 hash cache (keyed by file size + mtime secs/nanos) so unchanged sources aren’t re-read/re-hashed on every run.

This introduces a stable task_state_key to scope state under STATE/task-sources/, adds msgpack+zlib load/save with atomic *.part-* writes (and silent fallback on corrupt cache / trace-only write failures), and expands unit tests to cover cache reuse, pruning, and disk round-tripping.

^{Reviewed by Cursor Bugbot for commit f1f039b. Bugbot is set up for automated code reviews on this repo. Configure here.}

[greptile-apps]

Greptile Summary

Adds an on-disk per-file content-hash cache for task.source_freshness_hash_contents, keyed by (size, mtime_secs, mtime_nanos) (git's stat-info trick), stored as msgpack+zlib under STATE/task-sources/*-content-cache. Files whose stat info is unchanged between runs are skipped without re-reading their contents.

• Refactors the shared filename stem into task_state_key so both the checksum file and the new cache file are invalidated together when the task definition changes.
• Writes the cache atomically via a random *.part-XXXX temporary file and rename; the previous concern about ZlibEncoder's Drop silently discarding finalization errors is fixed by calling zlib.finish()? explicitly before the rename.
• Adds three unit tests covering cache reuse, pruning of removed sources, and corrupt-file fallback.

Confidence Score: 5/5

Safe to merge — the cache is keyed correctly, writes are atomic, and every error path falls back to a full re-hash so task freshness decisions remain correct even when the cache is absent or corrupt.

The ZlibEncoder::finish() concern from the prior review round is explicitly fixed. The mtime cast (as_secs() as i64) is symmetric between make_cache_entry and cached_entry_matches, so cache comparisons are always consistent. Pre-epoch mtimes and unreadable cache files both fall back conservatively to a full re-hash. Atomic rename prevents partial-file reads.

No files require special attention.

Important Files Changed

Filename	Overview
src/task/task_source_checker.rs	Introduces on-disk per-file content-hash cache with atomic writes, correct explicit ZlibEncoder finalization, consistent mtime cast on both store and compare sides, and conservative fallback (always re-hashes) for pre-epoch mtimes and unreadable caches.

_{Reviews (5): Last reviewed commit: "Merge remote-tracking branch 'origin/mai..." | Re-trigger Greptile}

[gemini-code-assist]

Code Review

This pull request implements a content hash cache for task sources, using file metadata to avoid redundant hashing and persisting the cache to disk with zlib compression and MessagePack. Feedback recommends optimizing memory usage by streaming data directly through the zlib encoder and decoder during serialization and deserialization instead of using intermediate buffers.

[gemini-code-assist]

Instead of reading the entire decompressed stream into a Vec before deserializing, you can deserialize directly from the ZlibDecoder stream. This is more memory-efficient, especially if the cache grows large.

        let zlib = ZlibDecoder::new(File::open(path)?);
        Ok(rmp_serde::from_read(zlib)?)

[gemini-code-assist]

Similarly to the loading logic, you can serialize directly to the ZlibEncoder instead of creating an intermediate Vec via to_vec_named. This avoids an unnecessary allocation.

        let mut zlib = ZlibEncoder::new(File::create(&partial)?, Compression::fast());
        rmp_serde::encode::write_named(&mut zlib, cache)?;

[jdx]

Thanks @gemini-code-assist, but I'm going to pass on both of these.

The buffered `read_to_end` → `from_slice` and `to_vec_named` → `write_all` pattern is the established convention everywhere else in this codebase that does `rmp_serde + zlib` (src/cache.rs:248-251, src/env_diff.rs:136, src/hook_env.rs:363). Diverging just here adds inconsistency for a theoretical memory win that doesn't materialize at this scale — a cache entry is ~100 bytes, so even a task with 10k source files is ~1MB uncompressed. Streaming msgpack also produces less precise error messages than `from_slice` on a flat byte buffer.

If we ever want to switch the convention across the codebase, that should be a separate PR touching all four call sites at once.

This comment was generated by an AI coding assistant.

[gemini-code-assist]

Thanks for the clarification, @jdx. I completely understand the need for consistency across the codebase, especially when the current pattern is well-established and the performance impact is negligible at this scale. I agree that a codebase-wide refactor would be the better approach if we decide to move away from this pattern later. I'll keep this in mind for future reviews.

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 1ef7b76. Configure here.}

[github-actions]

Hyperfine Performance

mise x -- echo

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
mise-2026.5.6 x -- echo	19.4 ± 0.9	17.5	24.6	1.00
mise x -- echo	20.1 ± 2.6	17.6	36.9	1.04 ± 0.14

mise env

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
mise-2026.5.6 env	20.2 ± 1.9	16.8	27.5	1.00
mise env	21.7 ± 1.5	18.0	27.4	1.08 ± 0.13

mise hook-env

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
mise-2026.5.6 hook-env	20.7 ± 1.1	18.7	24.9	1.01 ± 0.07
mise hook-env	20.5 ± 1.0	18.9	25.6	1.00

mise ls

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
mise-2026.5.6 ls	16.6 ± 0.8	15.2	22.7	1.00
mise ls	17.0 ± 0.9	15.2	22.7	1.03 ± 0.07

xtasks/test/perf

Command	mise-2026.5.6	mise	Variance
install (cached)	128ms	129ms	+0%
ls (cached)	61ms	61ms	+0%
bin-paths (cached)	65ms	65ms	+0%
task-ls (cached)	503ms	503ms	+0%