Repository navigation
perf(task): benchmark artifact cache phases - #11614
Conversation
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Central YAML (base), Organization UI (inherited) Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Greptile SummaryThe PR adds an isolated benchmark harness for task artifact-cache hashing, publication, restoration, and graph traversal, plus task-runner wiring and usage documentation.
Confidence Score: 4/5The PR is not ready to merge because the restore and cached-graph phases still measure the wrong behavior. The restore setup deletes the output but the current-output fast path skips before extraction, while source-less graph tasks do not receive cache configuration and therefore never create the entries the cached traversal phase claims to measure. Files Needing Attention: benchmarks/task-cache/benchmark.sh, TASK_CACHE_PARITY.md Important Files Changed
Reviews (10): Last reviewed commit: "perf(task): benchmark artifact cache pha..." | Re-trigger Greptile |
0183658 to
db16a64
Compare
Instruction counts
No instruction-count regression above 1%. Only instruction counts gate. Wall clock is shown for context — on identical hardware it moves 4-20% run to run. Measured by tak — instruction-counted CLI benchmarks, stored in this repository's git notes.
|
db16a64 to
c67b7b1
Compare
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
There are 2 total unresolved issues (including 1 from previous review).
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit c67b7b1. Configure here.
04a9c68 to
f338896
Compare
f338896 to
4711996
Compare
4711996 to
5f69079
Compare
5f69079 to
4711996
Compare
4711996 to
8cf46ff
Compare
8cf46ff to
03aa160
Compare
Dismissed because a newer commit was pushed; Greptile will re-review the current head.
## Summary - add an isolated task artifact cache benchmark harness - measure source hashing, archive publication, restoration, and cached graph traversal separately - generate configurable large source, artifact, and dependency-graph fixtures - document usage and complete the benchmark tracker item ## Validation - `TASK_CACHE_BENCH_SOURCES=10 TASK_CACHE_BENCH_ARTIFACT_BYTES=65536 TASK_CACHE_BENCH_TASKS=5 TASK_CACHE_BENCH_RUNS=2 MISE_BIN="$PWD/target/debug/mise" bash benchmarks/task-cache/benchmark.sh` - `mise run lint-fix` *AI-assisted — Tool: Codex; model: unavailable/unavailable; version: unavailable.* <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Low Risk** > Benchmark and documentation only; no changes to cache runtime behavior or production code paths. > > **Overview** > Adds an **end-to-end benchmark harness** for the experimental local task artifact cache, wired as `mise run perf:task-cache` (release build + `benchmarks/task-cache/benchmark.sh`). > > The script spins up a **temporary mise project** with configurable scale (defaults: 2k sources, 32 MiB artifact, 500-node graph) and reports **median wall-clock** times for four phases: source hashing / result-only publish, task run + archive creation, cache restore, and fully cached graph traversal. Fixture prep and cache warm-up sit outside the timed windows; `MISE_BIN` and `TASK_CACHE_BENCH_*` env vars tune binary and size. > > **Docs and tracking:** README describes usage and env overrides; `TASK_CACHE_PARITY.md` marks the hashing/archive/restore benchmark tracker item done; `tasks.md` documents the new task. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit 03aa160. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY -->

Summary
Validation
TASK_CACHE_BENCH_SOURCES=10 TASK_CACHE_BENCH_ARTIFACT_BYTES=65536 TASK_CACHE_BENCH_TASKS=5 TASK_CACHE_BENCH_RUNS=2 MISE_BIN="$PWD/target/debug/mise" bash benchmarks/task-cache/benchmark.shmise run lint-fixAI-assisted — Tool: Codex; model: unavailable/unavailable; version: unavailable.
Note
Low Risk
Benchmark and documentation only; no changes to cache runtime behavior or production code paths.
Overview
Adds an end-to-end benchmark harness for the experimental local task artifact cache, wired as
mise run perf:task-cache(release build +benchmarks/task-cache/benchmark.sh).The script spins up a temporary mise project with configurable scale (defaults: 2k sources, 32 MiB artifact, 500-node graph) and reports median wall-clock times for four phases: source hashing / result-only publish, task run + archive creation, cache restore, and fully cached graph traversal. Fixture prep and cache warm-up sit outside the timed windows;
MISE_BINandTASK_CACHE_BENCH_*env vars tune binary and size.Docs and tracking: README describes usage and env overrides;
TASK_CACHE_PARITY.mdmarks the hashing/archive/restore benchmark tracker item done;tasks.mddocuments the new task.Reviewed by Cursor Bugbot for commit 03aa160. Bugbot is set up for automated code reviews on this repo. Configure here.