Repository navigation
feat(task): stream remote cache artifacts - #11623
Conversation
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Central YAML (base), Organization UI (inherited) Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Greptile SummaryThe PR makes task-cache stores asynchronous and adds streamed, content-addressed HTTP artifact transport.
Confidence Score: 5/5The PR appears safe to merge within the scope of this follow-up review. No blocking failure remains from an eligible prior finding. Important Files Changed
Reviews (12): Last reviewed commit: "fix(task): verify and canonicalize remot..." | Re-trigger Greptile |
e6bad52 to
cb45fdc
Compare
cb45fdc to
6714f71
Compare
6714f71 to
183eea9
Compare
Instruction counts
No instruction-count regression above 1%. Only instruction counts gate. Wall clock is shown for context — on identical hardware it moves 4-20% run to run. Measured by tak — instruction-counted CLI benchmarks, stored in this repository's git notes.
|
183eea9 to
da1f8a7
Compare
da1f8a7 to
225b0d6
Compare
26bac7f to
5c12871
Compare
5c12871 to
d4e8afd
Compare
d4e8afd to
7bd9cf2
Compare
7bd9cf2 to
96e6d03
Compare
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 96e6d03. Configure here.
96e6d03 to
16390f1
Compare
## Summary - make the task cache store interface asynchronous for network-backed implementations - stream remote artifact uploads from disk and downloads into temporary files - publish artifacts before manifests and validate protocol keys, headers, and media types ## Tests - `cargo test task_cache_store::tests` - `cargo clippy --workspace --all-features --all-targets -- -D warnings` - `mise run test:e2e e2e/tasks/test_task_artifact_cache_concurrency e2e/tasks/test_task_artifact_cache_no_outputs e2e/tasks/test_task_artifact_cache_resilience` *AI-assisted — Tool: Codex; model: unavailable/unavailable; version: unavailable.* <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Medium Risk** > Touches core task cache restore/store paths and adds a large network/CAS layer; failures are mostly softened with warnings, but bad remote data or partial failures could still affect cache hits and artifact integrity checks. > > **Overview** > Adds an **async** `TaskCacheStore` and wires task execution to **await** cache get, restore, store, and current-output paths. > > **Remote HTTP backend** implements the v1 cache protocol: action results and content-addressed blobs (action bytes, canonical client metadata, directory trees, file chunks). Artifact **uploads stream from disk**; **downloads stream into temp files** before reassembly into a local `tar.zst` for restore. The composite store still promotes remote hits locally and mirrors writes, with lookup failures degrading to local-only. > > Cache keys now hash **canonical JSON** key material; commits pass the serialized **action** bytes so remote lookups are keyed by `(blake3 key, action size)`. Manifest/types are shared with the store layer; artifact paths use a small wrapper for on-disk vs temporary files. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit 16390f1. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY -->

Summary
Tests
cargo test task_cache_store::testscargo clippy --workspace --all-features --all-targets -- -D warningsmise run test:e2e e2e/tasks/test_task_artifact_cache_concurrency e2e/tasks/test_task_artifact_cache_no_outputs e2e/tasks/test_task_artifact_cache_resilienceAI-assisted — Tool: Codex; model: unavailable/unavailable; version: unavailable.
Note
Medium Risk
Touches core task cache restore/store paths and adds a large network/CAS layer; failures are mostly softened with warnings, but bad remote data or partial failures could still affect cache hits and artifact integrity checks.
Overview
Adds an async
TaskCacheStoreand wires task execution to await cache get, restore, store, and current-output paths.Remote HTTP backend implements the v1 cache protocol: action results and content-addressed blobs (action bytes, canonical client metadata, directory trees, file chunks). Artifact uploads stream from disk; downloads stream into temp files before reassembly into a local
tar.zstfor restore. The composite store still promotes remote hits locally and mirrors writes, with lookup failures degrading to local-only.Cache keys now hash canonical JSON key material; commits pass the serialized action bytes so remote lookups are keyed by
(blake3 key, action size). Manifest/types are shared with the store layer; artifact paths use a small wrapper for on-disk vs temporary files.Reviewed by Cursor Bugbot for commit 16390f1. Bugbot is set up for automated code reviews on this repo. Configure here.