perf: Bound remote-cache transfer memory instead of retaining whole artifacts - #14011
Merged
Merged
Conversation
…rtifacts Spool compressed artifacts through a SpooledTempFile (8 MiB threshold): small artifacts stay in memory while large ones roll to an anonymous temporary file, bounding retained payload memory for uploads and downloads. Artifact signatures are computed incrementally with bounded chunks, including as download chunks arrive. Verification still completes before any extraction and retries resend byte-identical content. Closes TURBO-6038
Contributor
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
anthonyshew
added a commit
that referenced
this pull request
Sep 9, 2026
## Summary Closes TURBO-5986. Builds on the spooled-artifact transfer work merged in #14011. With both local and remote cache writes enabled, output files were opened, read, and compressed twice: `FSCache::put` built its own archive while `HTTPCache::put` built a second copy for upload. The fetch direction had the mirror problem: a remote hit restored files and then `FSCache::put` re-read and re-compressed every restored output to populate the local cache. ## Changes - New shared `ArtifactBody` (in `artifact_body.rs`): the canonical compressed archive, held in memory below 8 MiB and spooled to an anonymous temp file above it. Built once via `ArtifactBody::from_files`; every consumer gets a fresh positioned reader or bounded stream over identical bytes. - `CacheMultiplexer::put` with local+remote writes now builds the archive once, atomically installs it into the local cache (temp + rename, 1 MiB-buffered copy, same publication semantics as `CacheWriter::create`), and uploads the same bytes. Single-destination paths are unchanged. - `CacheMultiplexer::fetch` on a remote hit with local writes installs the *downloaded* archive bytes (signature verification gates any restore or local install) instead of re-encoding the restored files. A rejected download can never become a local hit. - `HTTPCache::put` splits into archive construction and `put_body`; `fetch_with_archive` returns the verified body alongside restored files. Signing is unchanged: the tag covers exactly the uploaded bytes. ## Benchmarks End-to-end through `AsyncCache` against the repo's mock remote as a separate process; debug build; macOS. Five interleaved runs per scenario (order alternated), medians reported. Harness was temporary and is not committed. Debug-build disk I/O on this machine is noisy; the many-small-files case is the most deterministic and the multi-hundred-MiB case is dominated by page-cache variance, so phase timings from an instrumented run are included for the large case. | Scenario | Phase | Before | After | Change | | --- | --- | --- | --- | --- | | 10,000 × 100 B files | put (local+remote) | 1,980 ms | 916 ms | −54% | | 10,000 × 100 B files | remote-hit fetch + local install | 2,125 ms | 1,395 ms | −34% | | 100 × 1 MiB files | put (local+remote) | 207 ms | 148–263 ms (noisy) | ~wash to −29% | | 100 × 1 MiB files | remote-hit fetch + local install | 254 ms | 197 ms | −22% | | 1 × 512 MiB file | put (local+remote) | 3,460 ms | 2,498 ms | −28% | | 1 × 512 MiB file | remote-hit fetch + local install | 4,156–4,530 ms | 3,222–3,672 ms | −14 to −25% | Instrumented phase breakdown, 512 MiB remote-hit fetch: local archive install (buffered copy of verified bytes) 338–641 ms, replacing a full read + zstd re-encode of restored outputs (~1.1–1.2 s in the same environment). Restore time dominates and is identical in both paths. ## Correctness preserved - Local publication stays atomic (temp file + rename); metadata/manifest sidecars are written exactly as before. - Signing still covers exactly the uploaded bytes; retries after token refresh resend byte-identical content. - New tests: local install and remote upload are byte-identical after a combined put; a remote-hit fetch installs byte-identical archive bytes locally. - `cargo test -p turborepo-cache`: 159 passed, 0 failed. `cargo clippy -p turborepo-cache`: clean.
github-actions Bot
added a commit
that referenced
this pull request
Sep 10, 2026
## Release v2.10.13-canary.3 > [!CAUTION] > Versioned docs aliasing FAILED. [View logs](https://github.com/vercel/turborepo/actions/runs/34474193742) ### Changes - perf: Reuse resolver inference during file tracing (#13999) (`596ea4c`) - chore: Release Turborepo 2.10.13-canary.2 (#14000) (`ebfc808`) - chore: Exclude turbo and @turbo/* from minimum release age (#14005) (`5897420`) - perf: Binary search boundaries comments before each import (#14003) (`6a95c81`) - perf: Fetch only npm dist tags in the binary version endpoint (#14007) (`4db9ca0`) - perf: Compute Devtools graph depths without rescanning every edge (#14001) (`ef46772`) - perf: Reuse package manager detection during create startup (#14006) (`06f5900`) - perf: Borrow workspace entries during dependency lookup (#14002) (`ba822e1`) - perf: Cache LSP task index and dedupe identities in constant time (#14008) (`77f6232`) - perf: Deduplicate recursive glob prefixes before walking (#14004) (`90fd2d6`) - perf: Batch boundaries progress updates (#14009) (`3862f3e`) - perf: Reuse archive anchor during cache creation (#14010) (`f01afa9`) - perf: Bound remote-cache transfer memory instead of retaining whole artifacts (#14011) (`470ffb9`) - perf: Build cache archives once for local and remote writes (#14012) (`f0a0d77`) - perf: Make VS Code binary discovery and LSP probes nonblocking (#14015) (`8cc0c42`) - perf: Avoid serializing ASTs for path-only file dependency queries (#14016) (`96712c9`) - perf: Share source text across boundaries diagnostics instead of copying it per error (#14014) (`5223bd6`) - perf: Parse the lockfile once when opening a native Workspace with its graph (#14017) (`23ee440`) - docs: Document a focused Cargo build for CLI-only development (#14018) (`d21277e`) - perf: Run the docs MDX generator once per quality task graph (#14019) (`f7b030b`) - perf: Keep package bundling inside a cacheable build boundary (#14020) (`f3ae66f`) - perf: Skip speculative content hashing for size-changed unnormalized tracked files (#14021) (`a8b4ac2`) - perf: Move synchronous cache archiving off Tokio runtime workers (#14013) (`22b6577`) - perf: Index package directories for changed-file ownership lookups (#14022) (`05f16eb`) - perf: Bound TUI raw-output retention independently of scrollback (#14024) (`0cb6d58`) - perf: Coalesce package rediscovery while a scan is already running (#14023) (`c6eac09`) - perf: Replace repeated deferred-hash scans with dependency-driven readiness (#14025) (`9a98e64`) - perf: Read child output in bounded chunks instead of per-line (#14026) (`f6214c8`) - perf: Share one repo index across same-package spec rehashes in watch mode (#14027) (`171a0a3`) - test: Deflake rediscovery_coalesces_during_in_flight_scan (#14029) (`d41369b`) - perf: Reuse the package graph across partial watch reruns (#14028) (`9a42fe3`) - test: Deflake Go watch CI (#14031) (`0af9b4f`) Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
github-actions Bot
added a commit
that referenced
this pull request
Sep 14, 2026
## Release v2.10.13 > [!CAUTION] > Versioned docs aliasing FAILED. [View logs](https://github.com/vercel/turborepo/actions/runs/34866964811) ### Changes - chore: Release Turborepo 2.10.12 (#13844) (`32748f5`) - fix: Remove unsupported remote cache environment variable (#13845) (`b4c2eed`) - fix: Copy TUI selections locally over SSH (#13847) (`03df632`) - feat: Use uv workspace metadata (#13848) (`fa1ca7d`) - feat: Support Python virtual environments (#13849) (`7f66dbd`) - fix: Scope uv lockfile affectedness (#13850) (`1e074f3`) - test: Isolate uv prune configuration (#13851) (`9f2fd33`) - fix: Explain disabled uv task caching (#13852) (`0f59d11`) - fix: Explain uv identity probe failures (#13853) (`eabe73a`) - fix: Explain uncached Cargo library builds (#13855) (`35ce2fa`) - fix: Explain disabled Cargo task caching (#13854) (`39821f6`) - fix: Keep Cargo tasks runnable without compiler identity (#13857) (`31645fb`) - chore: Release Turborepo 2.10.13-canary.1 (#13858) (`5ac6ea4`) - feat: Support Cargo root packages (#13856) (`e732034`) - feat(repository): Expose flat lockfile package list for metrics (#13859) (`935ee2c`) - chore: Release Turbo repository packages 0.0.1-canary.25 (#13860) (`b0d4c71`) - docs: Upgrade Geistdocs to 1.25.1 (#13861) (`81b0414`) - feat: Expand repository lockfile package metadata (#13862) (`2c84c66`) - chore: Release Turbo repository packages 0.0.1-canary.26 (#13863) (`69e394b`) - fix: Restrict Factory to review comments (#13864) (`1a34319`) - fix: Pluralize package count in run prelude (#13865) (`a175bd9`) - chore: Update with-vite example (#13866) (`0e1ff63`) - chore: Add searchable workspace model picker (#13870) (`73b6268`) - fix: Restore docs client navigation (#13872) (`2995e17`) - feat: Improve Factory workspace statuses (#13869) (`35f03fa`) - chore: Clarify Factory pull request descriptions (#13871) (`5df4cb3`) - fix: Refresh Factory workspaces from main (#13873) (`efa4ae0`) - perf: Speed up plain package listings (#13868) (`f68d214`) - fix: Detect package-level turbo.jsonc (#13875) (`e945403`) - fix: Format Slack pull request notifications (#13877) (`15c15f3`) - feat: Link workspaces to pull requests (#13878) (`ae25f97`) - chore: Upgrade repository to pnpm 12 (#13879) (`4e11bad`) - chore: Update with-vite-react example (#13880) (`54cf466`) - chore: Improve Factory chat streaming (#13881) (`eebf1b0`) - fix: Show workspace failure details (#13884) (`c7661b8`) - chore: Add selectable Factory coding harnesses (#13887) (`6b5ca16`) - fix: Gracefully handle native lockfile failures (#13886) (`463a292`) - chore: Update merged PR Slack messages (#13888) (`66ebcac`) - fix: Show native tasks in task listings (#13889) (`7e4d9d3`) - docs: Clarify turbo ls stability in Agent Skill (#13892) (`8ee758c`) - chore: Resume workspace chat on refocus (#13894) (`7026627`) - chore: Preserve HarnessAgent bridge assets (#13896) (`26f3160`) - fix: Deduplicate multi-language watch hashes (#13897) (`5ade89b`) - fix: Normalize multi-language cache environment inputs (#13898) (`cfca6f2`) - chore: Add workspace diff view (#13900) (`813d54a`) - refactor: Migrate CLI parser to usage-rs (#13890) (`712e0e4`) - fix: Update --filter help links to canonical docs URL (#13891) (`6cf116d`) - chore: Update with-vue-nuxt example (#13901) (`d10e3e0`) - docs: Document missing reference flags and fix query typo (#13902) (`1f2acc5`) - docs: Align CONTRIBUTING Node and pnpm versions (#13883) (`a3bda61`) - perf: Skip Cargo resolution for workspace-only queries (#13903) (`b928ecc`) - fix: Bound memory usage for streamed task output (#13908) (`7f85401`) - fix: Clarify generator action reporting (#13910) (`7fe0201`) - fix: Bump tar to 7.5.22 in @turbo/releaser (CVE-2026-73566) (#13911) (`aba64e9`) - fix: Correct devtools --port default in CLI help (#13905) (`11f7ee6`) - fix: Resolve lint errors (#13912) (`d06cf32`) - fix: Stabilize Rust builds on macOS (#13913) (`7d14df2`) - docs: Clean up Rust docs (#13914) (`3225cee`) - fix: Preserve task metadata for command overrides (#13915) (`54bbf8a`) - perf: Resolve Git SCM state in one process (#13882) (`7c993ef`) - fix: Only treat files literally named .gitignore as ignore files in the untracked walk (#13916) (`ac32a79`) - fix: Pass merge_base and allow_unknown_objects in signature order for task-level filter ranges (#13917) (`48f64e1`) - fix: Hash deferred inputs after dependencies (#13919) (`4cc80f7`) - test: Make Cargo hash fixtures hermetic (#13920) (`32ff2f3`) - fix: Allow release versioning with pending changes (#13921) (`5114901`) - chore: Update to Rust 1.98.0 (#13821) (`ebef942`) - chore: Upgrade Factory to Fable 5.1 (#13924) (`a57476f`) - docs: Align Python guide with Rust guide (#13927) (`f166370`) - fix: Give the package changes watcher the future flags so root inputs reach the task filter (#13926) (`e5ccf55`) - fix: Support pnpm 6 single-package lockfiles (#13928) (`a3d52a6`) - feat: Add skipPackageGraph option to @turbo/repository Workspace.find (#13929) (`99b5a00`) - fix: Use pnpm 12 in library release containers (#13930) (`0b6aaf9`) - fix: Upgrade repository N-API build tooling (#13931) (`dd6c1c6`) - fix: Use Node 20 for musl library builds (#13933) (`61b1635`) - fix: Bootstrap musl builds with sh (#13934) (`f9ac0c9`) - fix: Use static library bootstrap shell (#13935) (`f819dca`) - fix: Allow unclean tree in bump-version (#13936) (`28562e7`) - fix: Skip git commit in bump-version (#13937) (`ea4ba17`) - fix: Expect napi 3 artifact paths (#13938) (`6b5eea3`) - chore: Release Turbo repository packages 0.0.1-canary.27 (#13939) (`5c54e50`) - perf: Batch package detail queries (#13923) (`c8347b1`) - docs: Touch-ups for Python and Rust docs (#13940) (`f20b415`) - chore: Update basic example (#13922) (`b40dd07`) - fix: Avoid oxlint false positive on Workspace.find options (#13941) (`3125eb1`) - chore: Update kitchen-sink example (#13945) (`fd8dc59`) - fix: Hash native workspace task inputs (#13947) (`17d0940`) - chore: Update with-angular example (#13951) (`3770d4c`) - fix: Recompute eager hashing flag after prepending global inputs (#13949) (`9cea33b`) - refactor: Generalize daemon repository discovery (#13946) (`350bd41`) - feat: Make daemon discovery toolchain generic (#13952) (`d65c615`) - feat: Add experimental Go workspace graph foundation (#13953) (`6687fa8`) - feat: Add native Go task tables (#13954) (`dccc890`) - feat: Add Go task input and output contracts (#13955) (`8501ce7`) - test: Cover native Go execution end to end (#13956) (`48d4322`) - feat: Add Go external resolution fingerprints (#13957) (`f6d5f18`) - feat: Fingerprint Go toolchain environment (#13958) (`ec18363`) - feat: Add Go change observations (#13959) (`5256ecc`) - test: Cover Go query and summary surfaces (#13960) (`a24ea48`) - feat: Add Go-aware prune planning (#13961) (`026d7fb`) - test: Add end-to-end Go prune and cache coverage (#13962) (`a1cbfaa`) - docs: Document experimental native Go workspaces (#13963) (`7f36a42`) - fix: Harden Go workspace diagnostics (#13964) (`8a6b0f7`) - fix: Correct Go workspace resolution and task caching (#13966) (`8ef6aaf`) - chore: Add Factory thinking controls and message queue (#13967) (`77e975c`) - chore: Remove ARCHITECTURE.md docs (#13969) (`4f8e693`) - fix: Add slug to OPTIONS request (#13943) (`63797e4`) - perf: Parse CLI exits before starting worker pools (#13971) (`6888882`) - perf: Reuse compiled workspace matchers (#13970) (`7aaf7cb`) - chore: Revert bundled startup optimizations (#13982) (`c11e327`) - perf: Fast-path numeric package manager versions (#13973) (`3b28640`) - perf: Stop inference after selecting a workspace root (#13980) (`a3c5b9c`) - perf: Reuse compiled workspace matchers (#13984) (`eb80d6f`) - perf: Skip unnecessary glob normalization (#13979) (`1840e84`) - perf: Cache workspace configuration lookups (#13975) (`f59661a`) - perf: Avoid regexes for simple environment matching (#13974) (`7e5d793`) - perf: Reuse canonical paths during repository discovery (#13976) (`2ba3e6e`) - perf: Skip work for disabled telemetry (#13972) (`267ed73`) - perf: Defer filesystem checks for absent configs (#13977) (`f7db9df`) - perf: Compile workspace glob combinators directly (#13978) (`223fa9f`) - docs(skills): fix the dependencies-only filter caret position (#13965) (`73ac1fa`) - chore: Update with-biome example (#13983) (`bd331cf`) - fix: Normalize bare directory outputs when filtering dependencyOutputs hashes (#13932) (`555f2bc`) - chore: Scope panic lint allows in hash crates to tests and generated code (#13810) (`e901550`) - perf: Use binary search for resolution states (#13986) (`6a7f1c6`) - perf: Use dense membership for task graph pruning (#13987) (`4ffbaa9`) - perf: Throttle TUI tick cadence (#13988) (`b74be22`) - perf: Fast-path package-only filter selectors with --only (#13990) (`8059b91`) - perf: Avoid materializing npm's legacy dependency tree (#13993) (`acd35c8`) - perf: Stream custom repository downloads to disk (#13994) (`a6b88ee`) - perf: Copy shared file dependencies once per destination during prune (#13995) (`efbe558`) - perf: Index npm workspace links during subgraph extraction (#13997) (`996a0bb`) - perf: Cache Turbo config reads across ESLint rule invocations (#13998) (`c02f97e`) - perf: Scope untracked-file discovery to filtered runs (#13991) (`ead0e19`) - perf: Limit turbo.json preloading to required task scope (#13992) (`f788cd8`) - ci: Restrict release signing to main (#13985) (`2258541`) - perf: Coalesce glob-watcher invalidation snapshots (#13996) (`3fec2fe`) - perf: Reuse resolver inference during file tracing (#13999) (`596ea4c`) - chore: Release Turborepo 2.10.13-canary.2 (#14000) (`ebfc808`) - chore: Exclude turbo and @turbo/* from minimum release age (#14005) (`5897420`) - perf: Binary search boundaries comments before each import (#14003) (`6a95c81`) - perf: Fetch only npm dist tags in the binary version endpoint (#14007) (`4db9ca0`) - perf: Compute Devtools graph depths without rescanning every edge (#14001) (`ef46772`) - perf: Reuse package manager detection during create startup (#14006) (`06f5900`) - perf: Borrow workspace entries during dependency lookup (#14002) (`ba822e1`) - perf: Cache LSP task index and dedupe identities in constant time (#14008) (`77f6232`) - perf: Deduplicate recursive glob prefixes before walking (#14004) (`90fd2d6`) - perf: Batch boundaries progress updates (#14009) (`3862f3e`) - perf: Reuse archive anchor during cache creation (#14010) (`f01afa9`) - perf: Bound remote-cache transfer memory instead of retaining whole artifacts (#14011) (`470ffb9`) - perf: Build cache archives once for local and remote writes (#14012) (`f0a0d77`) - perf: Make VS Code binary discovery and LSP probes nonblocking (#14015) (`8cc0c42`) - perf: Avoid serializing ASTs for path-only file dependency queries (#14016) (`96712c9`) - perf: Share source text across boundaries diagnostics instead of copying it per error (#14014) (`5223bd6`) - perf: Parse the lockfile once when opening a native Workspace with its graph (#14017) (`23ee440`) - docs: Document a focused Cargo build for CLI-only development (#14018) (`d21277e`) - perf: Run the docs MDX generator once per quality task graph (#14019) (`f7b030b`) - perf: Keep package bundling inside a cacheable build boundary (#14020) (`f3ae66f`) - perf: Skip speculative content hashing for size-changed unnormalized tracked files (#14021) (`a8b4ac2`) - perf: Move synchronous cache archiving off Tokio runtime workers (#14013) (`22b6577`) - perf: Index package directories for changed-file ownership lookups (#14022) (`05f16eb`) - perf: Bound TUI raw-output retention independently of scrollback (#14024) (`0cb6d58`) - perf: Coalesce package rediscovery while a scan is already running (#14023) (`c6eac09`) - perf: Replace repeated deferred-hash scans with dependency-driven readiness (#14025) (`9a98e64`) - perf: Read child output in bounded chunks instead of per-line (#14026) (`f6214c8`) - perf: Share one repo index across same-package spec rehashes in watch mode (#14027) (`171a0a3`) - test: Deflake rediscovery_coalesces_during_in_flight_scan (#14029) (`d41369b`) - perf: Reuse the package graph across partial watch reruns (#14028) (`9a42fe3`) - test: Deflake Go watch CI (#14031) (`0af9b4f`) - chore: Release Turborepo 2.10.13-canary.3 (#14032) (`cfd82d2`) - docs: Group experimental languages and link the Go RFC (#14034) (`6e8f7fe`) - chore: Update with-docker example (#14035) (`25fe817`) - perf: Avoid cloning package list in turbo ls (#14036) (`f3b0fa9`) - fix: Restore leading flags for implicit run commands (#14037) (`3235ec5`) - fix: Include experimentalCI in task hashes (#14038) (`b8c7370`) - chore: Release Turborepo 2.10.13-canary.4 (#14039) (`17b358a`) - docs: Explain circular package dependencies in boundaries (#14042) (`895337e`) - docs: Document circular dependencies in boundaries (#14041) (`3821e42`) - fix: Preserve Yarn resolution declaration order (#14045) (`54b74a7`) - fix: Avoid duplicate interrupts during task shutdown (#14046) (`f9fa885`) - fix: Preserve nested Bun package keys (#14050) (`b647afc`) - chore: Update with-mcp-servers example (#14049) (`2c11c4f`) - chore: Release Turborepo 2.10.13-canary.5 (#14051) (`cc1caca`) - fix: Preserve native task dependencies across entrypoints (#14047) (`ce4f045`) - fix: Pass through LOCALAPPDATA by default (#14056) (`258a08d`) - chore: Release Turborepo 2.10.13-canary.6 (#14057) (`bea9b6c`) - docs: Fix some typos (#14058) (`6a3b4c1`) - feat: Add Remix inference and `NUXT_ENV_*` wildcard (#14060) (`2167e74`) - fix: Return resolved package names for aliases (#14061) (`83f3ee1`) - chore: Release Turbo repository packages 0.0.1-canary.28 (#14062) (`e5094c7`) - feat: Add NEXT_DEPLOYMENT_ID to Next.js framework inference (#14065) (`36383cb`) --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
github-actions Bot
added a commit
that referenced
this pull request
Sep 18, 2026
## Release v2.11.0 > [!CAUTION] > Versioned docs aliasing FAILED. [View logs](https://github.com/vercel/turborepo/actions/runs/35366297249) ### Changes - feat: Add durable Factory workspaces (#13804) (`ccd79d3`) - feat: Stream Factory sandbox as terminal (#13807) (`33d8b24`) - fix: Restore Factory workspace creation (#13812) (`fd72cca`) - fix: Update Factory session network policy (#13814) (`c579fec`) - fix: Improve Factory terminal line spacing (#13815) (`a88e39b`) - fix: Install Factory publishing skill (#13817) (`7b8cb14`) - chore: Route Factory publishing through Eve (#13816) (`e05b81c`) - feat: Standardize Factory meta titles to Turborepo suffix (#13819) (`fd593b5`) - perf: Skip Factory chat verification (#13820) (`39137a6`) - fix: Restore Factory workspaces (#13823) (`0afd4b2`) - docs: Fix inconsistent Yarn command in basic example (#13825) (`3dd49d1`) - chore: Update non-monorepo example (#13808) (`d2a673f`) - perf: Batch package detail queries (#13809) (`331183e`) - chore: Update basic example (#13824) (`f06836d`) - fix: Include virtual tasks in affected query (#13805) (`89a9b78`) - fix: Add workspace approval controls (#13827) (`02dfd21`) - fix: Prevent chat SSH command overflow (#13828) (`d9eaff5`) - fix: Update Factory pull request branches (#13831) (`b670754`) - chore: Add operator chat model selector (#13833) (`bb2fcb3`) - fix: Move model selector to workspace creation (#13835) (`72805d7`) - chore: Skip redundant Factory PR approval (#13837) (`bbe5406`) - chore: Use geistdocs 1.23.1 (#13834) (`3787c06`) - chore: Handle feedback on Factory pull requests (#13836) (`a76330b`) - fix: Escape ampersands in RSS feed enclosure URLs (#13839) (`7107f26`) - chore: Add automatic issue handling (#13840) (`e1674e4`) - chore: Alert Slack for low-confidence issues (#13841) (`f153cda`) - feat: Require high confidence for issue fixes (#13842) (`c97782b`) - fix: Run pnpm directly on Windows (#13843) (`9d2b03b`) - chore: Release Turborepo 2.10.12 (#13844) (`32748f5`) - fix: Remove unsupported remote cache environment variable (#13845) (`b4c2eed`) - fix: Copy TUI selections locally over SSH (#13847) (`03df632`) - feat: Use uv workspace metadata (#13848) (`fa1ca7d`) - feat: Support Python virtual environments (#13849) (`7f66dbd`) - fix: Scope uv lockfile affectedness (#13850) (`1e074f3`) - test: Isolate uv prune configuration (#13851) (`9f2fd33`) - fix: Explain disabled uv task caching (#13852) (`0f59d11`) - fix: Explain uv identity probe failures (#13853) (`eabe73a`) - fix: Explain uncached Cargo library builds (#13855) (`35ce2fa`) - fix: Explain disabled Cargo task caching (#13854) (`39821f6`) - fix: Keep Cargo tasks runnable without compiler identity (#13857) (`31645fb`) - chore: Release Turborepo 2.10.13-canary.1 (#13858) (`5ac6ea4`) - feat: Support Cargo root packages (#13856) (`e732034`) - feat(repository): Expose flat lockfile package list for metrics (#13859) (`935ee2c`) - chore: Release Turbo repository packages 0.0.1-canary.25 (#13860) (`b0d4c71`) - docs: Upgrade Geistdocs to 1.25.1 (#13861) (`81b0414`) - feat: Expand repository lockfile package metadata (#13862) (`2c84c66`) - chore: Release Turbo repository packages 0.0.1-canary.26 (#13863) (`69e394b`) - fix: Restrict Factory to review comments (#13864) (`1a34319`) - fix: Pluralize package count in run prelude (#13865) (`a175bd9`) - chore: Update with-vite example (#13866) (`0e1ff63`) - chore: Add searchable workspace model picker (#13870) (`73b6268`) - fix: Restore docs client navigation (#13872) (`2995e17`) - feat: Improve Factory workspace statuses (#13869) (`35f03fa`) - chore: Clarify Factory pull request descriptions (#13871) (`5df4cb3`) - fix: Refresh Factory workspaces from main (#13873) (`efa4ae0`) - perf: Speed up plain package listings (#13868) (`f68d214`) - fix: Detect package-level turbo.jsonc (#13875) (`e945403`) - fix: Format Slack pull request notifications (#13877) (`15c15f3`) - feat: Link workspaces to pull requests (#13878) (`ae25f97`) - chore: Upgrade repository to pnpm 12 (#13879) (`4e11bad`) - chore: Update with-vite-react example (#13880) (`54cf466`) - chore: Improve Factory chat streaming (#13881) (`eebf1b0`) - fix: Show workspace failure details (#13884) (`c7661b8`) - chore: Add selectable Factory coding harnesses (#13887) (`6b5ca16`) - fix: Gracefully handle native lockfile failures (#13886) (`463a292`) - chore: Update merged PR Slack messages (#13888) (`66ebcac`) - fix: Show native tasks in task listings (#13889) (`7e4d9d3`) - docs: Clarify turbo ls stability in Agent Skill (#13892) (`8ee758c`) - chore: Resume workspace chat on refocus (#13894) (`7026627`) - chore: Preserve HarnessAgent bridge assets (#13896) (`26f3160`) - fix: Deduplicate multi-language watch hashes (#13897) (`5ade89b`) - fix: Normalize multi-language cache environment inputs (#13898) (`cfca6f2`) - chore: Add workspace diff view (#13900) (`813d54a`) - refactor: Migrate CLI parser to usage-rs (#13890) (`712e0e4`) - fix: Update --filter help links to canonical docs URL (#13891) (`6cf116d`) - chore: Update with-vue-nuxt example (#13901) (`d10e3e0`) - docs: Document missing reference flags and fix query typo (#13902) (`1f2acc5`) - docs: Align CONTRIBUTING Node and pnpm versions (#13883) (`a3bda61`) - perf: Skip Cargo resolution for workspace-only queries (#13903) (`b928ecc`) - fix: Bound memory usage for streamed task output (#13908) (`7f85401`) - fix: Clarify generator action reporting (#13910) (`7fe0201`) - fix: Bump tar to 7.5.22 in @turbo/releaser (CVE-2026-73566) (#13911) (`aba64e9`) - fix: Correct devtools --port default in CLI help (#13905) (`11f7ee6`) - fix: Resolve lint errors (#13912) (`d06cf32`) - fix: Stabilize Rust builds on macOS (#13913) (`7d14df2`) - docs: Clean up Rust docs (#13914) (`3225cee`) - fix: Preserve task metadata for command overrides (#13915) (`54bbf8a`) - perf: Resolve Git SCM state in one process (#13882) (`7c993ef`) - fix: Only treat files literally named .gitignore as ignore files in the untracked walk (#13916) (`ac32a79`) - fix: Pass merge_base and allow_unknown_objects in signature order for task-level filter ranges (#13917) (`48f64e1`) - fix: Hash deferred inputs after dependencies (#13919) (`4cc80f7`) - test: Make Cargo hash fixtures hermetic (#13920) (`32ff2f3`) - fix: Allow release versioning with pending changes (#13921) (`5114901`) - chore: Update to Rust 1.98.0 (#13821) (`ebef942`) - chore: Upgrade Factory to Fable 5.1 (#13924) (`a57476f`) - docs: Align Python guide with Rust guide (#13927) (`f166370`) - fix: Give the package changes watcher the future flags so root inputs reach the task filter (#13926) (`e5ccf55`) - fix: Support pnpm 6 single-package lockfiles (#13928) (`a3d52a6`) - feat: Add skipPackageGraph option to @turbo/repository Workspace.find (#13929) (`99b5a00`) - fix: Use pnpm 12 in library release containers (#13930) (`0b6aaf9`) - fix: Upgrade repository N-API build tooling (#13931) (`dd6c1c6`) - fix: Use Node 20 for musl library builds (#13933) (`61b1635`) - fix: Bootstrap musl builds with sh (#13934) (`f9ac0c9`) - fix: Use static library bootstrap shell (#13935) (`f819dca`) - fix: Allow unclean tree in bump-version (#13936) (`28562e7`) - fix: Skip git commit in bump-version (#13937) (`ea4ba17`) - fix: Expect napi 3 artifact paths (#13938) (`6b5eea3`) - chore: Release Turbo repository packages 0.0.1-canary.27 (#13939) (`5c54e50`) - perf: Batch package detail queries (#13923) (`c8347b1`) - docs: Touch-ups for Python and Rust docs (#13940) (`f20b415`) - chore: Update basic example (#13922) (`b40dd07`) - fix: Avoid oxlint false positive on Workspace.find options (#13941) (`3125eb1`) - chore: Update kitchen-sink example (#13945) (`fd8dc59`) - fix: Hash native workspace task inputs (#13947) (`17d0940`) - chore: Update with-angular example (#13951) (`3770d4c`) - fix: Recompute eager hashing flag after prepending global inputs (#13949) (`9cea33b`) - refactor: Generalize daemon repository discovery (#13946) (`350bd41`) - feat: Make daemon discovery toolchain generic (#13952) (`d65c615`) - feat: Add experimental Go workspace graph foundation (#13953) (`6687fa8`) - feat: Add native Go task tables (#13954) (`dccc890`) - feat: Add Go task input and output contracts (#13955) (`8501ce7`) - test: Cover native Go execution end to end (#13956) (`48d4322`) - feat: Add Go external resolution fingerprints (#13957) (`f6d5f18`) - feat: Fingerprint Go toolchain environment (#13958) (`ec18363`) - feat: Add Go change observations (#13959) (`5256ecc`) - test: Cover Go query and summary surfaces (#13960) (`a24ea48`) - feat: Add Go-aware prune planning (#13961) (`026d7fb`) - test: Add end-to-end Go prune and cache coverage (#13962) (`a1cbfaa`) - docs: Document experimental native Go workspaces (#13963) (`7f36a42`) - fix: Harden Go workspace diagnostics (#13964) (`8a6b0f7`) - fix: Correct Go workspace resolution and task caching (#13966) (`8ef6aaf`) - chore: Add Factory thinking controls and message queue (#13967) (`77e975c`) - chore: Remove ARCHITECTURE.md docs (#13969) (`4f8e693`) - fix: Add slug to OPTIONS request (#13943) (`63797e4`) - perf: Parse CLI exits before starting worker pools (#13971) (`6888882`) - perf: Reuse compiled workspace matchers (#13970) (`7aaf7cb`) - chore: Revert bundled startup optimizations (#13982) (`c11e327`) - perf: Fast-path numeric package manager versions (#13973) (`3b28640`) - perf: Stop inference after selecting a workspace root (#13980) (`a3c5b9c`) - perf: Reuse compiled workspace matchers (#13984) (`eb80d6f`) - perf: Skip unnecessary glob normalization (#13979) (`1840e84`) - perf: Cache workspace configuration lookups (#13975) (`f59661a`) - perf: Avoid regexes for simple environment matching (#13974) (`7e5d793`) - perf: Reuse canonical paths during repository discovery (#13976) (`2ba3e6e`) - perf: Skip work for disabled telemetry (#13972) (`267ed73`) - perf: Defer filesystem checks for absent configs (#13977) (`f7db9df`) - perf: Compile workspace glob combinators directly (#13978) (`223fa9f`) - docs(skills): fix the dependencies-only filter caret position (#13965) (`73ac1fa`) - chore: Update with-biome example (#13983) (`bd331cf`) - fix: Normalize bare directory outputs when filtering dependencyOutputs hashes (#13932) (`555f2bc`) - chore: Scope panic lint allows in hash crates to tests and generated code (#13810) (`e901550`) - perf: Use binary search for resolution states (#13986) (`6a7f1c6`) - perf: Use dense membership for task graph pruning (#13987) (`4ffbaa9`) - perf: Throttle TUI tick cadence (#13988) (`b74be22`) - perf: Fast-path package-only filter selectors with --only (#13990) (`8059b91`) - perf: Avoid materializing npm's legacy dependency tree (#13993) (`acd35c8`) - perf: Stream custom repository downloads to disk (#13994) (`a6b88ee`) - perf: Copy shared file dependencies once per destination during prune (#13995) (`efbe558`) - perf: Index npm workspace links during subgraph extraction (#13997) (`996a0bb`) - perf: Cache Turbo config reads across ESLint rule invocations (#13998) (`c02f97e`) - perf: Scope untracked-file discovery to filtered runs (#13991) (`ead0e19`) - perf: Limit turbo.json preloading to required task scope (#13992) (`f788cd8`) - ci: Restrict release signing to main (#13985) (`2258541`) - perf: Coalesce glob-watcher invalidation snapshots (#13996) (`3fec2fe`) - perf: Reuse resolver inference during file tracing (#13999) (`596ea4c`) - chore: Release Turborepo 2.10.13-canary.2 (#14000) (`ebfc808`) - chore: Exclude turbo and @turbo/* from minimum release age (#14005) (`5897420`) - perf: Binary search boundaries comments before each import (#14003) (`6a95c81`) - perf: Fetch only npm dist tags in the binary version endpoint (#14007) (`4db9ca0`) - perf: Compute Devtools graph depths without rescanning every edge (#14001) (`ef46772`) - perf: Reuse package manager detection during create startup (#14006) (`06f5900`) - perf: Borrow workspace entries during dependency lookup (#14002) (`ba822e1`) - perf: Cache LSP task index and dedupe identities in constant time (#14008) (`77f6232`) - perf: Deduplicate recursive glob prefixes before walking (#14004) (`90fd2d6`) - perf: Batch boundaries progress updates (#14009) (`3862f3e`) - perf: Reuse archive anchor during cache creation (#14010) (`f01afa9`) - perf: Bound remote-cache transfer memory instead of retaining whole artifacts (#14011) (`470ffb9`) - perf: Build cache archives once for local and remote writes (#14012) (`f0a0d77`) - perf: Make VS Code binary discovery and LSP probes nonblocking (#14015) (`8cc0c42`) - perf: Avoid serializing ASTs for path-only file dependency queries (#14016) (`96712c9`) - perf: Share source text across boundaries diagnostics instead of copying it per error (#14014) (`5223bd6`) - perf: Parse the lockfile once when opening a native Workspace with its graph (#14017) (`23ee440`) - docs: Document a focused Cargo build for CLI-only development (#14018) (`d21277e`) - perf: Run the docs MDX generator once per quality task graph (#14019) (`f7b030b`) - perf: Keep package bundling inside a cacheable build boundary (#14020) (`f3ae66f`) - perf: Skip speculative content hashing for size-changed unnormalized tracked files (#14021) (`a8b4ac2`) - perf: Move synchronous cache archiving off Tokio runtime workers (#14013) (`22b6577`) - perf: Index package directories for changed-file ownership lookups (#14022) (`05f16eb`) - perf: Bound TUI raw-output retention independently of scrollback (#14024) (`0cb6d58`) - perf: Coalesce package rediscovery while a scan is already running (#14023) (`c6eac09`) - perf: Replace repeated deferred-hash scans with dependency-driven readiness (#14025) (`9a98e64`) - perf: Read child output in bounded chunks instead of per-line (#14026) (`f6214c8`) - perf: Share one repo index across same-package spec rehashes in watch mode (#14027) (`171a0a3`) - test: Deflake rediscovery_coalesces_during_in_flight_scan (#14029) (`d41369b`) - perf: Reuse the package graph across partial watch reruns (#14028) (`9a42fe3`) - test: Deflake Go watch CI (#14031) (`0af9b4f`) - chore: Release Turborepo 2.10.13-canary.3 (#14032) (`cfd82d2`) - docs: Group experimental languages and link the Go RFC (#14034) (`6e8f7fe`) - chore: Update with-docker example (#14035) (`25fe817`) - perf: Avoid cloning package list in turbo ls (#14036) (`f3b0fa9`) - fix: Restore leading flags for implicit run commands (#14037) (`3235ec5`) - fix: Include experimentalCI in task hashes (#14038) (`b8c7370`) - chore: Release Turborepo 2.10.13-canary.4 (#14039) (`17b358a`) - docs: Explain circular package dependencies in boundaries (#14042) (`895337e`) - docs: Document circular dependencies in boundaries (#14041) (`3821e42`) - fix: Preserve Yarn resolution declaration order (#14045) (`54b74a7`) - fix: Avoid duplicate interrupts during task shutdown (#14046) (`f9fa885`) - fix: Preserve nested Bun package keys (#14050) (`b647afc`) - chore: Update with-mcp-servers example (#14049) (`2c11c4f`) - chore: Release Turborepo 2.10.13-canary.5 (#14051) (`cc1caca`) - fix: Preserve native task dependencies across entrypoints (#14047) (`ce4f045`) - fix: Pass through LOCALAPPDATA by default (#14056) (`258a08d`) - chore: Release Turborepo 2.10.13-canary.6 (#14057) (`bea9b6c`) - docs: Fix some typos (#14058) (`6a3b4c1`) - feat: Add Remix inference and `NUXT_ENV_*` wildcard (#14060) (`2167e74`) - fix: Return resolved package names for aliases (#14061) (`83f3ee1`) - chore: Release Turbo repository packages 0.0.1-canary.28 (#14062) (`e5094c7`) - feat: Add NEXT_DEPLOYMENT_ID to Next.js framework inference (#14065) (`36383cb`) - chore: Release Turborepo 2.10.13 (#14066) (`05a1d4b`) - fix: Fall back to the published tag when the dogfooded turbo version is unpublished (#14067) (`4d13cb5`) - feat: Expose compact task planning through turbo query (#14068) (`1e9336e`) - chore: Release Turborepo 2.10.14-canary.1 (#14069) (`9077648`) - fix: Discover native workspace metadata lazily (#14053) (`30e0250`) - fix: Run Go workspace formatting per module (#14070) (`81d62ff`) - fix: Run Go verification per module (#14072) (`e1eb388`) - fix: Use pnpm for non-monorepo deployments (#14071) (`53dadbe`) - chore: Update with-nestjs example (#14064) (`a534c7e`) - docs: Document per-toolchain task commands (#14073) (`bc9017a`) - fix: Infer all co-located packages from cwd (#14074) (`46d0fd6`) - fix: Preserve all co-located package change owners (#14075) (`064d74a`) - fix: Isolate task logs for co-located packages (#14077) (`16b312f`) - chore: Make filesystem watch cleanup idempotent (#14078) (`441f205`) - fix: Follow Go default binary output paths (#14079) (`a1ab64a`) - chore: Release Turborepo 2.10.14-canary.2 (#14080) (`c2ddd13`) - feat: Make package queries and pruning task-aware (#14081) (`012b091`) - chore: Release Turborepo 2.10.14-canary.3 (#14082) (`7b2abd7`) - chore: Update with-npm example (#14083) (`d7f89ba`) - feat: Expose static repository discovery and safe affected candidates (#14085) (`22a201d`) - fix: Preserve task env mode when pruning (#14087) (`7eda8d5`) - feat: Derive short names for Go workspace packages (#14086) (`b3cfa86`) - chore: Release Turbo repository packages 0.0.1-canary.29 (#14088) (`15830dc`) - chore: Release Turborepo 2.10.14-canary.4 (#14089) (`b66c30d`) - feat: Infer Cargo and uv affectedness from static package inputs (#14090) (`728a79d`) - chore: Release Turbo repository packages 0.0.1-canary.30 (#14091) (`438c45b`) - perf: Cache Windows archive anchor (#14093) (`0c3bc42`) - fix: Harden cache archive restoration (#14095) (`7e7ab44`) - fix: Defer dependency output validation for lazy scopes (#14097) (`d804909`) - chore: Release Turborepo 2.10.14-canary.5 (#14098) (`613b3ba`) - test: Fix Windows cache restore test (#14099) (`2c49c23`) - chore: Update with-prisma example (#14101) (`f21d73f`) --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes TURBO-6038.
Remote cache transfers previously retained entire compressed artifacts in memory: uploads compressed into a
Vecbefore streaming, and downloads collected the full response body before signature verification and extraction. Concurrent large, poorly compressible artifacts retained the sum of their compressed sizes.Changes
SpooledTempFilewith an 8 MiB threshold: small artifacts stay in memory (same path as before), while large ones roll to an anonymous temporary file that is unlinked on close and cleaned up on every exit path, including errors.Bytesslices in memory,ReaderStreamover the temp file on disk). Retries after token refresh re-create a fresh stream over the same spooled bytes, so resent content stays byte-identical and Content-Length is preserved.ArtifactSignatureAuthenticatorgains chunked reader/streaming tag variants that produce byte-identical tags to the in-memory API (the body is the final length-prefixed HMAC field, so prefixing the known length preserves the exact protocol semantics).Benchmarks
4 concurrent transfers of 64 MiB incompressible artifacts against the repo's mock server running as a separate process, so peak RSS reflects only cache client memory. Debug build, macOS,
/usr/bin/time -lmaximum resident set size. Benchmark harness was temporary and is not committed.Retained artifact payload memory is now bounded by the 8 MiB spool threshold per transfer rather than the full compressed artifact size.
Verification
cargo test -p turborepo-cache: 157 passed, 0 failed, including a new 16 MiB rolled-to-disk round-trip test through the mock server and streaming-tag equivalence tests.cargo clippy -p turborepo-cache: clean.signature-filtered env-var tests race each other when run in parallel; that flake exists onmainand is unrelated to this change.