Skip to content

Harden Skippy layer-package materialization cache - #583

Merged
IvGolovach merged 1 commit into
mainfrom
codex/cache-safe-skippy-materialization
May 20, 2026
Merged

Harden Skippy layer-package materialization cache#583
IvGolovach merged 1 commit into
mainfrom
codex/cache-safe-skippy-materialization

Conversation

@IvGolovach

Copy link
Copy Markdown
Collaborator

Summary

Skippy layer-package stages now materialize into the local derived-stage cache with a stronger cache-safety contract.

  • Adds a materialized-cache sidecar that binds a derived stage artifact to the request identity, manifest SHA, selected package parts, and final output metadata.
  • Serializes writers for the same materialized output with a per-output advisory lock.
  • Uses unique cache-owned staging paths so concurrent materialization attempts cannot collide on the same temporary file.
  • Validates the staged GGUF before publishing it as the final derived artifact.
  • Keeps cleanup paths honest by removing the sidecar record when derived stage artifacts are pruned or deleted.

Why

The risky part of layer-package transfer and reuse is not only whether package artifacts are downloaded correctly. The final materialized GGUF is also a cache boundary: if it is reused after a partial write, concurrent publish race, stale final artifact, or mismatched package identity, Skippy can silently load the wrong derived stage.

This PR makes that boundary explicit. A materialized stage is only treated as reusable when its sidecar still matches the current request, manifest, selected artifacts, and output metadata. Otherwise Skippy rematerializes instead of trusting a valid-looking file path.

Implementation

  • Extracts materialized stage cache handling into a dedicated skippy-runtime module.
  • Adds MaterializedCacheIdentity records for request identity, manifest SHA, and selected artifact path/size/SHA data.
  • Adds per-output locking around the reuse/write/publish sequence and rechecks the cache after acquiring the lock.
  • Writes materialization output through .staging/ with unique names rather than pid-only temp paths.
  • Publishes only after the staged GGUF opens successfully through the runtime metadata reader.
  • Exposes the sidecar path to host-runtime cleanup so cache records do not outlive the artifact they describe.

Compatibility

  • No mesh gossip, peer protocol, plugin protocol, or API contract changes.
  • Existing materialized .gguf files without sidecar records are treated as cache misses and rebuilt when needed.
  • Layer-package manifest and artifact verification behavior is preserved; this adds a second safety boundary around the derived materialized output.

Validation

Local validation passed with LLAMA_STAGE_BUILD_DIR set to the local stage ABI build directory:

  • git fetch --no-tags origin main:refs/remotes/origin/main
  • git diff --check
  • git diff --cached --check
  • cargo fmt --all
  • cargo fmt --all -- --check
  • cargo test -p skippy-runtime --lib materialized_cache - 3 passed
  • cargo test -p skippy-runtime --lib package::tests - 15 passed
  • cargo test -p skippy-runtime --lib - 36 passed
  • cargo test -p mesh-llm-host-runtime --lib materialized_stage_preview_matches_source_removal_candidates - 1 passed
  • cargo test -p mesh-llm-host-runtime --lib inference::skippy::materialization::tests - 16 passed, 2 ignored
  • cargo check -p mesh-llm

Not run: just build - not required for this Rust-only runtime/cache change; no UI, bundle, or release artifact changed.

Remote CI will provide final PR-sha proof after the PR is opened.

Rollback

Revert this PR. Existing package artifacts and source model caches are not migrated by this change.

Related

Follow-up to the Skippy layer-package / HF cache-safety discussion around #471 and #485.

@IvGolovach
IvGolovach requested a review from i386 May 18, 2026 18:45
Validation
* Validation tier: Tier 2R — narrow post-review/CI correction on top of the existing Skippy materialization cache-safety PR.
* git fetch --no-tags origin main:refs/remotes/origin/main: PASS
* git diff --check: PASS, no output
* git diff --cached --check: PASS, no output
* cargo fmt --all: PASS
* cargo fmt --all -- --check: PASS
* LLAMA_STAGE_BUILD_DIR=/Users/Funtland/Downloads/mesh-llm/.deps/llama-build/build-stage-abi-metal cargo clippy -p skippy-runtime --all-targets -- -D warnings: PASS
* LLAMA_STAGE_BUILD_DIR=/Users/Funtland/Downloads/mesh-llm/.deps/llama-build/build-stage-abi-metal cargo test -p skippy-runtime --lib materialized_cache: PASS, 3 passed
* LLAMA_STAGE_BUILD_DIR=/Users/Funtland/Downloads/mesh-llm/.deps/llama-build/build-stage-abi-metal cargo test -p skippy-runtime --lib package::tests: PASS, 15 passed
* LLAMA_STAGE_BUILD_DIR=/Users/Funtland/Downloads/mesh-llm/.deps/llama-build/build-stage-abi-metal cargo test -p skippy-runtime --lib: PASS, 36 passed
* LLAMA_STAGE_BUILD_DIR=/Users/Funtland/Downloads/mesh-llm/.deps/llama-build/build-stage-abi-metal cargo test -p mesh-llm-host-runtime --lib materialized_stage_preview_matches_source_removal_candidates: PASS, 1 passed
* LLAMA_STAGE_BUILD_DIR=/Users/Funtland/Downloads/mesh-llm/.deps/llama-build/build-stage-abi-metal cargo test -p mesh-llm-host-runtime --lib inference::skippy::materialization::tests: PASS, 16 passed, 2 ignored
* LLAMA_STAGE_BUILD_DIR=/Users/Funtland/Downloads/mesh-llm/.deps/llama-build/build-stage-abi-metal cargo check -p mesh-llm: PASS
* Ledger: not applicable — not required for selected validation tier/change family.
* Version: not applicable — not required for selected validation tier/change family.
* Not run: just build — not required for selected validation tier; no UI, bundle, or release artifact changed.

Rollback
* git revert HEAD
@IvGolovach
IvGolovach force-pushed the codex/cache-safe-skippy-materialization branch from bf57be6 to 375f74d Compare May 18, 2026 18:50
@IvGolovach
IvGolovach merged commit 005c528 into main May 20, 2026
21 checks passed
@IvGolovach
IvGolovach deleted the codex/cache-safe-skippy-materialization branch May 20, 2026 17:15
michaelneale added a commit that referenced this pull request May 21, 2026
Brings in #579 advisory capacity, #583 hardened materialization cache,
#562 version bump, #606/#604 Windows CUDA build fixes, #608 lint rule,
and #560 UI mockup.

* origin/main:
  fix(ci): small update for lint rule (#608)
  mockup: Reserves high-fidelity UI mockup (#560)
  Add advisory capacity evaluation for model targets (#579)
  Harden Skippy layer package materialization cache (#583)
  fix(release): pin Windows CUDA to sccache-compatible version (#606)
  fix(build-windows): tolerate dead sccache server in CUDA retry path (#604)
  chore(version): synchronize version bump everywhere (#562)
michaelneale added a commit that referenced this pull request May 21, 2026
* main:
  docs(AGENTS): add confidence-testing recipe for routing/MoA/gossip changes (#613)
  ci(sdk-smoke): install lld in macOS swift smoke job (#610)
  MoA: mesh mode and many inference critical fixes, and quic keep alive (#566)
  fix(ci): small update for lint rule (#608)
  mockup: Reserves high-fidelity UI mockup (#560)
  Add advisory capacity evaluation for model targets (#579)
  Harden Skippy layer package materialization cache (#583)
  fix(release): pin Windows CUDA to sccache-compatible version (#606)
  fix(build-windows): tolerate dead sccache server in CUDA retry path (#604)
  chore(version): synchronize version bump everywhere (#562)
  fix(mesh): skip filtered peers in gossip dial loop to unwedge `--auto` (#602)
  docs(agents): clarify just build vs release-build for serious testing (#599)
  build: ozempic — slim binary -42 MB / -47 MB (#592)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant