Skip to content
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions .claude/skills/rust-build/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ description: >-
incremental builds, the cross-worktree artifact-contamination hazard that
sharing creates (the phantom `E0063: missing field` on correct source), and the
wrapper (`scripts/rust-build.sh`) that shares by default and auto-isolates the
moment a worktree diverges a depended-on crate. Auto-triggers on a spurious
moment a worktree diverges a depended-on crate or `runtime/`. Auto-triggers on a spurious
cargo compile error that names a field/symbol absent from your checkout, on
"worktree build contamination", and on setting CARGO_TARGET_DIR for a worktree.
metadata:
Expand Down Expand Up @@ -57,12 +57,12 @@ error[E0063]: missing field `lockfile_legacy_basenames` in initializer of `aube_

Sharers are grouped by the **content** of their depended-on crates, hashed into the bucket name (`shared-target-<hash>`):

- **Depended-on crate sources unmodified → share** that content's bucket. Everyone in it agrees by construction. The common case: feature work in `nub-cli`, integration tests, docs, non-Rust files.
- **Diverged a depended-on crate → isolate** to a private per-worktree `target/` (removed with the worktree), CoW-seeded from the matching bucket so you rebuild only what differs.
- **Depended-on crate sources and `runtime/` unmodified → share** that content's bucket. Everyone in it agrees by construction. The common case: feature work in `nub-cli`, integration tests, docs.
- **Diverged a depended-on crate or `runtime/` → isolate** to a private per-worktree `target/` (removed with the worktree), CoW-seeded from the matching bucket so you rebuild only what differs. `runtime/` is hashed for a different reason than the crates: a dev binary resolves `runtime/*.cjs` from the tree that compiled `nub-core` (baked `CARGO_MANIFEST_DIR`), so a shared-bucket binary loads whichever sharer compiled last — a worktree with edited runtime files would silently test a sibling's copy.

**Content, not merge-base:** a merge-base proves only that *this* worktree made no local changes against *its own* base, so two worktrees whose bases straddle a `nub-core`/`aube` commit both pass while disagreeing on content. The key hashes the git **index**, which only matters on the shared branch (no local changes by definition), so it moves on rebase, never mid-edit.

Depended-on crates = every workspace/vendored crate **except leaf artifacts nothing links**: `crates/nub-cli` (bin), `crates/nub-native` (cdylib, own workspace), `crates/nub-phantom` (bin, own workspace). So: `crates/nub-core`, `crates/nub-cache-key`, `crates/nub-phantom-core`, `crates/nub-phantom-scan`, and all of `vendor/aube`. `nub-phantom-core`/`nub-phantom-scan` are **not** leaves — `nub-cli` depends on both — and the pathspec `:(exclude)crates/nub-phantom` matches only that directory, not those siblings.
The hashed set = every workspace/vendored crate **except leaf artifacts nothing links** — `crates/nub-cli` (bin), `crates/nub-native` (cdylib, own workspace), `crates/nub-phantom` (bin, own workspace) — **plus `runtime/`**. So: `crates/nub-core`, `crates/nub-cache-key`, `crates/nub-phantom-core`, `crates/nub-phantom-scan`, all of `vendor/aube`, and `runtime/`. `nub-phantom-core`/`nub-phantom-scan` are **not** leaves — `nub-cli` depends on both — and the pathspec `:(exclude)crates/nub-phantom` matches only that directory, not those siblings.

## Letting the wrapper choose IS the caching strategy

Expand Down
41 changes: 31 additions & 10 deletions scripts/rust-build.sh
Original file line number Diff line number Diff line change
Expand Up @@ -97,6 +97,13 @@ leaves=":(exclude)crates/nub-cli :(exclude)crates/nub-native :(exclude)crates/nu
# Both checks are deliberately broad (any path under a depended-on crate, not just
# *.rs): over-isolating on an irrelevant file costs one cold build; under-isolating
# risks the clobber. Depended-on = every workspace/vendored crate except nub-cli.
# `runtime/` is in the set for a different reason than the crates: the binary
# resolves `runtime/*.cjs` at run time from the tree that compiled nub-core (its
# baked CARGO_MANIFEST_DIR), so a shared-bucket binary loads whichever SHARER
# compiled last — a worktree with edited runtime files would test a sibling's
# copy, silently. Isolating on runtime divergence keeps every shared-bucket
# binary pointed at base-identical runtime content. (Observed: three worktrees'
# red/green verdicts flipped as siblings rebuilt the common bucket.)
# `-C "$root"` on the git queries so the pathspecs resolve from the repo root
# regardless of the CWD the wrapper was invoked from (a subdir would otherwise
# misread them). The final `exec cargo` still runs in the original CWD.
Expand All @@ -105,17 +112,17 @@ diverged=""
if [ -n "$base" ]; then
# shellcheck disable=SC2086 # $leaves must word-split into separate pathspecs
diverged=$(git -C "$root" diff --name-only "$base" -- \
vendor/aube crates $leaves 2>/dev/null || true)
vendor/aube crates runtime $leaves 2>/dev/null || true)
fi
# shellcheck disable=SC2086
untracked=$(git -C "$root" ls-files --others --exclude-standard -- \
vendor/aube crates $leaves 2>/dev/null || true)
vendor/aube crates runtime $leaves 2>/dev/null || true)

# The content key names the bucket AND, when isolating, names the seed to clone
# from — so it is computed unconditionally. `ls-files -s` emits the staged blob
# OIDs, so this is a pure content hash of the depended-on crates. ~0.2s.
# shellcheck disable=SC2086
key=$(git -C "$root" ls-files -s -- vendor/aube crates $leaves 2>/dev/null \
key=$(git -C "$root" ls-files -s -- vendor/aube crates runtime $leaves 2>/dev/null \
Comment thread
pullfrog[bot] marked this conversation as resolved.
Resolved
| shasum 2>/dev/null | cut -c1-12 || true)
if [ "$keyed" = 1 ]; then
bucket="$shared${key:+-$key}"
Expand Down Expand Up @@ -188,13 +195,27 @@ if [ -n "$diverged" ] || [ -n "$untracked" ]; then
else
target="$bucket" # shared fast path, content-keyed
why="shared bucket ${key:-none} — depended-on crate content"
# MIGRATION: content-keying renames the bucket, so on the first run after this
# change lands the warm legacy dir would be orphaned and every worktree would
# eat one cold build. Seed the new bucket from it (same CoW clone, ~0 cost).
# Self-retiring: the legacy dir is covered by the GC below, so once every live
# worktree has moved to a keyed bucket it ages out on its own.
if [ "$target" != "$shared" ] && seed_from "$shared" "$target"; then
why="$why (migrated from the legacy shared dir)"
# RE-KEY MIGRATION: any change that moves the content key — origin/main
# advancing a hashed path, or a pathspec addition like runtime/ — renames the
# bucket, and without a seed every worktree eats one cold build. Seed from the
# newest existing keyed bucket, falling back to the legacy un-keyed dir
# (pre-keying installs). Sound for the same reason the isolated-branch seed is:
# sharers of the NEW bucket agree on current content, and cargo rebuilds any
# seeded artifact whose fingerprint no longer matches. Self-retiring: orphaned
# sources age out via the GC below.
if [ "$target" != "$shared" ]; then
# An EMPTY bucket blocks seeding as hard as a warm one: `seed_from` refuses an
# existing destination, and the unconditional `mkdir -p` below (also reached
# via --print-target) creates exactly such a placeholder. `rmdir` clears it
# and cannot touch a bucket with contents.
rmdir "$target" 2>/dev/null || true
if [ ! -e "$target" ]; then
_seed=$(newest_bucket)
[ -n "$_seed" ] && [ -d "$_seed" ] || _seed="$shared"
if seed_from "$_seed" "$target"; then
Comment thread
pullfrog[bot] marked this conversation as resolved.
Resolved
why="$why (seeded from $(basename "$_seed"))"
fi
fi
fi
fi

Expand Down
Loading