fix(#6612): make the pinned Claude Code the one that runs in the sandbox

[waynesun09]

Summary

The OpenShell base image bundles its own Claude Code at /usr/local/bin/claude (installed upstream with the unpinned curl claude.ai/install.sh, so it is whatever was current when the base was built — 2.1.156 on the base we pin), and /usr/local/bin precedes npm's global bin (/usr/bin) on the sandbox PATH. The npm install pinned by CLAUDE_CODE_VERSION was therefore never the claude that ran: every Renovate bump of the pin has been a runtime no-op, and model aliases resolved from 2.1.156's table (sonnet → claude-sonnet-4-5@20250929 on Vertex, while the pi runtime maps the same alias to claude-sonnet-4-6). Every fleet run's → Agent: … (v2.1.156) line shows it.

This points /usr/local/bin/claude at the npm install right after it and asserts at build time that claude --version on PATH reports CLAUDE_CODE_VERSION, so a base-image change cannot silently bring the shadow back. fullsend-code inherits the fix from this image.

Related Issue

Fixes #6612

Changes

• images/sandbox/Containerfile: after npm install -g @anthropic-ai/claude-code@${CLAUDE_CODE_VERSION}, replace /usr/local/bin/claude with a symlink to $(npm prefix -g)/bin/claude and fail the build if claude --version ≠ CLAUDE_CODE_VERSION.
• internal/sandbox/sandbox_claude_image_test.go: TestSandboxImageClaudeCodePinWins guards both halves (symlink after the install, version assertion), in the style of the pi image guards.
• docs/contributing/runtime-implementation.md: restructured for readability (second commit) — "On this page" list, an explicit "Adding a runtime" checklist, the security matrix split into host-side / sandbox-hook / bootstrap tables with the shared PostToolUse chain described once, the hook contract and pi internals broken into subsections and tables, and a new Pinned runtime binaries in the sandbox image section that records the Claude Code pin/shadow rule from this PR next to the pi and extension pins. Same facts, no anchor changes (#sandbox-hook-contract, #pi-runtime-internals-6464 still resolve); every identifier/issue/ADR reference from the old text is still present (checked mechanically).

Not in this PR (tracked on #6612): deciding what sonnet/opus/haiku should mean on the Claude runtime once the real pin runs (explicit ids in harnesses / agents: entries, or ANTHROPIC_DEFAULT_*_MODEL), and asking upstream whether bundling agent CLIs in the base image is intentional.

Evidence

$ podman run --rm --entrypoint sh ghcr.io/fullsend-ai/fullsend-code@sha256:7b2bcbdb… -c 'claude --version; /usr/bin/claude --version; ls -la /usr/local/bin/claude'
2.1.156 (Claude Code)
2.1.234 (Claude Code)
-rwxr-xr-x. 1 root root 240301704 May 29 16:24 /usr/local/bin/claude
$ podman run --rm --entrypoint sh ghcr.io/nvidia/openshell-community/sandboxes/base@sha256:aeef1c63… -c '/usr/local/bin/claude --version'
2.1.156 (Claude Code)

Live: fullsend-ai/fullsend run 32895730379 (Agent: claude-opus-4-6 (v2.1.156)), pi-xai-vertex runs 32893364520 / 32894519933 (Model: sonnet → claude-sonnet-4-5@20250929, v2.1.156), konflux-ci/.fullsend triage/code/review/fix runs since Aug 18 — all v2.1.156.

Local check of the new step against the pinned fullsend-sandbox@sha256:46adf184… (matching pin builds, CLAUDE_CODE_VERSION=9.9.9 fails the assertion): results in a follow-up comment.

Testing

• go test ./internal/sandbox/ -run TestSandboxImage passes
• make lint passes
• Tests added for the new Containerfile invariant
• sandbox-images PR build (CI) — this is the real verification: the assertion runs inside the build

Checklist

• PR title follows Conventional Commits
• Commits are signed off (DCO) — human-directed session
• I wrote this contribution myself and can explain all changes in it

Note for reviewers

This moves every Claude-runtime agent from 2.1.156 to 2.1.243 in one step. The hook/tool-name contracts and --print JSON parsing have only ever been exercised against 2.1.156 in CI, so please let e2e/functional run fully rather than treating this as a trivial Containerfile edit. Agents' harness image pins move separately (fullsend-ai/agents, after the next release — #6607).

[waynesun09]

Local check of the new step (rootless Podman on Fedora, against the pinned fullsend-sandbox@sha256:46adf184…, which carries npm 2.1.234 + the base image's 2.1.156)

Matching pin (CLAUDE_CODE_VERSION=2.1.234) — builds:

before: 2.1.156 (Claude Code)
npm: 2.1.234 (Claude Code)
after: 2.1.234 (Claude Code) via /usr/lib/node_modules/@anthropic-ai/claude-code/bin/claude.exe
as sandbox user: 2.1.234 (Claude Code) PATH=/sandbox/.venv/bin:/usr/local/bin:/usr/bin:/bin
COMMIT fs6612-test

Wrong pin (--build-arg CLAUDE_CODE_VERSION=9.9.9) — the assertion fails the build as intended:

claude on PATH reports 2.1.234, expected CLAUDE_CODE_VERSION=9.9.9
Error: building at STEP "RUN NPM_CLAUDE=…": while running runtime: exit status 1

make lint clean; go test ./internal/sandbox/ -run TestSandboxImage passes.

Note on CI scope: the build-base job here proves the assertion inside a real build of this Containerfile, but the PR's e2e/functional-tests still run against the published images (the PR build doesn't push), so the first runtime exercise of Claude Code 2.1.243 is the main build after merge, followed by the agents-repo image repin (#6607). Worth watching the first fleet runs' Agent: … (v2.1.243) line after that repin.

[qodo-code-review]

PR Summary by Qodo

Ensure sandbox runs the pinned Claude Code version

🐞 Bug fix 🧪 Tests 📝 Documentation 🕐 10-20 Minutes

AI Description

• Ensure sandbox PATH resolves Claude Code to the pinned npm installation.
• Fail image builds when the resolved Claude Code version differs from the pin.
• Add regression coverage and document runtime-version verification.

Diagram

graph TD
  A["OpenShell Base"] --> B["Pinned npm Install"] --> C["Claude Symlink"] --> D["Sandbox PATH"] --> E["Version Check"]
  D --> F["Claude Runtime"]

Loading

High-Level Assessment

Replacing the shadowing executable with a symlink and asserting the resolved version is the most robust approach. Changing PATH alone would remain vulnerable to explicit /usr/local/bin/claude calls and future base-image ordering changes, while the assertion prevents silent regressions.

Files changed (3) +69 / -2

Bug fix (1) +21 / -1

ContainerfileRedirect Claude Code to the pinned npm binary +21/-1

Redirect Claude Code to the pinned npm binary

• Replaces the OpenShell-provided /usr/local/bin/claude with a symlink to the globally installed npm package. Fails the image build if the Claude Code version resolved on PATH does not match CLAUDE_CODE_VERSION.

images/sandbox/Containerfile

Tests (1) +47 / -0

sandbox_claude_image_test.goGuard pinned Claude Code executable resolution +47/-0

Guard pinned Claude Code executable resolution

• Adds a regression test that requires an explicit semantic version pin, the npm installation, the replacement symlink, and the build-time version assertion. It also verifies that symlink creation follows package installation.

internal/sandbox/sandbox_claude_image_test.go

Documentation (1) +1 / -1

runtime-implementation.mdDocument the effective Claude Code runtime pin +1/-1

Document the effective Claude Code runtime pin

• Explains why the npm version pin now controls the executable used in sandboxes. Identifies the run log's Agent version as the authoritative runtime check.

docs/contributing/runtime-implementation.md

[qodo-code-review]

Code Review by Qodo

🐞 Bugs (0) 📘 Rule violations (0) 📎 Requirement gaps (0)

Great, no issues found!

Qodo reviewed your code and found no material issues that require review

Tip of the day

💡 Did you know, you can start a comment with 'qodo' or '@qodo' to chat about any finding

More tips ↗ | Customize Qodo ↗ | Qodo docs ↗

[fullsend-ai-review]

🤖 Finished Review · ✅ Success · Started 2:31 PM UTC · Completed 2:49 PM UTC

Commit: 384ee27 · View workflow run →

Runtime: claude · Model: opus → claude-opus-4-6 · Effort: high · Cost: $5.22

[github-actions]

Site preview

Preview: https://23eeefeb-site.fullsend-ai.workers.dev

Commit: 66d724d1e7849ca4286b9bc36c90801eaef799ba

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[fullsend-ai-review]

Risk Assessment: moderate (2/5)

Details

Tier 1 signals unchanged from prior assessment. Containerfile remains a high-churn hotspot (24 commits/30d, 5 authors, 6 fix/revert in 90d) and issue carries security+priority/high labels, but these were already factored into the prior score of 2. Good test coverage, no dependency or CI workflow changes, and well-scoped bug fix continue to mitigate risk.

Previous run

Risk Assessment: moderate (2/5)

Details

Small, focused bug fix (3 files, 386 lines) with high churn on Containerfile (24 commits/30d, 6 authors, 14 fix/revert commits in 90d), elevated by security+priority/high issue labels, mitigated by good test coverage (new test file, TEST_FILE_RATIO=0.33), no dependency or CI workflow changes, and well-scoped changes matching issue requirements.

Previous run (2)

Risk Assessment: moderate (2/5)

Details

Small, focused bug fix (3 files, 71 lines) to ensure the pinned Claude Code version takes precedence over the base image's pre-installed version. Elevated by high churn on Containerfile (24 commits in 30d, 6 authors, 14 fix/revert commits in 90d) and security/priority-high issue labels. Mitigated by good test coverage (new test file verifying version pin), small blast radius, and well-scoped changes matching issue requirements. Overall moderate risk.

[fullsend-ai-review]

Review

Findings

Medium

• [protected-path] images/sandbox/Containerfile — This PR modifies images/sandbox/Containerfile, which is under the images/ protected path. The PR links to issue sandbox: the OpenShell base image ships its own Claude Code (2.1.156) which shadows the CLAUDE_CODE_VERSION pin #6612 and the description explains the rationale (fixing PATH shadowing of the pinned Claude Code version by the OpenShell base image's unpinned copy). Human approval is always required for protected-path changes, regardless of context.

Low

• [edge-case] images/sandbox/Containerfile:69 — The version assertion parses claude --version with awk '{print $1}', extracting only the first whitespace-delimited field. If a future Claude Code release changes the --version output format, the assertion would fail the build — which is the correct fail-closed behavior, forcing a human to update the parsing.

• [incomplete-doc] images/README.md:143 — The supply chain security table's Claude Code row lists verification as only "npm registry integrity check", but this PR adds a build-time version assertion (claude --version must equal CLAUDE_CODE_VERSION). The "Verification" column could mention the build-time assertion for completeness.

• [scope-creep] docs/contributing/runtime-implementation.md — The second commit restructures all of runtime-implementation.md (277 additions, 40 deletions) alongside a targeted Containerfile bug fix. The volume is large relative to the fix. Mitigated by clean commit split (docs: prefix) and the restructuring can be reviewed and reverted independently.

• [naming-convention] internal/sandbox/sandbox_claude_image_test.go — Variables install and link hold []int index pairs from FindStringIndex, but the names read as nouns. Names like installIdx and linkIdx would better convey these are byte offsets.

Previous run

Review

Findings

Medium

• [protected-path] images/sandbox/Containerfile — This PR modifies images/sandbox/Containerfile, which is under the images/ protected path. The PR links to issue sandbox: the OpenShell base image ships its own Claude Code (2.1.156) which shadows the CLAUDE_CODE_VERSION pin #6612 and the description explains the rationale (fixing PATH shadowing of the pinned Claude Code version by the OpenShell base image's unpinned copy). Human approval is always required for protected-path changes, regardless of context.

Low

• [edge-case] images/sandbox/Containerfile:69 — The version assertion parses claude --version with awk '{print $1}', extracting only the first whitespace-delimited field. If a future Claude Code release changes the --version output format, the assertion would fail the build — which is the correct fail-closed behavior, forcing a human to update the parsing.

• [incomplete-doc] images/README.md:143 — The supply chain security table's Claude Code row lists verification as only "npm registry integrity check", but this PR adds a build-time version assertion (claude --version must equal CLAUDE_CODE_VERSION). The "Verification" column could mention the build-time assertion for completeness.

Previous run (2)

Review

Findings

Medium

• [protected-path] images/sandbox/Containerfile — This PR modifies images/sandbox/Containerfile, which is under the images/ protected path. The PR links to issue sandbox: the OpenShell base image ships its own Claude Code (2.1.156) which shadows the CLAUDE_CODE_VERSION pin #6612 and the description explains the rationale (fixing PATH shadowing of the pinned Claude Code version by the OpenShell base image's unpinned copy). Human approval is always required for protected-path changes, regardless of context.

Low

• [edge-case] images/sandbox/Containerfile:69 — The version assertion parses claude --version with awk '{print $1}', extracting only the first whitespace-delimited field. If a future Claude Code release changes the --version output format, the assertion would fail the build — which is the correct fail-closed behavior, forcing a human to update the parsing.

[waynesun09]

Full-image verification (Fedora 44, rootless Podman 5.8.4, linux/amd64 — run by the pi-worker session; mac-mini repeat pending)

Build: podman build --platform linux/amd64 -f images/sandbox/Containerfile from 384ee27c → Successfully tagged localhost/fs6647-sandbox:latest (4.19 GB, 36 steps). The new step (STEP 7/36) is silent on success by design — it only prints on mismatch.

In-image, identical as root and as sandbox (uid 998): claude --version → 2.1.243 (Claude Code); command -v claude → /usr/local/bin/claude → symlink to /usr/bin/claude → /usr/lib/node_modules/@anthropic-ai/claude-code/bin/claude.exe; PATH=/sandbox/.venv/bin:/usr/local/bin:/usr/bin:/bin. Baseline on the published sha256:46adf184…: claude --version → 2.1.156 (regular file), /usr/bin/claude --version → 2.1.234 — #6612 reproduced exactly, and the PR fixes it.

Negative check: --build-arg CLAUDE_CODE_VERSION=9.9.9 on the full Containerfile fails first at the npm install (notarget), so the assertion was exercised via a mini-Containerfile FROM localhost/fs6647-sandbox / USER root / ARG CLAUDE_CODE_VERSION=9.9.9 / the identical RUN → claude on PATH reports 2.1.243, expected CLAUDE_CODE_VERSION=9.9.9, build fails; control with 2.1.243 → builds.

Agent run through the new image: not completed — host-environmental, not a PR defect. fullsend run triage with FULLSEND_SANDBOX_IMAGE=localhost/fs6647-sandbox reached Sandbox created and then failed in copying fullsend binary to sandbox (gzip: unexpected end of file); a control run with the published image and no override failed identically, and a bare openshell sandbox create on that host never reached Ready (host swap 100% full). Will be repeated on a healthy host.

Model matrix on Vertex (the fleet's Vertex project, CLOUD_ML_REGION=global, run as sandbox inside each image; resolved id = key of .modelUsage in claude -p … --output-format json --max-turns 1, not the reply text):

requested	2.1.243 (this PR's image)	2.1.156 (published image today)
sonnet	claude-sonnet-4-5@20250929 ok	claude-sonnet-4-5@20250929 ok
opus	claude-opus-4-8 ok	claude-opus-4-6 ok
haiku	claude-haiku-4-5@20251001 ok	claude-haiku-4-5@20251001 ok
claude-sonnet-4-6	ok	ok
claude-opus-4-6	ok	ok
claude-haiku-4-5	ok	ok
claude-sonnet-5	ok	ok
claude-opus-4-8	ok	ok
claude-opus-5	not enabled (expected: this project deliberately enables only Opus 4.6 and 4.8)	not enabled (expected)

⚠️ Reviewer heads-up: the fleet harnesses pin model: opus. Once this image is what runs (i.e. after the agents-repo image repin that follows the next release, #6607), opus moves from claude-opus-4-6 to claude-opus-4-8 — an alias-table change in 2.1.243, not a project-availability fallback (claude-opus-4-6 still works when named explicitly; the claude-opus-5 rows are the project's deliberate model set, not a finding). sonnet stays on 4-5 in both versions (never 4-6). Also measured: ANTHROPIC_DEFAULT_{SONNET,OPUS}_MODEL does not steer the request on Vertex with either version — the stream-json init event echoes the override, but .modelUsage (and billing) stay on the alias default. Explicit --model <id> is what actually steers. So the #6612 item-3 knob is explicit ids in harnesses / agents: entries, not that env var. I'd land this PR as-is and put the explicit-id pin into the same agents-repo PR that repins the images, so the fleet never sees the alias jump.

[fullsend-ai-review]

🤖 Finished Review · ✅ Success · Started 6:17 PM UTC · Completed 6:36 PM UTC

Commit: 3beaecb · View workflow run →

Runtime: claude · Model: opus → claude-opus-4-6 · Effort: high · Cost: $5.56

[fullsend-ai-review]

See the review comment for full details.

[fullsend-ai-retro]

🤖 Finished Retro · ✅ Success · Started 7:58 PM UTC · Completed 8:16 PM UTC

Commit: 66d724d · View workflow run →

Runtime: claude · Model: opus → claude-opus-4-6 · Effort: high · Cost: $5.41

[fullsend-ai-retro]

Retro: PR #6647 — make the pinned Claude Code the one that runs in the sandbox

PR: #6647 by waynesun09 (human-authored, fix/6612-claude-symlink)
Issue: #6612 — sandbox PATH shadowing caused the base image's bundled Claude Code 2.1.156 to silently override the npm-pinned 2.1.243
Agents repo: fullsend-ai/agents@beed20e7e85f

Workflow timeline

Time (UTC)	Event
Aug 25 20:59	Triage agent ran on #6612 ($0.73) — correctly identified PATH shadowing root cause
Aug 25 21:01–21:05	Code agent ran on #6612 ($1.41) — produced no PR (see finding below)
Aug 26 14:29	waynesun09 opened PR #6647 with manual fix
Aug 26 14:31–14:49	Review agent run 1 ($5.22, commit 384ee27) — 2 findings
Aug 26 18:17–18:36	Review agent run 2 ($5.56, commit 3beaecb) — 3 findings
Aug 27 10:21	rh-hemartin approved (no comments)
Aug 27 18:40–19:02	Review agent run 3 ($5.71, commit 66d724d) — 5 findings, post-approval
Aug 27 19:56	PR merged

Total agent cost: ~$18.63 ($1.41 code + $0.73 triage + $16.49 review across 3 runs).

Code agent failure: staged correct changes but never committed

The code agent run (32898527242) correctly understood the issue, created branch agent/6612-fix-claude-path-shadowing, edited images/sandbox/Containerfile with the right fix (remove base binary, symlink to npm install, add version assertion), ran git add and pre-commit run — then exited without running git commit. The harness found zero committed changes and reported "no changes needed." The agent completed 37 turns in ~2 minutes and $1.41 before the manual fix was needed.

Ironically, the code agent itself was running on v2.1.156 (the shadowed version) — the exact bug it was trying to fix.

A proposal is filed below for enforcing git commit as a mandatory step in the code agent's implementation flow.

Review agent: good quality, known operational issues

Quality was strong. Findings were accurate and appropriately low-severity: protected-path flag for images/ (correct policy), awk version-parsing edge case (correct observation, intentionally fail-closed), missing supply-chain doc entry, scope-creep on docs restructure (mitigated by commit split), and naming convention suggestion. No false positives, no bugs missed.

Known operational issues observed — all well-covered by existing open issues:

• Duplicate inline comments: 3 identical inline comments posted on Containerfile:69 across 3 runs. The minimizeStaleReviews() mechanism collapses old formal reviews but does not prevent duplicate inline comments. Evidence for #5007 and #5760.
• Post-approval re-review: Third review run triggered by rebase 8+ hours after human approval, costing $5.71. Evidence for #5759, #963, and #3025.
• Dismissed finding re-raised: The author dismissed the awk edge-case finding with an explanation on run 1; it was re-raised identically on runs 2 and 3, prompting a second identical dismissal. Evidence for #5265, #3515, and #4682.

Overall assessment

The PR was well-crafted with thorough author self-testing (local builds, negative tests, Vertex model-alias matrix). Review quality was good. The main workflow gap is the code agent's failure to commit its correct changes — a single missing git commit call turned a successful implementation into a silent no-op, requiring manual rework.

Proposals filed

• Code agent skill must enforce git commit after staging changes (in fullsend-ai/agents)