Avoid stale embedded Metal shaders in llama builds

[i386]

Summary

• avoid using sccache for the C compiler launcher on Darwin Metal llama.cpp builds
• keep C++/CUDA/HIP compiler launchers cached where they are safe
• explicitly clear CMake compiler launcher settings when caching is unavailable or disabled

Why

The embedded Metal shader library is included through an assembly .incbin object. sccache does not include that embedded payload in the assembly cache key, which can leave a llama.cpp build using stale Metal shader contents after shader source changes.

Validation

• bash -n scripts/build-llama.sh
• just build

Summary by CodeRabbit

• Bug Fixes
  • Improved compiler caching configuration for Apple Metal builds.
  • Prevented unsupported or unavailable caching settings from affecting C, C++, CUDA, and HIP compilation.
  • Updated build status messages to accurately reflect when caching is enabled only where safe.

[github-actions]

This pull request is currently a draft. Reviews will not take place until the PR is marked as ready for review.

[coderabbitai]

📝 Walkthrough

Walkthrough

The build script updates sccache handling for Darwin Metal builds and explicitly clears CMake compiler launcher variables when sccache is unavailable or disabled.

Changes

Sccache compiler launcher configuration

Layer / File(s)	Summary
Compiler launcher branch configuration scripts/build-llama.sh	Darwin Metal builds disable sccache for C while retaining it for C++; unavailable or disabled sccache clears C, C++, CUDA, and HIP launcher variables and updates the status output.

Estimated code review effort: 2 (Simple) | ~10 minutes

Suggested reviewers: ndizazzo

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly matches the main change: preventing stale embedded Metal shader contents in llama builds.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch jd/avoid-stale-metal-shader-cache

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@scripts/build-llama.sh`:
- Around line 228-233: When sccache is unavailable, update the CMAKE_ARGS
fallback block to explicitly add -DGGML_CCACHE=OFF alongside the cleared
compiler launcher settings. Keep the existing no-caching behavior and mirror the
corresponding setting in the else branch.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: ae6dca9c-5379-44f6-b2cb-79b65981d186

📥 Commits

Reviewing files that changed from the base of the PR and between efcae3e and 04deeb2.

📒 Files selected for processing (1)

• scripts/build-llama.sh

[coderabbitai]

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Explicitly disable GGML_CCACHE to prevent unintended fallback to ccache.

The log message on line 227 states that the build will run "without compiler caching." However, because llama.cpp defaults GGML_CCACHE to ON, it will automatically search for ccache and overwrite the explicitly cleared compiler launchers if ccache happens to be installed on the system.

To ensure caching is truly disabled when sccache is not found, pass -DGGML_CCACHE=OFF, mirroring the else branch below.

💻 Proposed fix

   CMAKE_ARGS+=(
+    -DGGML_CCACHE=OFF
     -DCMAKE_C_COMPILER_LAUNCHER=
     -DCMAKE_CXX_COMPILER_LAUNCHER=
     -DCMAKE_CUDA_COMPILER_LAUNCHER=
     -DCMAKE_HIP_COMPILER_LAUNCHER=
   )

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change

	CMAKE_ARGS+=(
	-DCMAKE_C_COMPILER_LAUNCHER=
	-DCMAKE_CXX_COMPILER_LAUNCHER=
	-DCMAKE_CUDA_COMPILER_LAUNCHER=
	-DCMAKE_HIP_COMPILER_LAUNCHER=
	)
	CMAKE_ARGS+=(
	-DGGML_CCACHE=OFF
	-DCMAKE_C_COMPILER_LAUNCHER=
	-DCMAKE_CXX_COMPILER_LAUNCHER=
	-DCMAKE_CUDA_COMPILER_LAUNCHER=
	-DCMAKE_HIP_COMPILER_LAUNCHER=
	)

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@scripts/build-llama.sh` around lines 228 - 233, When sccache is unavailable,
update the CMAKE_ARGS fallback block to explicitly add -DGGML_CCACHE=OFF
alongside the cleared compiler launcher settings. Keep the existing no-caching
behavior and mirror the corresponding setting in the else branch.

[michaelneale]

Holding this one - I don't think it does what it claims, so let's not merge as-is.

The .incbin payload compiles as an ASM-language source (ggml-metal-embed.s via enable_language(ASM)), so clearing CMAKE_C_COMPILER_LAUNCHER has no effect on it - that's a per-language var for C sources. And in our build the ASM step isn't wrapped by sccache at all: ggml's internal RULE_LAUNCH_COMPILE fallback is guarded by if(GGML_CCACHE AND NOT C_LAUNCHER AND NOT CXX_LAUNCHER), and we pass both launchers, so that block never runs. So there's no cache-key bug on that step to fix here.

The likely reason it appeared to fix stale shaders: changing CMAKE_ARGS changes the build stamp and forces a reconfigure/rebuild, so the stale object got relinked incidentally.

If we do want a real hardening, the minimal correct version keeps C+CXX cached and just disables ggml's fallback:

CMAKE_ARGS+=(
  -DGGML_CCACHE=OFF
  -DCMAKE_C_COMPILER_LAUNCHER="$SCCACHE_BIN"
  -DCMAKE_CXX_COMPILER_LAUNCHER="$SCCACHE_BIN"
)

(Note CMAKE_ASM_COMPILER_LAUNCHER is not a thing - CMake doesn't implement a launcher for ASM.)

Also minor: the 'sccache not found' branch clears the launchers but leaves GGML_CCACHE on, so on a box with ccache-but-no-sccache ggml would still enable global caching despite the 'without caching' message.

Separately - the failing Windows CPU check here is not caused by this PR; it's the pre-existing autoupdate.rs INSTALL_SCRIPT_URL cfg-gate breakage on main (fixed by #1030).

[ndizazzo]

Holding this one - I don't think it does what it claims, so let's not merge as-is.

The .incbin payload compiles as an ASM-language source (ggml-metal-embed.s via enable_language(ASM)), so clearing CMAKE_C_COMPILER_LAUNCHER has no effect on it - that's a per-language var for C sources. And in our build the ASM step isn't wrapped by sccache at all: ggml's internal RULE_LAUNCH_COMPILE fallback is guarded by if(GGML_CCACHE AND NOT C_LAUNCHER AND NOT CXX_LAUNCHER), and we pass both launchers, so that block never runs. So there's no cache-key bug on that step to fix here.

The likely reason it appeared to fix stale shaders: changing CMAKE_ARGS changes the build stamp and forces a reconfigure/rebuild, so the stale object got relinked incidentally.

If we do want a real hardening, the minimal correct version keeps C+CXX cached and just disables ggml's fallback:

CMAKE_ARGS+=(
  -DGGML_CCACHE=OFF
  -DCMAKE_C_COMPILER_LAUNCHER="$SCCACHE_BIN"
  -DCMAKE_CXX_COMPILER_LAUNCHER="$SCCACHE_BIN"
)

(Note CMAKE_ASM_COMPILER_LAUNCHER is not a thing - CMake doesn't implement a launcher for ASM.)

Also minor: the 'sccache not found' branch clears the launchers but leaves GGML_CCACHE on, so on a box with ccache-but-no-sccache ggml would still enable global caching despite the 'without caching' message.

Separately - the failing Windows CPU check here is not caused by this PR; it's the pre-existing autoupdate.rs INSTALL_SCRIPT_URL cfg-gate breakage on main (fixed by #1030).

Yeah, this checks out. Since the shader gets compiled as ASM, changing the C launcher won’t really affect that step. Let’s keep the C/C++ launcher flags, (could add -DGGML_CCACHE=OFF to disable the fallback), and dig into the stale shader issue separately. We could start with ninja -d explain -v to investigate the build obj.