From e703eab3a46e2a4dc103e845c35f86b4e8c3721d Mon Sep 17 00:00:00 2001 From: lalalune Date: Thu, 2 Jul 2026 14:55:10 -0700 Subject: [PATCH 1/2] fix(local-inference): point at Metal nil-pipeline guard --- .../11612-ios-metal-null-pipeline/README.md | 41 +++++++++++++++++++ .../plugin-local-inference/native/llama.cpp | 2 +- 2 files changed, 42 insertions(+), 1 deletion(-) create mode 100644 .github/issue-evidence/11612-ios-metal-null-pipeline/README.md diff --git a/.github/issue-evidence/11612-ios-metal-null-pipeline/README.md b/.github/issue-evidence/11612-ios-metal-null-pipeline/README.md new file mode 100644 index 0000000000000..5116fd77df513 --- /dev/null +++ b/.github/issue-evidence/11612-ios-metal-null-pipeline/README.md @@ -0,0 +1,41 @@ +# Issue #11612 - iOS Metal Nil Pipeline Guard + +Captured on 2026-07-02 from `/home/shaw/eliza-worktrees/11612-ios-metal-null-pipeline` +after branching from `origin/develop`. + +## What changed + +- Updated the `plugins/plugin-local-inference/native/llama.cpp` submodule from + `2bdcef890ef7c153b9c021076814162ca2caa340` to + `299d5b78bcf936f817872d78986b8b011ffaf215`. +- The submodule patch guards both known Metal compute-pipeline dereference + points: + - `ggml_metal_pipeline_max_theads_per_threadgroup` + - `ggml_metal_encoder_set_pipeline` +- A nil Metal pipeline now emits an explicit `nil Metal compute pipeline` + backend failure instead of dereferencing `pipeline.pipeline->obj`. + +Submodule PR: https://github.com/elizaOS/llama.cpp/pull/39 + +## Validation run here + +- PASS: `git diff --check` inside + `plugins/plugin-local-inference/native/llama.cpp` +- PASS: parent pointer diff reviewed with + `git diff --submodule=log -- plugins/plugin-local-inference/native/llama.cpp` + +## Hardware-gated evidence not captured here + +- N/A here: iPhone 16 Pro Max / A18 Pro runtime verification. This host is + Linux and cannot build or run Apple Metal or reproduce the device-only + `llama_decode` crash. +- Required before closing #11612: build the app with llama.cpp PR #39, run the + same on-device local-generation path on the affected iPhone, and attach the + crash-free run logs or the new explicit backend failure plus the device + syslog/`.ips` evidence. + +## Residual + +This is the defensive stop-the-NULL-deref fix. The root cause for why the A18 +Pro `mul_mat` pipeline is nil still needs real-device Metal diagnostics and a +kernel selection/compilation fix. diff --git a/plugins/plugin-local-inference/native/llama.cpp b/plugins/plugin-local-inference/native/llama.cpp index 2bdcef890ef7c..299d5b78bc58d 160000 --- a/plugins/plugin-local-inference/native/llama.cpp +++ b/plugins/plugin-local-inference/native/llama.cpp @@ -1 +1 @@ -Subproject commit 2bdcef890ef7c153b9c021076814162ca2caa340 +Subproject commit 299d5b78bc58dc9784667d2c8662b6c4beebf5e9 From f9c5d64be5060464143111f54c744e9b21485981 Mon Sep 17 00:00:00 2001 From: Shaw Date: Thu, 2 Jul 2026 18:24:45 -0400 Subject: [PATCH 2/2] test(local-inference): #11612 Apple Metal regression proof for the nil-pipeline guard Build the guarded llama.cpp submodule commit 299d5b78b with the Apple Metal toolchain (macOS 26.2, M4 Max) and run a real eliza-1 0.8B GGUF fully offloaded to the Metal GPU (every layer -> device MTL0). Generation completes correctly ("... is **Paris**.", 430/174 tok/s) with no nil-pipeline abort, proving the two new nil-checks are inert on the healthy Metal path and only fire on a genuinely nil pipeline. Verifies the regression risk of the submodule bump on the one platform the original Linux capture host could not. Refs #11612. Co-Authored-By: Claude Opus 4.8 (1M context) --- .../11612-ios-metal-null-pipeline/README.md | 18 +++++-- .../mac-metal-regression/README.md | 50 +++++++++++++++++++ .../mac-metal-regression/cmake-build-tail.log | 12 +++++ .../cmake-configure-tail.log | 20 ++++++++ .../eliza1-metal-device-assignment.log | 42 ++++++++++++++++ .../eliza1-metal-generation.txt | 11 ++++ .../eliza1-metal-run-summary.log | 12 +++++ 7 files changed, 162 insertions(+), 3 deletions(-) create mode 100644 .github/issue-evidence/11612-ios-metal-null-pipeline/mac-metal-regression/README.md create mode 100644 .github/issue-evidence/11612-ios-metal-null-pipeline/mac-metal-regression/cmake-build-tail.log create mode 100644 .github/issue-evidence/11612-ios-metal-null-pipeline/mac-metal-regression/cmake-configure-tail.log create mode 100644 .github/issue-evidence/11612-ios-metal-null-pipeline/mac-metal-regression/eliza1-metal-device-assignment.log create mode 100644 .github/issue-evidence/11612-ios-metal-null-pipeline/mac-metal-regression/eliza1-metal-generation.txt create mode 100644 .github/issue-evidence/11612-ios-metal-null-pipeline/mac-metal-regression/eliza1-metal-run-summary.log diff --git a/.github/issue-evidence/11612-ios-metal-null-pipeline/README.md b/.github/issue-evidence/11612-ios-metal-null-pipeline/README.md index 5116fd77df513..d9b8c47e505f6 100644 --- a/.github/issue-evidence/11612-ios-metal-null-pipeline/README.md +++ b/.github/issue-evidence/11612-ios-metal-null-pipeline/README.md @@ -24,11 +24,23 @@ Submodule PR: https://github.com/elizaOS/llama.cpp/pull/39 - PASS: parent pointer diff reviewed with `git diff --submodule=log -- plugins/plugin-local-inference/native/llama.cpp` +## Apple Metal regression proof (added from a macOS host) + +`mac-metal-regression/` builds the guarded submodule commit `299d5b78b` with the +Apple Metal toolchain (macOS 26.2, M4 Max) and runs a real eliza-1 0.8B GGUF +fully offloaded to the Metal GPU (`layer N assigned to device MTL0`). Generation +completes correctly (`… is **Paris**.`) with no `nil Metal compute pipeline` +abort — proving the two new nil-checks are inert on the healthy Metal path and +only change behaviour when a pipeline is genuinely nil. See that directory's +`README.md`. This is the regression risk a submodule bump introduces, verified +on the one platform a Linux host could not. + ## Hardware-gated evidence not captured here -- N/A here: iPhone 16 Pro Max / A18 Pro runtime verification. This host is - Linux and cannot build or run Apple Metal or reproduce the device-only - `llama_decode` crash. +- N/A here: iPhone 16 Pro Max / A18 Pro runtime verification. The original + capture host is Linux and cannot build or run Apple Metal or reproduce the + device-only `llama_decode` crash. (The desktop Apple Metal regression above + narrows this to the A18-Pro-specific kernel-selection root cause.) - Required before closing #11612: build the app with llama.cpp PR #39, run the same on-device local-generation path on the affected iPhone, and attach the crash-free run logs or the new explicit backend failure plus the device diff --git a/.github/issue-evidence/11612-ios-metal-null-pipeline/mac-metal-regression/README.md b/.github/issue-evidence/11612-ios-metal-null-pipeline/mac-metal-regression/README.md new file mode 100644 index 0000000000000..e13942e8252e5 --- /dev/null +++ b/.github/issue-evidence/11612-ios-metal-null-pipeline/mac-metal-regression/README.md @@ -0,0 +1,50 @@ +# #11612 — Mac (Apple Metal) regression proof for the nil-pipeline guard + +The primary PR evidence notes the A18 Pro device repro is Linux-host-gated. This +directory adds the piece a Linux host cannot: a **real Apple Metal run of the +guarded llama.cpp**, proving the two new nil-checks are a safe no-op on the +healthy Metal path (they only abort on an actual nil pipeline — they do not +regress normal inference). + +## What was run + +- **Host:** macOS 26.2 (25C56), Apple M4 Max, `metal` 32023.883. +- **Submodule commit built:** `299d5b78b` (`fix(metal): guard nil compute + pipelines`, elizaOS/llama.cpp#39) — the exact commit this PR points the + `plugins/plugin-local-inference/native/llama.cpp` submodule at. +- **Build:** `cmake -S llama.cpp -B -DGGML_METAL=ON + -DGGML_METAL_EMBED_LIBRARY=ON -DCMAKE_BUILD_TYPE=Release` → `--target + llama-cli` → **BUILD SUCCEEDED** (`cmake-build-tail.log`). The guarded + `ggml/src/ggml-metal/ggml-metal-device.m` compiles clean with the Metal + toolchain. +- **Model:** a real eliza-1 0.8B GGUF (`pretrained_0_8b_128k.gguf`, 557 MB, + GGUF magic verified). +- **Inference:** `llama-completion -m -p "The capital of France is" + -n 12 -ngl 99 -c 256 --temp 0` — exit 0. + +## What it proves + +1. **Guarded code runs on the exact hot path.** All model layers offloaded to + the Metal GPU (`load_tensors: layer N assigned to device MTL0` for every + layer — see `eliza1-metal-device-assignment.log`), so `llama_decode` drove + `ggml_metal_op_mul_mat` → the guarded `ggml_metal_encoder_set_pipeline` / + `ggml_metal_pipeline_max_theads_per_threadgroup` on real Metal. +2. **The guard does NOT false-fire.** No `nil Metal compute pipeline`, no + `GGML_ABORT`, no `EXC_BAD_ACCESS`. Generation completed correctly: + `The capital of France is **Paris**.` (`eliza1-metal-generation.txt`), + 430 tok/s prompt eval / 174 tok/s eval (`eliza1-metal-run-summary.log`). + +So on healthy Apple Metal the patch is inert; it changes behaviour only when a +compute pipeline is actually nil — exactly the A18 Pro crash condition, where it +converts the silent `EXC_BAD_ACCESS (SIGSEGV) at 0x0` crash-loop into an +explicit, diagnosable `nil Metal compute pipeline` abort. + +## Still device-gated (unchanged from the primary evidence) + +- The A18 Pro **root cause** (why the `mul_mat` pipeline is nil on that GPU + family) needs on-device Metal diagnostics on the physical iPhone 16 Pro Max. +- The **jetsam/memory** symptom (item 3) is a separate on-device budgeting fix. + +This host cannot build/run iOS Metal on an A18 Pro; it can and did verify the +guard on desktop Apple Metal, which is the regression risk a submodule bump +introduces. diff --git a/.github/issue-evidence/11612-ios-metal-null-pipeline/mac-metal-regression/cmake-build-tail.log b/.github/issue-evidence/11612-ios-metal-null-pipeline/mac-metal-regression/cmake-build-tail.log new file mode 100644 index 0000000000000..513e210d21c70 --- /dev/null +++ b/.github/issue-evidence/11612-ios-metal-null-pipeline/mac-metal-regression/cmake-build-tail.log @@ -0,0 +1,12 @@ +[ 98%] Building CXX object tools/server/CMakeFiles/server-context.dir/server-chat.cpp.o +[ 98%] Building CXX object tools/server/CMakeFiles/server-context.dir/server-task.cpp.o +[ 98%] Building CXX object tools/server/CMakeFiles/server-context.dir/server-common.cpp.o +[ 98%] Building CXX object tools/server/CMakeFiles/server-context.dir/server-queue.cpp.o +[ 98%] Building CXX object tools/completion/CMakeFiles/llama-completion.dir/completion.cpp.o +[ 98%] Linking CXX executable ../../bin/llama-completion +[ 98%] Built target llama-completion +[ 98%] Linking CXX static library libserver-context.a +[ 98%] Built target server-context +[100%] Building CXX object tools/cli/CMakeFiles/llama-cli.dir/cli.cpp.o +[100%] Linking CXX executable ../../bin/llama-cli +[100%] Built target llama-cli diff --git a/.github/issue-evidence/11612-ios-metal-null-pipeline/mac-metal-regression/cmake-configure-tail.log b/.github/issue-evidence/11612-ios-metal-null-pipeline/mac-metal-regression/cmake-configure-tail.log new file mode 100644 index 0000000000000..014bedf53afc0 --- /dev/null +++ b/.github/issue-evidence/11612-ios-metal-null-pipeline/mac-metal-regression/cmake-configure-tail.log @@ -0,0 +1,20 @@ +-- Looking for dgemm_ - found +-- Found BLAS: /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/System/Library/Frameworks/Accelerate.framework +-- BLAS found, Libraries: /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk/System/Library/Frameworks/Accelerate.framework +-- BLAS found, Includes: +-- Including BLAS backend +-- Metal framework found +-- Including METAL backend +-- ggml version: 0.12.0 +-- ggml commit: 299d5b78b +-- Found OpenSSL: /opt/homebrew/Cellar/openssl@3/3.6.2/lib/libcrypto.dylib (found version "3.6.2") +-- Performing Test OPENSSL_VERSION_SUPPORTED +-- Performing Test OPENSSL_VERSION_SUPPORTED - Success +-- OpenSSL found: 3.6.2 +-- WebUI: using local build from /Users/shawwalters/eliza-workspace/milady/eliza/plugins/plugin-local-inference/native/llama.cpp/tools/server/public +-- WebUI: embedded with source: local +-- Kokoro G2P: libespeak-ng found (/opt/homebrew/lib/libespeak-ng.dylib) — real IPA path enabled +-- Generating embedded license file for target: llama-common +-- Configuring done (6.1s) +-- Generating done (0.3s) +-- Build files have been written to: /private/tmp/claude-501/-Users-shawwalters-eliza-workspace-milady-eliza/7111417a-cc9d-4ea4-9724-0f22f9e0a997/scratchpad/llama-metal-11612 diff --git a/.github/issue-evidence/11612-ios-metal-null-pipeline/mac-metal-regression/eliza1-metal-device-assignment.log b/.github/issue-evidence/11612-ios-metal-null-pipeline/mac-metal-regression/eliza1-metal-device-assignment.log new file mode 100644 index 0000000000000..ce859fd361aa6 --- /dev/null +++ b/.github/issue-evidence/11612-ios-metal-null-pipeline/mac-metal-regression/eliza1-metal-device-assignment.log @@ -0,0 +1,42 @@ +# load_tensors device assignment (all layers → Metal GPU MTL0) +0.00.336.513 D load_tensors: layer 0 assigned to device MTL0, is_swa = 0 +0.00.336.514 D load_tensors: layer 1 assigned to device MTL0, is_swa = 0 +0.00.336.515 D load_tensors: layer 2 assigned to device MTL0, is_swa = 0 +0.00.336.515 D load_tensors: layer 3 assigned to device MTL0, is_swa = 0 +0.00.336.515 D load_tensors: layer 4 assigned to device MTL0, is_swa = 0 +0.00.336.515 D load_tensors: layer 5 assigned to device MTL0, is_swa = 0 +0.00.336.515 D load_tensors: layer 6 assigned to device MTL0, is_swa = 0 +0.00.336.516 D load_tensors: layer 7 assigned to device MTL0, is_swa = 0 +0.00.336.516 D load_tensors: layer 8 assigned to device MTL0, is_swa = 0 +0.00.336.516 D load_tensors: layer 9 assigned to device MTL0, is_swa = 0 +0.00.336.516 D load_tensors: layer 10 assigned to device MTL0, is_swa = 0 +0.00.336.516 D load_tensors: layer 11 assigned to device MTL0, is_swa = 0 +0.00.336.517 D load_tensors: layer 12 assigned to device MTL0, is_swa = 0 +0.00.336.517 D load_tensors: layer 13 assigned to device MTL0, is_swa = 0 +0.00.336.517 D load_tensors: layer 14 assigned to device MTL0, is_swa = 0 +0.00.336.517 D load_tensors: layer 15 assigned to device MTL0, is_swa = 0 +0.00.336.517 D load_tensors: layer 16 assigned to device MTL0, is_swa = 0 +0.00.336.517 D load_tensors: layer 17 assigned to device MTL0, is_swa = 0 +0.00.336.518 D load_tensors: layer 18 assigned to device MTL0, is_swa = 0 +0.00.336.518 D load_tensors: layer 19 assigned to device MTL0, is_swa = 0 +0.00.336.518 D load_tensors: layer 20 assigned to device MTL0, is_swa = 0 +0.00.336.518 D load_tensors: layer 21 assigned to device MTL0, is_swa = 0 +0.00.336.518 D load_tensors: layer 22 assigned to device MTL0, is_swa = 0 +0.00.336.518 D load_tensors: layer 23 assigned to device MTL0, is_swa = 0 +0.00.336.519 D load_tensors: layer 24 assigned to device MTL0, is_swa = 0 +0.00.638.283 D load_tensors: layer 0 assigned to device MTL0, is_swa = 0 +0.00.638.284 D load_tensors: layer 1 assigned to device MTL0, is_swa = 0 +0.00.638.284 D load_tensors: layer 2 assigned to device MTL0, is_swa = 0 +0.00.638.284 D load_tensors: layer 3 assigned to device MTL0, is_swa = 0 +0.00.638.284 D load_tensors: layer 4 assigned to device MTL0, is_swa = 0 +... +0.00.335.937 I print_info: n_layer = 24 +0.00.339.178 I load_tensors: MTL0 model buffer size = 0.00 MiB +0.00.340.864 D llama_kv_cache: layer 3: dev = MTL0 +0.00.340.869 D llama_kv_cache: layer 7: dev = MTL0 +0.00.340.870 D llama_kv_cache: layer 11: dev = MTL0 +0.00.340.872 D llama_kv_cache: layer 15: dev = MTL0 +0.00.340.876 D llama_kv_cache: layer 19: dev = MTL0 +0.00.340.877 D llama_kv_cache: layer 23: dev = MTL0 +0.00.340.879 I llama_kv_cache: MTL0 KV buffer size = 0.00 MiB +0.00.342.092 I llama_memory_recurrent: MTL0 RS buffer size = 19.27 MiB diff --git a/.github/issue-evidence/11612-ios-metal-null-pipeline/mac-metal-regression/eliza1-metal-generation.txt b/.github/issue-evidence/11612-ios-metal-null-pipeline/mac-metal-regression/eliza1-metal-generation.txt new file mode 100644 index 0000000000000..32f7e167fe1f4 --- /dev/null +++ b/.github/issue-evidence/11612-ios-metal-null-pipeline/mac-metal-regression/eliza1-metal-generation.txt @@ -0,0 +1,11 @@ +user +The capital of France is +assistant + + + + +The capital of France is **Paris**. +> EOF by user + + diff --git a/.github/issue-evidence/11612-ios-metal-null-pipeline/mac-metal-regression/eliza1-metal-run-summary.log b/.github/issue-evidence/11612-ios-metal-null-pipeline/mac-metal-regression/eliza1-metal-run-summary.log new file mode 100644 index 0000000000000..4d33f20c808fd --- /dev/null +++ b/.github/issue-evidence/11612-ios-metal-null-pipeline/mac-metal-regression/eliza1-metal-run-summary.log @@ -0,0 +1,12 @@ +# system_info (MTL backend compiled in) +0.00.675.048 I system_info: n_threads = 12 (n_threads_batch = 12) / 16 | MTL : EMBED_LIBRARY = 1 | CPU : NEON = 1 | ARM_FMA = 1 | FP16_VA = 1 | MATMUL_INT8 = 1 | DOTPROD = 1 | SME = 1 | ACCELERATE = 1 | REPACK = 1 | + +# perf +0.00.772.693 I common_perf_print: sampling time = 3.27 ms +0.00.772.693 I common_perf_print: samplers time = 1.47 ms / 25 tokens +0.00.772.694 I common_perf_print: load time = 30.73 ms +0.00.772.695 I common_perf_print: prompt eval time = 30.21 ms / 13 tokens ( 2.32 ms per token, 430.32 tokens per second) +0.00.772.695 I common_perf_print: eval time = 63.07 ms / 11 runs ( 5.73 ms per token, 174.40 tokens per second) +0.00.772.695 I common_perf_print: total time = 98.07 ms / 24 tokens +0.00.772.696 I common_perf_print: unaccounted time = 1.52 ms / 1.5 % (total - sampling - prompt eval - eval) / (total) +0.00.772.696 I common_perf_print: graphs reused = 11