Mac-native memory/RAG model hosting on mac-coordinator-64gb-20260925 (#379): measured capacity, receipt, host-roles proposal

adoption/host-roles.json

@@ -24,10 +24,13 @@
       "request_label": "host:mac-coordinator",
       "owns": [
         "macos-acceptance",
-        "coordination"
+        "coordination",
+        "model-hosting",
+        "memory-e2e",
+        "rag-e2e"
       ],
       "credentials": [],
-      "note": "The replacement coordinator Mac."
+      "note": "The replacement coordinator Mac. It also owns model-hosting, memory-e2e and rag-e2e for its own native memory/RAG foundation layer (user decision 2026-09-27, request #379; measured basis in docs/decisions/2026-09-27-mac-native-model-hosting.md). GPU runtime qualification, cross-host model qualification and independent review stay with the workstation. Routing still follows only the request label."
     }
   },
   "peers": [

adoption/platforms/macos-arm64.md

@@ -112,7 +112,14 @@ plans for is the labelled projection `macos-arm64-64gb-projected` in
 [`adoption/hardware-profiles.json`](../hardware-profiles.json) (about 38 GB of
 unified memory as the generation budget, `full` semantic-RAG tier, both drawn
 from one shared pool); for this page's embedding choice it follows the 48 GB
-rules. None of these sizes has a real qualification run yet.
+rules. The projections themselves have no qualification run. The measured
+64 GB host `mac-coordinator-64gb-20260925` has had one since 2026-09-27. The
+pinned RAG embedder, the memory stack's Ollama models and a 27B-class 4-bit
+generation model ran together under coordinator load with no swap growth and
+no memory kills. A large Node production build running alongside that set did
+push the host into swap (peak about 14 GB, no kills), so the shared-pool
+warning still applies to work beyond that set
+([#379 evidence](../../evidence/artifacts/mac-model-hosting-20260927/README.md)).
 
 ## Prerequisites
 
@@ -715,8 +722,14 @@ currently loaded from that same destination path, rename into place,
 confirmed loaded from its own destination path, or not loaded at all with a
 file present to clean up). On the hosted runner (run `35875188590`, "What a
 hosted run proves" above) `launchd-agents.sh` bootstrapped and booted out the
-`qdrant` and `llama-embed` agents; `ai-memory` has not run, and none of the
-three has run on a Mac workstation.
+`qdrant` and `llama-embed` agents; `ai-memory` has not run there. On
+2026-09-27 the `llama-embed` agent began running on the coordinator Mac
+`mac-coordinator-64gb-20260925`. It passes the embedding acceptance there,
+with all layers on Metal ([#379 evidence](../../evidence/artifacts/mac-model-hosting-20260927/README.md)).
+That host's pre-existing `local.agent-ecosystem.qdrant` and
+`local.agent-ecosystem.ai-memory` agents serve the other two roles. The
+`qdrant` and `ai-memory` templates therefore still have not run on a Mac
+workstation.
 
 **2026-09-23 decision: brew-services semantics, no backup or reconcile.**
 

docs/decisions/2026-09-27-mac-native-model-hosting.md

@@ -0,0 +1,79 @@
+# Decision: the 64 GB coordinator Mac hosts its own memory/RAG models natively (2026-09-27)
+
+**Decided by:** the user, 2026-09-27, in host request
+[#379](https://github.com/seathatflowsinourveins/native-agent-stack/issues/379): memory/RAG is a native
+foundation layer on every host, not a service borrowed from the workstation. This record gives the
+measured basis on `mac-coordinator-64gb-20260925` and the proposed role change. It was written by the Mac
+coordinator session on branch `claude/mac-model-hosting-20260927` (base `origin/main@8bb52541`).
+
+**Scope:** [`adoption/host-roles.json`](../../adoption/host-roles.json) (the proposal below) and the
+evidence in
+[`evidence/artifacts/mac-model-hosting-20260927/`](../../evidence/artifacts/mac-model-hosting-20260927/README.md)
+with its host receipt. Lane: `lane:foundation`. Nothing here changes a pin, a production service or the
+memory-layer decision: retrieval quality belongs to the S3 preregistration (#390).
+
+## Context
+
+Measured on this Mac (Apple M5 Pro, 64 GB unified memory, macOS 27.0) with a normal coordinator session
+active. The full numbers and their evidence classes are in the artifact README.
+
+- **RAG embedder.** The pinned profile's `com.native-stack.llama-embed` LaunchAgent (llama.cpp b11057
+  `--embedding`, embeddinggemma-300M-Q8_0, port 8232) passes
+  `tools/adoption/embed_acceptance.py` against the Linux reference: 768 dimensions, cosine 0.99966. All 25
+  layers run on Metal (`MTL0`), per llama-server's own log.
+- **Memory stack.** ai-memory, Ollama (qwen3-embedding:4b, qwen3.5-9b-64k) and Qdrant already ran here as
+  `local.agent-ecosystem.*` agents. No second Qdrant or ai-memory was started.
+- **Generation.** `unsloth/Qwen3.8-27B-GGUF@4ca7207` UD-Q4_K_M (16.46 GB, 27.8B dense) runs fully on
+  Metal.
+  - llama-bench: pp512 371.5, pp4096 341.7, tg128 15.9 tokens/s.
+  - Serving at a 32K context: peak RSS 22.9 GiB, swap unchanged, 0 memorystatus kills.
+  - li26 on 360 real 8-K filings, descriptive: micro-F1 0.9909 and JSON-valid 1.0 for C0 on b11057 and b11146
+    and for C2 with MTP drafting on b11146. Predictions are identical on all 360 filings. Median decode is
+    15.7, 15.8 and 23.9 tokens/s respectively, peak RSS stays at or under 17.6 GiB, and there are 0 memory
+    kills.
+- **Worst case.** The C2 profile at a 32K context on b11214 (19.4 GiB peak RSS) and ai-memory's two Ollama
+  models (12.5 GiB) were resident and driven at once, next to the RAG embedder and Qdrant. System free
+  memory was 53%, swap did not grow, there were 0 kills, and every request succeeded.
+- **The one swap event.** Swap grew only once, peaking at 13.9 GB for about 40 s with no kills. That was
+  when an OmniRoute Next.js production build and ai-memory's model load overlapped a li26 run.
+- **Workstation embedder.** Nemotron-3-Embed-1B has no supported Mac runtime. Its `Ministral3Model` is not
+  registered in llama.cpp's converter at b11146, and NVIDIA publishes no GGUF or MLX artifact.
+
+## Decision
+
+1. **This Mac hosts, natively and together:**
+   - the pinned RAG embedder (embeddinggemma-300M on llama.cpp Metal, port 8232);
+   - the memory stack's own models (Ollama qwen3-embedding:4b and qwen3.5-9b-64k);
+   - Qdrant;
+   - one 27B-class 4-bit generation model on llama.cpp Metal.
+   The measured budget for this co-resident set is a generation server of up to 19.4 GiB RSS at a 32K context, plus 12.5 GiB of memory-stack models, the embedder and Qdrant, with 53% of system memory still free, no swap growth and no memorystatus kills.
+2. **`adoption/host-roles.json`:** `mac-coordinator` also owns `model-hosting`, `memory-e2e` and `rag-e2e`
+   for its own foundation layer. GPU runtime qualification, cross-host model qualification and independent
+   review stay with the workstation. Routing is unchanged, because `scripts/host_requests.py` routes on the
+   request label alone. `owns` records what the host accepts.
+3. **What exceeds this Mac** (the trigger for a larger one): a 27B-class LLM as ai-memory's reranker at 8K tokens, which runs 27-28 s on llama.cpp Metal and 18.5 s median on Ollama's MLX engine against the 20 s gate. Also a heavy build overlapping the full model set, which pushed the host into swap (peak 13.9 GB, no kills). A model with no supported Mac runtime, such as Nemotron-3-Embed today, is a runtime gap, not a memory limit. The 30B-class challengers would fit the 38 GB generation allowance alone, which is projected, not measured
+
+## Alternatives considered
+
+- **Borrow the workstation's models over the network.** Rejected by the user's decision, and by the
+  memory-stack rule "never depend on another host's localhost service".
+- **Serve the workstation's embedder (Nemotron-3-Embed-1B) on the Mac.** It has no supported Mac runtime
+  (step 2). Community conversions are not upstream and are untested for this model's pooling.
+- **Ollama's MLX engine for generation.** This Mac's Ollama 0.34.4 already holds nvfp4 builds of
+  Qwen3.8-27B. The frozen li26 harness cannot score them, because `eval_arm.py` checks llama.cpp's
+  `/props`. Only its latency was measured, for the reranker gate: at about 7.5K prompt tokens it answered in 18.5 s median against llama.cpp Metal's 27.9 s, 1.5× faster at prefill but still borderline against the 20 s gate (1 of 3 calls over)
+- **MLX-LM or LM Studio.** Considered in the macOS page's embedding decision and not activated. There is
+  still no measured comparison on this project's workload.
+- **Re-pin llama.cpp to b11146 on macOS for MTP drafting.** Recorded as a finding with this host's b11146
+  runs as native evidence, but not done here. A pin change is its own reviewed change.
+
+## Overturn conditions
+
+This decision is revisited when any one of these is measured on this Mac:
+
+- swap growth or a memorystatus kill under normal coordinator load with the co-resident set above;
+- a required model set whose weights and KV cache exceed the measured headroom;
+- the chosen memory reranker LLM missing ai-memory's 20 s gate at 4-8K tokens (more than 5% fallback)
+  while a larger machine would meet it;
+- a Mac-native Nemotron-3-Embed (llama.cpp registration or an NVIDIA GGUF/MLX) that beats embeddinggemma
+  on this project's own retrieval comparison, with the headroom to serve it.

docs/harness-defaults.md

@@ -90,7 +90,7 @@ When a claim or action proves wrong, correct it where it was relayed and record
 | 2026-09-25 | Reading a filtered command's output as the raw output | Through rtk 0.50.0's Claude hook, a large `git show <rev>:<path>` blob is windowed to about 8 KiB, so a following `tail` reads the window rather than the file end, and `diff` on a missing file exits 1 instead of 2 | Exclude those forms with rtk's four-entry `[hooks] exclude_commands` value and confirm with `rtk hook check`; read an exact blob with native `git show` or `rtk proxy git show` | [Claude hook exclusions at 0.50.0](../recipes/README.md#native-context-mode-and-hooks); the `adoption/bootstrap-linux.sh` reminder; `evidence/receipts/rtk-050-qualification-20260925.json` |
 | 2026-09-25 | Letting a harness feature rewrite shared repository configuration | Launching an agent with `isolation: "worktree"`, or an agent type that declares it such as the shipped `isolated-builder`, rewrote the shared `core.hooksPath` from the relative `scripts/git-hooks` to the main checkout's absolute path; whenever the main checkout's branch lacked `scripts/git-hooks`, every worktree committed without the gitleaks pre-commit scan | Create the worktree yourself (`git worktree add --no-track <path> -b <branch> origin/main`) and pass its path; run `git config --show-origin --get core.hooksPath` before relying on hooks | Coordinator briefs; the `secret-scan` job in `.github/workflows/validate.yml` scans every pull request. Appended 2026-09-27: no shipped agent declares `isolation` any more, and `examples/claude-native/workflows/test-envelope.mjs` fails one that does ([record](decisions/2026-09-27-claude-harness-settings.md)) |
 | 2026-09-25 | A wait loop whose pattern matches itself | `until ! pgrep -f "unittest discover"; do sleep 3; done` never ended: pgrep never reports its own process (procps-ng `pgrep(1)`), but the waiting shell's command line contains the pattern; one waiter ran for almost ten hours | Bracket one character (`"[u]nittest discover"`), anchor the pattern to the command name, or wait on a PID or exit-marker file, with a timeout | This log; an anchored pattern in `blueprints/gap-wave2-20260923/foundation__quality-evaluation/run_codex_author.sh` |
-| 2026-09-25 | Recording evidence against an unpublished commit | Host receipts recorded in a worktree on a local-only commit named it as `catalog_revision`; validation passed locally because worktrees share one object store, while a clone of the branch alone fails | Record receipts only at a commit on GitHub (`git merge-base --is-ancestor <rev> origin/main`) and validate in `git clone --no-local --single-branch --branch <branch>` | Partly, in CI: `.github/workflows/validate.yml` runs `scripts/host_receipts.py validate` on a full-history checkout from GitHub, so a never-pushed commit fails there ("is not a commit present in this checkout"). A local worktree accepts a local-only commit, because `git cat-file -e` finds it in the shared object store, and whether the commit is on `main` stays a manual check |
+| 2026-09-25 | Recording evidence against an unpublished commit | Host receipts recorded in a worktree on a local-only commit named it as `catalog_revision`; validation passed locally because worktrees share one object store, while a clone of the branch alone fails | Record receipts only at a commit on GitHub (`git merge-base --is-ancestor <rev> origin/main`) and validate in `git clone --no-local --single-branch --branch <branch>` | Partly, in CI: `.github/workflows/validate.yml` runs `scripts/host_receipts.py validate` on a full-history checkout from GitHub, so a never-pushed commit fails there ("is not a commit present in this checkout"). A local worktree accepts a local-only commit, because `git cat-file -e` finds it in the shared object store, and whether the commit is on `main` stays a manual check. Appended 2026-09-27: a commit that is published only on a PR branch fails the same way after the squash merge, because the repository deletes merged head branches (`delete_branch_on_merge: true`) and CI fetches only heads and tags. #391's receipts turned `main` red this way. Before merging, tag every such revision with an annotated `receipt-revision/<sha8>` tag, and check it in a full `git clone --no-local --single-branch --branch main` plus tags; a `--filter=blob:none` clone hides the failure |
 | 2026-09-25 | Inline code inside a guarded shell command | The PreToolUse secret guard tokenizes inline Python and heredocs, and a segment starting with `set(` after a pipe was blocked as `environment_dump` | Put non-trivial Python in a script file and run that file; never bypass or edit the guard | `scripts/hooks/secret_path_guard.py` (`is_environment_dump`); `evidence/artifacts/token-e2e-ultracode-20260925/README.md` |
 | 2026-09-25 | Taking a pipeline's exit code as the command's result | A unittest run piped into `tail` returned tail's exit 0 and dropped the summary printed on stderr, so a "suite passed" log rested on tail; a `git push` piped into `tail` hid a refused push | Write stdout and stderr to files and read the command's own exit code and summary; read a push back with `git ls-remote origin refs/heads/<branch>` | This log |
 | 2026-09-26 | Trusting a flag's name instead of the request it sends | Against a local stand-in provider (cases W1 to W3 on Codex 0.155.1 and 0.157.1, local integration evidence), `codex --search exec` sent `external_web_access: false` (Codex's cached web index), the same as no flag, and only `-c web_search="live"` sent `true`; OpenAI parity rests on source reading, and a live `-c web_search="live"` search seen on OpenAI was not retained. That the landscape sweep's GPT-6 lanes, started with `codex --search exec`, searched a cached index is therefore an inference | Pass `-c web_search="live"` when a lane needs current upstream facts, and verify a flag by the request or configuration the tool actually sends | `evidence/artifacts/landscape-sweep-20260926/README.md` (cases W1 to W3); `docs/decisions/2026-09-26-workstation-tool-refresh.md`; `tools/sota-convergence/landscape-sweep/codex_job.py` still passes `--search` at this base, with its fix pending |