Add Apple provider host supervision

[i386]

Stack

• Builds on Add executable provider runtime artifact contract #1250, which adds the carrier-neutral executable-provider artifact and resolver.
• Tracks Proposal: add a macOS Core AI whole-model inference backend #1246.
• This is the experimental Phase 2B host-supervisor layer. It does not advertise apple/system to mesh peers or ship the provider in release products.

Why

Eligible Apple silicon Macs already have a system model delivered and optimized by Apple. Supervising that model as a whole-request provider gives MeshLLM a private, zero-checkpoint-download inference path backed by Apple's native accelerator stack, without linking Foundation Models into the backend-neutral Rust host or introducing Skippy pipeline parallelism.

System-model identity

apple/system is the rolling alias for the system model currently installed by Apple. The provider also reports Apple's documented OS-aligned generation and exposes the matching resolved route:

• model_version=27.0
• version_source=apple_os_release_band
• versioned_model_id=apple/system@27.0

This is not presented as an immutable checkpoint: Foundation Models exposes no public model build identifier, and MeshLLM cannot install, pin, or roll back the system model. Unknown future OS generations expose no Apple system model until Apple documents their mapping.

There is intentionally no backward-compatibility lane for unversioned Apple providers. All three fields are mandatory and must agree. The host rejects missing, unsupported, or contradictory metadata. Both apple/system and apple/system@27.0 resolve to the same whole-model process, while the exact ID only exists for the installed generation.

What changed

• Adds a Rust supervisor for the packaged kind=apple, model=apple/system, protocol 0.1 provider.
• Starts it only from host-capable mesh-llm serve processes; mesh-llm client does not launch a local provider.
• Resolves the artifact from a product bundle, immutable cache, or explicitly enabled release index.
• Revalidates manifest hashes and macOS signing policy before execution. Ad-hoc signatures are rejected unless the local QA-only override is explicit.
• Launches the Swift sidecar on an ephemeral loopback port with the real MeshLLM parent PID and common credential variables removed from the child environment.
• Requires structured readiness plus passing /health and /v1/models probes before registering apple/system as an ordinary local inference target.
• Exposes the supervised PID, port, health, backend, and context length through /api/runtime/processes.
• Withdraws the exact provider route on unavailability or failure, restarts unexpected exits with bounded backoff, and terminates the child during host shutdown.
• Reuses the existing MeshLLM OpenAI proxy for completion, SSE, tools, usage, errors, and disconnect cancellation; there is no Apple-specific proxy path.
• Adds just apple::mesh as the Golden Gate end-to-end lifecycle test and updates the design/roadmap docs.

Experimental requirements

You must have macOS Golden Gate (macOS 27).

You also need:

1. Apple silicon.
2. Full Xcode 27 selected with xcode-select (Command Line Tools alone are insufficient).
3. Apple Intelligence enabled.
4. The system model downloaded and reported as available.

Confirm the toolchain:

xcode-select -p
xcodebuild -version
xcrun --sdk macosx --show-sdk-version

The developer path should end in Xcode.app/Contents/Developer (or the beta equivalent), and the SDK should be 27.x.

Try it

From the repository root:

1. Run the complete supervised path:

   just apple::mesh

   This packages the sidecar, builds the normal dynamic host, starts an isolated mesh-llm serve, waits for apple/system on its ordinary /v1/models, checks /api/runtime/processes, forces a provider restart, runs completion/SSE/tool/cancellation probes through MeshLLM, then proves the child exits with the host.

2. For a manual session, package and build:

   just apple::package
   just build

3. Start MeshLLM with the local ad-hoc QA artifact:

   MESH_LLM_PROVIDER_RUNTIME_BUNDLE_DIR="$PWD/target/apple-runtime/package/meshllm-apple-runtime-darwin-arm64" \
   MESH_LLM_PROVIDER_RUNTIME_CACHE_DIR="$PWD/target/apple-runtime/manual-cache" \
   MESH_LLM_APPLE_PROVIDER_ALLOW_AD_HOC=1 \
     ./target/debug/mesh-llm --log-format json serve \
       --port 9337 --console 3131 --headless

   MESH_LLM_APPLE_PROVIDER_ALLOW_AD_HOC=1 is for local development only. Product artifacts must have a trusted signature.

4. Confirm discovery and process ownership:

   curl -s http://127.0.0.1:9337/v1/models | jq
   curl -s http://127.0.0.1:3131/api/runtime/processes | jq

5. Send a completion through MeshLLM's REST API:

   curl -s http://127.0.0.1:9337/v1/chat/completions \
     -H 'content-type: application/json' \
     -d '{
       "model": "apple/system",
       "messages": [{"role":"user","content":"Reply with exactly: apple runtime REST ready"}],
       "temperature": 0,
       "max_tokens": 32
     }' | jq

6. Exercise the deterministic tool path:

   curl -s http://127.0.0.1:9337/v1/chat/completions \
     -H 'content-type: application/json' \
     -d '{
       "model": "apple/system",
       "messages": [{"role":"user","content":"Use the tool with key: rest-demo"}],
       "tools": [{
         "type": "function",
         "function": {
           "name": "mesh_fixture_lookup",
           "description": "Look up a fixture",
           "parameters": {
             "type": "object",
             "properties": {"key": {"type": "string"}},
             "required": ["key"]
           }
         }
       }]
     }' | jq

Captured Golden Gate output over MeshLLM REST

{
  "status": "pass",
  "model": "apple/system",
  "versioned_model": "apple/system@27.0",
  "completion_content": "apple runtime REST ready",
  "tool_executions": [{
    "name": "mesh_fixture_lookup",
    "arguments": {"key": "rest-demo"},
    "result": "mesh-fixture-value-for-rest-demo"
  }],
  "stream_done": true,
  "client_disconnect_cancelled": true,
  "provider_reported_in_management_api": true,
  "provider_restarted_after_crash": true,
  "provider_exited_with_meshllm": true
}

The live management record reported name=apple/system, instance_id=provider:apple/system, backend=apple, status=ready, and context_length=4096.

Validation

• just apple::mesh
• just apple::live
• just apple::carriers
• just with-lld cargo test -p mesh-llm-provider-runtime — 11 passed
• just with-lld cargo test -p mesh-llm-host-runtime provider_supervisor --lib — 7 passed
• just with-lld cargo clippy -p mesh-llm-host-runtime --lib -- -A unfulfilled-lint-expectations -D warnings
• just with-lld cargo fmt --all -- --check
• bash -n providers/apple/QA/mesh.sh providers/apple/QA/rest.sh
• shellcheck providers/apple/QA/mesh.sh providers/apple/QA/rest.sh
• git diff --check

The unmodified host crate currently has existing unfulfilled lint-expectation warnings, so the strict Clippy run explicitly allows only that lint while denying every other warning.

Next stack

Phase 2C will put this artifact/lifecycle behind the Rust, Swift, Node/Electron, and Kotlin/JVM SDK distribution surfaces. Phase 3 will add private-mesh advertisement, load-aware routing, failover, affinity, withdrawal, and mixed-version validation.

[github-actions]

This pull request is currently a draft. Reviews will not take place until the PR is marked as ready for review.

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: cb047eea-2f73-4285-b39f-682b98682060

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[ndizazzo]

Needs revision. The provider process gets too much of the host environment, and the stacked branch is currently unmergeable.

Follow-ups:

• Rebase onto the rewritten base and resolve the 16 conflicting paths before asking CI for another result.
• Add CI that builds providers/apple and runs the Swift tests. The current Apple ownership slices don't cover either one.
• The exact head has no meaningful CI run; the older failed run is stale.

[ndizazzo]

Removing a short list of known names still gives the downloaded provider executable the rest of the host environment. Credentials such as AWS_ACCESS_KEY_ID, AWS_SESSION_TOKEN, Google/Azure keys, and organization-specific secrets remain visible. Please clear the child environment and add back a small allowlist of required non-secret variables, with tests for credential scrubbing.

[i386]

Collapsed into #1444, which rebases the whole Apple Core AI stack onto current main and folds in Nick's review (credential-scrubbing fix + Apple provider CI). Closing in favor of that single PR.