feat(provider): vMLX local inference for Apple Silicon airgap mode

[maurice-jobst]

Summary

Adds a first-class vMLX model-provider plugin (v0.1.0) for fully airgapped
Hermes operation on Apple Silicon, plus an official setup guide.

The plugin is self-contained under plugins/model-providers/vmlx/ and built
against the v0.13.0 ProviderProfile contract introduced in
#20324 and
documented in
#20749. It is gated
to macOS via an ImportError guard in __init__.py, so it has zero impact
on Linux, Windows, or non-Apple-Silicon contributors — the discovery loop in
providers/__init__.py skips the directory on non-Darwin.

Motivation

CONTRIBUTING.md ranks cross-platform compatibility as the #2 contribution
priority. macOS is currently the weakest production target for Hermes because
the bundled custom/Ollama profile is the only local-server precedent in
plugins/model-providers/, and it requires the user to know they should set
base_url=http://localhost:…/v1 and discover their own auxiliary-routing
pattern. On Apple Silicon, MLX is the native, Metal-accelerated path — there
is no first-class provider for it.

This PR closes that gap with the airgap profile real users on regulated or
offline workloads need today: $0 token cost, zero outbound network during
inference, and a clean primary/janitor split so auxiliary work
(compression, summarization, memory writes, skill curation) does not steal
context window from the main agent loop.

Related issues / community context:

• #467 — existing
  community fork with local MLX inference; this brings the pattern upstream
  cleanly.
• #523 — request
  for a local model setup skill / guide; the new airgap doc partially
  addresses this.
• #7800 —
  community tip on faster local inference on Mac via MLX.
• Sister contributions in flight:
  #3498
  (mlx_whisper STT) and
  #5350
  (parakeet-mlx STT) — same Apple Silicon / MLX direction, complementary scope.

Compatibility

• Hermes Agent: requires v0.13.0 (the ProviderProfile plugin contract).
• Platform: Apple Silicon only (M1 / M2 / M3 / M4 / M5).
• Plugin semver: pre-1.0 (v0.1.0). The public surface (provider names,
  default base URLs, aliases) is open to change in response to maintainer
  feedback during review; once merged, the plugin will graduate to 1.0 and
  semver applies. See
  plugins/model-providers/vmlx/CHANGELOG.md.

Changes

• plugins/model-providers/vmlx/plugin.yaml — manifest, kind: model-provider,
  v0.1.0. Following the canonical schema documented in docs: pluggable surfaces coverage — model-provider guide, full plugin map, opt-in fix #20749.

• plugins/model-providers/vmlx/__init__.py — module-level
  register_provider() calls for two ProviderProfile instances:

  • vmlx — primary, http://localhost:8000/v1, aliases
    mlx, mlx-server, apple-mlx, vmlx-primary.
  • vmlx-janitor — auxiliary, http://localhost:8001/v1, aliases
    vmlx-aux, mlx-janitor.

  Both profiles use env_vars=() and fallback_models=() — no API key,
  no cloud fallback. Apple Silicon platform gate via ImportError on
  non-Darwin so the plugin is invisible elsewhere.

• plugins/model-providers/vmlx/README.md — provider-level docs covering
  hardware sizing, model acquisition (mlx-community), serving primary on
  :8000 and janitor on :8001, the matching Hermes config, and a 5-row
  troubleshooting table.

• plugins/model-providers/vmlx/CHANGELOG.md — release history with the
  pre-1.0 stability note and references to feat(providers): ProviderProfile ABC + plugins/model-providers/ (salvage of #14424 + pluggable migration) #20324 / docs: pluggable surfaces coverage — model-provider guide, full plugin map, opt-in fix #20749.

• website/docs/guides/macos-airgap.md — new Docusaurus guide under
  guides/ with full setup walkthrough, copy-pasteable config.yaml,
  two launchd plists for auto-start (with absolute-path discipline so
  the plists survive on Apple Silicon Homebrew, pip-user, and venv
  installs), verification commands, and a 7-row troubleshooting table.

• tests/test_vmlx_provider.py — pytest suite covering both profile
  registrations, the default base URLs, and the non-Darwin import guard.
  Uses unittest.mock only (no new test deps) and skips properly on
  non-Darwin runners.

No core files modified. agent/, hermes_cli/, providers/,
gateway/, run_agent.py, and cli.py are untouched — the v0.13.0
discovery loop in providers/__init__.py auto-picks up plugins under
plugins/model-providers/<name>/, so no resolver branch is needed.

Out of scope

• Linux MLX support (MLX is Apple Silicon only; revisit if/when upstream MLX
  adds CUDA/ROCm).
• Windows local inference (route through vLLM or llama.cpp instead).
• Non-MLX Apple Silicon backends (llama.cpp Metal, MPS-only PyTorch).
• Subclassing ProviderProfile for vMLX-specific quirks (extra_body fields,
  non-standard auth, etc.) — none observed in current vmlx serve; if any
  emerge in review, they ship in a follow-up.
• Custom provider_health_check hook — the v0.13.0 doctor automatically
  probes {base_url}/models for any auth_type="api_key" profile, so a
  plugin-side hook would be redundant.

Testing

Automated:

uv pip install -e ".[all,dev]"
pytest tests/test_vmlx_provider.py -v

Manual end-to-end on M5 Max, 48 GB unified memory:

• pip install vmlx
• vmlx serve --model ~/models/primary --port 8000 --ctx-size 65536
• vmlx serve --model ~/models/janitor --port 8001 --ctx-size 16384
• hermes doctor reports both profiles registered and both /models
  probes [ok]
• hermes run completes a tool loop with no outbound network traffic
  (verified via sudo tcpdump -i any -nn 'host not 127.0.0.1 and host not ::1')
• Killing the janitor server: primary still responds; auxiliary routes
  log warnings via the existing doctor probe instead of crashing
• pytest tests/ -v passes on darwin
• pytest tests/test_vmlx_provider.py -v skips (does not fail) on a
  Linux CI runner

Reviewer checklist

• plugin.yaml schema (name, kind: model-provider, version,
  description, author) matches the canonical form in docs: pluggable surfaces coverage — model-provider guide, full plugin map, opt-in fix #20749
• No core files modified (only plugins/, website/docs/, tests/)
• ProviderProfile fields used: name, aliases, display_name,
  description, env_vars, base_url, fallback_models
• No subclassed ProviderProfile hooks — vMLX matches the default
  OpenAI-compatible chat-completions wire protocol
• Tests skip rather than fail on non-Darwin
• MDX frontmatter renders correctly in Docusaurus v3 preview
• All commands in docs are copy-pasteable

Suggested labels

comp/provider, platform/macos, type/feature, P2

Author

maurice-jobst — developed and validated on M5 Max / 48 GB unified
memory in production use before opening this PR.