feat(provider): vMLX local inference for Apple Silicon airgap mode

plugins/model-providers/vmlx/CHANGELOG.md

@@ -0,0 +1,36 @@
+# Changelog
+
+All notable changes to the `vmlx` provider plugin are documented here.
+Format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/);
+versioning follows [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [0.1.0] — Initial release
+
+### Added
+- Two `ProviderProfile` registrations under `plugins/model-providers/vmlx/`:
+  - `vmlx` — primary, `http://localhost:8000/v1`, aliases
+    `mlx`, `mlx-server`, `apple-mlx`, `vmlx-primary`.
+  - `vmlx-janitor` — auxiliary, `http://localhost:8001/v1`, aliases
+    `vmlx-aux`, `mlx-janitor`.
+- Both profiles use `env_vars=()` and `fallback_models=()` — no API key,
+  no cloud fallback (airgap by construction).
+- Apple Silicon platform gate via `ImportError` on non-Darwin so the plugin
+  is invisible to Linux/Windows contributors.
+- README with hardware sizing, dual-port serving instructions, Hermes config,
+  and 5-row troubleshooting table.
+- pytest suite covering profile registration, default base URLs, and the
+  non-Darwin import guard.
+
+### Compatibility
+- Built against the v0.13.0 plugin contract introduced in
+  [#20324](https://github.com/NousResearch/hermes-agent/pull/20324)
+  (ProviderProfile ABC + `plugins/model-providers/`).
+- Follows the canonical pattern documented in
+  [#20749](https://github.com/NousResearch/hermes-agent/pull/20749)
+  (`website/docs/developer-guide/model-provider-plugin.md`).
+- Apple Silicon (M1 / M2 / M3 / M4 / M5 family) only.
+
+### Stability
+- Pre-1.0: provider names, aliases, and default base URLs may shift in
+  response to PR-review feedback. Once the plugin ships in an upstream
+  release, semver applies and graduation to 1.0 freezes the surface.

plugins/model-providers/vmlx/README.md

@@ -0,0 +1,142 @@
+# vMLX provider — local Apple Silicon inference
+
+`vmlx` is a Hermes model-provider plugin that runs MLX-format LLMs on Apple
+Silicon and serves them through an OpenAI-compatible chat-completions endpoint.
+It is the recommended backend for fully airgapped Hermes operation on macOS.
+
+> **Status:** v0.1.0 — initial release. The public surface (provider names,
+> base URLs, aliases) may evolve in response to PR-review feedback before
+> the plugin graduates to 1.0. See [`CHANGELOG.md`](./CHANGELOG.md).
+
+## Why this exists
+
+The bundled [`custom`](../custom/) profile already covers generic local
+OpenAI-compatible servers (Ollama, vLLM, llama.cpp). `vmlx` is a sibling
+that adds:
+
+- **Sane localhost defaults** — `base_url` is pre-filled, no config needed.
+- **Apple Silicon platform gate** — `ImportError` on non-Darwin so the
+  plugin is invisible to Linux/Windows contributors.
+- **Primary/janitor split out of the box** — two profiles registered side
+  by side so auxiliary work (compression, memory writes, summarization,
+  skill curation) can run on a separate `vmlx serve` instance and stop
+  consuming the primary model's context window.
+
+## What gets registered
+
+The plugin's `__init__.py` calls `register_provider()` twice:
+
+| Profile name   | Default base URL              | Suggested role          |
+|----------------|-------------------------------|-------------------------|
+| `vmlx`         | `http://localhost:8000/v1`    | Primary agent loop      |
+| `vmlx-janitor` | `http://localhost:8001/v1`    | Auxiliary / aux model   |
+
+Aliases: `mlx`, `mlx-server`, `apple-mlx`, `vmlx-primary` resolve to `vmlx`;
+`vmlx-aux`, `mlx-janitor` resolve to `vmlx-janitor`.
+
+## Hardware requirements
+
+| Model class | Quantization | Recommended unified memory |
+|-------------|--------------|----------------------------|
+| 3–8 B       | 4-bit        | 16 GB                      |
+| 13–14 B     | 4-bit        | 24 GB                      |
+| 30–34 B     | 4-bit        | 36 GB                      |
+| 70 B        | 4-bit        | 64 GB                      |
+
+Apple Silicon (M1 / M2 / M3 / M4 / M5 family) only. Intel Macs are not
+supported by MLX itself.
+
+## Installation
+
+```bash
+pip install vmlx
+```
+
+Discovery is automatic — `providers/__init__.py._discover_providers()` picks
+up the directory the next time `get_provider_profile()` is called.
+
+## Model acquisition
+
+Suggestions; any MLX-quantized chat model in the same size class works.
+
+```bash
+mkdir -p ~/models
+huggingface-cli download mlx-community/Qwen2.5-32B-Instruct-4bit \
+    --local-dir ~/models/primary
+huggingface-cli download mlx-community/gemma-3-4b-it-4bit \
+    --local-dir ~/models/janitor
+```
+
+## Serving both models
+
+```bash
+vmlx serve --model ~/models/primary --port 8000 --ctx-size 65536
+vmlx serve --model ~/models/janitor --port 8001 --ctx-size 16384
+```
+
+For permanent setup with `launchd`, see the [macOS airgap guide](../../../website/docs/guides/macos-airgap.md).
+
+## Hermes config
+
+Minimal — picks up the primary profile's defaults:
+
+```yaml
+model:
+  provider: vmlx
+  name: primary
+  context_length: 65536
+  temperature: 0.2
+
+fallback_providers: []
+```
+
+`fallback_providers: []` is the explicit airgap declaration: Hermes will
+never reach for a cloud provider on inference failure.
+
+For the primary/janitor split, point auxiliary tasks at the `vmlx-janitor`
+profile via your Hermes version's auxiliary-routing config (see the
+provider-runtime developer guide for resolution precedence). A common
+pattern:
+
+```yaml
+auxiliary_routes:
+  compression:    { provider: vmlx-janitor, name: janitor }
+  memory_write:   { provider: vmlx-janitor, name: janitor }
+  skill_curation: { provider: vmlx-janitor, name: janitor }
+  context_summary:{ provider: vmlx-janitor, name: janitor }
+```
+
+If your Hermes release does not support per-task `provider` overrides yet,
+serve both models on `:8000` and use `default_aux_model` instead.
+
+## Verification
+
+```bash
+hermes doctor
+```
+
+Expected (relevant lines — the `/models` probe is automatic for any
+`api_key`/empty-`env_vars` profile):
+
+```
+[ok]   provider 'vmlx' registered
+[ok]   provider 'vmlx-janitor' registered
+[ok]   http://localhost:8000/v1/models reachable
+[ok]   http://localhost:8001/v1/models reachable
+```
+
+## Troubleshooting
+
+| Symptom | Likely cause | Fix |
+|---------|--------------|-----|
+| `vmlx` not in `hermes doctor` output | Plugin import failed (likely non-Darwin) | This plugin is Apple Silicon only; check `python -c "import platform; print(platform.system())"` |
+| `connection refused` on /models probe | `vmlx serve` not listening on the expected port | `lsof -nP -iTCP:8000 -sTCP:LISTEN`; restart the server |
+| `context length exceeded` mid-loop | `--ctx-size` lower than `model.context_length` | Restart server with matching `--ctx-size` |
+| Janitor crashes on first request | OOM (model + primary both loaded) | Smaller janitor model or 4-bit quantization |
+| Override the bundled defaults | Plugin lives at `<repo>/plugins/model-providers/vmlx/` (bundled) | Drop a same-named directory under `$HERMES_HOME/plugins/model-providers/vmlx/` — user plugins win because `register_provider()` is last-writer-wins |
+
+## Contributing
+
+Issues and PRs welcome — see [CONTRIBUTING.md](../../../CONTRIBUTING.md). The
+plugin's macOS gate lives in `__init__.py` (`ImportError` on non-Darwin), so
+non-Mac contributors can hack on Hermes without it loading.

plugins/model-providers/vmlx/__init__.py

@@ -0,0 +1,55 @@
+"""vMLX provider profiles — Apple Silicon local inference.
+
+Registers two profiles against the v0.13.0 ``providers/`` plugin contract:
+
+  * ``vmlx``         — primary, http://localhost:8000/v1, large context.
+  * ``vmlx-janitor`` — auxiliary (compression / summarization / memory writes /
+                       skill curation), http://localhost:8001/v1, short context.
+
+Both speak the OpenAI chat-completions wire protocol against a local
+``vmlx serve`` instance. Apple Silicon only; the ``platform.system()`` guard
+below makes the import fail cleanly on Linux/Windows so the discovery loop
+in ``providers/__init__.py`` skips the directory.
+
+Sibling of the bundled ``custom``/Ollama profile, but with sane localhost
+defaults baked in for vMLX so airgap setups need zero configuration.
+"""
+from __future__ import annotations
+
+import platform
+
+from providers import register_provider
+from providers.base import ProviderProfile
+
+if platform.system() != "Darwin":
+    raise ImportError("vmlx provider requires macOS (Apple Silicon)")
+
+vmlx_primary = ProviderProfile(
+    name="vmlx",
+    aliases=("mlx", "mlx-server", "apple-mlx", "vmlx-primary"),
+    display_name="vMLX (Apple Silicon local inference)",
+    description=(
+        "MLX-format LLMs served locally via `vmlx serve` on Apple Silicon. "
+        "Airgap-friendly: no API key, no cloud fallback."
+    ),
+    env_vars=(),
+    base_url="http://localhost:8000/v1",
+    fallback_models=(),
+)
+
+vmlx_janitor = ProviderProfile(
+    name="vmlx-janitor",
+    aliases=("vmlx-aux", "mlx-janitor"),
+    display_name="vMLX janitor (Apple Silicon, auxiliary tasks)",
+    description=(
+        "Smaller MLX model on a separate `vmlx serve` instance — drives "
+        "compression, summarization, memory writes, and skill curation so the "
+        "primary model's context window stays free for the agent loop."
+    ),
+    env_vars=(),
+    base_url="http://localhost:8001/v1",
+    fallback_models=(),
+)
+
+register_provider(vmlx_primary)
+register_provider(vmlx_janitor)

plugins/model-providers/vmlx/plugin.yaml

@@ -0,0 +1,5 @@
+name: vmlx
+kind: model-provider
+version: 0.1.0
+description: vMLX (Apple Silicon local inference) — OpenAI-compatible MLX server with primary/janitor split
+author: "maurice-jobst"

tests/test_vmlx_provider.py

@@ -0,0 +1,91 @@
+"""Tests for the vMLX model-provider plugin.
+
+Validates the v0.13.0 ``providers/`` contract: two ``ProviderProfile``
+instances are registered at module import (vmlx + vmlx-janitor), and the
+plugin's ``platform.system()`` guard fails the import cleanly on non-Darwin.
+
+The plugin lives at ``plugins/model-providers/vmlx/`` (hyphen — not a valid
+Python identifier) so we load it via ``importlib.util.spec_from_file_location``
+the same way ``providers/__init__.py._discover_providers()`` does.
+"""
+from __future__ import annotations
+
+import importlib.util
+import sys
+from pathlib import Path
+from typing import Any
+from unittest import mock
+
+import pytest
+
+PLUGIN_INIT = (
+    Path(__file__).resolve().parents[1]
+    / "plugins"
+    / "model-providers"
+    / "vmlx"
+    / "__init__.py"
+)
+
+
+def _load_plugin(register_provider_mock: Any) -> Any:
+    sys.modules.pop("vmlx_plugin_under_test", None)
+    spec = importlib.util.spec_from_file_location(
+        "vmlx_plugin_under_test", PLUGIN_INIT
+    )
+    assert spec is not None and spec.loader is not None, (
+        f"plugin not found at {PLUGIN_INIT}"
+    )
+    module = importlib.util.module_from_spec(spec)
+    with mock.patch("providers.register_provider", register_provider_mock):
+        spec.loader.exec_module(module)
+    return module
+
+
+@pytest.mark.skipif(sys.platform != "darwin", reason="vMLX is macOS-only")
+def test_plugin_registers_two_profiles() -> None:
+    register_provider = mock.MagicMock()
+    _load_plugin(register_provider)
+
+    assert register_provider.call_count == 2
+    profiles = [c.args[0] for c in register_provider.call_args_list]
+    names = {p.name for p in profiles}
+    assert names == {"vmlx", "vmlx-janitor"}
+
+
+@pytest.mark.skipif(sys.platform != "darwin", reason="vMLX is macOS-only")
+def test_primary_profile_has_correct_defaults() -> None:
+    register_provider = mock.MagicMock()
+    _load_plugin(register_provider)
+
+    primary = next(
+        c.args[0]
+        for c in register_provider.call_args_list
+        if c.args[0].name == "vmlx"
+    )
+    assert primary.base_url == "http://localhost:8000/v1"
+    assert primary.env_vars == ()
+    assert primary.fallback_models == ()
+    assert "mlx" in primary.aliases
+
+
+@pytest.mark.skipif(sys.platform != "darwin", reason="vMLX is macOS-only")
+def test_janitor_profile_on_separate_port() -> None:
+    register_provider = mock.MagicMock()
+    _load_plugin(register_provider)
+
+    janitor = next(
+        c.args[0]
+        for c in register_provider.call_args_list
+        if c.args[0].name == "vmlx-janitor"
+    )
+    assert janitor.base_url == "http://localhost:8001/v1"
+    assert janitor.env_vars == ()
+    assert janitor.fallback_models == ()
+
+
+def test_plugin_raises_importerror_on_non_darwin() -> None:
+    register_provider = mock.MagicMock()
+    with mock.patch("platform.system", return_value="Linux"):
+        with pytest.raises(ImportError, match="macOS"):
+            _load_plugin(register_provider)
+    register_provider.assert_not_called()