feat(moa): expose MoA presets as selectable virtual models

[teknium1]

Summary

MoA is now a virtual provider: each named preset appears as a selectable model under provider moa, and the preset's aggregator is the acting model that answers and calls tools.

What changed

• Removed the old mixture_of_agents model tool and moa toolset from registry/setup/tool config/ACP/distributions/docs.
• Added moa as a virtual provider in the provider/model inventory.
• Exposed configured MoA preset names as models under the Mixture of Agents provider in model picker payloads.
• Runtime provider=moa, model=<preset> now builds a MoA client facade instead of a network OpenAI client.
• The MoA facade runs references first, then calls the configured aggregator with the normal Hermes tool schema; the aggregator response is returned to the agent loop as the actual model response.
• /moa is now shortcut behavior over model selection:
  • /moa switches to the default MoA preset.
  • /moa <preset> switches to that preset when it exactly matches a configured preset.
  • /moa <prompt> temporarily switches to the default MoA preset for one turn, sends the prompt, then restores the previous model.
• Dashboard and Desktop settings manage named presets; those presets now appear in normal model picker surfaces.
• Docs updated for virtual-provider / aggregator-as-actor semantics.

How MoA works now

1. The selected model is provider=moa, model=<preset>.
2. Hermes resolves <preset> from moa.presets.
3. Reference models are called without tool schemas.
4. Their outputs are appended as private context for the aggregator.
5. The configured aggregator receives the normal Hermes messages and full tool schema and acts as the model (references get only the conversation's user/assistant text — no system prompt or tool transcript).
6. The aggregator writes the assistant response or emits tool calls.
7. If tools run, their results enter history and the next model iteration repeats the MoA process.

Verified behavior

Claim	Evidence
MoA presets show as model choices	build_models_payload() injects a virtual moa provider row with preset names as models.
Aggregator is the acting model	Unit test asserts provider=moa, model=review returns the aggregator response as the final assistant response.
Aggregator receives tools	Unit test uses the file toolset and asserts the moa_aggregator call receives tools.
Mixed providers still work	MoA preset supports openai-codex:gpt-5.5 reference plus OpenRouter-hosted aggregator/reference models.
Legacy tool footprint removed	mixture_of_agents tool and moa toolset are gone.
References get a trimmed view	_reference_messages() strips the system prompt + tool/tool_calls messages; reference calls receive user/assistant text only.
enabled flag is live	A preset with enabled: false skips the reference fan-out — the aggregator acts alone (unit test).

Complete live task proof from earlier in this PR

A live MoA task completed with this mixed-provider setup:

• Reference: openai-codex:gpt-5.5
• Reference: openrouter:deepseek/deepseek-v4-pro
• Aggregator: openrouter:anthropic/claude-opus-4.8

Task result:

• completed=true
• failed=false
• api_calls=5
• moa_iterations=5
• assistant_tool_calls=16
• verifier output: VERIFIED_COMPLETE_TASK

Validation

Check	Result
Final targeted pytest	804 passed
Python compile	clean for touched Python entrypoints
Dashboard build	cd web && npm run build passed
Desktop focused lint/typecheck	eslint passed; npm --workspace apps/desktop run typecheck passed
Diff whitespace	git diff --check clean

HermesBench results

A two-model MoA preset — claude-opus-4.8 aggregating over a gpt-5.5 reference — outscores either model run solo on HermesBench:

Model	HermesBench score
Opus aggregator (opus-4.8 + gpt-5.5 reference) — MoA	0.8202
anthropic/claude-opus-4.8	0.7607
openai/gpt-5.5	0.7412

MoA beats its strongest component (opus-4.8) by ~6 points.

Infographic

[github-actions]

🔎 Lint report: hermes/hermes-afb6a269 vs origin/main

ruff

Total: 0 on HEAD, 0 on base (➖ 0)

🆕 New issues: none

✅ Fixed issues: none

Unchanged: 0 pre-existing issues carried over.

ty (type checker)

Total: 11284 on HEAD, 11274 on base (🆕 +10)

🆕 New issues (46):

Rule	Count
invalid-argument-type	15
unsupported-operator	15
unresolved-attribute	14
not-subscriptable	1
invalid-assignment	1

First entries

tests/run_agent/test_in_place_compaction.py:257: [unresolved-attribute] unresolved-attribute: Attribute `get` is not defined on `str`, `list[Unknown]`, `list[str]`, `None`, `int`, `float` in union `str | dict[Unknown, Unknown] | list[Unknown] | ... omitted 35 union elements`
agent/moa_loop.py:294: [invalid-argument-type] invalid-argument-type: Argument to function `call_llm` is incorrect: Expected `str`, found `Any | None`
tests/hermes_cli/test_destructive_slash_confirm_gate.py:32: [unresolved-attribute] unresolved-attribute: Attribute `get` is not defined on `str`, `list[Unknown]`, `list[str]`, `None`, `int`, `float` in union `str | dict[Unknown, Unknown] | list[Unknown] | ... omitted 35 union elements`
tests/gateway/test_whatsapp_reply_prefix.py:119: [unsupported-operator] unsupported-operator: Operator `>=` is not supported between objects of type `str | dict[Unknown, Unknown] | list[Unknown] | ... omitted 35 union elements` and `int`
hermes_cli/mcp_startup.py:67: [invalid-argument-type] invalid-argument-type: Argument to constructor `float.__new__` is incorrect: Expected `str | Buffer | SupportsFloat | SupportsIndex`, found `str | dict[Unknown, Unknown] | list[Unknown] | ... omitted 35 union elements`
hermes_cli/config.py:5178: [unresolved-attribute] unresolved-attribute: Attribute `items` is not defined on `int`, `str`, `list[Unknown]`, `float`, `None` in union `Unknown | int | str | ... omitted 17 union elements`
tests/cron/test_suggestions.py:213: [unsupported-operator] unsupported-operator: Operator `in` is not supported between objects of type `Literal["monitor"]` and `str | dict[Unknown, Unknown] | list[Unknown] | ... omitted 35 union elements`
gateway/run.py:8569: [unresolved-attribute] unresolved-attribute: Unresolved attribute `_moa_restore_override` on type `MessageEvent`
tools/browser_tool.py:1196: [invalid-argument-type] invalid-argument-type: Argument to constructor `int.__new__` is incorrect: Expected `str | Buffer | SupportsInt | SupportsIndex | SupportsTrunc`, found `Unknown | int | str | ... omitted 17 union elements`
gateway/run.py:8578: [unresolved-attribute] unresolved-attribute: Unresolved attribute `_moa_disable_after_turn` on type `MessageEvent`
tests/agent/test_auxiliary_config_bridge.py:284: [unsupported-operator] unsupported-operator: Operator `in` is not supported between objects of type `Literal["provider"]` and `Unknown | int | str | ... omitted 17 union elements`
tests/cli/test_fast_command.py:484: [invalid-argument-type] invalid-argument-type: Argument to bound method `TestCase.assertIn` is incorrect: Expected `Iterable[Any] | Container[Any]`, found `str | dict[Unknown, Unknown] | list[Unknown] | ... omitted 35 union elements`
tests/tools/test_browser_lightpanda.py:242: [unsupported-operator] unsupported-operator: Operator `in` is not supported between objects of type `Literal["engine"]` and `str | dict[Unknown, Unknown] | list[Unknown] | ... omitted 35 union elements`
tests/tools/test_browser_console.py:341: [unsupported-operator] unsupported-operator: Operator `in` is not supported between objects of type `Literal["record_sessions"]` and `str | dict[Unknown, Unknown] | list[Unknown] | ... omitted 35 union elements`
tests/agent/test_curator.py:1105: [unsupported-operator] unsupported-operator: Operator `in` is not supported between objects of type `Literal["curator"]` and `str | dict[Unknown, Unknown] | list[Unknown] | ... omitted 35 union elements`
tests/tools/test_web_providers.py:217: [unsupported-operator] unsupported-operator: Operator `in` is not supported between objects of type `Literal["backend"]` and `str | dict[Unknown, Unknown] | list[Unknown] | ... omitted 35 union elements`
tests/hermes_cli/test_aux_config.py:47: [unsupported-operator] unsupported-operator: Operator `not in` is not supported between objects of type `Literal["session_search"]` and `str | dict[Unknown, Unknown] | list[Unknown] | ... omitted 35 union elements`
agent/moa_loop.py:297: [invalid-argument-type] invalid-argument-type: Argument to function `call_llm` is incorrect: Expected `int`, found `Any | None`
tools/process_registry.py:514: [invalid-argument-type] invalid-argument-type: Argument to constructor `float.__new__` is incorrect: Expected `str | Buffer | SupportsFloat | SupportsIndex`, found `Unknown | int | str | ... omitted 17 union elements`
tests/agent/test_curator.py:1110: [unsupported-operator] unsupported-operator: Operator `>` is not supported between objects of type `Unknown | int | str | ... omitted 6 union elements` and `Literal[0]`
tests/cli/test_moa_command.py:69: [not-subscriptable] not-subscriptable: Cannot subscript object of type `None` with no `__getitem__` method
tests/cli/test_resume_display.py:716: [unsupported-operator] unsupported-operator: Operator `in` is not supported between objects of type `Literal["resume_display"]` and `str | dict[Unknown, Unknown] | list[Unknown] | ... omitted 35 union elements`
hermes_cli/config.py:5159: [unresolved-attribute] unresolved-attribute: Attribute `items` is not defined on `str`, `list[Unknown]`, `list[str]`, `None`, `int`, `float` in union `str | dict[Unknown, Unknown] | list[Unknown] | ... omitted 35 union elements`
tests/tools/test_refresh_agent_mcp_tools.py:257: [invalid-argument-type] invalid-argument-type: Argument to constructor `float.__new__` is incorrect: Expected `str | Buffer | SupportsFloat | SupportsIndex`, found `str | dict[Unknown, Unknown] | list[Unknown] | ... omitted 35 union elements`
agent/moa_loop.py:299: [invalid-argument-type] invalid-argument-type: Argument to function `call_llm` is incorrect: Expected `dict[Unknown, Unknown]`, found `Any | None`
... and 21 more

✅ Fixed issues (37):

Rule	Count
unsupported-operator	15
invalid-argument-type	10
unresolved-attribute	9
unresolved-import	1
invalid-parameter-default	1
invalid-return-type	1

First entries

hermes_cli/config.py:5132: [unresolved-attribute] unresolved-attribute: Attribute `get` is not defined on `str`, `list[Unknown]`, `list[str]`, `None`, `int`, `float` in union `str | dict[Unknown, Unknown] | list[Unknown] | ... omitted 34 union elements`
hermes_cli/mcp_startup.py:67: [invalid-argument-type] invalid-argument-type: Argument to constructor `float.__new__` is incorrect: Expected `str | Buffer | SupportsFloat | SupportsIndex`, found `str | dict[Unknown, Unknown] | list[Unknown] | ... omitted 34 union elements`
tests/tools/test_mixture_of_agents_tool.py:6: [unresolved-import] unresolved-import: Cannot resolve imported module `pytest`
tools/mixture_of_agents_tool.py:185: [invalid-parameter-default] invalid-parameter-default: Default value of type `None` is not assignable to annotated parameter type `int`
tests/gateway/test_whatsapp_reply_prefix.py:119: [unsupported-operator] unsupported-operator: Operator `>=` is not supported between objects of type `str | dict[Unknown, Unknown] | list[Unknown] | ... omitted 34 union elements` and `int`
tools/mixture_of_agents_tool.py:111: [invalid-return-type] invalid-return-type: Function can implicitly return `None`, which is not assignable to return type `tuple[str, str, bool]`
tests/hermes_cli/test_kanban_core_functionality.py:3369: [unresolved-attribute] unresolved-attribute: Attribute `get` is not defined on `str`, `list[Unknown]`, `list[str]`, `None`, `int`, `float` in union `str | dict[Unknown, Unknown] | list[Unknown] | ... omitted 34 union elements`
tools/process_registry.py:514: [invalid-argument-type] invalid-argument-type: Argument to constructor `float.__new__` is incorrect: Expected `str | Buffer | SupportsFloat | SupportsIndex`, found `Unknown | int | str | ... omitted 16 union elements`
tests/hermes_cli/test_mcp_reload_confirm_gate.py:33: [unresolved-attribute] unresolved-attribute: Attribute `get` is not defined on `str`, `list[Unknown]`, `list[str]`, `None`, `int`, `float` in union `str | dict[Unknown, Unknown] | list[Unknown] | ... omitted 34 union elements`
tools/browser_tool.py:1196: [unresolved-attribute] unresolved-attribute: Attribute `get` is not defined on `str`, `list[Unknown]`, `list[str]`, `None`, `int`, `float` in union `str | dict[Unknown, Unknown] | list[Unknown] | ... omitted 34 union elements`
hermes_cli/config.py:5141: [unresolved-attribute] unresolved-attribute: Attribute `items` is not defined on `int`, `str`, `list[Unknown]`, `float`, `None` in union `Unknown | int | str | ... omitted 16 union elements`
tests/run_agent/test_in_place_compaction.py:257: [unresolved-attribute] unresolved-attribute: Attribute `get` is not defined on `str`, `list[Unknown]`, `list[str]`, `None`, `int`, `float` in union `str | dict[Unknown, Unknown] | list[Unknown] | ... omitted 34 union elements`
tests/tools/test_browser_console.py:341: [unsupported-operator] unsupported-operator: Operator `in` is not supported between objects of type `Literal["record_sessions"]` and `str | dict[Unknown, Unknown] | list[Unknown] | ... omitted 34 union elements`
tests/tools/test_refresh_agent_mcp_tools.py:257: [invalid-argument-type] invalid-argument-type: Argument to constructor `float.__new__` is incorrect: Expected `str | Buffer | SupportsFloat | SupportsIndex`, found `str | dict[Unknown, Unknown] | list[Unknown] | ... omitted 34 union elements`
tests/hermes_cli/test_destructive_slash_confirm_gate.py:32: [unresolved-attribute] unresolved-attribute: Attribute `get` is not defined on `str`, `list[Unknown]`, `list[str]`, `None`, `int`, `float` in union `str | dict[Unknown, Unknown] | list[Unknown] | ... omitted 34 union elements`
tests/agent/test_auxiliary_config_bridge.py:284: [unsupported-operator] unsupported-operator: Operator `in` is not supported between objects of type `Literal["provider"]` and `Unknown | int | str | ... omitted 16 union elements`
tests/cli/test_reasoning_command.py:552: [invalid-argument-type] invalid-argument-type: Argument to bound method `TestCase.assertIn` is incorrect: Expected `Iterable[Any] | Container[Any]`, found `str | dict[Unknown, Unknown] | list[Unknown] | ... omitted 34 union elements`
gateway/run.py:16327: [invalid-argument-type] invalid-argument-type: Argument to bound method `AIAgent.run_conversation` is incorrect: Expected `str | None`, found `Any | str | int | float | list[dict[str, Any]]`
gateway/run.py:16327: [invalid-argument-type] invalid-argument-type: Argument to bound method `AIAgent.run_conversation` is incorrect: Expected `int | float | None`, found `Any | str | int | float | list[dict[str, Any]]`
tests/cli/test_resume_display.py:716: [unsupported-operator] unsupported-operator: Operator `in` is not supported between objects of type `Literal["resume_display"]` and `str | dict[Unknown, Unknown] | list[Unknown] | ... omitted 34 union elements`
tests/tools/test_web_providers.py:218: [unsupported-operator] unsupported-operator: Operator `in` is not supported between objects of type `Literal["search_backend"]` and `str | dict[Unknown, Unknown] | list[Unknown] | ... omitted 34 union elements`
tests/tools/test_web_providers.py:217: [unsupported-operator] unsupported-operator: Operator `in` is not supported between objects of type `Literal["backend"]` and `str | dict[Unknown, Unknown] | list[Unknown] | ... omitted 34 union elements`
gateway/run.py:16327: [invalid-argument-type] invalid-argument-type: Argument to bound method `AIAgent.run_conversation` is incorrect: Expected `str`, found `Any | str | int | float | list[dict[str, Any]]`
gateway/restart.py:16: [invalid-argument-type] invalid-argument-type: Argument to constructor `float.__new__` is incorrect: Expected `str | Buffer | SupportsFloat | SupportsIndex`, found `Unknown | int | str | ... omitted 16 union elements`
tests/hermes_cli/test_aux_config.py:47: [unsupported-operator] unsupported-operator: Operator `not in` is not supported between objects of type `Literal["session_search"]` and `str | dict[Unknown, Unknown] | list[Unknown] | ... omitted 34 union elements`
... and 12 more

Unchanged: 5910 pre-existing issues carried over.

Diagnostics are surfaced as warnings — this check never fails the build.

[PodJamz]

Can we call it Mable? Mixture of Fables, just to take the piss out of Anthropic?

[PodJamz]

Really like where this lands MoA — slash-command mode in the normal loop is the right call. I built a similar reference-aggregator loop in my own agent (8gent Code), where the constraint was free-and-local-by-default, so the harness was the only lever I had. The one thing that moved the needle most there was making the ensemble learn who to trust instead of asking everyone every turn. Offering that back here as a tentative, opt-in addition on top of this PR — entirely your call whether it's useful.

Branch: PodJamz:feat/verdict-routed-moa-clean (cherry-picked onto this PR's head 4c367df)
Diff (6 files, +438/-2): 4c367df...PodJamz:feat/verdict-routed-moa-clean

What it adds

An opt-in policy: learned for /moa. policy defaults to static, so with nothing merged the behavior is byte-identical to what you have now.

1. Bucket the turn into a task class (code / writing / research / general) with a cheap deterministic keyword heuristic — no extra model call.
2. Route, don't broadcast. A per-task-class epsilon-greedy bandit picks the top-k references for the turn. Unseen models get an optimistic prior so they're tried at least once.
3. Free verdict. The aggregator already reads every reference, so it ends with MOA_VERDICT: <n> naming the one it relied on. That line is the reward signal — zero extra calls — and it's stripped out before the guidance reaches the main agent.
4. Credit + persist to ~/.hermes/moa_router.json. Next turn of that class routes to what's actually been helping. If a provider degrades, its verdict share drops and the router routes around it on its own — self-healing without anyone touching config.

Files

• agent/moa_router.py (new) — task classifier, bandit, verdict parse, JSON store
• hermes_cli/moa_config.py — policy, max_reference_models, exploration (backward compatible)
• agent/moa_loop.py — learned selection + verdict request + credit; static path unchanged
• agent/conversation_loop.py — pass the new fields through
• hermes_cli/moa_cmd.py — show policy in hermes moa list
• tests/run_agent/test_moa_router.py (new) — 8 tests

Validation

Check	Result
MoA test suite (loop + router + config)	14 passed
ruff (touched files)	clean
py_compile (touched files)	clean
Static-path behavior	unchanged (existing test_aggregate_moa_context_calls_mixed_providers still green)

What I haven't proven yet

The unit tests show it routes and learns, not that it's better. The real proof is a side-by-side on a fixed task set — static vs learned, same models — measuring quality at equal-or-lower cost once the router warms up. Happy to run that and attach numbers if you'd want to take this further. If it doesn't win or match at lower cost, it stays behind the flag.

Open questions: is the aggregator verdict enough or would you want an optional second judge for high-stakes classes; worth making the task-class bucketing pluggable; and is ~/.hermes/moa_router.json the right home for the store.

---

[kshitijk4poor]

Took a thorough pass at this — particularly the prompt-caching interaction, since that's the one invariant worth being paranoid about. Verified empirically against the branch (drove the real aggregate_moa_context + the injection block in conversation_loop.py, stubbing only the network boundary).

Caching is safe ✅ — api_messages is rebuilt fresh each iteration with per-message .copy(), the MoA guidance is appended only to that per-call copy's last user message (same pattern as memory prefetch / plugin pre_llm_call), and the system-prompt cache prefix + persisted history are byte-identical before/after. Nothing leaks to the session DB, and memory + plugin + MoA stack cleanly into one user message. No regression.

A few non-blocking observations:

1. Reference models get the main model's full system prompt + tool history. aggregate_moa_context (moa_loop.py:60-67) passes api_messages verbatim, which at the callsite already includes the ~8K-token Hermes system prompt and the assistant.tool_calls/tool-role transcript. For advisory reference calls that's (a) re-billed per ref per iteration and (b) a correctness risk — strict providers (Mistral/Fireworks) may reject tool_calls the ref never emitted / orphan tool messages. A trimmed view (user/assistant text, MoA-specific or no system prompt) would be cheaper and cleaner.

2. MoA fans out N+1 calls before every main-model iteration. It runs over a growing history each pass, so a K-iteration turn = K×(N+1) auxiliary calls with no cross-iteration reuse. The docs say "increases model-call count" but it's per-iteration, not per-turn — worth calling out explicitly, and maybe worth gating to every-K-iterations as a future lever.

3. The per-preset enabled flag looks dead. It's normalized/persisted (moa_config.py:41,57,108) and surfaced in the API + dashboard + desktop, but no execution path reads it — activation is governed entirely by the session toggle. So enabled: false in config/dashboard has no effect. Either wire it into the injection gate or drop the field.

Minor: the toggle + "resolve active config before run_conversation" logic is copy-pasted across CLI/gateway/TUI (4 near-identical spots) — a shared resolve_active_moa_for_session() helper would DRY it. And the desktop model-menu checkmark can desync from the server-side active preset on remount.

Tests green (the only 3 failures are unrelated macOS PTY-WebSocket flakes), old mixture_of_agents tool removed cleanly, config wiring complete. Nothing blocking here — feel free to push back on any of it.

[WolframRavenwolf]

I like this direction. One follow-up thought on the current reference-message trimming:

The default of stripping system/tool messages for reference calls makes sense to me, but for power-user setups, it would be very useful to make the reference context strategy configurable per preset. Beyond raw conversation history, persona/operating-frame files such as SOUL.md (or an equivalent user-defined context file) can be critical for agents whose behavior depends on a carefully defined identity, voice, priorities, or user-alignment model. Likewise, project instruction files such as AGENTS.md, CLAUDE.md, or .cursorrules can materially affect what the "correct" advice is: allowed files, test strategy, commit/push rules, security constraints, repo conventions, etc.

The default can stay exactly as-is, but presets could opt into something like:

• reference_system: none | custom | full
• reference_context_files: none | selected
• reference_context_file_names: optional list, e.g. SOUL.md / AGENTS.md / CLAUDE.md / .cursorrules
• reference_history: last_user | text | full
• reference_prompt: optional advisory instruction

That would make the approach substantially more flexible while preserving the current safe/cheap default.

I hope something like this can make it in, either in this PR or as a small follow-up. If you like the direction, I'd be happy to help implement it.

[LarryLemonBot]

This PR is exactly the direction I was hoping Hermes would take: MoA as a first-class model/provider surface, not a separate tool bolted onto the side.

I've been dogfooding Hermes hard from a real Windows + Telegram gateway setup, using GPT-5.5/OpenAI and xAI lanes for local profile-fusion MoA workflows. The current virtual-provider design in this PR maps much better to how I actually want to use it remotely: /moa should feel like model selection from CLI/TUI/Telegram/gateway sessions, while the aggregator remains the acting model that can use tools.

A few concrete follow-up contributions I'd be happy to help with, if aligned with the maintainers' direction:

1. Opt-in learned routing for presets
   Keep policy: static as the default/no-behavior-change path, but allow policy: learned where a small local router learns which reference models help per task class. The aggregator can emit a stripped hidden verdict/reward signal, so the router can reduce broadcast cost and route around degraded providers over time.

2. Preset-level reference context strategy
   The current trimmed reference view is the right safe/cheap default. For power users, presets could opt into controlled context modes such as reference_system: none|custom|full, reference_history: last_user|text|full, and selected project context files like AGENTS.md, CLAUDE.md, .cursorrules, or a user-defined operating-frame file. That would preserve the default while making coding/project MoA much more accurate.

3. Gateway/Telegram MoA behavior + docs
   Since this PR makes MoA a virtual provider, gateway sessions should treat /moa, /moa <preset>, and /moa <prompt> as model-selection/session behavior rather than tool enablement. I can help test/document this from a real Telegram setup.

4. Static-vs-learned benchmark harness
   Before claiming learned routing is better, I can contribute a small repeatable harness: same task set, static vs learned, same model pool, compare quality/cost after warm-up. If it doesn't win or match at lower cost, it stays behind the flag.

I'm here because I want to become a useful contributor to Hermes Agent, not just request features. Happy to split any of this into the smallest acceptable follow-up PR after #46081 lands, or adapt to whatever shape the maintainers prefer.