fix(moa): restore virtual runtime after fallback by ildunari · Pull Request #53802 · NousResearch/hermes-agent

ildunari · 2026-06-27T19:50:23Z

What does this PR do?

Fixes MoA primary-runtime restoration after provider fallback.

MoA is a virtual provider: it uses a MoAClient facade and intentionally does not have real OpenAI client kwargs. If a long-lived MoA session falls back to a concrete provider, then the next turn restores provider=model back to MoA. Before this fix, restoration could try to rebuild a real OpenAI client from MoA's empty/virtual client kwargs, causing restore failure instead of recreating the MoA facade.

The restore path now special-cases provider == "moa" and recreates MoAClient(agent.model or "default") instead of calling the OpenAI client builder.

Related Issue

No linked issue. Found while testing MoA sessions with fallback/restore behavior.

Type of Change

Bug fix (non-breaking change that fixes an issue)
Tests (adding or improving test coverage)

Changes Made

agent/agent_runtime_helpers.py
- Rebuilds the virtual MoA facade when restoring the primary runtime to provider="moa".
- Avoids constructing a real OpenAI client for MoA's virtual runtime.
tests/run_agent/test_moa_loop_mode.py
- Adds a regression test that simulates fallback to Z.AI, then verifies primary restore returns to MoA without calling _create_openai_client.

How to Test

python -m py_compile agent/agent_runtime_helpers.py
pytest -q tests/run_agent/test_moa_loop_mode.py tests/run_agent/test_primary_runtime_restore.py

Results locally:

py_compile: passed
MoA loop + primary runtime restore tests: 40 passed
Additional adversarial review reran the focused restore test plus primary restore suite: 32 passed

Checklist

Code

I've read the Contributing Guide
My commit messages follow Conventional Commits (fix(scope):, feat(scope):, etc.)
I searched for existing PRs to make sure this isn't a duplicate
My PR contains only changes related to this fix/feature (no unrelated commits)
I've run pytest tests/ -q and all tests pass
I've added tests for my changes (required for bug fixes, strongly encouraged for features)
I've tested on my platform: macOS 26.5

Documentation & Housekeeping

I've updated relevant documentation (README, docs/, docstrings) — or N/A
I've updated cli-config.yaml.example if I added/changed config keys — or N/A
I've updated CONTRIBUTING.md or AGENTS.md if I changed architecture or workflows — or N/A
I've considered cross-platform impact (Windows, macOS) per the compatibility guide — or N/A
I've updated tool descriptions/schemas if I changed tool behavior — or N/A

Screenshots / Logs

The regression test guards the original failure mode by making _create_openai_client raise if the MoA restore path tries to build a real OpenAI client.

teknium1

Thanks for identifying the restore path; the current-main premise is valid: restore_primary_runtime() restores MoA's empty client_kwargs and then calls _create_openai_client() at agent/agent_runtime_helpers.py:1193-1198.

Problems

The proposed MoAClient(...) reconstruction drops the reference_callback installed during MoA initialization (agent/agent_init.py:846-889). That callback is the only route for moa.reference / moa.aggregating display events (agent/moa_loop.py:791-798, 937-958), so a restored MoA session would lose progress output.
The same virtual-runtime reconstruction remains in try_recover_primary_transport() at agent/agent_runtime_helpers.py:1001-1026; the conversation loop calls it before fallback (agent/conversation_loop.py:3925-3946). A MoA primary hitting a transient transport recovery would still attempt the empty-kwargs OpenAI rebuild.

Suggested changes

Reuse a shared MoA-facade factory or preserve the initialized callback when recreating the facade, and add an event-delivery regression test after restore.
Cover the primary transport-recovery sibling path with the same no-OpenAI-client invariant.

Automated hermes-sweeper review.

teknium1 · 2026-07-15T06:38:05Z

+            # OpenAI client kwargs; restoring it after a fallback must recreate
+            # the facade, not call OpenAI() with an empty api_key.
+            from agent.moa_loop import MoAClient
+


MoA initialization passes reference_callback=_moa_reference_relay (agent/agent_init.py:846-889), but this reconstruction omits it. Please preserve that callback or use a shared facade factory; otherwise moa.reference and moa.aggregating progress events stop after a fallback restore.

teknium1 · 2026-07-15T06:38:05Z

@@ -1018,7 +1018,15 @@ def restore_primary_runtime(agent) -> bool:
        )


Please apply the same virtual-MoA reconstruction to try_recover_primary_transport (agent/agent_runtime_helpers.py:1001-1026). It restores the same empty MoA snapshot and still calls _create_openai_client; the conversation loop invokes it before fallback for transient failures.

#53802) Follow-up to the salvaged core of #53802: a naive MoAClient(preset) rebuild restores a working facade but silently drops the reference_callback relay wired in agent_init, so moa.reference / moa.aggregating display events stop reaching every frontend for the rest of the session. Introduce agent.moa_loop.build_moa_facade(agent, preset) as the single construction point for the MoA facade and use it at: - initial client construction (agent_init.py) - turn-start fallback restore (restore_primary_runtime) - transient transport recovery (try_recover_primary_transport — previously fell through to _create_openai_client with MoA's empty client_kwargs and died with 'api_key client option must be set') - mid-session model switches (switch_model) The relay reads agent.tool_progress_callback at emit time, so callbacks attached after construction are picked up automatically. Adds test_moa_restored_facade_still_emits_reference_events covering event delivery through a restored facade.

teknium1 · 2026-07-24T01:41:23Z

Merged via cluster PR #70280 (commit 74a56b7) — your core commit cherry-picked with authorship preserved, plus a follow-up that turns the fix into a shared build_moa_facade() factory used at init/restore/recover and preserves the reference_callback (a bare MoAClient() reconstruction would have silenced advisor display events after restore). Thanks for the restore-path diagnosis and regression test!

- model_setup_flows: _model_flow_moa (always show presets), _model_flow_bedrock_api_key (mantle region endpoint) - codex migration: _query_codex_plugins (app-server RPC) - slack parser: build_slack_parser (manifest + write) - delegate_tool: _resolve_child_credential_pool (custom endpoint identity NousResearch#7833) - relay: _platform_is_fronted (back-compat alias) - secrets_cli: cmd_token (verify-then-persist, 0. prefix warn) - skills_hub: _github_publish (fork → branch → tree → PR) - main: _tui_need_npm_install (content compare, prebuilt bundle), note_turn_start (interleave tripwire), _build_gateway_vbs_script (wscript no-console NousResearch#45599), _write_full_zip_backup (sqlite safe-copy), _login_openai_codex (device code + reuse), _frame_renderable (rich Group), systemd_install (legacy removal + --force pre-sync), build_moa_facade (reference relay NousResearch#53802), build_plugins_parser, _run_anthropic_oauth_flow (claude-code link), _render_distribution_plan (non-distribution warning), _detect_venv_python_processes (refuse-don't-kill)

…_recover_primary_transport) Whole-function owner tests for transient transport recovery: skip guards (fallback-active / non-transient / OpenRouter / Nous-Portal-OpenAI-wire) + the Portal-native-anthropic-wire exception, wire-aware rebuild from _primary_runtime (openai / anthropic-client-nulled / moa-facade NousResearch#53802), FD-safe retire (NousResearch#70773, skipped when no client, exception swallowed), transport-cache clear, and rebuild-exception -> False. Function (1248-1349) 100%.

…ry_runtime) Whole-function owner tests for turn-scoped primary restore: NousResearch#20465 index reset, rate-limit + reset-aware skip gates (future/already-logged/exception-fallthrough), wire-aware rebuild (moa-facade NousResearch#53802 / anthropic-nulled / openai), _cache_disabled survival (NousResearch#33555), transport-cache clear, pool rebind (mismatch-reload / prefetched-reuse / reload-exception-cleared), credential re-select NousResearch#25205 (match / mismatch / no-key / select-None) incl. custom:<name> disambiguation NousResearch#56885 (match/exception both blocks), reasoning restore, fallback reset, and rebuild fail-safe. Function (1449-1732) 100%. NOTE: an earlier grep-artifact hid the custom-provider blocks as false-covered; caught via re-roast, added the missing cases, switched to a python range filter.

NousResearch#53802) Follow-up to the salvaged core of NousResearch#53802: a naive MoAClient(preset) rebuild restores a working facade but silently drops the reference_callback relay wired in agent_init, so moa.reference / moa.aggregating display events stop reaching every frontend for the rest of the session. Introduce agent.moa_loop.build_moa_facade(agent, preset) as the single construction point for the MoA facade and use it at: - initial client construction (agent_init.py) - turn-start fallback restore (restore_primary_runtime) - transient transport recovery (try_recover_primary_transport — previously fell through to _create_openai_client with MoA's empty client_kwargs and died with 'api_key client option must be set') - mid-session model switches (switch_model) The relay reads agent.tool_progress_callback at emit time, so callbacks attached after construction are picked up automatically. Adds test_moa_restored_facade_still_emits_reference_events covering event delivery through a restored facade.

fix(moa): restore virtual runtime after fallback

9927d81

alt-glitch added type/bug Something isn't working comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P3 Low — cosmetic, nice to have labels Jun 27, 2026

helix4u mentioned this pull request Jun 27, 2026

fix(moa): keep virtual provider on MoA client #53809

Closed

14 tasks

teknium1 reviewed Jul 15, 2026

View reviewed changes

teknium1 mentioned this pull request Jul 23, 2026

fix(moa): thought-signature integrity, stream close on interrupt, facade restore, advisor attribution (cluster salvage) #70280

Merged

teknium1 closed this Jul 24, 2026

datanerdie mentioned this pull request Jul 26, 2026

MoA reference fan-out gives zero UI feedback — a slow advisor is indistinguishable from a hang (and reference_timeout defaults to 900s) #71243

Open

burak33bb mentioned this pull request Aug 4, 2026

fix(moa): preserve facade across client rebuilds #78409

Closed

18 tasks

x1051445024 mentioned this pull request Aug 10, 2026

fix(agent): preserve MoA facade when rebuilding primary client (stream retry, rotation, fallback+restore) #83088

Merged

haizaar mentioned this pull request Aug 14, 2026

Feat/anthropic on vertex #61859

Open

14 tasks

Navigation Menu

Uh oh!

fix(moa): restore virtual runtime after fallback - #53802

fix(moa): restore virtual runtime after fallback#53802
ildunari wants to merge 1 commit into
NousResearch:mainfrom
ildunari:fix/moa-restore-after-fallback-upstream

ildunari commented Jun 27, 2026

Uh oh!

teknium1 left a comment

Uh oh!

teknium1 Jul 15, 2026

Uh oh!

teknium1 Jul 15, 2026

Uh oh!

teknium1 commented Jul 24, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

		@@ -1018,7 +1018,15 @@ def restore_primary_runtime(agent) -> bool:
		)

Uh oh!

Conversation

ildunari commented Jun 27, 2026

What does this PR do?

Related Issue

Type of Change

Changes Made

How to Test

Checklist

Code

Documentation & Housekeeping

Screenshots / Logs

Uh oh!

teknium1 left a comment

Choose a reason for hiding this comment

Problems

Suggested changes

Uh oh!

teknium1 Jul 15, 2026

Choose a reason for hiding this comment

Uh oh!

teknium1 Jul 15, 2026

Choose a reason for hiding this comment

Uh oh!

teknium1 commented Jul 24, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants