fix(agent): preserve MoA facade when rebuilding primary client (stream retry, rotation, fallback+restore) - #83088
Merged
Conversation
…m retry, rotation, fallback+restore) When agent.provider == "moa", the MoAClient facade *is* the client - there is no real OpenAI wire endpoint behind the moa://local placeholder. Client rebuilds (_replace_primary_openai_client: stream-retry pool cleanup, credential rotation, dead-connection cleanup, fallback+restore) go through create_openai_client and produce a native OpenAI client while provider stays "moa". The next primary call then either raises a `_moa_prepared_request` TypeError (NousResearch#78382) or, when _client_kwargs carry an unrelated relay base_url, leaks the request to a foreign gateway (observed as HTTP 503 "group ... no available channel" from an unrelated new-api relay right after an aggregator empty-stream retry). Fix: in create_openai_client, when provider is "moa", return build_moa_facade(agent, model) instead of a native client. This covers every rebuild entry point. The three already-fixed call sites (restore_primary_runtime, try_recover_primary_transport, switch_model) assign the facade directly and do not go through create_openai_client, so they are unaffected. Closes NousResearch#78382
…e client
`_moa_prepared_request` is a private handshake between the conversation
loop and MoAChatCompletions.create. It is attached whenever
agent.provider == "moa", on the assumption that agent.client is still the
in-process MoA facade.
Credential rotation, provider fallback and dead-connection cleanup all
rebuild agent.client from _client_kwargs between attempts, and
pending_moa_prepared_request deliberately carries a prepared request
across exactly that boundary. The rebuilt client is a native OpenAI
client while provider stays "moa", so the key reaches an SDK that has
never heard of it:
TypeError: Completions.create() got an unexpected keyword argument
'_moa_prepared_request'
That error is non-retryable, so every remaining turn on the session
fails. Both dispatch paths are affected: the non-streaming one calls
agent.client directly, and _create_request_openai_client returns
agent.client unchanged for provider "moa".
Re-check the live client at the point the key is attached, which covers
both paths at once. When the facade is gone, send the prepared prompt
without the handshake and log the downgrade.
…client After a client replacement (credential rotation, dead-connection cleanup, or fallback+restore), agent.client may become a native OpenAI client while agent.provider stays "moa". The _moa_prepared_request key was passed through to the native SDK, causing TypeError on every turn. Pop the key at the dispatch point (chat_completion_helpers.py:509). The MoAClient facade already handles a missing key by falling through to its normal resolution path. Closes NousResearch#78382
…not the MoA facade The unconditional pop from the previous commit also stripped the key when agent.client was still the real MoA facade, forcing the facade to re-prepare from scratch — a duplicate reference fan-out per turn (caught by test_moa_virtual_provider_aggregator_is_actor). Gate the defensive strip on the same prepare()-capability probe used at the injection site so the handshake survives on the facade while a swapped-in native client is still protected.
teknium1
force-pushed
the
fix/moa-facade-rebuild
branch
from
August 15, 2026 03:59
3c3086e to
81c8243
Compare
This was referenced Aug 15, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Issue #78382 covers the MoA facade being dropped when
agent.clientis replaced — but the fix there only covers three call sites (restore_primary_runtime,try_recover_primary_transport,switch_model). The generic rebuild path_replace_primary_openai_client(used by stream-retry pool cleanup, credential rotation, dead-connection cleanup, and fallback+restore) still goes throughcreate_openai_client, which builds a native OpenAI client whileagent.providerstays"moa".Second symptom (this PR's report): with a MOA preset active, when the aggregator returns an empty stream (
EmptyStreamError, HTTP 200 / zero valid chunks), the stream-retry machinery calls_replace_primary_openai_client(reason="stream_retry_pool_cleanup"). The rebuilt native client picks up whatever relaybase_urlhappens to be in_client_kwargsat that point, and the retried request — still carrying the MOA preset name as the model — is sent to an unrelated relay gateway, which answers:No such group exists anywhere in the user's config; the request simply leaked to a foreign gateway. On top of that, the facade is gone for the rest of the turn, so subsequent primary calls keep hitting the wrong endpoint (or raise the
_moa_prepared_requestTypeError from #78382).Fix
In
create_openai_client, whenagent.provider == "moa", returnbuild_moa_facade(agent, model)instead of building a native OpenAI client. Since every rebuild entry point funnels throughcreate_openai_client, this covers_replace_primary_openai_clientand any future rebuild path. The three already-fixed call sites assign the facade directly and never reachcreate_openai_client, so they are unaffected.build_moa_facadealso re-wires the reference relay (see #53802), so display events (moa.reference/moa.aggregating) survive rebuilds too.Verification
python -m py_compilepasses on the modified file.create_openai_client(SimpleNamespace(provider="moa", model=preset, ...), {}, reason="stream_retry_pool_cleanup")returns aMoAClient— facade preserved.Notes