fix(agent): stop the MoA prepared request reaching a swapped-in native client

[Drexuxux]

Addresses #78382. Two PRs are already open on that issue — see "Relation to the open PRs" at the bottom before spending review time here.

What

_moa_prepared_request is a private handshake: agent/conversation_loop.py attaches it to api_kwargs, and only MoAChatCompletions.create in agent/moa_loop.py pops it back off. It is attached on one condition — agent.provider == "moa" — which assumes agent.client is still the in-process MoA facade.

That assumption does not survive a client swap. _replace_primary_openai_client rebuilds agent.client from _client_kwargs on credential rotation, provider fallback and dead-connection cleanup, and it leaves agent.provider at "moa". The rebuilt client is a native OpenAI client, so the key reaches an SDK that has never heard of it:

TypeError: Completions.create() got an unexpected keyword argument '_moa_prepared_request'

The error is non-retryable, so it is not a lost turn — every remaining turn on that session fails the same way.

The timing is not incidental. pending_moa_prepared_request deliberately carries a prepared request across attempt boundaries, and a client swap is exactly what happens at an attempt boundary. Preparation reads the facade; dispatch, one rotation later, does not.

Both dispatch paths are affected, for the same reason. The non-streaming one calls agent.client.chat.completions.create(**api_kwargs) directly, and the streaming one goes through _create_request_openai_client, which returns the primary client unchanged when the provider is "moa".

This is the gap left by 3e86df2 (2026-07-27, fix(agent): redecorate prompt-cache breakpoints after provider failover). That commit already recognised, in this same function, that per-client request state has to be re-derived after a failover — it just re-derived the cache breakpoints and not the MoA handshake, which is attached a few lines later and swaps out under the same event.

The fix

Re-read the live client at the point the key is attached, rather than trusting the one that prepared the request. Only the facade exposes prepare(), which is the same duck-type the preparation step above already uses, so no import and no new coupling:

• facade still in place — attach the key, behaviour unchanged
• facade gone — send the prepared prompt without the handshake and log the downgrade

Guarding at the attachment point rather than at the dispatch point covers the streaming and non-streaming paths with one check. The messages in api_kwargs are already the prepared ones, so the rebuilt client serves a well-formed request instead of raising.

Tests and results

New tests/agent/test_moa_prepared_request_client_swap.py — 4 passed:

• a native-signature completions object is not treated as a facade
• a real MoAChatCompletions is
• a client with no chat attribute, and None, are not
• the premise itself: the native signature raises TypeError naming _moa_prepared_request, and the same call without the key succeeds

tests/agent -k moa — 68 passed, 3 skipped, no failures.

Reproduced the original error before the change by dispatching with provider="moa" and a native-signature client, and confirmed the same dispatch is clean once the key is withheld.

Relation to the open PRs

I found #78409 and #80132 only after opening this. Recording the differences so a reviewer can pick one rather than diff three.

#78409 — preserve the facade across client rebuilds (run_agent.py). This is the root-cause repair: it restores the invariant that provider == "moa" implies client is the facade, instead of teaching callers to cope when it does not hold. If it holds up, it is the better fix and this PR is unnecessary. It is also the largest behavioural change of the three, since it alters what rotation produces.

#80132 — pop the key at the non-streaming dispatch (agent/chat_completion_helpers.py). Same intent as this PR, and it would stop the reported TypeError. Two things a reviewer should weigh:

• The pop is unconditional, so it also strips the key when the facade is still in place. MoAChatCompletions.prepare documents the contract it relies on — "when the loop supplies the returned private object back to create(), the advisor fan-out is not repeated" — and rebase_prepared_request exists specifically to avoid "a second costly fan-out". Dropping the key on the healthy path sends create() down its normal resolution branch, so the advisors run a second time on every non-streaming MoA turn.
• It guards the non-streaming dispatch only. The streaming path reaches the same swapped-in client through _create_request_openai_client, which returns the primary client unchanged for provider "moa", and still carries the key.

This PR withholds the key only when the facade is actually gone, and does it once for both paths. I have no stake in which lands — if #78409 is the direction, close this.

[teknium1]

Thanks! Your injection-site _moa_client_consumes_prepared_request guard and tests were cherry-picked verbatim into #83088 with your authorship preserved in git history. Closing as superseded by #83088, which combines your guard with facade preservation at the client-rebuild choke point so the swap can no longer happen at all.