Skip to content

fix(auxiliary): evict async wrappers on poisoned client (follow-up to #23482) - #23606

Closed
wuli666 wants to merge 1 commit into
NousResearch:mainfrom
wuli666:fix/evict-async-aux-wrappers-on-poison
Closed

fix(auxiliary): evict async wrappers on poisoned client (follow-up to #23482)#23606
wuli666 wants to merge 1 commit into
NousResearch:mainfrom
wuli666:fix/evict-async-aux-wrappers-on-poison

Conversation

@wuli666

@wuli666 wuli666 commented May 11, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?

Closes the async-side gap left by #23482. That fix solved cache poisoning on the sync auxiliary path — which is what the original #23432 reporter reproduced via context_compressor.pycall_llm. The async side has the identical bug class
but was not covered.

The cache key includes async_mode (see _client_cache_key), so the sync and async clients for the same provider live in two distinct cache entries pointing at the same underlying transport. The #23482 fix walked the sync wrapper's
_real_client correctly, but the async wrappers — AsyncCodexAuxiliaryClient, AsyncAnthropicAuxiliaryClient, AsyncGeminiNativeClient — never exposed _real_client, so the async entry survived eviction and kept handing out the poisoned
client.

Async-only callers that are NOT covered by #23482:

Code path Affected feature
tools/vision_tools.py (5 call sites) vision_analyze for image inputs
tools/session_search_tool.py session_search over chat history
tools/web_tools.py LLM-backed web tools
agent/plugin_llm.py Any plugin that delegates to an aux LLM
plugins/teams_pipeline/pipeline.py Teams gateway pipeline
trajectory_compressor.py Trajectory compression (separate from context_compressor)

For any of these, one timeout permanently poisons every subsequent async aux call with Connection error until gateway restart — even while the sync route recovered as designed in #23482.

Fix

Mirror the sync wrapper's _real_client onto each async wrapper so the existing eviction helper finds them. Three lines, one per wrapper:

Wrapper Added Note
AsyncCodexAuxiliaryClient self._real_client = sync_wrapper._real_client Underlying OpenAI client
AsyncAnthropicAuxiliaryClient same shape Underlying native Anthropic client
AsyncGeminiNativeClient self._real_client = sync_client Gemini's native facade is itself the leaf — no OpenAI client beneath it, so we point at the sync GeminiNativeClient directly

Plus a one-line docstring update on _evict_cached_client_instance to reflect that it now covers both sync and async wrappers via the same attribute walk.

Why this approach (vs. modifying the eviction helper)

The async wrappers already store enough to reach the leaf (via self._sync.chat.completions._sync._client or similar). I considered teaching _evict_cached_client_instance to walk those chains, but mirroring _real_client is:

  • Symmetric with the sync wrappers — same attribute, same semantics
  • Smaller diff — 3 one-line additions vs. a multi-branch helper
  • Self-documenting — the attribute name signals the contract this class participates in
  • Future-proof — any new helper that uses _real_client (e.g. a future "is this client healthy?" probe) automatically covers async too

Related Issue

Refs #23482 (parent fix), #23432 (original issue — closed by #23482; sync side fully resolved, this PR addresses the parallel bug class on the async side).

Type of Change

  • 🐛 Bug fix (non-breaking change that fixes an issue)
  • ✨ New feature
  • 🔒 Security fix
  • 📝 Documentation update
  • ✅ Tests (adding or improving test coverage)
  • ♻️ Refactor (no behavior change)
  • 🎯 New skill (bundled or hub)

Changes Made

  • agent/auxiliary_client.py (+18 / -3)
    • AsyncCodexAuxiliaryClient.__init__: mirror _real_client (with explanatory comment)
    • AsyncAnthropicAuxiliaryClient.__init__: same
    • _evict_cached_client_instance docstring: now covers async wrappers
  • agent/gemini_native_adapter.py (+6)
    • AsyncGeminiNativeClient.__init__: mirror _real_client pointing at sync_client
  • tests/agent/test_auxiliary_client.py (+36)
    • New test_evict_cached_client_instance_walks_async_wrapper: seeds both sync and async cache entries pointing at the same leaf, asserts a single eviction call drops both. Without the wrapper changes the test fails with the assertion message
      "async cache entry survived eviction — wrapper is missing _real_client".

How to Test

# 1. New test passes on this branch
pytest 'tests/agent/test_auxiliary_client.py::TestAuxiliaryClientPoisonedCacheEviction' -v
# → 8 passed

# 2. Regression discipline — revert the wrapper changes only:
git stash push -- agent/auxiliary_client.py agent/gemini_native_adapter.py
pytest 'tests/agent/test_auxiliary_client.py::TestAuxiliaryClientPoisonedCacheEviction::test_evict_cached_client_instance_walks_async_wrapper' -v
# → FAIL: "async cache entry survived eviction — wrapper is missing _real_client"
git stash pop
pytest 'tests/agent/test_auxiliary_client.py::TestAuxiliaryClientPoisonedCacheEviction::test_evict_cached_client_instance_walks_async_wrapper' -v
# → PASS again

# 3. Full file: 147/148 pass; the one pre-existing failure
#    (test_async_call_llm_retries_nous_after_401) is environmental
#    (httpx rejecting socks:// proxy URLs from local env), unrelated to this change.

Checklist

Code

  • I've read the Contributing Guide
  • My commit messages follow Conventional Commits (fix(auxiliary): ...)
  • I searched for existing PRs — no PR addresses the async-side gap
  • My PR contains only changes related to this fix — single commit, +60 / -3 across 3 files
  • I've run pytest tests/ -q and all tests pass — ran tests/agent/test_auxiliary_client.py (147/148 pass; the one pre-existing failure is environmental and present on main); did not run the full repository suite locally. Relying on CI
    to validate the full suite.
  • I've added tests for my changes — 1 new regression test with verified discipline
  • I've tested on my platform: Ubuntu 24.04 (kernel 6.8), Python 3.13.5

Documentation & Housekeeping

  • I've updated relevant documentation — _evict_cached_client_instance docstring updated to cover async wrappers
  • I've updated cli-config.yaml.example if I added/changed config keys — N/A
  • I've updated CONTRIBUTING.md or AGENTS.md if I changed architecture or workflows — N/A
  • I've considered cross-platform impact (Windows, macOS) — purely Python attribute additions; platform-agnostic
  • I've updated tool descriptions/schemas if I changed tool behavior — N/A (internal cache plumbing only; no tool-facing schema change)

…ousResearch#23482)

NousResearch#23482 fixed cache poisoning in the sync path: when a Codex auxiliary
timeout closes the underlying OpenAI client, _evict_cached_client_instance
walks CodexAuxiliaryClient wrappers via their _real_client attribute and
drops the cache entry so the next aux call rebuilds.

The cache key includes async_mode (see _client_cache_key), so the sync and
async clients for the same provider live in two distinct entries pointing
at the same underlying transport. The fix walked the sync wrapper's
_real_client correctly but the async wrappers
(AsyncCodexAuxiliaryClient, AsyncAnthropicAuxiliaryClient,
AsyncGeminiNativeClient) never exposed _real_client at all, so the async
entry survived eviction and kept handing out the poisoned client.

Effect on async aux callers: one timeout now poisons every subsequent
async aux call (compression, vision, session_search, title_generation)
with 'Connection error' until gateway restart -- even while the sync
route recovered as designed in NousResearch#23482.

Mirror the sync wrapper's _real_client onto each async wrapper so the
existing eviction helper finds them. Three changes, one per wrapper:

- AsyncCodexAuxiliaryClient: self._real_client = sync_wrapper._real_client
  (the underlying OpenAI client)
- AsyncAnthropicAuxiliaryClient: same shape
- AsyncGeminiNativeClient: self._real_client = sync_client (Gemini's
  native facade is itself the leaf; no OpenAI client beneath it)

Update _evict_cached_client_instance docstring to reflect that it now
covers both sync and async wrappers via the same attribute walk.

Test: TestAuxiliaryClientPoisonedCacheEviction.test_evict_cached_client_instance_walks_async_wrapper
seeds both sync and async cache entries pointing at the same leaf and
asserts both are dropped on a single eviction call. Verified the test
fails without the wrapper changes ("async cache entry survived
eviction -- wrapper is missing _real_client") and passes with them.

Refs NousResearch#23482, NousResearch#23432
@alt-glitch alt-glitch added type/bug Something isn't working comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P2 Medium — degraded but workaround exists labels May 11, 2026
@teknium1

Copy link
Copy Markdown
Contributor

Salvaged via #23931 — cherry-picked onto current main with your authorship preserved in git log. Thanks for the fix and the well-disciplined regression test! cc @Teknium

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P2 Medium — degraded but workaround exists type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants