fix(web): scope Parallel async clients to extraction loops

[goslingmanagment]

What does this PR do?

Prevents Parallel web-extraction clients from crossing event-loop ownership boundaries during concurrent tool execution.

AsyncParallel owns an HTTPX async transport whose connections are event-loop-affine, but it was cached process-wide in tools.web_tools._async_parallel_client. Concurrent tool workers each run on their own thread-local event loop (model_tools._get_worker_loop), so a single cached client ends up with sockets bound to a loop other workers don't own.

AsyncParallel is now created per extraction and closed in finally on the same loop that performed the request. The synchronous Parallel client keeps its cache — it has no loop affinity.

Related Issue

Addresses the async Parallel portion of #24736.

This corrects only the async Parallel portion of #83786; that PR's sync Parallel and Exa singleton-lock changes remain valid and are not touched here.

Type of Change

• 🐛 Bug fix (non-breaking change that fixes an issue)
• ✅ Tests (adding or improving test coverage)

Root Cause

On first concurrent use, every worker observes an empty cache, constructs its own client, and races to publish one. The clients that lose the race are dropped while still holding live keep-alive connections bound to their worker loops. When they are finalized later, the SDK's AsyncHttpxClientWrapper.__del__ schedules aclose() on whatever loop happens to be running — prompt_toolkit's — while the transports belong to loops that are gone:

RuntimeError: Event loop is closed
  httpx/_client.py aclose -> _transports/default.py aclose
  -> httpcore connection_pool/connection/http11 -> anyio tls
  -> asyncio selector_events.close -> base_events.call_soon -> _check_closed

Three concurrent first-use extractions reproduce it deterministically: three clients constructed, two unpublished, two closed-loop errors. The fetches themselves succeed — the crash lands afterwards, at finalization, and prompt_toolkit surfaces it as Unhandled exception in event loop followed by Press ENTER to continue....

Note on the alternative: serializing construction with a lock removes the orphaned clients but forces one loop-affine client to be shared across worker loops, which is the condition that makes the transports unusable in the first place.

Changes Made

• plugins/web/parallel/provider.py — _get_async_client() is now a factory rather than a cache; extract() owns its client and closes it in finally on the owning loop.
• plugins/web/parallel/provider.py — a cleanup failure can no longer discard a completed extraction. close() funnels into httpx.aclose() -> transport.aclose(), which can raise after the response is fully materialized; that failure is masked from the caller but logged at warning level so a regression of this ownership fix stays visible. except Exception is deliberate: CancelledError is a BaseException and must keep propagating so cancellation semantics are preserved.
• plugins/web/parallel/provider.py — _reset_clients_for_tests() no longer clears an async slot.
• tools/web_tools.py — removed the obsolete _async_parallel_client cache slot.
• tests/tools/test_parallel_async_client_lifecycle.py — new regression coverage.

How to Test

1. On clean 165c889, run three first-use Parallel extractions concurrently (a constructor barrier makes the publication race deterministic). The old implementation constructs three clients, leaves two unpublished and unclosed, and produces two RuntimeError: Event loop is closed failures at finalization.
2. Apply this change and repeat: three clients, each closed on its owner loop, zero closed-loop errors. Verified against live network with the three URLs from the original report.
3. Run the tests:

python -m pytest -q tests/tools/test_parallel_async_client_lifecycle.py
python -m pytest -q tests/tools/test_web_tools_config.py tests/plugins/web/

All three new tests fail on 165c889 and pass with this change:

• test_concurrent_extract_closes_each_client_on_its_owner_loop — one client per extraction, closed on the same loop that used it, no process-wide publication.
• test_close_failure_does_not_discard_a_successful_extraction — asserts the close was actually attempted and exactly one warning emitted, so deleting the cleanup cannot make it pass.
• test_extraction_failure_outranks_a_close_failure — a secondary cleanup failure must not overwrite the primary error.

Local results: 69 passed for the provider/config suites, 25/25 repeated runs of the new file with no flakes, ruff check and git diff --check clean.

Checklist

Code

• I've read the Contributing Guide
• My commit messages follow Conventional Commits (fix(scope):, feat(scope):, etc.)
• I searched for existing PRs to make sure this isn't a duplicate
• My PR contains only changes related to this fix/feature (no unrelated commits)
• I've run pytest tests/ -q and all tests pass
• I've added tests for my changes (required for bug fixes, strongly encouraged for features)
• I've tested on my platform: macOS 26.6, Python 3.11.15

Documentation & Housekeeping

• I've updated relevant documentation (README, docs/, docstrings) — module and function docstrings in the provider now state the ownership rule
• I've updated cli-config.yaml.example if I added/changed config keys — N/A
• I've updated CONTRIBUTING.md or AGENTS.md if I changed architecture or workflows — N/A
• I've considered cross-platform impact (Windows, macOS) — no platform-specific APIs; the change is pure asyncio/HTTPX lifecycle
• I've updated tool descriptions/schemas if I changed tool behavior — N/A (no user-visible tool surface change)

[Enough1122]

AI code review — automated review for reference, author can ignore or act on any point.

fix(web): scope Parallel async clients to extraction loops

Making the async client request-scoped and closing it on the owning loop is the right fix for loop-affine httpx transports. Observations:

1. plugins/web/parallel/provider.py — _reset_clients_for_tests() no longer clears _async_parallel_client, and the module-level slot _async_parallel_client was removed from tools/web_tools.py entirely. Any in-tree caller or test that still reads tools.web_tools._async_parallel_client directly will now hit AttributeError (the new test uses monkeypatch.setattr(..., raising=False) to preserve compatibility, which suggests this was anticipated). Worth a quick grep to confirm no remaining direct readers; if any exist they should be updated to the factory pattern.
2. _get_async_client() now constructs a fresh client per extract call — every extraction pays full connection-pool setup with no reuse even within the same event loop. For high-volume extract usage this is a throughput regression; acceptable given loop-affinity constraints, but a per-loop client cache keyed by id(get_running_loop()) could recover reuse without the cross-loop leak the old code had. At minimum the docstring should call out the per-call cost.
3. The finally block correctly prioritizes the primary exception (close failure is masked + logged, and except Exception lets CancelledError propagate), and test_extraction_failure_outranks_a_close_failure pins that ordering — good.
4. Minor: the _get_async_client docstring says "The caller must close the client on the same event loop that uses it" — the only caller (extract) honors this, but _get_async_parallel_client is still re-exported from tools/web_tools.py as a backward-compat name. Callers of that alias now own a client they may not know they must close; consider a deprecation note on the alias.