fix(aiohttp): dispose recycled client sessions deterministically

[anmolg1997]

Relevant issues

Fixes #24230

Builds on the analysis in #24231 (credit to @alilxxey for the report and the first PR, closed as stale with an unsigned CLA). This PR additionally disposes of sessions in the loop-inspection fallback branch and handles the cross-loop lifecycles that were previously left to GC.

Pre-Submission checklist

Please complete all items before asking a LiteLLM maintainer to review your PR

• I have added meaningful tests
• My PR passes all CI/CD checks (e.g., lint, format, unit tests)
• My PR's scope is as isolated as possible; it only solves 1 specific problem
• I have requested a Greptile review by commenting @greptileai and received a Confidence Score of at least 4/5 before requesting a maintainer review

Screenshots / Proof of Fix

1. Verbatim repro script from #24230

Before (current main / v1.88.1):

recycled session replaced: True
old session closed immediately: False
old session closed after loop tick: False   <- session leaked to GC

After (this branch):

recycled session replaced: True
old session closed immediately: False
old session closed after loop tick: True

2. End-to-end A/B harness - real HTTP requests through LiteLLMAiohttpTransport (via httpx.AsyncClient(transport=...)) against a local aiohttp server, across 20 event-loop churn cycles (asyncio.run per cycle, module-cached transport, like proxy/eval workloads), with 5 loop-inspection failures induced exactly as in the issue repro:

build	Unclosed client session / Unclosed connector dumps on stderr	dead-loop session closed at recycle return
litellm v1.88.1 vanilla	9	False
v1.88.1 + this patch	0	True

The same stderr dumps are what production deployments see as ERROR-severity asyncio log entries (our GKE deployment surfaced them daily; Cloud Logging classifies them red).

3. Test suite

• pytest tests/test_litellm/llms/custom_httpx/test_aiohttp_transport.py -> 24 passed (5 new regression tests; the two GC-behavior tests fail on current main and pass with the fix)
• pytest tests/test_litellm/llms/custom_httpx/ -> 174 passed
• ruff check / ruff format --check clean on both changed files

Type

🐛 Bug Fix

Changes

LiteLLMAiohttpTransport replaced its cached aiohttp.ClientSession in three places without reliably closing the previous session:

1. loop-mismatch recycle: asyncio.create_task(old_session.close()) discarded the task reference, so the close task could be garbage-collected before it ran;
2. the (RuntimeError, AttributeError) fallback branch: replaced the session without closing it at all (the exact repro in [Bug]: LiteLLMAiohttpTransport can leak recycled aiohttp ClientSession instances #24230);
3. cross-loop sessions: a session bound to a closed loop was abandoned to GC ("rely on GC"), and a session bound to a loop running in another thread was closed from the wrong loop.

This PR adds one disposal helper, _close_recycled_session(), used by all three replacement sites (loop-mismatch, fallback branch, and the "Session is closed" retry in handle_async_request). It handles the three lifecycles a recycled session can be in:

• current running loop: schedule the async close and keep a strong reference in a class-level registry until the task completes (pruned via done-callback, close errors logged at debug);
• loop running elsewhere (another thread): hand the close to the session's own loop with asyncio.run_coroutine_threadsafe;
• loop closed / no running loop: dispose synchronously through the connector teardown aiohttp's own finalizer uses (BaseConnector._close), which releases pooled connections and flips the closed flags that ClientSession.__del__ / BaseConnector.__del__ check - so nothing is left for the GC to report.

No public API changes; the happy path (valid session on the current loop) is untouched.

[CLAassistant]

All committers have signed the CLA.

[anmolg1997]

@greptileai

[codspeed-hq]

Merging this PR will not alter performance

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 1 improved benchmark
❌ 1 regressed benchmark
✅ 28 untouched benchmarks

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

	Benchmark	BASE	HEAD	Efficiency
❌	test_completion_simple_message	3.2 ms	4.2 ms	-23.47%
⚡	test_completion_multi_turn	4.2 ms	3.1 ms	+33.31%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.

---

_{Comparing anmolg1997:fix/aiohttp-recycled-session-leak (18e8ef0) with main (88e03e5)}

[greptile-apps]

Greptile Summary

This PR fixes deterministic disposal of recycled aiohttp.ClientSession objects in LiteLLMAiohttpTransport, addressing the "Unclosed client session / connector" GC warnings that surface as ERROR-level asyncio log entries in production deployments.

• Adds _close_recycled_session() with three branches: schedule an async close via create_task (current loop, with a class-level strong-reference set to prevent GC before the task runs), hand the close to the session's own loop via run_coroutine_threadsafe (foreign running loop), or synchronously tear down the connector via BaseConnector._close (stopped or closed loop — the same path aiohttp's own __del__ uses).
• Applies the helper to all three replacement sites: the loop-mismatch recycle path, the (RuntimeError, AttributeError) fallback branch (the verbatim repro from [Bug]: LiteLLMAiohttpTransport can leak recycled aiohttp ClientSession instances #24230), and the "Session is closed" retry in handle_async_request; the retry handler now correctly disposes the local client_session variable rather than self.client, avoiding premature closure of a concurrently installed replacement.
• Includes seven new regression tests, all mock-only, covering each disposal branch and the concurrent-replacement race.

Confidence Score: 5/5

The change is a targeted, self-contained cleanup fix that does not touch any request-processing hot path or modify any public API; the happy path (valid session on the current running loop) is structurally unchanged.

All three session-replacement sites now deterministically dispose the recycled session, with correct handling for every loop lifecycle (current, foreign-running, stopped, closed). The concurrent-replacement race in the exception handler is resolved by operating on the captured local variable instead of self.client. Strong references prevent GC from collecting close tasks before they execute, and the done-callback prunes completed tasks. Seven new regression tests cover each branch, all mocked without real network calls. No functional defects were found in the new code.

No files require special attention.

Important Files Changed

Filename	Overview
litellm/llms/custom_httpx/aiohttp_transport.py	Adds _close_recycled_session helper and _background_close_tasks class registry to deterministically dispose replaced ClientSession objects across all three replacement sites (loop-mismatch recycle, fallback branch, "Session is closed" retry). All three lifecycle branches (current loop, foreign running loop, stopped/closed loop) are handled correctly.
tests/test_litellm/llms/custom_httpx/test_aiohttp_transport.py	Adds seven regression tests for the session-leak fix: fallback branch closure, GC warning suppression, dead-loop synchronous disposal, strong-reference registry, foreign-loop threadsafe close, concurrent-replacement safety, and stopped-loop synchronous disposal. All tests are properly mocked with no real network calls.

_{Reviews (5): Last reviewed commit: "fix(aiohttp): dispose recycled client se..." | Re-trigger Greptile}

[greptile-apps]

Greptile Summary

This PR fixes aiohttp ClientSession leaks in LiteLLMAiohttpTransport by introducing a _close_recycled_session helper that deterministically disposes replaced sessions across three lifecycle paths: async task on the current loop (with a strong reference to prevent GC of the task), run_coroutine_threadsafe for sessions owned by a foreign running loop, and a synchronous BaseConnector._close teardown for sessions whose loop is already gone.

• The loop-mismatch recycle path and the (RuntimeError, AttributeError) fallback branch in _get_valid_client_session both now call _close_recycled_session instead of either discarding the task or doing nothing.
• The "Session is closed" retry branch in handle_async_request also calls _close_recycled_session, but captures self.client — a mutable instance field — rather than the local client_session that holds the session which actually faulted; under concurrency a concurrent task can replace self.client with a healthy new session during the preceding await, causing the exception handler to close the wrong session.

Confidence Score: 3/5

The core fix is sound for its primary target scenarios, but the "Session is closed" retry path in handle_async_request can close an unrelated healthy session under concurrent load, causing failures on subsequent requests.

The disposal helper itself is well-designed and the tests validate each lifecycle branch. The risk is in handle_async_request: old_session is read from self.client after an await point, so a concurrent task that already replaced self.client can cause the exception handler to close the freshly-created replacement session rather than the one that actually errored.

litellm/llms/custom_httpx/aiohttp_transport.py — specifically the exception handler in handle_async_request around line 410.

Important Files Changed

Filename	Overview
litellm/llms/custom_httpx/aiohttp_transport.py	Adds _close_recycled_session helper that correctly handles three lifecycle paths (async on current loop, threadsafe on foreign loop, sync on dead loop); however the "Session is closed" retry branch in handle_async_request captures self.client instead of the local client_session variable, which can close the wrong session under concurrent load.
tests/test_litellm/llms/custom_httpx/test_aiohttp_transport.py	Adds five well-scoped regression tests covering all three disposal paths and the task-registry lifecycle; no real network calls; test_close_task_strongly_referenced_until_done asserts on class-level state (_background_close_tasks) without resetting it first, which could be noisy in a parallel test run.

_{Reviews (2): Last reviewed commit: "fix(aiohttp): dispose recycled client se..." | Re-trigger Greptile}

[codecov]

Codecov Report

❌ Patch coverage is 74.19355% with 16 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
litellm/llms/custom_httpx/aiohttp_transport.py	74.19%	16 Missing ⚠️

📢 Thoughts on this report? Let us know!

[anmolg1997]

@greptileai review the latest commit please - addressed all three findings: the retry handler now disposes the local faulted session instead of self.client (with a concurrency regression test), stopped-but-not-closed foreign loops take the synchronous teardown path, and the degraded private-API path logs instead of silently no-op'ing.

[anmolg1997]

@greptileai

[anmolg1997]

@greptileai review

[anmolg1997]

Hi @yassin-berriai, it would be great if you could review this, to unblock us!

Thanks!