fix(gateway): stop leaking Codex app-server sessions - #73429
Conversation
|
Confirmed related to #55450 and #66678. The additional production finding here is that dynamic webhook deliveries have unique one-shot session keys but remain in the general agent cache (cap 128, with finite-reset sessions potentially retained until expiry), so lifecycle cleanup is not invoked soon enough to prevent OOM. This PR adds webhook completion eviction as the second half of the fix, plus detach-before-close semantics and regression coverage for both layers. I am happy to consolidate whichever lifecycle implementation maintainers prefer; the webhook eviction should remain paired with one of them. |
|
Thanks for covering both the agent-owned Codex runtime and one-shot webhook cache lifecycle. Suggested changes
The implementation ordering is sound: the gateway handler clears its running-agent slot in Automated hermes-sweeper review. |
What does this PR do?
Closes and detaches an agent-owned Codex app-server session whenever
AIAgentreleases its clients or performs full cleanup, and evicts one-shot webhook agents as soon as their delivery finishes.Long-running gateways create short-lived agents for cron and webhook work. Evicted agents already released their LLM clients, child agents, and tool resources, but
_codex_sessionwas omitted from both cleanup paths. Each eviction could therefore retain a Codex app-server and its MCP subprocesses until the gateway itself restarted.Webhook deliveries have unique session keys, but their completed agents also remained in the general agent cache. That cache permits 128 entries and can defer idle eviction for finite-reset sessions, so webhook runtimes could consume memory long before normal cache pressure cleaned them up.
The lifecycle helper detaches the session before best-effort close, making cleanup idempotent and preventing a close failure from retaining the session reference. The webhook completion hook now evicts its unique one-shot cache key after ending the persisted session; interactive chat caching is unchanged.
Related Issue
N/A — no matching issue or PR was found.
Type of Change
Changes Made
_close_codex_session()lifecycle helper inrun_agent.py.close()implementation.webhook_completesession close.How to Test
tests/**/test_*codex*.pyfiles.codex_app_server, complete repeated cron and webhook runs and verify nocodex app-serveror MCP child remains in the gateway cgroup after each one-shot run.Checklist
Code
pytest tests/ -qand all tests pass. The complete suite has three pre-existing macOS ACP failures on untouchedmain: ACP edit-approval tests classify pytest's/private/var/...temp directory as a sensitive system path.Documentation & Housekeeping
cli-config.yaml.example: N/A; no config changes.CONTRIBUTING.md/AGENTS.md: N/A; no architecture or workflow changes.Test Results
802 passed— every Codex-named test file across agent, transport, CLI, cron, tools, plugins, and TUI.268 passed— gateway webhook completion/cache/shutdown and cron shutdown lifecycle tests.348 passed, 3 failed; the same three failures reproduce on untouched upstreammain.