fix(weixin): track fire-and-forget tasks to prevent GC mid-flight - #11998
Closed
alexzhu0 wants to merge 1 commit into
Closed
fix(weixin): track fire-and-forget tasks to prevent GC mid-flight#11998alexzhu0 wants to merge 1 commit into
alexzhu0 wants to merge 1 commit into
Conversation
_poll_loop spawned _process_message_safe with a bare asyncio.create_task(...) for every inbound message, and _process_message spawned _maybe_fetch_typing_ticket the same way. Python's event loop only keeps a weak reference to tasks returned by create_task — the Python docs explicitly warn that untracked tasks can be garbage-collected mid-flight. The weixin long-poll drains buffered messages in tight bursts, so a GC pass at the wrong moment silently drops a user's message after the sync buffer has already advanced. Mirror the tracking pattern the DingTalk adapter uses: - Add self._bg_tasks: Set[asyncio.Task] in __init__ - Add _spawn_bg(coro) helper that create_task's, adds to the set, and installs a done-callback to discard the task on completion - Route both fire-and-forget sites (inbound dispatch + typing-ticket fetch) through _spawn_bg - Cancel + await _bg_tasks in disconnect() before closing HTTP sessions, so no orphaned tasks keep writing after teardown Two regression tests cover the tracking lifecycle: a single task is added and discarded on completion, and multiple concurrent tasks are all captured simultaneously.
This was referenced Apr 18, 2026
haileymarshall
added a commit
to haileymarshall/hermes-agent
that referenced
this pull request
Apr 18, 2026
`handle_401` spawned a dedup'd recovery coroutine via `asyncio.create_task(_do_handle())` and discarded the returned task reference. Python's event loop only keeps weak references to tasks, so the coroutine could be garbage-collected before it called `pending.set_result(...)`. Every concurrent caller awaiting that future then hangs forever, and the `finally: entry.pending_401.pop(...)` cleanup never runs — so subsequent 401s for the same key latch onto the dead future too. Same pattern the adapter-side fixes address (NousResearch#11997, NousResearch#11998, NousResearch#12000, NousResearch#12001, NousResearch#12006). Hold the task in a process-wide set on the manager and discard it via `add_done_callback` once it completes. Regression test covers both the structural invariant (task tracked, then removed on completion) and a concurrent dedup path with a forced `gc.collect()` between the handler's await points.
3 tasks
alt-glitch
added
type/bug
3 tasks
Contributor
Author
|
Closing as part of post-mortem cleanup of an early-batch proactive audit that did not get review traction. The patch still applies if anyone wants to repurpose it. My contribution methodology has moved to alexzhu0/echo-agent — not pursuing this individual fix further. |
teknium1
pushed a commit
that referenced
this pull request
Jun 30, 2026
`handle_401` spawned a dedup'd recovery coroutine via `asyncio.create_task(_do_handle())` and discarded the returned task reference. Python's event loop only keeps weak references to tasks, so the coroutine could be garbage-collected before it called `pending.set_result(...)`. Every concurrent caller awaiting that future then hangs forever, and the `finally: entry.pending_401.pop(...)` cleanup never runs — so subsequent 401s for the same key latch onto the dead future too. Same pattern the adapter-side fixes address (#11997, #11998, #12000, #12001, #12006). Hold the task in a process-wide set on the manager and discard it via `add_done_callback` once it completes. Regression test covers both the structural invariant (task tracked, then removed on completion) and a concurrent dedup path with a forced `gc.collect()` between the handler's await points.
dtera
pushed a commit
to dtera/hermes-agent
that referenced
this pull request
Jul 1, 2026
`handle_401` spawned a dedup'd recovery coroutine via `asyncio.create_task(_do_handle())` and discarded the returned task reference. Python's event loop only keeps weak references to tasks, so the coroutine could be garbage-collected before it called `pending.set_result(...)`. Every concurrent caller awaiting that future then hangs forever, and the `finally: entry.pending_401.pop(...)` cleanup never runs — so subsequent 401s for the same key latch onto the dead future too. Same pattern the adapter-side fixes address (NousResearch#11997, NousResearch#11998, NousResearch#12000, NousResearch#12001, NousResearch#12006). Hold the task in a process-wide set on the manager and discard it via `add_done_callback` once it completes. Regression test covers both the structural invariant (task tracked, then removed on completion) and a concurrent dedup path with a forced `gc.collect()` between the handler's await points.
waefrebeorn
pushed a commit
to waefrebeorn/slermes
that referenced
this pull request
Jul 2, 2026
`handle_401` spawned a dedup'd recovery coroutine via `asyncio.create_task(_do_handle())` and discarded the returned task reference. Python's event loop only keeps weak references to tasks, so the coroutine could be garbage-collected before it called `pending.set_result(...)`. Every concurrent caller awaiting that future then hangs forever, and the `finally: entry.pending_401.pop(...)` cleanup never runs — so subsequent 401s for the same key latch onto the dead future too. Same pattern the adapter-side fixes address (NousResearch#11997, NousResearch#11998, NousResearch#12000, NousResearch#12001, NousResearch#12006). Hold the task in a process-wide set on the manager and discard it via `add_done_callback` once it completes. Regression test covers both the structural invariant (task tracked, then removed on completion) and a concurrent dedup path with a forced `gc.collect()` between the handler's await points.
Jasper6439
pushed a commit
to Jasper6439/hermes-agent
that referenced
this pull request
Jul 5, 2026
`handle_401` spawned a dedup'd recovery coroutine via `asyncio.create_task(_do_handle())` and discarded the returned task reference. Python's event loop only keeps weak references to tasks, so the coroutine could be garbage-collected before it called `pending.set_result(...)`. Every concurrent caller awaiting that future then hangs forever, and the `finally: entry.pending_401.pop(...)` cleanup never runs — so subsequent 401s for the same key latch onto the dead future too. Same pattern the adapter-side fixes address (NousResearch#11997, NousResearch#11998, NousResearch#12000, NousResearch#12001, NousResearch#12006). Hold the task in a process-wide set on the manager and discard it via `add_done_callback` once it completes. Regression test covers both the structural invariant (task tracked, then removed on completion) and a concurrent dedup path with a forced `gc.collect()` between the handler's await points.
habarmc1223-sudo
pushed a commit
to habarmc1223-sudo/hermes-agent-fluxmem
that referenced
this pull request
Jul 8, 2026
`handle_401` spawned a dedup'd recovery coroutine via `asyncio.create_task(_do_handle())` and discarded the returned task reference. Python's event loop only keeps weak references to tasks, so the coroutine could be garbage-collected before it called `pending.set_result(...)`. Every concurrent caller awaiting that future then hangs forever, and the `finally: entry.pending_401.pop(...)` cleanup never runs — so subsequent 401s for the same key latch onto the dead future too. Same pattern the adapter-side fixes address (NousResearch#11997, NousResearch#11998, NousResearch#12000, NousResearch#12001, NousResearch#12006). Hold the task in a process-wide set on the manager and discard it via `add_done_callback` once it completes. Regression test covers both the structural invariant (task tracked, then removed on completion) and a concurrent dedup path with a forced `gc.collect()` between the handler's await points.
santhreal
pushed a commit
to santhreal/hermes-agent
that referenced
this pull request
Jul 13, 2026
`handle_401` spawned a dedup'd recovery coroutine via `asyncio.create_task(_do_handle())` and discarded the returned task reference. Python's event loop only keeps weak references to tasks, so the coroutine could be garbage-collected before it called `pending.set_result(...)`. Every concurrent caller awaiting that future then hangs forever, and the `finally: entry.pending_401.pop(...)` cleanup never runs — so subsequent 401s for the same key latch onto the dead future too. Same pattern the adapter-side fixes address (NousResearch#11997, NousResearch#11998, NousResearch#12000, NousResearch#12001, NousResearch#12006). Hold the task in a process-wide set on the manager and discard it via `add_done_callback` once it completes. Regression test covers both the structural invariant (task tracked, then removed on completion) and a concurrent dedup path with a forced `gc.collect()` between the handler's await points.
Gravezzz
pushed a commit
to Gravezzz/hermes-agent
that referenced
this pull request
Jul 21, 2026
`handle_401` spawned a dedup'd recovery coroutine via `asyncio.create_task(_do_handle())` and discarded the returned task reference. Python's event loop only keeps weak references to tasks, so the coroutine could be garbage-collected before it called `pending.set_result(...)`. Every concurrent caller awaiting that future then hangs forever, and the `finally: entry.pending_401.pop(...)` cleanup never runs — so subsequent 401s for the same key latch onto the dead future too. Same pattern the adapter-side fixes address (NousResearch#11997, NousResearch#11998, NousResearch#12000, NousResearch#12001, NousResearch#12006). Hold the task in a process-wide set on the manager and discard it via `add_done_callback` once it completes. Regression test covers both the structural invariant (task tracked, then removed on completion) and a concurrent dedup path with a forced `gc.collect()` between the handler's await points.
leewenjie
pushed a commit
to leewenjie/hermes-agent
that referenced
this pull request
Aug 7, 2026
`handle_401` spawned a dedup'd recovery coroutine via `asyncio.create_task(_do_handle())` and discarded the returned task reference. Python's event loop only keeps weak references to tasks, so the coroutine could be garbage-collected before it called `pending.set_result(...)`. Every concurrent caller awaiting that future then hangs forever, and the `finally: entry.pending_401.pop(...)` cleanup never runs — so subsequent 401s for the same key latch onto the dead future too. Same pattern the adapter-side fixes address (NousResearch#11997, NousResearch#11998, NousResearch#12000, NousResearch#12001, NousResearch#12006). Hold the task in a process-wide set on the manager and discard it via `add_done_callback` once it completes. Regression test covers both the structural invariant (task tracked, then removed on completion) and a concurrent dedup path with a forced `gc.collect()` between the handler's await points.
melon-xf
added a commit
to melon-xf/hermes-agent
that referenced
this pull request
Sep 3, 2026
`handle_401` spawned a dedup'd recovery coroutine via `asyncio.create_task(_do_handle())` and discarded the returned task reference. Python's event loop only keeps weak references to tasks, so the coroutine could be garbage-collected before it called `pending.set_result(...)`. Every concurrent caller awaiting that future then hangs forever, and the `finally: entry.pending_401.pop(...)` cleanup never runs — so subsequent 401s for the same key latch onto the dead future too. Same pattern the adapter-side fixes address (NousResearch#11997, NousResearch#11998, NousResearch#12000, NousResearch#12001, NousResearch#12006). Hold the task in a process-wide set on the manager and discard it via `add_done_callback` once it completes. Regression test covers both the structural invariant (task tracked, then removed on completion) and a concurrent dedup path with a forced `gc.collect()` between the handler's await points.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What & why
`_poll_loop` spawned `_process_message_safe` with a bare `asyncio.create_task(...)` for every inbound message, and `_process_message` did the same for `_maybe_fetch_typing_ticket`. Python's event loop only keeps a weak reference to tasks returned by `create_task`; the Python docs explicitly warn that untracked tasks can be garbage-collected mid-flight.
The weixin long-poll drains buffered messages in tight bursts, so a GC pass at the wrong moment silently drops the user's message after the sync buffer has already been advanced — the lost message cannot be re-polled.
Same root cause as the DingTalk fix in PR #11997 (separate PR because the adapters don't share infrastructure for this).
Change
Mirror the tracking pattern the DingTalk adapter uses:
How to test
```bash
pytest tests/gateway/test_weixin.py -q
```
→ 44 passed (2 new). Regression tests in `TestWeixinBackgroundTaskTracking`:
Platforms tested
Related
Found during a proactive audit of untracked `asyncio.create_task` sites across platform adapters. Companion to PR #11997 (dingtalk). Further untracked sites exist in `qqbot`, `bluebubbles`, `rl_training_tool` — keeping each adapter its own PR to keep reviews scoped.