fix(gateway): offload all blocking atomic_json_write calls from async paths - #83951
Merged
kshitijk4poor merged 4 commits intoAug 11, 2026
Conversation
atomic_json_write() calls os.fsync(), which blocks until the write reaches stable storage. build_channel_directory() already offloads its builders with asyncio.to_thread (NousResearch#60794) but still called the persist step directly on the loop, so the Discord heartbeat waited on a disk flush.
Mirrors test_discord_builder_runs_off_event_loop_thread. Verified to FAIL against unpatched v0.19.0 and pass with the fix.
Completes the bug class from NousResearch#83906 — the same blocking fsync-on-event-loop pattern existed in two more async gateway paths: - slash_commands.py _handle_restart_command: two atomic_json_write calls for .restart_notify.json and .restart_last_processed.json were blocking on fsync inside an async function. Now offloaded via asyncio.to_thread. - run.py _clear_restart_failure_count: called from _handle_message_with_agent (async, per-turn path) after a successful agent turn. Made the method async and offloaded the atomic_json_write call via asyncio.to_thread. Caller updated to await. Shutdown-path calls in _stop_impl_body (_increment_restart_failure_counts, planned restart notification marker) are intentionally left synchronous — the event loop is draining/stopping and offloading adds complexity for no benefit.
alt-glitch
added
type/perf
Contributor
Related: this is a broader competing repair to #83906. It includes that channel-directory offload and also moves the restart-path writes off the gateway event loop. |
kshitijk4poor
enabled auto-merge (rebase)
August 11, 2026 14:56
1 task
This was referenced Aug 12, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Completes the bug class from #83906 — offloads all remaining blocking
atomic_json_writecalls that run inside async gateway paths, using the sameawait asyncio.to_thread(...)pattern already established inbuild_channel_directory().Root cause:
atomic_json_writecallsos.fsync(), which blocks until the write reaches stable storage. When called directly inside an async function on the gateway event loop, discord.py's heartbeat task waits behind the flush.Changes
gateway/channel_directory.py: offload the persist step withasyncio.to_thread(@landaun's original commit, cherry-picked from fix(gateway): run the channel-directory write off the event loop (completes #60794) #83906)tests/gateway/test_channel_directory.py: test asserting the write runs off-loop (@landaun's original commit, cherry-picked from fix(gateway): run the channel-directory write off the event loop (completes #60794) #83906)gateway/slash_commands.py: offload twoatomic_json_writecalls in_handle_restart_command()—.restart_notify.jsonand.restart_last_processed.jsonwere blocking on fsync inside an async functiongateway/run.py:_clear_restart_failure_count()made async +atomic_json_writeoffloaded viaasyncio.to_thread; caller in_handle_message_with_agentupdated toawaitSites intentionally left synchronous
Two
atomic_json_writecalls inside_stop_impl_body(shutdown path:_increment_restart_failure_countsand planned restart notification marker) are left synchronous — the event loop is draining/stopping during shutdown and offloading adds complexity for no benefit.Validation
Closes #83906
Completes #60794