fix(coding-agent): single-dump kernel snapshots - #1945
Merged
Merged
Conversation
The kernel snapshot pipeline serialized the aggregate payload dict into an in-memory buffer, copied it out with getvalue(), and only then wrote the file - on top of the per-variable blob dict, a ~3.9x-payload transient memory spike on every debounced snapshot (measured 963MB overhead for a 250MB namespace). The aggregate dict now pickles straight into the staged temp file through a size-capped pass-through writer (measured 253MB overhead, ~1x payload); the strict max_bytes contract and the largest-fitting-prefix fallback are preserved by rewinding the temp file between prefix attempts. _SnapshotBuffer is replaced by the same _CappedWriter over a plain BytesIO for the per-variable dumps.
xeophon
approved these changes
Sep 1, 2026
olety
added a commit
to oneiron-dev/prime-agent
that referenced
this pull request
Sep 1, 2026
Takes upstream's event-driven supervisor roster ledger + push (PrimeIntellect-ai#1897, PrimeIntellect-ai#1900), direct TUI<->worker transport (ENG-5817), daemon startup/recovery hardening (PrimeIntellect-ai#1929, PrimeIntellect-ai#1909), single-dump kernel snapshots (PrimeIntellect-ai#1945), empty-draft eviction (PrimeIntellect-ai#1946), rlm_child_update suppression (PrimeIntellect-ai#1944), bash-skill preview (PrimeIntellect-ai#1911). Fork laws re-expressed on the roster architecture: - stable-target follow-up honesty kept (capability proof via worker hello, target_unavailable never not_found when ownership unproven) - schema revision 26 (union of fork rev-24 stable-target + upstream rev-24/25 roster+transport); digest minted by the repo's own algorithm - summary freshness reuse + single-flight + staleness + root-omission rejection restored on upstream's refresh pull - adoption/recovery never fails a live worker on a slow or root-omitting catalog: get_state root seed + stale mark + bounded background rehydration - repl.py keeps fork prune-on-aggregate-overflow - delete handlers keep fork persistence reporting; eviction fence test keeps the stronger two-worker contention variant Known test debt (deferred to post-Wave cleanup per owner): roster-era fixture migrations in daemon-supervisor-monitor (2), plus un-triaged failures in package-command-paths, agent-session-recursion, daemon-runtime-stress, 4600-supervisor-singleton, 4603-worker-recovery, 4606-update-restart- coordinator, agents-view-roster. Production laws preserved; failures are fixture-era artifacts or mechanism assertions to rewrite.
ketema
added a commit
to ketema/prime-agent
that referenced
this pull request
Sep 1, 2026
- Direct session transport between TUI and worker (ENG-5817, PrimeIntellect-ai#1926) - Event-driven supervisor agent roster with push subscriptions (PrimeIntellect-ai#1897, PrimeIntellect-ai#1900, PrimeIntellect-ai#1895) - Hardened daemon startup, recovery ownership, and worker launch diagnostics (PrimeIntellect-ai#1929, PrimeIntellect-ai#1918) - Python REPL runtime single-dump snapshots and bash preview tool (PrimeIntellect-ai#1945, PrimeIntellect-ai#1911) - Non-blocking RLM subagent deletion and snapshot update suppression (PrimeIntellect-ai#1954, PrimeIntellect-ai#1944) - Saved catalog loading on Agents View open (PrimeIntellect-ai#1960) - Advanced Anthropic prompt caching marker across tool results (PrimeIntellect-ai#1927) - TUI process replacement on update and empty draft eviction (PrimeIntellect-ai#1631, PrimeIntellect-ai#1946, PrimeIntellect-ai#1920)
paralin
pushed a commit
to paralin/prime-agent
that referenced
this pull request
Sep 2, 2026
* fix(coding-agent): single-dump kernel snapshots (ENG-5819) The kernel snapshot pipeline serialized the aggregate payload dict into an in-memory buffer, copied it out with getvalue(), and only then wrote the file - on top of the per-variable blob dict, a ~3.9x-payload transient memory spike on every debounced snapshot (measured 963MB overhead for a 250MB namespace). The aggregate dict now pickles straight into the staged temp file through a size-capped pass-through writer (measured 253MB overhead, ~1x payload); the strict max_bytes contract and the largest-fitting-prefix fallback are preserved by rewinding the temp file between prefix attempts. _SnapshotBuffer is replaced by the same _CappedWriter over a plain BytesIO for the per-variable dumps. * chore(coding-agent): strip narrative comments from the snapshot dump path
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Saving kernel state — the namespace snapshot that lets a resumed session keep its variables — buffered the whole pickled namespace in memory up to three times over before writing it to disk: each variable was pickled into a dict of blobs, that dict was pickled again into a second in-memory buffer, and that buffer was copied once more for the file write. A snapshot runs shortly after every successful cell, so long sessions with sizeable variables repeatedly spike to roughly 4x the payload in transient RSS. This is the kernel-side memory spike tracked in ENG-5819 (kernel processes ballooning to 1.3-1.9GB and getting OOM-killed).
The fix deletes the in-memory aggregate stage: the dict of blobs now pickles directly into the staged temp file through a small size-capped writer (the same class replaces the old per-variable buffer). The on-disk format, the manifest, the
max_bytescap semantics, and the overflow fallback that keeps the largest prefix that fits are all unchanged; overflow retries rewind and rewrite the temp file instead of allocating fresh buffers. No host or protocol changes.Measured on a 250MB namespace (25 x 10MB variables, fresh process per side, 3 snapshots,
ru_maxrss):The remaining ~1x overhead is the per-variable blob dict, which pickling the aggregate genuinely needs. The snapshot still runs synchronously on the kernel loop; moving it off-loop is out of scope here because that ordering is what keeps the namespace stable while it pickles.
Validation:
prime-agent-runtimepython -m unittest test.test_repl95/95 with no test changes (the cap and overflow-fallback behaviors are already pinned there); coding-agent kernel suites (repl-kernel-state-roundtripagainst the real runtime, execute/abort/shutdown/startup/protocol-corruption,npm run test:kernel) green; rootnpm run checkgreen.Linear: ENG-5819
Note
Medium Risk
Touches the kernel snapshot write path used after every successful cell; behavior is intended to be equivalent but mistakes could truncate or mis-record state under size caps.
Overview
Fixes ENG-5819 by changing how
_snapshot_statewrites the aggregate namespace payload so the kernel no longer holds multiple full-size serialized copies in RAM on every debounced snapshot.The in-memory
_SnapshotBufferpath is replaced with_CappedWriter, which enforces per-dump byte limits while streamingdilloutput into a sink (stillBytesIOfor per-variable blobs, the staged temp file for the full payload dict). Aggregate overflow handling keeps the same largest-prefix binary search and skip reasons, but retriesseek/truncate/redumpon that temp file instead of re-pickling into fresh buffers.Snapshot caps, manifest fields, on-disk format, and host protocol are unchanged; this is a memory/latency optimization inside the REPL kernel.
Reviewed by Cursor Bugbot for commit bffb62d. Bugbot is set up for automated code reviews on this repo. Configure here.
Note
Stream kernel snapshot payload directly to temp file instead of memory
_SnapshotBuffer(BytesIO wrapper) inprime-agent-runtime/src/rlm/repl.pywith_CappedWriter, a capped writer that streams writes to an arbitrary sink while enforcing a byte limit._snapshot_statenow serializes the payload directly into the staged temp file using_CappedWriterinstead of callingserialize()to build a full in-memory copy first. This reduces kernel memory spikes during namespace snapshots.seek(0)/truncate();bytes_writtenis sourced from the writer's running count rather thanlen(serialized)._SnapshotBufferis removed; any code importing it directly will break. Per-variable serialization now usesio.BytesIO+_CappedWriterinstead of_SnapshotBuffer.Macroscope summarized bffb62d.