refactor(coding-agent): extract the append-only event-log substrate from the RLM spawn ledger - #1987
Merged
Merged
Conversation
…rom the RLM spawn ledger One EventLog owns the shared crash-safety mechanics: single O_APPEND writes with optional fsync, bounded fail-closed reads, torn-final-line tolerance on replay, and repair-on-append (byte-offset truncate for an unparseable tail, newline completion for a parseable one). RlmSpawnLedger keeps spawn semantics only; its public API and test suite are unchanged.
…vent-log substrate Reject unserializable events with a TypeError before any byte (including repair) is written; truncate EVERY unterminated tail instead of newline-completing a JSON-parseable one, which would hand a line a strict consumer parser rejects to every later replay as fail-closed interior poison; read through a bounded descriptor so a concurrent grow between size check and read cannot bypass maxBytes.
sethkarten
self-requested a review
September 2, 2026 15:22
sethkarten
approved these changes
Sep 3, 2026
This was referenced Sep 3, 2026
sethkarten
added a commit
that referenced
this pull request
Sep 7, 2026
) * refactor(coding-agent): move the semantic-edge ledger onto the event-log substrate The recorder's private append/replay/repair IO is deleted; EventLog owns it, the same move #1987 made for the RLM spawn ledger. One durability rule is unified in the substrate rather than dropped: an unterminated final line is an uncommitted append, skipped on read and truncated before the next append — never newline-completed and never surfaced to a consumer whose next append destroys it. * fix(coding-agent): make the explicit ledger reader's ENOENT contract atomic readSemanticEdgeLedger probed with statSync before reading through EventLog, which swallows ENOENT; a ledger deleted between the two returned [] instead of throwing. The missing-file decision now lives at the single open (replaySync missingFileThrows), so no check-then-read window exists. * docs(coding-agent): state the event-log tail rule once The unterminated-tail contract was restated four times (module doc, replaySync doc, two test comments). It now lives once in the module doc; the method doc keeps only its own parse/missing-file semantics and the test comments reference the contract. * fix(coding-agent): write event-log appends fully and gate appends on tail repair writeSync may write short (ENOSPC after a prefix); appendSync now loops until the payload is fully on disk so write-before-action callers never act on a torn record reported as success. A tail-repair failure (e.g. append-only ACL permitting O_APPEND but not r+) now propagates instead of being swallowed: writing through an unrepaired torn tail would weld it to the new record as permanent interior corruption. ENOENT and the concurrent-writer instability path keep their existing semantics. * fix(coding-agent): reclaim short event-log writes instead of completing them The rlm spawn ledger is multi-writer by documented design (supervisor plus each worker over one file), so completing a short O_APPEND write with a second write could interleave with a rival append and weld two records. A short write now truncates its own torn prefix back off (only while this writer still owns the tail) and fails the append; a torn tail is read-tolerated, a weld is permanent corruption. The append fd opens a+ so the ownership check can read the tail. * fix(coding-agent): leave the torn tail on a short write instead of reclaiming it The tail-match reclaim could truncate a rival's committed record whose final bytes coincide with our torn prefix - committed-data loss, strictly worse than the torn tail it prevented. A short write now just fails the append: the torn tail is the one tolerated shape, skipped on read and truncated by any writer's next repair (verified for both topologies: a resumed single-writer recorder repairs on its first append; every rlm-ledger writer repairs before each append). * refactor(coding-agent): compress event-log comments * fix(ai): omit the default service tier, reprice cache writes from message_delta, repoint the zai default Incorporates #2032 at f82c7fa. * fix(tui,coding-agent): survive lone surrogates in table cells and terminate the WebP EXIF scan Incorporates #2033 at a3d1139. * fix(coding-agent): restart dead kernels on ensure() and read mcp>=2 tool schemas Incorporates #2034 at 749e216. * fix: one crash-safe owner for durable state writes Incorporates #2035 at f0f02d2. * fix(coding-agent): one zombie-aware process-liveness probe Incorporates #2041 at 92a0eac. * fix(coding-agent): snapshot transfer ids from the materialized cursor; mismatches settle the transfer, not the worker channel Incorporates #2044 at 5af3bbe. * fix(coding-agent): failed workers recover on touch; roster gaps answer a structured recovering error Incorporates #2047 at 77b747a. * fix(coding-agent): seven session and IO correctness defects Incorporates #2037 at 41b5d72. * fix(coding-agent): coalesce child-usage attribution and gate agent-status persistence on real changes Incorporates #2050 at 6b0af5d. * fix(coding-agent): incremental single-flight session metadata scans Incorporates #2043 at df032c1. * fix(coding-agent): memoize the passive RLM topology derivation Incorporates #2051 at 0ee114c. * fix(coding-agent): preserve accounting and metadata across deferred updates Keep durable child-usage aggregates separate from pending sibling usage. Retry optional topology metadata after transient reads. Completes #2050 and #2051 integration. * fix: preserve session accounting and read-only persistence boundaries --------- Co-authored-by: Seth <seth@primeintellect.ai>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The daemon records every agent spawn, rename, and delete in an append-only JSONL spawn ledger: one file per sessions dir, appended concurrently by the supervisor and every session worker, and replayed to rebuild family and roster topology instead of re-deriving it from session files. Any of those writers can die mid-append, so the ledger carries a real crash-safety discipline — appends must be single atomic writes, a crashed writer's torn final line must be tolerated on read and repaired before the next append, and reads must be bounded so a bloated or hostile file can never trigger an unbounded allocation.
That discipline is not spawn-specific, and it had already been written a second time, independently, for another append-only ledger. Two private copies of subtle crash-recovery code is how the guarantees drift apart. This PR extracts the discipline into one shared substrate,
src/core/event-log.ts(184 lines), and migrates the spawn ledger onto it.rlm-ledger.tsshrinks from 884 to 784 lines and keeps only spawn semantics; its public API is unchanged, and its entire 26-test suite passes unchanged — that unchanged suite is the migration proof. Net: +84 lines of src (+184 substrate, −100 in the ledger), plus 52 lines of substrate tests and a changelog entry.The substrate
EventLogtakes a path and optional bounds (maxBytes,maxRecords, and alogsink for repair and tolerance messages), and offers three operations, each carrying one crash-safety guarantee:Validated appends. Every event is serialized before the file is touched, and an unserializable event (
JSON.stringifyreturnsundefined) throws aTypeErrorbefore any byte — including repair — is written. The payload, plus a caller-supplied header when this append creates the file, goes out as oneO_APPENDwrite, optionally fsynced. The multi-writer design relies on the atomicity of single smallO_APPENDwrites, so the substrate writes exactly once per append and never splits records across writes.Repair on append — never on read. A crashed writer can leave a final line missing its newline. Before appending to an existing file, an unterminated tail is truncated at its byte offset, guarded by a double-read stability check so a log that is changing under us is left alone; the byte bound is checked before the repair reads anything, and any repair failure leaves the tail for the reader's tolerance. Because repair happens only on append, a viewer can always replay a live writer's log without mutating it.
Bounded, fail-closed replay. Replay reads through the file descriptor, so the size check and the allocation see the same fd and a log that grows concurrently cannot bypass
maxBytes;maxRecordsfails closed the same way. A missing file replays as empty, a malformed interior line fails closed, and exactly one torn final line — rejected by the consumer's parser and unterminated, the signature of a crashed writer — is logged and tolerated. What counts as valid is the consumer's call: replay takes a parse callback that throws to reject a line or returnsundefinedto deliberately skip one (the ledger skips records with unknown ops).All offsets are byte offsets on raw buffers: string indices diverge from byte offsets as soon as any record carries multi-byte UTF-8, and
ftruncatetakes bytes.The union of safety behaviors
The rule that shaped the substrate: when two implementations of the same crash-safety discipline are unified, the survivor must keep the stronger guarantee of each — the union of their safety behaviors, never the intersection. Three properties of the substrate exist because one side was weaker there; one weakness was the other implementation's, two sat in the spawn ledger's own pre-extraction code and are visible in this diff's deletions:
Truncate, never complete. The friendlier-looking repair for an unterminated tail that still parses as JSON is to complete it with a newline. But a completed line that the consumer's strict parser rejects — well-formed JSON with the wrong fields, like a record missing
v/at— becomes permanent fail-closed interior poison for every later replay. Truncation is safe for strict and lenient parsers alike, and the torn bytes were never readable data. An earlier revision of this change completed parseable tails; review caught the poison, and every unterminated tail is now truncated.Validate before any byte. An append path that interpolates
JSON.stringify(record)unchecked writes the literal textundefinedinto the log for an unserializable record, poisoning every replay. The substrate throws before the file is touched.Read through the fd, everywhere. The old replay checked the size with
statand then read the file by path — a window in which a concurrent grow turns a bounded read into an unbounded allocation. The repair path already read through the descriptor; replay now does too.All three are pinned in
test/event-log.test.ts: theTypeErrorfires before any byte reaches the log, a JSON-parseable unterminated tail is truncated so a strict replay stays clean, and an oversized log fails closed through the descriptor on both the replay and append paths.The migration
RlmSpawnLedgernow delegates its file mechanics and keeps its semantics:appendSyncwithdurable: true(the ledger has always fsynced) and anonCreatehook that emits the meta header, so header and first record leave in the same single write.replaySync(parseLedgerLine)under the ledger's own bounds (RLM_LEDGER_MAX_BYTES= 32 MiB,RLM_LEDGER_MAX_RECORDS= 100,000); the edge-map reconciliation stays in the ledger.link()publish, and family/sibling topology.The public API is unchanged — the export list is identical, and the only src consumers,
daemon-mode.tsanddaemon-supervisor.ts, are untouched.test/rlm-ledger.test.ts— 26 tests covering torn-final-line tolerance (but not mid-file), byte-offset repair that preserves multi-byte UTF-8 records, fail-closed malformed lines, byte and record bounds, unknown-op skipping, durable-append admission failure, and the full seeding matrix — is untouched by this PR and passes as-is.Validation
1481de4a2), sanitized environment: the newevent-logsuite (3), the unchangedrlm-ledgersuite (26), and the ledger's consumers —daemon-agent-roster,agent-roster,daemon-session-list,daemon-supervisor-monitor,daemon-supervisor-lazy-subagents(200 tests across those 7 files) plusdaemon-mode(184): 384 tests, 0 failures.npm run check(biome, tsgo, installer render, browser smoke) passes..changes/event-log-substrate.md.Linear: https://linear.app/primeintellect/issue/res-1254