Skip to content

W1-C: NodeKeyedGraphArtifact schema + value-size fold authority + sharing witnesses - #6481

Merged
briansrls merged 7 commits into
mainfrom
session/still-seal-748
Jul 11, 2026
Merged

briansrls merged 7 commits into
mainfrom
session/still-seal-748

Conversation

@gunbai-bot

@gunbai-bot gunbai-bot Bot commented Jul 11, 2026 •

Copy link
Copy Markdown
Contributor

Lands the settled W1-C encoding authority (agreement bound 2026-07-10 between the S2a/eval-memo/space-lens lanes; gentle-hawk-426 claimed the format authority then archived — this PR inherits and first-authors it).

Why (the defect this kills)

The resolved-graph cache write path serializes the Rc-shared graph with a tree codec (graph.clone() + serde_json::to_vec), which un-shares every shared subtree: a ~3.4GiB working set expands to a 17GiB encode (bisect receipt 2026-07-10). The codec ruling: serde-streaming and packed-binary-tree keep the unsharing — the correct artifact form is an interned content-keyed node table: each node encoded exactly once, keyed by its content hash, children referenced by hash and never inline; decode rebuilds the sharing. That form is the S2b node-store persistence shape, landing early.

Schema (v2.workflow.realization_runner, additive beside NodeKeyedStore)

  • NodeKeyedGraphRow — content_hash: ContentHash + row-local encoded_length: ByteSize + child_refs: List<ContentHash>. Transitive footprint is derived, never stored (a stored transitive size would be a dual representation of the child-ref topology, §2/§3).
  • NodeKeyedGraphArtifact — interned row table + (store_node, value_root) entries: the persistent/streamed form of S2a move 1: layer-2 runner + node-keyed store; migrate CI floor (byte-identical plan receipt) #6444's NodeKeyedStore.
  • node_keyed_graph_transitive_bytes — the reachable-set fold, the single value-size authority: the eval-memo byte-cap admission (extdeps.realization.eval_memo, slated consumer) and the space-lens predictor bind to it; no parallel size walk may exist. Shared rows count once; a missing child ref refuses with the hash (typed, never a fabricated size); Unsettled is the fuel-bound backstop with its unreachability argument stated on the carrier note.
  • Hashes ground on the fnv1a64 authority (std.content_hash / v1 intrinsics — DESIGN §3 convergence thread; no third surface minted).

Host codec kernel (resolved_graph_cache.rs, hand-maintained)

NodeKeyedGraphEncode trait + node_keyed_graph_encode/node_keyed_graph_decode/node_keyed_graph_row_facts. One bottom-up walk computes hash + row-local size + row (the agreed one-fold condition); rows emit child-before-parent, interned by content hash. The hash-consed single-pass decode rebuilds structural sharing by construction and refuses a forward/missing child ref — an encoded cycle is unrepresentable. Content verified on read (row hash recomputed; mismatch refuses). The cache's cross_process_write rewire onto this encoding is the held follow-on slice (eager-ram-612's consumer lane), not this PR.

Verified by execution

  • .dag witnesses (node_keyed_graph_artifact_witnesses, discovery-enrolled): diamond fixture pins shared-row-counted-once == 25 (RED control run: expecting the tree double-count 32 goes false), subgraph + entry-bound folds, missing-child/missing-root refusals, interning check true/false on interned/duplicated fixtures. Run green via gunbc run --claim-run (both test fns).
  • Host witnesses (7/7 green, v1-compiler-tests node_keyed_graph): decode-rebuilds-sharing via Rc::ptr_eq at both diamond parents with a deliberately unshared tree-decode RED control; reencode byte-identical; equal-content distinct allocations intern to one row and decode shared; row-local length sum matches the .dag fold semantics; bad-magic/truncation/payload-corruption refuse; non-digest store key refuses at encode.
  • Pre-existing surfaces green: cross-process cache suite 11/11, cache-purity 5/5, size-bound 2/2, ir-fixture-seam 3/3, type/func-env scope-chain suites; tree-wide gunbc compile --target dag gate run on the branch; cargo fmt clean.

🤖 Generated with Claude Code

briansrls and others added 3 commits July 11, 2026 03:55
…itnesses

Schema (v2.workflow.realization_runner, additive beside NodeKeyedStore):
NodeKeyedGraphRow = content_hash + row-local encoded_length + child hash
refs (never inline subtrees); NodeKeyedGraphArtifact = interned row table +
store entries. Transitive footprint is DERIVED by the reachable-set fold
(node_keyed_graph_transitive_bytes) — the single value-size authority the
eval-memo byte-cap admission and space lens bind to — never a stored field.
Refusals typed: missing child ref refuses with the hash; Unsettled is the
fuel-bound backstop.

Host codec kernel (resolved_graph_cache.rs, hand-maintained): interned
content-keyed node table — one bottom-up walk computes hash + size + row;
hash-consed single-pass decode rebuilds sharing and refuses forward/missing
refs (encoded cycles unrepresentable). Kills the tree-codec un-sharing
class (serde_json over the Rc-shared graph: ~3.4GiB working set -> 17GiB).

Witnesses by execution: .dag diamond fixture pins shared-row-counted-once
(RED-controlled against the tree double-count), missing-ref refusal,
interning check; host tests pin decode-rebuilds-sharing (Rc ptr-eq, with a
deliberately unshared tree-decode RED control), reencode byte-identity,
content interning across distinct allocations, corrupt/truncated refusals.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@gunbai-bot gunbai-bot Bot changed the title W1-C NodeKeyedGraphArtifact schema + sharing witnesses. W1-C: NodeKeyedGraphArtifact schema + value-size fold authority + sharing witnesses Jul 11, 2026
@gunbai-bot
gunbai-bot Bot marked this pull request as ready for review July 11, 2026 04:14
@gunbai-bot

gunbai-bot Bot commented Jul 11, 2026

Copy link
Copy Markdown
Contributor Author

Addressed the cursor REQUEST_CHANGES in 58af9ce (all three findings valid):

1+2 (hand-Rust scaffold discipline): the codec kernel and its test module are now a declared §7 seed-retained frontier row — node_keyed_graph_codec_seed_disposition (Scaffold, dissolves_to: RealizationDispatch, bound to the NodeKeyedGraphArtifact declaration) plus node_keyed_graph_codec_seed_note carrying the reason (byte-level IO and Rc pointer identity are realization facts the substrate cannot express today — no bytes carrier, and pointer sharing is deliberately not model-carried) and the concrete dissolution trigger: when the §4 one-grammar-both-directions emission rows extend to a binary medium (bytes carrier + row table for this format), the kernel becomes row-derived emission dispatched from the modeled schema and retires with the seed. Matching seed-retained/dissolve-on markers added to both Rust surfaces.

3 (witness-home misroute): the artifact note now routes precisely — size/topology/refusal witnesses in v2.test.claim.realization_runner_witness; the two sharing witnesses are pointer-identity facts only a host realization can observe, so they live in the seed test module node_keyed_graph_artifact_codec_test.rs, and migrate with the kernel when its dissolve-on fires.

Re-verified after the edits: .dag witnesses green via claim-run, 7/7 host codec tests green, fmt clean.

— sent from still-seal-748

@gunbai-bot

gunbai-bot Bot commented Jul 11, 2026

Copy link
Copy Markdown
Contributor Author

CI red on this PR (run 29139666284) is a fleet-wide infra class, not this diff: claim_executor panics with 'build_module_path_index: path .../srv2-02/_work/... is not under workspace .../srv2-04/_work/...' — a binary with a compile-time-baked CARGO_MANIFEST_DIR from one runner slot served to a job in another slot. Main shows the identical signature since 04:15Z (run 29139313054, srv2-01 vs srv2-02), before this PR's head existed. All PR witnesses are green locally (.dag claim-run + 7/7 host codec tests + tree-wide compile gate, 0 diagnostics). Escalated to the coordination point; will rerun CI once the runner cache skew is fixed. — sent from still-seal-748

@gunbai-bot

gunbai-bot Bot commented Jul 11, 2026

Copy link
Copy Markdown
Contributor Author

CI red on the new head 03fdb37 (run 29143294136) is a different pre-existing fleet class than the earlier slot-mismatch panic: build job passed (2m11s), the ci floor job cleared startup and the batch walk, then hit the 60-minute job timeout mid-corpus — the same timeout class main itself has shown since 01:40Z (the #6469 evidence-doc incident). No witness failed; no fix exists in this diff (its additions execute in milliseconds: one .dag claim fn + 7 host tests). Merge criteria otherwise all green: 2/2 distinct approvals (non-stale across the main-integration merge), no request-changes, mergeable. Will rerun when the fleet floor-performance work (or #6482 for the panic class) lands. — sent from still-seal-748

@briansrls
briansrls merged commit ccf153e into main Jul 11, 2026
1 of 2 checks passed
@briansrls
briansrls deleted the session/still-seal-748 branch July 11, 2026 19:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant