Close the V4.1 Engram membership against the full index (stacked on #12010)

[gunbai-bot]

Closes the V4.1 Engram membership against the full index. The .engram. selection is now read from the pinned model.safetensors.index.json by extdeps.languages.json parse_json_document, with no scaffold.

What changed in this revision (operator ruling, option A). Earlier heads read the index with a token scan, a stated divergence kept because the parser could not read a 7.4 MB index in the interpreter. After #12066 the ruling was to drop the scaffold rather than admit it.

• The scan is gone. The token scan, its local diagnostic and the parser probe (v41_index_parse_json_probe_wet) are deleted. gunbc.spark.v41_index_engram_selection v41_select_index_document is a fold over the parsed weight_map. It refuses a non-JSON document, a missing or duplicated weight_map, a map that is not an object, any value that is not a .safetensors shard name, and an empty map. Escaped keys are decoded, where the scan refused any backslash. The result type is renamed from V41IndexScan to V41IndexParse so the old name does not change meaning.
• The index read is its own step. gunbc.spark.v41_checkpoint_materialize v41_index_selection_ci_wet runs as a second step of spark_v41_checkpoint_materialize under always(). A parse stopped for memory therefore cannot take the fetch's per-file receipts with it. It verifies the index against its manifest row itself (v41_observe) before reading it.
• One CI bracket. Both entries share v41_group_a_ci_run (target, Group A admission, reach, credential, trust, ssh key), instead of two copies of that setup.
• The parse runs only inside a hard memory fence. v41_memory_fence reads /proc/self/cgroup for the process that will parse, plus memory.max and memory.high at every bounding level. It takes the tightest hard limit using extdeps.linux.cgroup_v2_memory cgroup_tightest_hard_memory_limit, the kernel's minimum-over-ancestors rule. memory.high alone is a located refusal, because it is a throttle an allocation may cross, not a fence. The fence (directory, bytes, and every level read) goes in the step's receipt, so the first run that parses also proves what bounded it.

Why not gunbc.memory_cgroup_binding. It was tried and dropped. cgroup_bind_decision returns UnnecessaryLimitAlreadyBinds under any existing hard memory.max, and elsewhere it creates its leaf at the cgroup root, which a non-root runner cannot do. On the executor it would have been a no-op or a refusal.

Follow-up, not in this PR (B). Extend gunbc.memory_cgroup_binding to nest a tighter child cgroup under an existing hard limit. That would give a read a bound of its own inside the runner slot, rather than relying on the slot's grant.

Evidence so far. On a remote runner, the parse-fold witnesses over v41_index_engram_selection (selection, escaped key, not-a-shard and empty-map refusals) evaluated true. Closures that include the materialize module were OOM-killed on that 7 GB runner, so CI's floor is the check for those. The live proof is the next srv5 run after this lands (srv5 holds the verified index).

🤖 Generated with Claude Code

[gunbai-bot]

Addressed review 69728 in 61355eb. The finding was right: the guards compared the pin's symbols with themselves, and the only evidence was prose.

• The selection is now an executed observation, and it lives in the observing layer. gunbc.spark.v41_index_engram_selection v41_index_engram_selection_read_wet digests a local copy of the index and scans it inside the corpus. The scan reads exactly the published shape and refuses anything else: an escape, a different opening, or a value that isn't a shard. The receipt's own values are transcribed as literals into v41_index_engram_selection_reading: sha256 74b0686a…, 7,470,294 bytes, 96,085 names and the 12 members, from the run on 2026-09-21.
• The guards now discriminate. deepseek_v4_1_flash_engram_membership_closure_of compares that reading against the extdeps pin, so a reading of another file or over fewer names stays scoped (only_a_reading_of_the_pinned_file_over_every_name_closes). A 13th member refuses by name (a_reading_with_a_thirteenth_engram_name_refuses).
• extdeps carries no live selection. Its closure is back to MembershipScopedToReadRows, and the candidate consumes v41_engram_population_observed().
• Honest limit, stated on the carrier: the reading is a transcription of an executed receipt, like the span readings. The scan diverges from extdeps.languages.json with a stated reason: parse_json_document had not finished the 7.4 MB file when a 15-minute timeout stopped it. The scan dissolves into that parser once it can read a file this size.

— sent from sleek-bee-348

[gunbai-bot]

Addressed review 69785 in 71afc99. Both findings were right.

1. index_bytes is now a ByteSize, and the use site no longer re-wraps it.
2. The transcribed reading is gone. My local diagnostic run of the scan is not an enrolled route, so its output no longer lands: v41_index_engram_selection_reading is Unread, and its obligation names the route. That route is now real. spark_v41_checkpoint_materialize verifies the index against its manifest row, reads its digest and size back on the host, fetches the bytes and runs v41_scan_index_text over them. The receipt's index-selection lines are the transcription source, with the same standing as the span readings in DS4.1 checkpoint on Group A: spark_v41_checkpoint_materialize mode and the published row-store encoding key #12010. Until a dispatched run's receipt lands, the observed population stays scoped and the candidate's Engram axis stays unestablished. This PR no longer keys anything by itself.
   • What stays: the extdeps fold (a reading compared against the pin, where a 13th member refuses), the scan with its refusals, and the candidate pair, which is now driven by a supplied reading.

— sent from sleek-bee-348

[gunbai-bot]

Addressed review 69820 in a82ca86.

1. Quadratic pairing. Fixed. v41_pairs_of is now one fold with constant state: the open name, the name count, the non-shard count and only the selected members. It neither re-indexes the token list per element nor copies a per-pair accumulator. Over the pinned index it completes in about 2m12s (96,085 names, 12 members). An intermediate fold that pushed every pair did not finish in 15 minutes, which confirmed the cost shape was real. A name left without a shard refuses.
2. The size read failed open. Fixed. V41IndexSelectionRead.bytes is now a ByteSize read by v41_wc_bytes. wc output that isn't a count leaves the index unread, a typed refusal, rather than producing an empty size on a line that counts as success. Control: an_unreadable_index_size_is_no_size.

— sent from sleek-bee-348

[gunbai-bot]

Review 70102 — no code change, and deliberately so. The one blocking item is the scaffold admission, and the doctrine says that admission cannot come from this diff.

docs/plans/scaffold-admission-doctrine.md: "Approval is external to the diff: a row the PR authors itself is not approval — the author could write an approved flag as easily as the scaffold." So writing an admission row here would defeat the check it is meant to satisfy. The review reads that correctly and the PR is waiting on it rather than routing around it.

Status of that admission: filed as an operator card, bound to this exact head 5fb181505d3. The doctrine invalidates admission on any later push, which is why the six-traversal repair landed first and the card followed — and it is why I am replying here rather than pushing anything now. Nothing further is coming from me on this PR while the card is open.

The premise is no longer asserted. When the divergence was first proposed it rested on a transcribed figure with no producer, which review 70075 correctly refused under §6. Both sides have since been executed against the pinned artifact — sha256 74b0686a3d2891980d5e303251b075a3bccae2c2ff650747db2620a649b98fa8, 7,470,294 bytes, matching deepseek_v4_1_flash_safetensors_index_file on both size and digest:

• parse_json_document: 32:02 elapsed, 8.1 GiB RSS, no return, stopped deliberately at a declared point (not an OOM, not a timeout).
• v41_scan_index_text: returned in 2m42s, names=96085, the 12 expected members in shards 47 and 48.

Re-derive either with v41_index_parse_json_probe_wet / v41_index_engram_selection_read_wet, GUNBC_V41_INDEX_PATH naming a local copy. The artifact is named by its digest and is re-fetchable from hub_resolve_url at the pinned revision — no path in my container is the reference.

That measurement is also what takes RejectAndFinishNow off the table: folding over parse_json_document would put a non-returning call on the converge mode's success gate. Review 70086 anticipated exactly this branch — "if the parser reads it, the scan deletes; if it does not, the finding moves upstream to the parser." It did not read it, and that upstream finding is now its own dispatched lane rather than something patched around here.

Review 70086's other finding is fixed in 5fb181505d3: the six materialisations of the ~192k-element token list are one fold, with quoted carried as fold state. Worth stating plainly — it did not make it faster (2m42s → 2m47s, byte-identical receipt both runs). The estimate I would have made was falsified and the null is kept; the change stands on §6 bare-minimum-cost, which fixes a cost-shape defect at any n without needing a percentage.

— sent from sleek-bee-348

[gunbai-bot]

The emit-build red on d337d67 is not this PR's. That lane landed in #12043 as present and not yet required, and it is failing fleet-wide right now: the same step (emit and build //gunbc/instruments:self-host, exit 2) is red on royal-otter-431, cool-dove-221, nimble-boar-636-pkg16, nimble-eagle-216-step3, bold-lynx-559-pkg4-lifecycle, sleek-bat-538 and adoring-edison-o4nz5e, none of which touch the V4.1 modules. This PR's required lanes are compiler, clippy, floor and witnesses; compiler and clippy pass, floor is running.

— sent from sleek-bee-348