V4.1: Engram arms by image config digest (A upstream pinned, B production file-backed); production image receipt row; differential bound to it; DeepSelect as image capability

[gunbai-bot]

Follow-up to #12303, based on proud-deer-538's report that the arm A image is built.

• Arm A is bound. The binding points at the pre-V4.1 Engram file-backed: release verify page cache (fail-closed on residue); batched fetch_rows gather #12304 srv8 image, fleet-converge run 36207135528, config sha256:ea39410ed01d396caad86339d877c1adb181a983baadffd22c722069109ea664. Arms are now keyed on the image config digest, which is the same digest every rank's startup receipt carries (V41OciImageDigest), so a receipt's arm can be read off its own image. Previously they were keyed on a patch digest. An image with zero or more than one binding has no arm. v41_engram_arm_binding_frontier now names only arm B, the image built from main after V4.1 Engram file-backed: release verify page cache (fail-closed on residue); batched fetch_rows gather #12304.
• DeepSelect as an image capability fact. The image logged Failed to import the DeepSelect extension (vllm._deepselect_C), which is the indexer top-k kernel, so vLLM may take a slower fallback that could dominate TTFT at 262k. Each rank's startup reading now carries V41ImageExtensionStanding: ExtensionLoaded, ExtensionImportFailed { logged } (recorded and admitted), or ExtensionUnobserved, which refuses by name like every other missing reading.

Evidence: claim_batch … --entry dag/test/claim/spark/v41_capacity_measurement_witness_test.dag gives 25/25 PASS in-session. Two new claims cover the arm resolution (bound versus unbound image) and the DeepSelect refusal.

🤖 Generated with Claude Code

[gunbai-bot]

Review 71462 (/api/reviews/71462/artifacts/stdout.log). Taking the two observations in order:

• DeepSelect result has no reader: fixed in b33399f. V41CapacityReport now carries indexer_kernel (via v41_report_indexer_kernel). If any rank's import failed, the report states it with that rank's logged line. A refused startup receipt leaves it ExtensionUnobserved. A new claim covers both arms; 26/26 pass in-session.
• Arm A digest typed in by hand: keeping it as is. It is the full config digest from the build run's report, with the build run (36207135528) recorded beside it as provenance. Reading it from the image directly belongs to the wet fleet mode, which reads each rank's image digest into its startup receipt. v41_engram_arm_of then joins that observed digest against this binding, so a mistyped digest makes every arm-A cell unresolvable. It cannot resolve silently to the wrong arm.
  — sent from clever-gull-48

[gunbai-bot]

Hold: do not merge. The arm A binding here is a false fact. The image from run 36207135528 does not serve file-backed Engram: on main, ParallelEngramEmbedding._allocate_weights still constructs upstream DPSharedEngramStorage (engram_storage_backed.patch only imports the file-backed names). So binding it as EngramCacheRetained misnames what that image runs. I've proposed a fix to proud-deer-538: rename the arm to upstream-pinned, and make the _allocate_weights cutover its own PR. I'll rework this PR once that's decided.
— sent from clever-gull-48

[gunbai-bot]

Reworked and un-held at 47c53a3. main is merged in, and the arms are renamed to what the images actually serve, as agreed with proud-deer-538:

• EngramUpstreamPinned (arm A): the image from run 36207135528. Every image built before the design-B cutover (Design-B Engram cutover: file-backed lookup bound in the overlay, launch manifest derived, differential fleet mode (the RED) #12359) serves upstream DPShared/pinned storage, whatever file-backed code it carries.
• EngramFileBackedReleased (arm B): the first image built after the cutover.
  The false EngramCacheRetained binding is gone. There's deliberately no file-backed arm that keeps the cache, because the release is unconditional. 28/28 v41_capacity_measurement_witness_test pass in-session.
  — sent from clever-gull-48

[gunbai-bot]

Extended at 1545847 with the production V4.1 image (fleet-converge run 36327364664 on srv8: tag gunbc-vllm-dsv41-gb10:3cedb66f0ec0869e, config sha256:5de3c355…, source_head d2d649e6, worktree_diff_sha256 e967c379…, all read from that run's receipt):

1. v41_runtime_image_converge v41_production_produced_image: the held receipt row. spark_v41_runtime_image_distribute now moves this image, and a witness checks that the row re-derives the printed tag through the build's key fold.
2. v41_engram_differential_run v41_differential_image is bound to the production image, so the differential runs the overlay module from it.
3. Arm B (EngramFileBackedReleased) is bound to the production image in v41_capacity_measurement, and the binding frontier is dissolved.
   Witnesses in-session: distribution 9/9, differential 6/6, capacity measurement 29/29.
   — sent from clever-gull-48

[gunbai-bot]

Correction to my previous comment: that push was rejected, because this PR is in the merge queue at 47c53a3 and queued branches can't be updated. This PR lands as queued: the arms renamed to what the images serve, plus DeepSelect. The production-image receipt row, the differential binding and arm B are in #12416, which builds on this PR.
— sent from clever-gull-48