Provider spine: native request families, persistent local materialization store, typed file-failure channel

[gunbai-bot]

What this changes

This is the provider spine the replacement native CI job runs on (operator rulings 2026-09-11, as corrected by design review C2/C3). The interface is std.materialization_provider. std.artifact_store is not touched: retention, capacity and eviction stay under the provider and are not built here.

1. Model: three request families, with manifests (std.materialization_provider)

• ArtifactRequest gets NativeCompilerArtifactRequest, UniverseArtifactRequest and NativeModuleVerdictBundleRequest(NativeModuleVerdictBundleSubject { route_binary_identity, module_identity, canonical_module_import_closure_identity, ordered_test_identity_population, eval_interpretation_identity }). MaterializedArtifact gets the matching variants, each with required per-kind parts: executable + build diagnostics, test population + import closures, and verdicts + preparation diagnostics.
• The three new kind rows are in std.cache_identity.
• Every field is an identity digest. No request or artifact type names a path, mount, rename or permission.
• ArtifactManifest is a projection (artifact_manifest). artifact_from_manifest is its inverse and is the only place a stored claim becomes an artifact again. A manifest with an unknown kind, an undeclared part or a repeated part refuses. A missing part decodes to the incompleteness variant, so the contract's own completeness check refuses it.
• RealizationReceipt { request_key, kind, content_digest, provider: CacheInterfaceId, store_instance, disposition }. Provider identity lives in the receipt, never in the key. A hermetic control shows two providers give the same key and the same content digest with different receipts.

2. Division of authority (std.materialization_object)

The native route emits typed requests. The provider contract owns request↔artifact correctness. This new module is the realization-agnostic half of the store:

• the stored object format (one self-describing object per request key: manifest plus percent-encoded payloads)
• the two observation coproducts a realization produces (StoreObjectRead = present | absent | unavailable, StoreObjectPublish = published | occupied | refused)
• the lookup and commit decisions over those observations

StoreLookupMiss is reachable only from StoreObjectAbsent. Unavailable, malformed/torn, uninterpretable manifests, and contract refusals (wrong kind, wrong artifact, wrong content, incomplete) all refuse and are never reclassified as a miss. A commit settles only on a verified read-back:

• published + same digest → Committed
• occupied + same digest → ConvergedOnPriorCommit
• different digest → Conflict
• anything else → ReadBackRefused

3. First real realization: local persistent filesystem (extdeps.realization.materialization_store_local)

• This is a thin transport handler: it checks the grant, names files, calls Filesystem.Read / Filesystem.WriteCreateNew, and maps the results onto the observations. It never consults a directory listing.
• A store is opened by reading its identity marker. A root with no marker is unavailable (LocalStoreNotInitialized): a missing volume is never read as an empty store, and nothing is written anywhere else, so there is no shadow store.
• Batch lookups do one open, then one exact read per request.
• Durable root grant (std.materialization_store_grant). DurableHostVolumeRoot has no path field. Its root is the declared /var/lib/gunbc/materialization-store, and a caller cannot hand in /tmp under that name. WitnessScratchRoot admits only /tmp/gunbc_*, and its durability is derived as EphemeralToFrame.
• Declared frontier: attaching the gunbc-owned host volume as a Firecracker drive at that path is the runner lane's job. Until then, a durable open on a guest refuses as unavailable.
• Its CacheInterfaceCatalogFacts row is added to extdeps.cache cache_catalog. The cache witnesses (104 fns across cache_layer_planner, cache_key_completeness, host_build_cache_provision_design and both provider/store witness files) all pass locally.

4. Prerequisite: a typed failure channel on the file transport (seed maintenance, v2-serving)

Without this, a miss could not be told apart from an unavailable object except by reading error text.

• Filesystem.Read and Filesystem.WriteCreateNew now declare error_kind (not_found | already_exists | permission_denied | other, projected from std::io::ErrorKind).
• The folds are filesystem_exact_read / filesystem_create_new. They carry typed absent/occupied arms, and an unrecognized kind is refused rather than folded into other.
• This discharges the next-rung trigger that the WriteCreateNew annotation named.
• Realized in both directions:
  • the interpreter: v1_interpreter dispatch_file / io_error_kind_name
  • the emitter: 05_emit FileChanErrorKind, and 05_emit_rust file_io_error_kind_fn bound once per file call
• Mirrors regenerated. claim_executor --required-regen now reports first_generation_equal=true, and --required-regen-fixed-point reports fixed_point_equal=true. Only v1_compiler_emit.rs and v1_compiler_emit_rust.rs changed.
• The existing Rust race test (concurrent_same_target_creates_produce_one_winner_and_no_residue) now also checks that every loser maps to already_exists.

5. GCS interface-ready; NFS later

• extdeps.cloud.gcp.storage has interface rows only: objects.get (404 = absent), objects.insert with ifGenerationMatch=0 (412 = occupied), and the status meanings. There is no handler. Declared frontier; trigger: the operator names the GCS realization lane.
• NFS is not built. Nothing here assumes a single host: publication is put-if-absent on one name per key, and correctness never lists.

BuildBuddy is not a provider, and there are no CI workflow edits.

v1 seed growth receipt (PURPOSE admission, gunbc.v1_maintenance_standing v1_seed_standing)

Measured from the merge base, never the moving tip: git diff 199aee77ab..HEAD -- 'src/v1/stage0/src/*.rs'.

• Hand-Rust items added: 1 production item. v1_interpreter io_error_kind_name projects std::io::ErrorKind onto the closed error_kind roster.
• Existing items modified:
  • the FileResult struct gains an error_kind field;
  • dispatch_file's arms populate the field;
  • map_file_outputs answers the "error_kind" key;
  • two existing tests are edited: the_emitted_listing_producer_refuses_too binds the five-tuple, and concurrent_same_target_creates_produce_one_winner_and_no_residue gains one assertion.
  • No test fn is added.
• Regeneration, not growth: the v1_compiler_emit.rs and v1_compiler_emit_rust.rs deltas are mirrors of src/v1/05_emit.dag and src/v1/05_emit_rust.dag. claim_executor --required-regen reports first_generation_equal=true, and --required-regen-fixed-point reports fixed_point_equal=true.
• Purpose: the channel is consumed by v2-program code. extdeps.realization.materialization_store_local goes through extdeps.filesystem.filesystem_io filesystem_exact_read / filesystem_create_new, which is how a store miss is told apart from an unavailable store without reading error text. The emitter carries the same channel, so the emitted native route can compile that module.
• Refused classes:
  • NewLanguageBehavior: no language semantics change; one transport output key is added.
  • NewEscapeHatchOrAdmissionRow: none.
  • NewCompatibilityObligation: none; the new key is additive, and existing consumers that don't declare it are unchanged.
  • PublicSurfaceGrowth: io_error_kind_name is private to v1_interpreter.

Controls, executed

Hermetic controls live in test.claim.materialization_store_witness (17 fns, floor-discovered). Wet controls live in test.claim.materialization_store_local_wet_witness (8 fns, real filesystem in a throwaway /tmp/gunbc_matstore.*). The wet file is enrolled on the required floor's local-repo wet lane with the full triple: the local_repo_wet_schedule rows, floor_route_gap_expectation_chunk_08, and the ci_layer_roots LocalRepoWetLane row. All 25 were executed with gunbc run against this tree and hold. Nothing was left behind in /tmp.

Brief control	Hermetic	Wet (real fs)	Discriminating mutation → red
commit + exact read-back	a_committed_object_reads_back_as_a_hit_with_its_payloads, commit_settles_only_on_a_verified_read_back	commit_then_read_back_by_real_execution	commit settles without the digest compare → commit_settles_only_on_a_verified_read_back reds
corruption → integrity refusal, no recompute	a_corrupted_payload_is_an_integrity_refusal, a_tampered_part_size_is_an_integrity_refusal, a_truncated_object_is_malformed_not_incomplete	corruption_is_an_integrity_refusal_by_real_execution (also asserts not-a-miss)	trust the manifest digest instead of re-deriving it from the bytes → both integrity controls red
wrong kind	an_object_of_another_kind_refuses_as_kind_mismatch (+ wrong artifact)	wrong_kind_refuses_by_real_execution	—
incomplete	an_object_missing_a_required_part_refuses_as_incomplete, the_write_door_refuses_before_publishing	incomplete_refuses_by_real_execution	—
concurrent writers, one key → one verified artifact	converged / conflict arms of commit_settles_only_on_a_verified_read_back	a_second_writer_converges_on_the_committed_artifact_by_real_execution, a_divergent_second_writer_conflicts_and_changes_nothing_by_real_execution	(race itself: Rust concurrent_same_target_creates_produce_one_winner_and_no_residue)
provider unavailable → typed refusal, no shadow	an_unavailable_object_refuses_and_is_not_a_miss, a_store_opens_only_on_its_own_marker	an_unavailable_store_refuses_and_leaves_no_shadow_by_real_execution (uninitialized, ungranted, batch; no marker or object written)	unavailable→miss → reds; NotFound→unreadable in the transport fold → the wet unavailable and batch controls red
batch	—	a_batch_answers_each_request_by_real_execution	—
key identity	every_verdict_bundle_identity_is_load_bearing_on_the_key (each of the 5 fields), native_families_key_apart_over_identical_digests, provider_identity_is_in_the_receipt_and_not_in_the_key	—	—

Concurrency, stated precisely. The wet second writer runs sequentially. That exercises exactly the loser's arm (create-new answers already_exists, then a read-back), because publication is a single link(2). The real 8-thread race is executed at the primitive by the Rust test named above.

Declared frontiers (DESIGN §3c)

• First consumer: the emitted native route (the successor of gunbc.witness.v2_native_route's operator-invoked instrument), in a later lane. Trigger: the emitted route builds these requests from its receipt identities and serves or commits through local_store_lookup_batch / local_store_commit instead of re-preparing every module per run. Emission already supports the new file channel, so that route can compile this module.
• Host volume attachment at the durable root: runner lane.
• Retention/capacity: the catalog row is CapacityUnobserved until std.artifact_store accounting binds to the store.
• GCS handler: when the operator says.

Known limits

• Part sizes are string_length, which counts characters for non-ASCII text. That is consistent on both doors, but it is a character count, not a byte count.
• Keys are 64-bit structural digests (fnv), inherited from the existing contract.

🤖 Generated with Claude Code

https://claude.ai/code/session_016AYera4CKxLqZJn5YW5Mzw

[gunbai-bot]

Re review 63841.

Finding 1 (GCS rows have no consumer): kept, and the frontier is now typed. This is not discretionary debt. The operator's 2026-09-11 provider ruling puts the GCS interface rows in extdeps on day one, with the handler later. The ruled dispatch order is: 1 recut #10940, 2 ancestry, 3 provider spine (this PR), 4 route integration, 5 NFS/GCS realization, 6 incubation. So under §3c this is the middle state: a named consumer that lands in a named later change. The module now says that as data, not prose.

extdeps.cloud.gcp.storage gcs_interface_consumer_frontier is a std.roster_frontier FrontierRow:

• subject: gcs_object_status_facts
• dissolution: bound_dissolution on extdeps.realization.materialization_store_gcs gcs_store_read_observation

That consumer is the GCS handler. It sits beside materialization_store_local and binds these rows to the std.materialization_object observations, the same way the local handler binds the filesystem. dissolution_status reports it as DissolutionPending until that declaration appears. I checked by execution that the row resolves and is well-formed.

Your true half stands, and the module says so too. GCS-readiness today comes from the path-free observation types in std.materialization_object, and those are consumed now. These rows are the cited upstream facts (API v1, upstream's own names), so the handler binds them rather than coining its own.

Finding 2 (misindented line in v1_interpreter.rs): real, fixed. Note that cargo fmt --all --check passed with the bad line in place, so rustfmt is not reaching that region.

Both are in 837cf78. CI on the previous head 50cf46c was fully green: build, floor, and the aggregate.

— sent from sharp-koi-253

[gunbai-bot]

Re review 63874: agreed, and fixed in 68e755b.

The frontier row is now a group, extdeps.cloud.gcp.storage gcs_storage_frontier_rows. It is enrolled in gunbc.census_closure_frontier census_closure_frontier_row_groups next to the Backblaze B2 and Cloudflare R2 groups. gunbc.dissolution_census now folds it, and that is its consumer.

Checked by execution against this tree: the 29 test fns in the three census-consuming witness files all pass (annotation_carrier_witness_test, which includes census_closure_frontier_rows_well_formed, the long dissolution_census_witness_test, and cloudflare_r2_token_witness_test).

The dissolution stays unbound until the handler exists. The previous head, ba3a82a, is fully green in CI. On 837cf78 the declarations phase refused a bound dissolution that cited the not-yet-existing extdeps.realization.materialization_store_gcs. The row's description names that consumer declaration and ruled dispatch step 5; the handler's change rebinds it.

— sent from sharp-koi-253

[gunbai-bot]

Re review 63888: all five findings were real, and all five are fixed in 27eed91.

1. ByteSize from a code-point count. payload_part now sizes a part by its UTF-8 byte count (std.bytes utf8_encode_bytes → bytes_octets → count), not by string_length. New RED: a_part_size_counts_utf8_bytes_not_code_points. "é€" is 5 bytes and 2 code points, and the test asserts both, so it would red on the old derivation.
2. Predicate matching StoreObjectLine directly. store_object_line_fold is now the only match over StoreObjectLine. store_object_line_is_end, store_object_last_is_end, the malformed-line, parts and payloads projections, and the end count all derive from it.
3. One frontier row per unconsumed declaration. gcs_storage_frontier_rows now enrolls gcs_objects_get, gcs_objects_insert_if_absent and gcs_object_status_facts. The cited names are literal decl_refs so the declarations phase resolves each one.
4. Re-minted endpoint. gcs_objects_get now names extdeps.cloud.gcp.gcp GcpStorage (GcsJsonApiEndpoint { service: GcpStorage }) and carries only the resource path. The base literal lives only in api_endpoints. The media-upload endpoint is a different upstream URL that api_endpoints doesn't carry, so it lives once, on GcsMediaUploadEndpoint, with its citation.
5. Stale comment. The write_create_new comment now describes the error_kind channel.

Checked by execution against this tree:

• the 18 hermetic and 8 wet store controls pass, with nothing left in /tmp
• all 47 census/store witness fns pass, including census_closure_frontier_rows_well_formed over the three new rows

The floor red on 68e755b was an infrastructure refusal on runner srv4-15, not a claim failure: MemoryStallRefusedPageThrash while resolving the terminal ledger.

— sent from sharp-koi-253

[gunbai-bot]

Re review 63898.

Finding (durable root expressed twice): real, fixed in bed50e8. materialization_store_durable_root is now absolute_path_position(path: materialization_store_durable_root_path). The grant root and the admission target come from the same datum, so they can no longer drift apart. The hand-written segment list is gone.

I checked this by execution:

• the derived segments are var|lib|gunbc|materialization-store, unchanged;
• the durable root admits Write;
• a WitnessScratchRoot pointed at the durable path is denied.

Advisory (v1 seed-growth receipt): added to the PR body under "v1 seed growth receipt". It is measured from the merge base, 199aee7. Its contents:

• added: one private hand-Rust item, io_error_kind_name;
• modified: FileResult gains error_kind, the dispatch arms populate it, and map_file_outputs answers the key;
• tests: two existing tests edited, none added;
• emit mirrors: these are regeneration, not growth (first_generation_equal=true, fixed_point_equal=true);
• purpose: the v2-program consumer is extdeps.realization.materialization_store_local;
• refused classes: each one is answered.

The previous head, 27eed91, is fully green in CI.

— sent from sharp-koi-253

[gunbai-bot]

srv2 closure receipt for the provider spine at head 27eed91 (overlay on main c560821; instrument closure_pr.sh, pinned producer claim_executor_pinned built from the overlay, route --v2-native-route --source-root dag --source-root src/v2).

Regressed files vs the main baseline (closure_main.sh at 199aee7; 29 refusals): none. Stage shifts: none.

• Emitted closure: 172 files, digest bcc917047abc… (changed from main's c17ed878…, as expected: dag/std/materialization_provider.dag is a closure member). Emitted binary sha256 dd41d62015b1c302.
• Closure root 167 modules; native fold EXIT=0, _terminal: complete, wall 325 s, rss 1.23 GB.
• After-set: 29 file refusals (28 parse_g0_tokens_remain, 1 normalize_reason_post_normalize_not_well_formed on src/v2/std/runtime.dag), joined by path against the baseline's 29: 0 new, 0 cleared.

Carry-over to the current head bed50e8: the delta from 27eed91 is one file, dag/std/materialization_store_grant.dag (+4/−3), which is not in the 167-module closure root of this overlay (no importer reachable from src/v2/compiler/00_compile.dag), and a body edit to a non-member cannot add it to the closure. The receipt therefore binds to bed50e8 unchanged; if a later push touches a closure member it is re-taken.

Receipt-neutral by execution; nothing blocks landing on the closure side.

[gunbai-bot]

Re review 63921: real finding, fixed in a2c0b08.

The defect. A second payload line for the same declared output id was resolved by payload_for → first(), so the lookup served the first copy instead of refusing. The manifest side already refuses a repeated part, so the two sides disagreed.

The fix. It now refuses as StoreLookupPayloadRepeated, a typed refusal of its own that tags as payload_repeated. The three payload-line checks (undeclared, repeated, missing) share one helper, store_payload_refusal, rather than nesting a third match.

New control. a_repeated_payload_line_is_refused_not_resolved sits beside the other corruption controls. It asserts that the duplicated object refuses with payload_repeated, and that the same object without the duplicate still hits.

Checked by execution:

• all 19 hermetic controls pass;
• with store_payload_repeated mutated to always answer none, exactly the new control goes red.

The previous head, bed50e8, is fully green in CI.

— sent from sharp-koi-253

[gunbai-bot]

Re review 63940: real finding, fixed in 9d4c04c.

filesystem_fault in extdeps.realization.materialization_store_local is the consumer you pointed to. It now builds StoreFault.detail as <kind>: <host message> through filesystem_failure_kind_name, so both the read and the publish refusals carry the kind the channel classified.

New control: an_unreadable_object_fault_names_the_host_kind. An unreadable object with FilesystemPermissionDenied must reach the store as unavailable, with a detail that starts with permission_denied: .

Checked by execution: all 20 hermetic and 8 wet store controls pass, and nothing is left in /tmp. filesystem_io itself is unchanged. The previous head, a2c0b08, is fully green in CI.

— sent from sharp-koi-253

[gunbai-bot]

Re review 63959: real finding, fixed in 2c6f702. Each value is now computed once and shared by every arm:

• store_lookup_decide binds let decode = artifact_from_manifest(m: m), and the three uninterpretable arms carry that decode.
• store_commit_prepare binds the decode, offered_digest and binding. The four Binding* arms and the prepared object reuse them, and the three manifest arms carry decode.
• store_commit_verify (the same shape) binds committed once for both the comparison and the conflict refusal.

After the change, artifact_from_manifest and request_artifact_binding each appear exactly once per function. Checked by execution: all 20 hermetic and 8 wet store controls pass, and nothing is left in /tmp. The previous head, 9d4c04c, is fully green in CI (run 34633043229).

— sent from sharp-koi-253

[gunbai-bot]

srv2 closure receipt at d5bf048 (this head overlaid on main bdf9823, producer built from the overlay, native fold over the 167-module closure of src/v2/compiler/00_compile.dag):

• native emit + build: EXIT=0, 172 files emitted, closure 2f71b52c….
• refusal after-set: 14 rows, identical row-for-row to the main baseline at bdf9823 (13 parse_g0_tokens_remain + 1 normalize_reason_post_normalize_not_well_formed on src/v2/std/runtime.dag). Regressed: none. Fixed: none.
• closure manifest: identical to main's (167 modules).

So the composed emitter change (#10990's walls plus this PR's error_kind delta on 05_emit_rust.dag) is closure-neutral on the emitted compiler. Remaining gates are CI at this head (rerun in flight after a contended-slot cost refusal on witnesses outside this diff) and an approval on d5bf048.

Instrument: closure_pr.sh 11059 on srv2; runs/closure_pr_11059_b.out, p-11059/refusals_after.txt.

— sent from eager-raven-113

[gunbai-bot]

srv2 closure receipt for #11059 @ 128185c (overlay on main 6cdebf5, instrument closure_pr.sh 11059, producer built from the overlay, route and fold invoked with GITHUB_SHA=<overlay tree>): clean. Fold rc=0; after-set 14 rows, identical to the main baseline at 2090c1b (13 parse_g0_tokens_remain + 1 normalize_reason_post_normalize_not_well_formed on src/v2/std/runtime.dag, all pre-existing); regressed 0, fixed 0; manifest 167 modules, identical to main's.

Caveat stated rather than hidden: #11105 landed on main (bb31b63) after this overlay was taken, touching dag/std/compiler_entry.dag and src/v1/**; a sixth baseline is queued on srv2 and this receipt will be re-joined against it if it differs from the previous five (which were all identical).

— sent from eager-raven-113