Native product reuse: key and reuse the self-host and v2-native-cli emitted products (CI row R3)

[gunbai-bot]

Programme item 2 of docs/plans/ci-end-to-end-accounting.md (PR #13505), row R3.

What: prepare_emitted_compiler_for_entry keys each entry's emitted executable by the six PreMaterializationIdentity axes (seed source tree, source closure + declaration universe, target, toolchain, build config, lens contract). A verified hit skips reconcile/emit/build/red; the instrument's door and refusal controls run against the reused executable. The two entries stay separate products (one key per closure). Entries commit atomically only after the discriminating red held; reads re-hash the executable and refuse+remove+rebuild on mismatch. Unset GUNBC_NATIVE_CACHE_ROOT = no store. emit-build sets a host-level root.

Purpose: serves the v2 self-host lane; seed growth justified in native_product_cache_seed_growth.

Controls (seed unit tests): hit under key / miss under changed key; corrupted executable refused and removed; entry without red refused; every axis reaches the digest.

Known narrowing left: the declaration universe is hashed at content grain (conservative, errs to miss); a span-insensitive heads hash is the declared next step.

Before/after runner-minutes (measured, run 37549915920, head f92b247)

• Cold (MISS) emit-build: 52m00s on srv3-07, 52m21s on srv3-03 (attempt 3 on srv4-05 still running when written). Both entries logged native product MISS then committed. generated 19m36s, floor 39m58s, rust-unit-tests 6m20s. First run on a host is a miss, so no regression.
• Keys are identical across hosts (self-host 78fa374e..., v2-native-cli 9c6dacc7...): the identity is host-independent.
• No HIT demonstrated yet, so no saving is claimed. The store is per runner host ($RUNNER_TOOL_CACHE) and each rerun landed on a different host, so a hit needs a job to land on a host already holding the products. A store shared across runner hosts is needed for the <=29 runner-min target to be met reliably; raised with the parent.

🤖 Generated with Claude Code

Shared store (declared frontier, per parent decision): gunbc.native_product_shared_store is the R2 realization of the same keyed request (address = the request key, put-if-absent via put_cache_blob, read via get_cache_blob; no new backend). It owns which run may use the store: fork PR and unattributed runs have no access (per-host L1 only); same-repo PR and merge group may read and write. Witness test.claim.native_product_shared_store shows the standing arms, the key-to-address mapping, and that a standing read refuses closed (Unauthorized naming the unminted credential) rather than reading as a miss. Trigger (capability): the cache-blobs read/write tokens minted and pinned in r2_origin fabric_cache_blobs_standing AND the required emit-build job carrying a workload identity (id-token grant + federation step). Until then the per-host L1 is the only tier and hits depend on host affinity. Residual accepted: a same-repo PR writer controls the manifest it is verified against.

Standing (parent decision B): this PR lands the keyed product, the verify-on-read path and the merge_group-only publish rule as a DECLARED FRONTIER. No CI saving is claimed. A per-host directory is writable by PR-controlled code, so it cannot be the wall; the live wall is the shared R2 store with credential asymmetry (PR jobs never receive the write token), a small follow-up once the cache-blobs tokens are pinned (#13542 mints them) and emit-build carries WIF. Reads re-hash and refuse but never delete; publication is admitted only for a merge_group run whose red+restore completed. Unit-level deletion control: only_a_completed_red_on_the_merge_queue_publishes. Review 77664 (store inert) is accepted as stated.

---

Full stack (folds in #13562, closed as superseded)

This PR now carries the whole change, so CI runs on it and the shared-store consumer is in the diff:

1. Keyed product: as above.
2. Restore/save (gunbc.native_product_shared_transfer): on an L1 miss the seed spawns the restore entry into a job-private dir under RUNNER_TEMP; the archive listing must be exactly the two entry files; the caller's lookup re-hashes. Save is merge_group only, admitted entries only. Any R2 or auth failure logs NativeProductRestoreUnavailable cause=... and is counted as a MISS; the job builds cold. Pull requests are attributed by head repo vs run repo (fork, unattributed: no access).
3. C1 read grant / C2 write federation (gunbc.native_product_federation): native-product-read SA via the existing github provider; a dedicated merge_group-only write pool, provider and SA (github-native-product-write). Both are gcp_iam_converge targets. Witnessed in native_product_federation_witness_test.
4. Wiring: compiler_gate_workflow adds id-token: write to emit-build, a read auth step (same-repo PR or merge_group), a write auth step and save step (merge_group), all continue-on-error; witnesses.yml regenerated. merge_group also reads: the read token is the least privilege and a queue hit is the common case.
5. Seed growth: ~70 lines in native_product_cache.rs plus one call site serve the v2 self-host lane (the seed derives the key, so only it can invoke restore); decisions stay in .dag.

Operator step required: run fleet-converge gcp_iam_converge to create the two SAs and the write pool. Until then the auth steps fail soft and jobs build cold (no saving claimed; before/after runner-minutes to be filled from the first cross-host HIT run).

Measured on head 95512b9 (run 37758989760, all green): emit-build 46m26s, floor 46m20s, generated 14m39s, seed 4m44s. Both entries logged NativeProductRestoreUnavailable cause=auth then native product MISS (typed, counted, built cold): the GCP accounts do not exist until the operator runs gcp_iam_converge. No saving is claimed; this is the cold baseline for the first cross-host HIT run.

---

Convergence-account hardening (defects 1–4 from the parent review)

1. Convergence impersonation closed. fleet-cloud-convergence (holds the R2 write-token accessor) was reachable by any same-repo job through the repo-wide github-oidc provider; this has been open on main since fleet-converge: r2_cache_object_read_mint / r2_cache_object_write_mint modes #13542. It is now reached only through a dedicated pool/provider github-fleet-convergence (gunbc.auth.fleet_convergence_federation), whose attribute condition pins job_workflow_ref = gunb-ai/gunbc/.github/workflows/fleet-converge.yml@refs/heads/main AND event_name = workflow_dispatch AND repository_id. gcp_iam_converge_targets now includes it, so the modeled converge ADDS the grant and verifies it by readback. Witnesses: the dispatch at main is admitted; PR, queue, other-workflow and other-ref runs are not; condition names workflow + dispatch; the account is reached only via the pinned pool.
2. Restore into a fresh root creates the store dir. 3. A refused entry is replaced by the verified build and marked committed; save is gated on the marker. 4. MISS/HIT/refused are one counted receipt (receipt.jsonl), not an eprintln.

Pinned consumers (the only workflows on this account via github-oidc; both regenerated onto the new provider in fleet-converge.yml): jobs fleet-converge and rlm-launch-deployment-receipt, both workflow_dispatch.

Rollout (corrected): (1) after merge, the operator runs the local iam_bootstrap_plan / apply from the updated tree, which installs the permissions for the new targets; (2) the operator REVOKES the old repo-wide binding (converge never deletes extras) and verifies it is gone:
gcloud iam service-accounts remove-iam-policy-binding fleet-cloud-convergence@gunbai-secrets.iam.gserviceaccount.com --project=gunbai-secrets --role=roles/iam.workloadIdentityUser --member='principalSet://iam.googleapis.com/projects/582015116396/locations/global/workloadIdentityPools/github-actions/attribute.repository/gunb-ai/gunbc'
(3) dispatch gcp_iam_converge on main: it authenticates through the existing github-gcp-iam-converge provider, so there is no cycle with the new pool, and it adds and verifies the pinned grant by readback; (4) cache reuse is live.

[gunbai-bot]

Review 77253: (1) fixed — the justification is now imported and added to seed_growth_justification_roster(), and it also names probe_toolchain_identity_for_key. (2) witnesses.yml is a generated projection and I could not regenerate it in-session (no local gunbc build); the PR's generated-job drift repair is expected to emit the GUNBC_NATIVE_CACHE_ROOT env, and I'll confirm on the next CI run — sent from tidy-wren-808

[gunbai-bot]

Review 77261: (1) fixed — emit-build now binds the fleet's own ci_native_cache_root_step ($RUNNER_TOOL_CACHE/gunbc-native/); the /var/tmp literal and job env are gone. (2) fixed — derive_key now folds through assemble_key and the control runs over that function, so dropping an axis turns it red. witnesses.yml still needs regeneration by the drift repair — sent from tidy-wren-808

[gunbai-bot]

Review 77420: verified, the finding is right. pre_materialization_digest is an Fnv fold (last two axes combined as a pair) and request_key is a 16-hex evaluation-bucket ContentHash, while the seed left-folds six SHA-256 digests into a 64-hex key, so "folded as pre_materialization_digest folds them" and "is the request key" asserted a sameness that does not hold. Fixed in the head after this comment: the seed module doc and the shared-store module now call it a seed-local product key over the same six axes (axis identity and order are the shared fact), state the reason for the divergence (a collision serves a wrong compiler, so the store address must be cryptographic; the provider does not serve the native artifact yet so there is no request_key to inhabit), and name its discharge (provider_serve becoming the only producer). Comments only, no behavior change. — sent from tidy-wren-808

[gunbai-bot]

Review 77424: verified and fixed. derive_key hashed a hand-copied string of the cargo flags while the build spawned a separate argv, so a flag change would not have moved the key. The flags are now spelled once (probe_cargo_build_flags), used by the build command and by probe_build_configuration_for_key (flags plus both rustflags channels), which derive_key hashes. Test the_native_product_key_names_exactly_the_flags_the_build_runs builds the real invocation and asserts its flags equal the shared list and appear in the axis; ran green with the existing native_product_cache tests, clippy clean. The new function is added to the seed-growth row. — sent from tidy-wren-808

[gunbai-bot]

review 77433: fixed in 3451d86. The merge group now has neither read nor write on the native product store: native_product_store_access returns NoAccess for MergeGroupRun, and the generated 'Derive native-cache root' step carries if: github.event_name == 'pull_request', so GUNBC_NATIVE_CACHE_ROOT is unset on the queue and it builds its own compiler. Controls: the two shared-store witnesses and the_native_cache_root_step_runs_on_pull_requests_and_never_on_the_merge_queue all return true. Cost: no reuse on merge_group runs; that tension with 'write on same-repo PR/merge_group' is raised with the parent. — sent from tidy-wren-808

[gunbai-bot]

Trust model changed per the operator direction: the merge group is the only writer of the shared native product store, same-repo pull requests read only and hold no write credential, forks have no access. Controls: only_the_merge_group_writes_and_pull_requests_only_read and the closed-refusal witness return true. Still a declared frontier until the cache-blobs tokens are pinned and emit-build carries a workload identity. Review 77446 needed no change. — sent from tidy-wren-808

[gunbai-bot]

Addressing the side-chat review at b5298f5, merged main (e58840d line). (2) Publication now requires a COMPLETED red+restore (red_completed, never the hard-coded true) AND merge_group standing, so a skipped-experiment PR miss builds and runs its controls but never publishes. (1) A read no longer mutates the store: lookup reports a refused entry and leaves it; only a protected writer may publish. Stated limit: PR code can set any variable in its own job and a per-host directory cannot be physically read-only to it, so the enforceable boundary is the credential asymmetry of the shared R2 store (read token vs write token, merge_group-only WIF), which waits on the mint (#13542) and pin. Until then the per-host store can only be written by a merge_group run, and merge_group is not given the root, so nothing publishes. Measured HIT still pending. — sent from tidy-wren-808

[gunbai-bot]

Review 77664: accepted. The per-host store is inert by design for now: a per-host directory is writable by PR-controlled code, so it cannot be the wall (parent ruling, B). Head now declares the frontier in the seed-growth row, claims no CI saving, drops the hit/miss-counter control, and adds the unit-level deletion control only_a_completed_red_on_the_merge_queue_publishes (publish rule extracted as publication_admitted). The R2 cutover is the follow-up once tokens are pinned. — sent from tidy-wren-808

[gunbai-bot]

Review 77673: fixed. Lookup::Refused doc, the operator log line and the test comment now say the entry is left in place, matching the never-delete read. Comment-and-string only; no logic change (unit tests unchanged). — sent from tidy-wren-808

[gunbai-bot]

Review 77675: the finding is accurate and was already accepted as the declared frontier (parent ruling B). What it correctly adds is that the §3c trigger was not stated beside the declaration; the seed-growth row's current_boundary now names the consumer (the credentialed shared R2 store over the same key, verify-on-read and merge_group-only publish rule) and its capability trigger (cache-blobs tokens pinned in r2_origin fabric_cache_blobs_standing AND emit-build carrying a workload identity). Those parts survive into that consumer; only the per-host directory is replaced, and the provider_serve deletion applies after. Tokens are minted via #13542, so the consumer follow-up is gated on the operator's dispatch, not on design. If the reviewer still holds that landing before the consumer is a scaffold, the alternative is to hold this PR until the R2 follow-up and land them together; the parent has chosen to land it behind the frontier. — sent from tidy-wren-808

[gunbai-bot]

Review 77747: correct that no CI run both fills and reads this store today, and the PR body/row say so (no saving claimed). I'm not disputing the observation; the disposition is parent ruling B: the per-host directory can never be the trust wall (PR code writes it), so the consumer is the R2 shared store, whose credentials are now minted and pinned (#13551, review 77739 approve). The remaining consumption path is stacked, in order: restore/save .dag step (restore into a job-private RUNNER_TEMP dir, save on merge_group only, one request_key shared with the seed), then a read-only identity for PR jobs and a merge_group-only write federation, then generator wiring. Two cost points I do take: the PR-path derive_key hashing is the price of the keyed lookup that the R2 restore will serve, and I will state its measured runner-time in the PR body once the first HIT run exists. If the operator prefers not to carry the interim, the alternative is to hold #13509 until that wiring lands as one change; that is a landing-order call, not mine. — sent from tidy-wren-808

[gunbai-bot]

Review 77747: held, not landing. The store has no executing consumer today (DESIGN 3c), so #13509 will land together with its first real consumer: the restore/save step stacked on it, then the read wiring so PR jobs actually read, with the merge_group write federation making it fillable. Please re-review once that wiring is stacked on this branch. Meanwhile the branch is kept merged with main (head just regenerated after #13515). — sent from tidy-wren-808

[gunbai-bot]

Review 77960: that review read head 7d67be6, which changed only the .dag. witnesses.yml was regenerated in 95512b9 and now carries the dissolve-on marker in the native-cache-root step's run body (one added comment line, same shape as the rustup default steps). No further change needed. — sent from tidy-wren-808