Converge R2 entitlement + bucket existence for every allocated origin purpose

[gunbai-bot]

Closes the gap behind #11713. gunbc.cloudflare.r2_origin allocated a bucket, but nothing ever checked whether the account had R2 or whether the bucket existed. That went unnoticed for nine days.

What lands

• extdeps.cloudflare.r2: the cited readings, the error codes, and the account-scoped storage grant and policy (r2_account_storage_create_request, over r2_account_resource_name).
• extdeps.cloudflare.r2_buckets (new module, one per client/v4 resource, like account_api_tokens)
  • Adds cloudflare.R2Buckets with Get and Create against /accounts/{id}/r2/buckets.
  • It is a separate module for a concrete reason too: a cloudflare.* service declared in r2 shadowed the vendor value cloudflare in modules that import r2, and test.claim.cloudflare_r2_origin_mint_run stopped compiling.
  • Adds classify_r2_bucket_get, which turns an HTTP outcome into one of R2BucketPresent | R2BucketAbsent | R2AccountNotEntitled | R2BucketReadRefused.
  • Only an exact cited code produces a fact: 403 + 10042 means not entitled, and 404 + 10006 means absent. Any other refusal stays unclassified.
• extdeps.cloudflare.client_v4: cloudflare_error_codes reads the v4 envelope's errors[].code properly instead of searching the body for a substring.
• gunbc.cloudflare.r2_bucket_ensure
  • Uses the existing std.upsert_decision vocabulary (Noop / Apply / Refuse); no new convergence algebra.
  • Walks every row of gunbc_fleet_r2_bucket_allocations: FabricDurableOrigin and FabricBootOrigin.
  • Observes each bucket, and creates it only when absence is established and the purpose's standing declares the default jurisdiction.
  • Reads back with a second, independent Get, which must classify as Converged.
  • Refuses for a non-default jurisdiction, because the create surface has no header for it.
  • Signs with the bucket-admin token only, never the bootstrap (operator ruling, 2026-09-19). An unminted credential refuses before any network call and names run_bucket_admin.
• Bucket-admin token, minted through the existing fold
  • Adds a third mint profile, R2AccountBucketAdmin, to gunbc.cloudflare.r2_token_mint_run (plus the run_bucket_admin entry). No second fold.
  • Its shape (cloudflare_r2_bucket_admin_token_shape) is one account-scoped grant: the observed Workers R2 Storage Write group (r2_permission_group_observe cloudflare_r2_storage_write_observed_permission_group_id, from the 2026-09-10 listing). It holds no object grant.
  • It gets its own custody container, cloudflare-r2-bucket-admin-token, and three IAM cells in r2_mint_secret_access (viewer, versionAdder, accessor).
  • Its standing is r2_origin fleet_r2_bucket_admin_credential, currently Unminted.
• r2_origin: adds bucket_purpose_origin_standing, an exhaustive per-purpose standing.
• fleet-converge: adds modes r2_bucket_admin_mint and r2_bucket_ensure (a step plus an always() receipt upload) to the existing dispatch-only job. No new job.

Enabling R2 is not possible through the API, so the ensure refuses

Cited readings, fetched with curl from the index.md projections and transcribed verbatim:

• developers.cloudflare.com/r2/get-started/: "You need a Cloudflare account with an R2 subscription… Complete the checkout flow to add an R2 subscription" (a dashboard flow).
• developers.cloudflare.com/api/resources/accounts/subresources/subscriptions/methods/create/: the rate_plan.id values are free/lite/pro/pro_plus/business/enterprise/partners_* only. There is no R2 plan.

So an unentitled account gets Inaccessible + Refuse with the dashboard step (r2_subscription_dashboard_step). The ensure never purchases anything.

Known limit: the 10042/10006 codes come from r2/api/error-codes/, which covers the Workers and S3 APIs. Applying them to v4 REST is an inference, and it is typed as one: a wrong guess produces a refusal, never an "absent" or "unentitled" verdict.

Evidence

• Witness test.claim.cloudflare_r2_bucket_ensure: 16/16 PASS locally (claim_batch --entry … --functions …).
  • Failing-case controls: an absent bucket leads to Apply of the declared name; an unentitled account leads to Refuse with the dashboard step.
  • Positive control: a present default bucket leads to Noop.
  • Other cases covered: jurisdiction conflict; absent non-default bucket; a 403 whose code is not 10042 is not read as unentitled; a 404 without 10006 is not read as absent; a transport failure is not read as absent; the real roster reaches the classifier.
• Live run (operator GCP token, token file deleted afterwards). Receipt:

  fabric-durable-origin	unknown-refused	refuse	THE BUCKET READ WAS REFUSED, NOTHING CREATED: status 403 carrying 10000, which names neither NotEntitled nor NoSuchBucket
  fabric-boot-origin	unknown-refused	refuse	THE BUCKET READ WAS REFUSED, NOTHING CREATED: status 403 carrying 10000, which names neither NotEntitled nor NoSuchBucket
  

  The bootstrap token has no R2 permission. That first run predates the ruling. The ruling (do not widen the bootstrap) is what this PR now implements.
• Live sequence, executed 2026-09-19 (receipts are in the PR comments):
  1. run_bucket_admin created its own custody container, minted token 6f287fde…, stored it at v1 and verified it. It is now pinned in fleet_r2_bucket_admin_credential.
  2. ensure found the durable origin present and did nothing. It found the boot-origin bucket gunbai-fabric-boot absent, created it, and the readback converged.
  3. A second ensure run found both present and did nothing.
• The mint flow now ensures its own custody container (operator direction): it observes the container and creates it when absent, before any Cloudflare effect. An identity without the project-level secretmanager.secrets.create permission (the fleet SA) refuses and names that permission. An ambiguous create is not retried.

Still open

• Token-mint witnesses: test.claim.cloudflare_r2_origin_mint_run 26/26, test.claim.r2_mint_secret_access_witness_test 5/5, test.claim.cloudflare_r2_bucket_ensure 16/16 (locally).
• The gcloud IAM converge cells in gunbc.cloudflare.r2_mint_secret_access have not been run: the four original ones plus the three new bucket-admin cells. Neither the session containers nor srv1 have gcloud.

Please merge by operator; I will not self-merge.

🤖 Generated with Claude Code

[gunbai-bot]

Addressed review 68490 in f820b55:

• upsert_decision_label is now generic over the plan (fn upsert_decision_label<P>); the receipt line takes the classification and derives both words from it, so no decision word is spelled at an emit site. The existing consumer (test.claim.roadmap_dispatch_environment) is still 7/7.
• observation_verdict_label moved onto the sum in std.upsert_decision, beside the decision label.
• The roster witness now asserts held.length() > 0 plus no failures, instead of the transcribed 2.
• Removed the unused R2BucketAllocation and UpsertDecision imports.

The same push applies the operator's ruling on credentials: the ensure signs with a new, separately minted account-scoped bucket-admin token, never the bootstrap (see the updated body). It also merges main, which fixes the heal-generated-artifacts failure: the branch predated dag/gunbc/heal_candidate.dag.

— sent from sharp-ant-20

[gunbai-bot]

Live receipts, 2026-09-19 ~17:11–17:14Z. Operator GCP access token, read from an owner-only file and deleted afterwards.

1. gunbc.cloudflare.r2_token_mint_run run_bucket_admin (exit 0), which now ensures its own custody container:

GET  secretmanager …/secrets/cloudflare-r2-bucket-admin-token        (absent)
POST secretmanager …/projects/gunbai-secrets/secrets                 (container created, automatic replication)
GET  …/cloudflare-account-api-token-bootstrap/versions/1:access
POST api.cloudflare.com …/accounts/<fleet>/tokens                     (one Create)
POST …/cloudflare-r2-bucket-admin-token:addVersion
GET  …/cloudflare-r2-bucket-admin-token/versions/1:access            (exact-version readback)
token_id=6f287fdeeffcfd409f7a8f2450fa2d4f
version_resource=projects/582015116396/secrets/cloudflare-r2-bucket-admin-token/versions/1

The version and token id are now pinned in gunbc.cloudflare.r2_origin fleet_r2_bucket_admin_credential.

2. gunbc.cloudflare.r2_bucket_ensure ensure, first run (exit 0):

fabric-durable-origin	converged	noop	gunbai-fabric-origin
fabric-boot-origin	absent	apply	create ok; readback converged

The boot-origin bucket gunbai-fabric-boot did not exist. This is the same kind of unobserved gap as #11713, on the purpose #11603 added. The ensure created it, and a second independent Get confirmed it.

3. ensure, second run (exit 0; three reads, no Create):

fabric-durable-origin	converged	noop	gunbai-fabric-origin
fabric-boot-origin	converged	noop	gunbai-fabric-boot

— sent from sharp-ant-20

[gunbai-bot]

Review 68550: agreed, and fixed in b91d374. The step creates an absent container, so it no longer observes. Renamed:

• observe_r2_mint_custody_container → ensure_r2_mint_custody_container (including the DischargedAt decl_ref)
• ObserveCustodyContainer → EnsureCustodyContainer
• R2MintCustodyContainerUnobserved → R2MintCustodyContainerNotEnsured

The refusal texts and annotations now say "ensured". No behaviour change. test.claim.cloudflare_r2_origin_mint_run passes 26/26.

— sent from sharp-ant-20

[gunbai-bot]

Review 68589: added the explicit import extdeps.transports.rest { RestOk, RestStatusRefused, RestTransportRefused, RestBodyUndecodable } in c3b1c62, matching the corpus convention. The module still compiles and runs the entry up to its credential refusal.

For the record, the symbols did already resolve. The live ensure run on this PR executed exactly that Apply arm of apply_r2_bucket_create (receipt: fabric-boot-origin absent apply create ok; readback converged), so the route was exercised against the account, not only by the classifier witnesses. The explicit import is still the right spelling, because it no longer depends on transitive visibility.

— sent from sharp-ant-20

[gunbai-bot]

Merged main again (36d19e1; main had added ApprovalKeyringConverge and MtCollins1Boot to the same one-line mode list). The .dag resolution is the union of all four modes.

One thing is knowingly incomplete in this commit: .github/workflows/fleet-converge.yml is main's copy and does not yet carry the two r2_* steps. I could not regenerate it here — generated_artifact_gate main_wet_one resolves the whole corpus, and after this merge that run exceeded 40 minutes and was killed three times on a shared 64 GiB host (currently ~49 GiB in use by other sessions). Rather than hand-edit a generated file or stall, I am letting heal-generated-artifacts regenerate it, which is what that lane is for. If it does not, the drift gate will fail and I will retry the regeneration on a runner with headroom.

Everything else is unchanged and already approved on the pre-merge head (review 68753).

— sent from sharp-ant-20

[gunbai-bot]

Review 68904: confirmed, and the root cause is one step earlier than the review reads it.

The stale projection is real. But it was not only lost in a merge: after merging main, the corpus did not resolve at all. main #11736 made fleet_converge_mode_fleet_ssh_key_demand exhaustive over the mode sum, and my merge added R2BucketEnsure/R2BucketAdminMint without arms there. So no regeneration could succeed — not mine, and not heal-generated-artifacts, which is why that lane failed instead of repairing the file:

error: resolve failed for dag/gunbc/instruments/generated_artifact_gate.dag
  cause: dag/gunbc/fleet/fleet_converge_workflow.dag:330:3: error: non-exhaustive match: missing variant(s) R2BucketEn…

854e773 declares both modes FleetSshKeyNotConsumed: each reaches api.cloudflare.com and secretmanager.googleapis.com over HTTPS as the run's own federated identity and opens no host session, so neither should hold the fleet SSH key. That restores resolution, which is the precondition for regenerating the YAML.

On the projection itself: I have tried to regenerate it locally five times since the merge. generated_artifact_gate main_wet_one resolves the whole corpus, and each run either exceeded 40 minutes or was killed — the session host is shared and has been sitting at 46–56 GiB of its cap while other sessions build. With resolution fixed, heal-generated-artifacts should now regenerate and publish it; if it does not, the drift gate stays red and I will regenerate it somewhere with headroom rather than hand-edit a generated file.

— sent from sharp-ant-20

[gunbai-bot]

Review 68939: confirmed, and I traced it further. Two corrections to the shared picture, then where it stands.

1. No CI lane will fix this, and none fails on it. I assumed heal-generated-artifacts would regenerate the projection. It will not. Its repair candidate for head 1cfe38c is:

{"expected_head_sha":"1cfe38c…","repair_declaration":"","entries":[]}

and generated_artifact_gate heal_repair_declaration only renders repair trailers for design-ledger rows. So heal passes while the file is stale, and so do clippy/compiler/witnesses — CI is fully green on this head with the drift present. Both reviews found it by reading the diff; no gate did.

2. The model is correct at HEAD. All four step functions exist and are in fleet_converge_job(), which is in the Workflow value expected_fleet_converge_yml renders. The only missing action is running the generator.

Why it is not yet regenerated — the generator refuses, twice, with typed causes:

• Here: admitted budget=33578549248 bytes, source=cgroup memory.max … Refusing rather than holding. Remedy: run where the admitted budget is genuinely available (a larger runner), while typechecking gunbc.plans.demand_engine_program. Six attempts, 40+ minutes each; several were OOM-killed by other sessions sharing the host.
• BuildBuddy: HostBudgetUnreadable (no cgroup binds the process); forwarding GUNBC_MEMORY_BUDGET_BYTES only degrades it to a declared-unverified 4000MB cap, and that runner has 7 GiB total.

The same entry runs fine inside the heal job on the self-hosted arm64 runners, so the fleet has hosts that can do it. I have asked the operator for one. I am deliberately not hand-writing the file: the emitted options: list follows the merged mode-roster order, so a textual reconstruction of the lost 328374f hunk would look right and still disagree with the authority — the same defect, harder to see.

— sent from sharp-ant-20

[gunbai-bot]

Merged main again (ab841c6). main reclassified MtCollins1Boot to FleetSshKeyNotConsumed and rewrote that declaration's reasoning, adding the rule that a mode added after it derives its arm from its operation route. My two R2 modes are exactly that case and already sat on the answer that rule gives: they reach api.cloudflare.com and secretmanager.googleapis.com over HTTPS as the run's own federated identity and open no host session. I took main's text and extended it to name them, and took main's MtCollins1Boot arm.

I also re-ran the per-mode sweep against main's copy (the check that caught the dropped arms in 1cfe38c): every mode's occurrence count agrees, and each of the four R2 modes appears in the same six constructs.

On the generated file, stated plainly: the merge driver again took main's side for fleet-converge.yml, dropping the regenerated hunks. Rather than ask for another srv1 run for a one-token delta, I resolved it as a 3-way merge of three generated versions — 0ac0900's (the operator's regeneration), the merge base, and main's — whose only divergence is main removing mtcollins1_boot from the fleet-key condition. No mode was added, so the emitted options: ordering is untouched. The result is main's file plus exactly the 41-line delta from 0ac0900, and the fleet-key condition now excludes precisely the five FleetSshKeyNotConsumed modes (the two org reads, mtcollins1_boot, and the two R2 modes), which is what the model says.

That is a textual merge of generator output, not a generator run. If anyone wants byte-certainty, re-running main_wet_one for this path on srv1 should be a no-op; if it is not, its output wins over mine.

— sent from sharp-ant-20

[gunbai-bot]

Review 69065: agreed, fixed in eb24535 — and I took the first option rather than the declared-fork row.

parse_r2_jurisdiction now folds over r2_jurisdictions() and compares through r2_jurisdiction_wire, so the five tokens are spelled exactly once. It also moved from extdeps.cloudflare.r2_buckets into extdeps.cloudflare.r2, beside the table it inverts; the operation module imports it. A sixth jurisdiction now needs one arm in the wire match and one row in the inhabitant list, and the parse follows without a third edit.

I did not add the mergeable_state style dissolution row, because that row exists to admit a fork that still stands. There is no fork left here to declare.

test.claim.cloudflare_r2_bucket_ensure passes 16/16, including the two cases that exercise the parse (a matching default converges; an unparseable token stays UnknownRefused rather than being assumed default).

— sent from sharp-ant-20

[gunbai-bot]

Review 69429 finding fixed in 91efb52.

Confirmed exactly as reported: git diff origin/main...HEAD -- .github was empty and the dispatch options: list ended at app_key_version_verify. The regeneration had landed earlier on the branch (328374f, 4a20129) and was lost across the later main merges.

Regenerated with tools.generated_artifact_gate main_wet against the merged roster — that fold rewrote only .github/workflows/fleet-converge.yml (git status after the run shows one modified path), +40/-1: the two options in the dispatch list, the two gunbc run steps and their two receipt-upload steps. The r2_bucket_ensure / r2_bucket_admin_mint entries now appear in the emitted job, so the new mode arms, ci_spec targets and gunbc.cloudflare.r2_bucket_ensure have their executing consumer.

Also on this head: the earlier merge conflict is resolved (f1ec9c9) — it was confined to the §4c annotation above FleetSshKeyDemand, and both sides were kept.