Skip to content

chore(OMN-16799): prod-promotion grant for the onex-api cutover + api.omninode.ai hostname reclaim - #7418

Merged
jonahgabriel merged 7 commits into
mainfrom
hotfix/omn-16799-prod-promotion-grant
Aug 28, 2026
Merged

jonahgabriel merged 7 commits into
mainfrom
hotfix/omn-16799-prod-promotion-grant

Conversation

@jonahgabriel

@jonahgabriel jonahgabriel commented Aug 28, 2026 •

Copy link
Copy Markdown
Contributor

What the approver is being asked to do

Merge this PR. That merge is the approval act (OMN-13418) — an agent structurally cannot complete a CODEOWNERS-gated merge to main, which is exactly why the merge, not any field in the file, is the anti-self-issue anchor. No agent will merge this.

Ticket: OMN-16799.

After this lands, the operator surface is two more things, in order: un-draft + merge omninode_infra#1044, then have the dispatcher fire the deploy. Nothing else.


What this grant authorizes

Exactly one gated deploy-onex-prod.yml dispatch, run with deploy_target=onex-api AND reclaim_api_hostname=true in the same run. Those are deliberately coupled: omninode_infra#1044's Validate reclaim_api_hostname input step refuses reclaim_api_hostname=true unless deploy_target is onex-api/all, precisely so api.omninode.ai can only ever be pointed at an onex-api digest this gate has already cleared. Promotion and hostname reclaim ride one dispatch.

It authorizes no other prod mutation, and no prod mutation has occurred.

Two entries, both mechanically required

deploy-onex-prod.yml runs the OMN-14209 grant gate twice on a deploy_target=onex-api dispatch:

Step Guard Digest it resolves Covered by
Enforce prod-promotion grant — runtime image none — runs unconditionally RUNTIME_IMAGE_REF from k8s/onex-prod/runtime/kustomization.yaml → sha256:13c96028… grant-7ebd7674-9b25-466c-920d-a74a0f3efadb
Enforce prod-promotion grant — onex-api image if: deploy_target != 'runtime-only' ECR lookup on the dispatched commit_sha → sha256:bc95b9bc… grant-7013b49e-f714-4ead-8910-e6ffda84df56

The runtime step's own comment states it: "Runtime image is deployed unconditionally for every deploy_target value … so this check also runs unconditionally." A single-entry grant covering only the onex-api digest would have blocked at the first gate before ever reaching the second. Entry 1 is a same-digest authorization — sha256:13c96028… is what onex-prod's runtime plane is already running and what is pinned on omninode_infra main (8a4ebe69) and dev (ab92b08b) alike, so kubectl apply -k is a no-op for the runtime image and no runtime pod is rescheduled.

Digest provenance (entry 2 — the real promotion)

  • Registry: 272493677981.dkr.ecr.us-east-1.amazonaws.com/omnicloud/core
  • Digest: sha256:bc95b9bc5a1860dd666ecea4e82e278ff4ddb2d08da5d280ba8472b7041309ab
  • Built by Build and push onex-api image to ECR run 33179880143 — conclusion success, 2026-08-28T14:23:12Z, headBranch: main, headSha: 8a4ebe6919a52f8de272fb908e90866e8514e3cd (the current main HEAD), tagged with that commit SHA.
  • Trivy CRITICAL/HIGH clean. Contains docker/onex-api/tenant_event_outbox.py.
  • deploy-onex-prod.yml resolves it via resolve-ecr-digest in exact-match-only mode (fallback-to-latest is not passed, fail-on-missing: 'true'), so a wrong commit_sha fails loud rather than silently resolving some other image.

It replaces sha256:b4c325545e1012aef53772d1a3f5a30eb9af3809b9a4036e28a156db906581bc, the digest pinned in k8s/onex-prod/api/deployment.yaml.

expires_at is a PROPOSAL

expires_at: 2026-08-30T16:00:00Z (~49h from staging) is a proposal — the approver may amend it before merge; merging is approving. Same for approved_by.

approved_by — disclosed, not claimed away

As staged, the PR author and approved_by are both jonahgabriel. On this branch's own header rule ("The approver MUST differ from the PR author") that is self-approval as staged, and it is written down here rather than argued around. The approver closes it by confirming or replacing the value at merge time.

It is a real login and not a placeholder string for the reason OCC#7213 recorded: the schema requires a non-empty string, and the consumer enforces approved_by != requested_by against github.actor — a synthetic placeholder would compare unequal to every dispatcher and would silently disable the only mechanical anti-self-approval check on this path.

Anti-self-approval is satisfied on the dispatcher side, not here. With the staged value the workflow_dispatch actor must be daniyalabbas96 — the same identity that dispatched the proven-shape run 33117155819 on 2026-08-27 against the jonahgabriel-approved batch-92a794cd. If approved_by changes at merge time, re-derive the permitted dispatcher.

Validation — run locally, both directions, against the real implementations

Producer side (this repo's own schema suite, includes the OMN-13424 at-rest invariant that refuses any expired entry):

uv run pytest tests/test_prod_promotion_grants.py -q
18 passed in 0.03s

Consumer side — omninode_infra/scripts/validate_prod_promotion_grant.py, the exact script the workflow runs, driven offline against this file via its own --grants-file path:

dispatcher digest result
daniyalabbas96 sha256:13c96028… PASS: grant grant-7ebd7674… approved_by=jonahgabriel expires_at=2026-08-30T16:00:00Z
daniyalabbas96 sha256:bc95b9bc… PASS: grant grant-7013b49e… approved_by=jonahgabriel expires_at=2026-08-30T16:00:00Z
jonahgabriel sha256:13c96028… BLOCKED (self_granted)
jonahgabriel sha256:bc95b9bc… BLOCKED (self_granted)

The BLOCKED rows are the point: the interlock is live, not asserted.

Pruned in this PR — and why nothing is lost

The two OMN-16626 entries added by OCC#7213 (grant-3c885a33… runtime, grant-9c23ed5d… omniintelligence) both carry expires_at: 2026-08-28T20:00:00Z — hours from this staging. The OMN-13424 invariant refuses to let any expired entry remain, so this file cannot pass CI with them present once that timestamp passes, which is likely before the approver reaches it. They are removed rather than silently extended.

  • grant-3c885a33… was consumed in substance by run 33117155819 (2026-08-27, runtime-only, dispatcher daniyalabbas96) — its runtime gate passed and the apply succeeded. Entry 1 re-authorizes the identical digest, so the runtime gate loses no coverage.
  • grant-9c23ed5d… (omniintelligence sha256:a66ff063…) was never mechanically read by any gate — its own reason says so — and the pin it recorded is already merged on omninode_infra main (docs(OMN-11024): add delegation wire DTO evidence #1016, f127b84) and was applied by that same run. If a fresh omniintelligence authorization is wanted it must be re-issued in its own PR; this batch authorizes an onex-api cutover and nothing else.

Header lines 1–46 (the durable schema / invariant / four-step process block) are carried forward byte-identical from main. The only text dropped is the OMN-16626 per-request paragraph, which documents the entries this PR prunes.

Rollback

The workflow's own Rollback deployment on failure step covers deployment/onex-api (returning it to the prior ReplicaSet on sha256:b4c32554…) plus objects newly created by a partially-failed apply.

The Ingress DELETE is not covered by that rollback. Re-applying the onex-dev/onex-api Ingress by hand would restore the pre-existing dual-claim defect, not a healthy state — so the declared rollback target for the hostname half is "leave onex-prod/onex-api as sole claimant and fix forward". That is precisely why #1044 refuses the delete unless onex-prod's onex-api is Available=True, its onex-api-tls Certificate is Ready=True (verified Ready=True on 2026-08-28), and onex-prod already claims the host.

Routing

Consumer is omninode_infra's scripts/validate_prod_promotion_grant.py (k3s/EKS cloud plane, OMN-14209) — not the .201 compose-lane node_redeploy_orchestrator path. No raw SSH/kubectl retag/recreate path is used or implied.

Evidence-Ticket: OMN-16799


main-target-guard trailers — what the two fields below actually point at

main-target-guard (the version resolved from main) accepts a main-targeting PR from a hotfix/* head only when the body carries both a hotfix-evidence: and a backmerge: line. Both are stated literally rather than satisfied with a placeholder:

  • hotfix-evidence: OCC-7420 — OCC#7420, the hand-authored evidence companion for omninode_infra#1044 on this same ticket. It carries the behaviour proof for the gated hostname reclaim that this grant authorises the other half of.
  • backmerge: #7420 — the same PR, and this needs saying plainly: there is no content backmerge of the grants file into dev, deliberately. onex_change_control@dev carries a grants registry with zero grant_id entries (verified: git show origin/dev:grants/prod_promotion_grants.yaml | grep -c grant_id: finds only header/schema prose, no entries; main has the live entries). The registry lives on main on purpose — that is the branch the consumer resolves from. Backmerging grant rows into dev would create a second registry that no gate reads and that the OMN-13424 at-rest expiry invariant would then start failing on independently. #7420 is named here as the dev-side PR of record for OMN-16799, which is the same convention OCC#7213 (OMN-16626) used on this identical base (hotfix-evidence: OCC-7211 / backmerge: #7211, both naming that ticket's dev-side companion).

CI fix applied after the first push (aaa0a0f56)

Contract Compliance Check hard-BLOCKed on dod-prod-promotion-grant-consumer-gate-omn16799: its command was rooted at $OMNINODE_INFRA, an env var the hosted gate does not set, so it resolved to python3 /scripts/validate_prod_promotion_grant.py → No such file or directory, exit 2.

Fixed by declaring the check cross-repo the way the runner asks for (OMN-16824): cwd: "${OMNI_HOME}/omninode_infra", with the grants-file argument moved to a required OCC_GRANTS_FILE env var so it fails loud instead of silently validating the wrong checkout. The hosted gate checks out exactly one product repo and leaves ${OMNI_HOME} empty, so it now reports NOT-EVALUATED [cwd] for this item — non-blocking, and an honest "this gate did not evaluate it" rather than a verdict about a tree it was never pointed at. Same shape OCC#7403 (OMN-16838) ran live on this gate earlier today.

The evidence itself did not weaken: all four consumer-gate rows were re-executed against the new command shape with real exit codes read from the processes (daniyalabbas96→exit=0 on both digests; jonahgabriel→exit=1 BLOCKED (self_granted) on both), and contract_entry_sha256 / contract_sha256 were recomputed.

hotfix-evidence: OCC-7420
backmerge: #7420

…omninode.ai hostname reclaim

Two entries under a new promotion_batch_id batch-404af076-32a6-449c-b60a-1e8848e3fe3c,
authorizing exactly one gated deploy-onex-prod.yml dispatch with
deploy_target=onex-api AND reclaim_api_hostname=true:

  grant-7ebd7674  runtime same-digest  sha256:13c96028...  (mechanically required)
  grant-7013b49e  onex-api promotion   sha256:bc95b9bc...  (the actual change)

Entry 1 is not optional. deploy-onex-prod.yml's "Enforce prod-promotion grant
- runtime image (OMN-14209)" step carries NO `if:` guard, so it runs on a
deploy_target=onex-api dispatch too and resolves RUNTIME_IMAGE_REF from
k8s/onex-prod/runtime/kustomization.yaml (sha256:13c96028 on both omninode_infra
main and dev). A single-entry grant covering only the onex-api digest would have
blocked at the first gate. Because the committed runtime pin equals what is
live, the apply is a no-op for the runtime image.

Digest provenance for entry 2, verified rather than asserted: ECR
272493677981.dkr.ecr.us-east-1.amazonaws.com/omnicloud/core, built by "Build and
push onex-api image to ECR" run 33179880143 (success, 2026-08-28T14:23:12Z) from
omninode_infra main at headSha 8a4ebe6919a52f8de272fb908e90866e8514e3cd and
tagged with that SHA - which is how resolve-ecr-digest (exact-match only;
fallback-to-latest is not passed) resolves it at dispatch.

approved_by is disclosed as a PROPOSAL, not an approval. As staged the PR author
and approved_by are both jonahgabriel, which is self-approval on main's own
header rule; that is written down rather than claimed away. Consumer-side
anti-self-approval is satisfied only by dispatcher != approved_by, so the
dispatcher must be daniyalabbas96 as staged. Verified both directions by running
the consumer's own script offline against this file:

  daniyalabbas96 + either digest -> PASS
  jonahgabriel   + either digest -> BLOCKED (self_granted)

expires_at 2026-08-30T16:00:00Z is a PROPOSAL (~49h); the approver may shorten
it at merge time.

PRUNED: the two OMN-16626 entries (grant-3c885a33 runtime, grant-9c23ed5d
omniintelligence) both expire 2026-08-28T20:00:00Z, hours from this staging. The
OMN-13424 invariant refuses any expired entry, so the file cannot pass CI with
them present past that timestamp. grant-3c885a33 was already spent in substance
by run 33117155819 and its digest is re-authorized identically by entry 1;
grant-9c23ed5d was never mechanically read by any gate (its own reason says so)
and the pin it recorded is already on omninode_infra main (#1016) and applied.
Re-issue it in its own PR if a fresh omniintelligence authorization is wanted.

Header lines 1-46 carried forward byte-identical from main.

PREP ONLY - must be landed on main by a human operator (CODEOWNERS). That merge
is the approval act; no agent merges this.

Evidence-Ticket: OMN-16799
@jonahgabriel
jonahgabriel requested a review from a team as a code owner August 28, 2026 14:50
@coderabbitai

coderabbitai Bot commented Aug 28, 2026 •

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 07807e02-6f0a-4586-8a4b-3ab5a1a9324b

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Comment @coderabbitai help to get the list of available commands.

…t eligibility resolves

occ-preflight / eligibility and verify / verify resolve OCC evidence from the PR
diff for onex_change_control's OWN PRs (design invariant I3, in-tree checkout) --
call-occ-preflight.yml sets sha=$(git rev-parse HEAD) and reads contracts/ and
drift/dod_receipts/ against the PR head checkout, never against the base branch.
So the pinned OCC evidence for #7418 IS #7418's own tree, and the contract has to
land on the branch itself. Landing it on OCC dev would not be read at all; this
PR targets main.

Adds, on the branch:
  contracts/OMN-16799.yaml
  drift/dod_receipts/OMN-16799/dod-prod-promotion-grant-values-omn16799/command.yaml
  drift/dod_receipts/OMN-16799/dod-prod-promotion-grant-validation-omn16799/command.yaml
  drift/dod_receipts/OMN-16799/dod-prod-promotion-grant-consumer-gate-omn16799/command.yaml
  drift/dod_receipts/OMN-16799/occ-self-bind-pr-7418/command.yaml

Same shape OCC#7084 (OMN-16205) and OCC#7213 (OMN-16626) used for grant PREP on
this same base: contract + receipts in the grant PR itself. All three substantive
check_values are real falsifiable probes, never `grep -q '^status: PASS$' <the
receipt itself>`, which tests/test_evidence_admissibility.py lists as a REFUSED
shape.

  - grant-values: three-clause grep for both grant_ids plus the shared
    promotion_batch_id. Exits 0 here, non-zero on onex_change_control@main,
    which carries neither grant_id.
  - grant-validation: uv run pytest tests/test_prod_promotion_grants.py -q
    (18 passed), which includes the OMN-13424 at-rest invariant test that fails
    on any expired entry -- the falsifying test for the OMN-16626 prune.
  - consumer-gate: the CONSUMER's own script
    (omninode_infra/scripts/validate_prod_promotion_grant.py, the exact one
    deploy-onex-prod.yml runs) driven offline via --grants-file, all four
    combinations. daniyalabbas96 + either digest -> exit 0 PASS;
    jonahgabriel + either digest -> exit 1 BLOCKED (self_granted). The BLOCKED
    rows are the substance: they prove the anti-self-approval interlock is live
    on this path rather than asserted in prose, and they establish that with
    approved_by staged as jonahgabriel the dispatcher MUST be daniyalabbas96.

Receipts carry contract_entry_sha256 (OMN-13888 per-entry binding) plus the
whole-file contract_sha256.

No skip tokens. No gate edits. No change to grants/prod_promotion_grants.yaml in
this commit. #7418 remains for a CODEOWNERS operator to merge -- that merge is
the approval act (OMN-13418), and no agent merges it.

Evidence-Ticket: OMN-16799
…ted OCC gate declines it

The dod-prod-promotion-grant-consumer-gate-omn16799 check rooted its command at
$OMNINODE_INFRA, an env var the hosted Contract Compliance gate does not set, so
it resolved to /scripts/validate_prod_promotion_grant.py and hard-BLOCKed the PR.

Declare cwd ${OMNI_HOME}/omninode_infra instead (OMN-16824): the hosted gate
checks out exactly one product repo, leaves ${OMNI_HOME} empty, and DECLINES the
check as NOT-EVALUATED rather than rerouting it at a tree it was never pointed at.
Proven live on this same gate by OCC#7403 (OMN-16838).

The grants-file argument moves to a required OCC_GRANTS_FILE env var so the
command fails loud rather than silently validating the wrong checkout.

Receipt re-run locally against the new shape: all four probe rows (2 dispatchers
x 2 digests) re-executed with real exit codes, contract_entry_sha256 and
contract_sha256 recomputed.
…receipt

ModelDodReceipt forbids extra inputs; occ-preflight rejected the receipt with
'cwd Extra inputs are not permitted'. cwd is a CONTRACT check field (OMN-16824),
not a receipt field — the contract keeps it, the receipt states the cross-repo
execution context in actual_output instead.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant