Skip to content

fix(OMN-16508): resolve the governing PR instead of vacuously passing contract-compliance on empty PR_NUMBER - #2926

Merged
github-actions[bot] merged 1 commit into
devfrom
jonah/omn-16508-contract-compliance-fail-closed
Aug 27, 2026
Merged

github-actions[bot] merged 1 commit into
devfrom
jonah/omn-16508-contract-compliance-fail-closed

Conversation

@jonahgabriel

@jonahgabriel jonahgabriel commented Aug 27, 2026 •

Copy link
Copy Markdown
Collaborator

Closes OMN-16508.

Defect

ci.yml's contract-compliance job runs run_contract_compliance_check.py --pr <n>, where <n> is github.event.pull_request.number. Only pull_request populates that — but the job's if: also admits push and merge_group. The step met the empty case with exit 0:

if [ -z "${PR_NUMBER:-}" ]; then
  echo "::notice::No PR number (event=${{ github.event_name }}); DoD check_values are PR-scoped. Skipping."
  exit 0
fi

That is a vacuous pass on a required gate, not a cosmetic skip. "Contract Compliance Check" is a GATE_JOBS entry in scripts/ci/ci_summary_gate.py:200, and CI Summary is this repo's sole required branch-protection context — so the umbrella poller counted a run that evaluated zero DoD check_values as a provable pass. Unlike a red gate, a green one that checked nothing never prompts investigation.

Scope correction vs. the ticket body

The ticket claimed the gate "reports green on every dev push" and framed this as "a false-signal risk across the whole omnibase_infra dev branch." That is not the live shape, and the PR body records it rather than quietly shipping against a wrong premise. This workflow's trigger is:

on:
  push:
    branches: [main]

ci.yml never runs on a dev commit at all. The two real fail-open events are:

  1. push to main — the release-synced fast-forward commits.
  2. merge_group — unexercised today (verified 2026-08-24: zero registry repos are queue-controlled, mergeQueue(branch:"dev") is null fleet-wide), but a latent bypass of the sole required context the instant a queue returns, with zero further code change. This is the higher-severity half.

The ticket also cited "the OMN-16346 omnibase_core diff as the template implementation." That diff is not landed — OMN-16346 is still In Progress and its own body records the fix as "committed ... NOT YET PUSHED". What is live on core dev is the earlier #1556 change (plain exit 0 → exit 1, no PR resolution). So this implements the pattern from the ticket's description rather than porting a diff. Full premise-correction comment is on the ticket.

Fix: resolution, not narrowing

Excluding push/merge_group with a job-level if: would publish a skipped check run, which branch protection counts as passing — the same skip-vector class the reject-required-check-skip-vector hook (OMN-14863) exists to reject. So the job keeps running on all three events, and a new step resolves which PR's check_values apply via scripts/ci/resolve_contract_compliance_pr.py:

Event Resolution API call
pull_request github.event.pull_request.number, verbatim none — behaviourally identical to pre-fix
merge_group the pr-<n>- segment of refs/heads/gh-readonly-queue/<base>/pr-<n>-<sha> none
push merged source PR via gh api repos/{repo}/commits/{sha}/pulls one
unresolved exit 1 — fail closed —

A transient gh failure resolves to nothing and fails closed: a flake must never be indistinguishable from a green gate. The DoD step keeps its own exit 1 floor as defence in depth.

Verification

Live end-to-end against the real GitHub API, before landing:

Input Resolved Exit
dev merge commit 8ce11d518 (push) 2923 0
main commits e3cb9969 / 559ee461 / 166dc960 2837 / 2823 / 2835 0
--pr-number 2915 (pull_request) 2915 0
refs/heads/gh-readonly-queue/dev/pr-2923-8ce11d5 2923 0
all-zero SHA (no stdout) 1

Every recent main commit resolves, so the push path now evaluates a real scope rather than trading a false green for a spurious red.

TDD, red before green. With ci.yml reverted to its pre-fix bytes, the three workflow-shape guards fail:

FAILED test_dod_step_no_longer_vacuously_passes_on_an_empty_pr_number
FAILED test_job_resolves_the_pr_rather_than_reading_only_the_event
FAILED test_dod_step_consumes_the_resolved_pr_not_the_raw_event_number
3 failed, 1 passed

Incident replay (OMN-15547). The resolver is newly wired enforcement, so the coverage guard's default-deny correctly refused the first commit ([DEFAULT-DENY] scripts/ci/resolve_contract_compliance_pr.py ... carries no incident replay case). It ships with two verbatim gh api captures rather than a baseline exemption:

54 passed across the three related modules; pre-commit run --all-files exits 0 (including Incident-replay coverage and Reject skip vectors on required-check workflows); mypy --strict clean on the new module; pre-push governed selector escalated to the full unit suite fail-closed and passed.

Risk

The pull_request path is unchanged in behaviour. The push/merge_group paths move from "always green" to "evaluates the originating PR's check_values, or fails closed" — which is the point. Fail-closed on main pushes is the intended direction and is empirically not spurious: all recent main commits resolve.

Evidence-Ticket: OMN-16508
Evidence-Source: OCC#7288

… contract-compliance

`ci.yml`'s `contract-compliance` step met an empty `PR_NUMBER` with `exit 0`.
That was a vacuous pass on a required gate: "Contract Compliance Check" is a
GATE_JOBS entry in `scripts/ci/ci_summary_gate.py`, and CI Summary is this
repo's sole required branch-protection context, so the umbrella poller counted
a run that evaluated zero DoD check_values as a *provable* pass. Unlike a red
gate, a green one that checked nothing never prompts investigation.

Scope correction against the ticket body, which claimed "every dev push":
`on.push.branches` here is `[main]`, so this never fired on a dev commit. The
two live fail-open events are push-to-main (release-synced fast-forwards) and
merge_group -- the latter unexercised today (no registry repo has a queue on
dev as of 2026-08-24) but a latent bypass of the sole required context the
instant a queue returns, with zero further code change.

The fix is resolution, not narrowing. Excluding push/merge_group with a
job-level `if:` would publish a `skipped` check run, which branch protection
counts as passing -- the OMN-14863 skip-vector class. So the job keeps running
on all three events and `scripts/ci/resolve_contract_compliance_pr.py` answers
which PR's check_values apply: the event's own number on pull_request (no API
call, behaviourally identical to pre-fix), the `pr-<n>-` segment of the
gh-readonly-queue ref on merge_group (no API call), or the merged source PR of
the pushed commit via `gh api repos/{repo}/commits/{sha}/pulls`. Anything
unresolved -- including a transient gh failure -- exits 1.

Verified live against the real API before landing: dev 8ce11d5 -> #2923, and
the three most recent main commits -> #2837/#2823/#2835, so the push path now
evaluates a real scope rather than going spuriously red. An all-zero SHA exits
1 with no stdout.

Ships with the OMN-15547 incident replay its own default-deny rule demands:
two verbatim `gh api` captures under tests/fixtures/omn16508/ -- the empty
association list GitHub really returns for a commit no merged PR produced
(6d7090d, head of closed-unmerged #2890), which the guard must reject, and a
21KB merged-PR response (5b904d8 -> #2378) as the control that stops a
hard-wired `return None` from replaying the incident while failing every
legitimate push-to-main closed.
@coderabbitai

coderabbitai Bot commented Aug 27, 2026

Copy link
Copy Markdown

Warning

Review limit reached

Next included review available in 22 minutes.

View limit details

Limit details: You’ve used the included review currently available. Your 130 included PR review attempts over the past 7 days set your current allowance at 1 review per hour.

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 74a155a8-f085-40b0-aec2-8607e960c255

📥 Commits

Reviewing files that changed from the base of the PR and between 8ce11d5 and 7545cfb.

📒 Files selected for processing (7)
  • .github/workflows/ci.yml
  • scripts/ci/resolve_contract_compliance_pr.py
  • tests/ci/test_incident_replay_omn16508.py
  • tests/ci/test_resolve_contract_compliance_pr.py
  • tests/fixtures/omn16508/commit-5b904d88-pulls.gh-api.json.captured
  • tests/fixtures/omn16508/commit-6d7090da-pulls.gh-api.json.captured
  • tests/incident_replays/registry.yaml

Comment @coderabbitai help to get the list of available commands.

@github-actions

Copy link
Copy Markdown
Contributor

⚠️ Hostile Reviewer — DEGRADED (informational)

Blocking findings (critical): 0
Total findings: 0
Models succeeded: none

Note: All reviewer models failed or were unavailable. Degraded results are informational during the pilot phase (OMN-8468/OMN-8524) and do not block merge. Error: all review endpoints [192.168.86.201:8000 192.168.86.201:8000 ] unreachable — preflight short-circuit (no models available)


Gate semantics (pilot phase)

Verdict Meaning Blocks merge?
passed No critical findings No
blocked CRITICAL findings found Yes
degraded All models unavailable (infra) No (pilot)

Powered by omniintelligence.review_pairing.cli_review — multi-model adversarial review (OMN-8468/OMN-8524)

jonahgabriel pushed a commit to OmniNode-ai/onex_change_control that referenced this pull request Aug 27, 2026
#7288)

* evidence: OCC companion pass 1 for OmniNode-ai/omnibase_infra#2926

* evidence: OCC companion self-bind for #7288

---------

Co-authored-by: node-occ-companion-effect <occ-companion-effect@omninode.ai>
@github-actions
github-actions Bot merged commit a638e61 into dev Aug 27, 2026
185 of 193 checks passed
@github-actions
github-actions Bot deleted the jonah/omn-16508-contract-compliance-fail-closed branch August 27, 2026 12:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant