ci(OMN-14127): run required CI Summary context as a no-needs GitHub-hosted fail-closed poller so it never wedges under self-hosted saturation

[jonahgabriel]

Summary

Ports the proven omniclaude #1870 wedge fix to omnibase_infra. The required branch-protection context CI Summary (the single umbrella gate for this repo, OMN-4497) was a needs-gated aggregator over ~20 self-hosted jobs. A needs-gated job gets no GitHub check-run at all until its needs reach a terminal state, so under self-hosted fleet saturation the gate jobs never terminalized and CI Summary was absent (not failing, not pending) — the PR wedged BLOCKED forever with 0 failing / 0 pending checks and no auto-recovery. Same wedge class fixed in omniclaude #1870.

Closes OMN-14127 (workflow-structure half; runner-fleet capacity half remains OMN-13932 / OMN-14080).

Fix — no-needs, GitHub-hosted, fail-closed poller

ci-summary is now a NO-needs, runs-on: ubuntu-latest, if: always(), fail-closed bounded-deadline poller:

• Its check-run instantiates immediately (the required context can never be absent), then it polls the current run's job conclusions via the GitHub jobs API until a terminal verdict (or a bounded deadline → fail-closed). Zero self-hosted / LAN dependency → immune to fleet saturation.
• name: CI Summary preserved byte-for-byte. A rename would name-mismatch branch protection and re-wedge every PR.
• DEADLINE_MINUTES: 90, timeout-minutes: 100 (proven omniclaude values) — comfortably above the tests-gate 30m budget + self-hosted queueing, and under the run-cancel window so the required context always posts terminal first.

Gating logic — scripts/ci/ci_summary_gate.py (exit 0/1/2 = success/fail/pending)

Built from infra's ACTUAL ci.yml (not copied blindly). It reproduces the exact strictness of the old needs-based condition and adds a strictly-stronger safety net:

• Strict gates (13, == success) — unconditional in ci.yml, never legitimately skip; a skip fails closed (matches old == "success"): CI Tests Gate, Lint, ONEX Validators, Infra Node Handler Ownership, Migration Freeze Check, Fingerprint Check, Demo Loop Gate, Topic Enum Drift Check, Topic Naming Lint, Topic Drift Check, Arch Invariants (OMN-3343), Kafka Schema Handshake (OMN-3411), Writer-Migration Coupling Check.
• Skippable gates (4, success||skipped) — carry a legitimate skip path (docs-only / event-scoped): Migration Integration Test, Contract Compliance, Contract Compliance Check, Contract Sync Gate (Wave C) [OMN-8915].
• Default-deny sweep over every other completed job, minus a small soft-allowlist of genuinely-non-gating jobs (verified against ci.yml needs + the pass/fail condition + dev branch-protection required contexts): Test-Failure Ratchet Gate (advisory OMN-13867), Version Pin Compliance (in needs, never checked), Runtime Boot Smoke (compose) (advisory OMN-9120), Cross-Repo Migration Conflicts (not required), Kafka Boundary Compat (OMN-3256) (advisory/xfail-drift), AI-Slop Pattern Check (strict, PR diff), zone-filter. Matching is prefix-aware so reusable-workflow callers (zone-filter / …, Runtime Boot Smoke (compose) / …) are covered.

Strictly stronger, not a rubber-stamp: the old tests-gate greens when test-parallel is skipped, so a failure in detect-changes / plugin-env-service-completeness / compose-required-env-coverage / contract-path-preflight (all of which skip test-parallel) used to slip through silently. The default-deny sweep catches them.

Dead-var scope correction (evidence, not assumption)

OMNI_REQUIRED_CI_RUNS_ON_JSON is 404 at both repo and org level (gh api …/actions/variables/OMNI_REQUIRED_CI_RUNS_ON_JSON → Not Found) → a silent no-op today. It is NOT a targeted ci-summary mitigation — it is the uniform OMNI_RUNNER_SELECTOR_V1 merge_group runner-override on ~28 self-hosted jobs, and it is test-enforced by test_merge_group_and_docker_workflows_have_runner_pool_overrides (any self-hosted job's runs-on MUST contain it).

This PR removes it from the ci-summary job only (natural, via the ubuntu-latest conversion — repo-wide count 29 → 28), which is what supersedes it for the wedge: the required context is now runner-independent. A repo-wide removal is deliberately NOT bundled here — it would break the enforcing test across ~28 jobs and change runner-placement policy for every heavy job, which is a separate decision, not part of the wedge fix.

Local verification

• python3 YAML parse → 30 jobs; ci-summary has needs: absent, runs-on: ubuntu-latest, if: always(), permissions: {actions: read, contents: read}, 2 steps.
• scripts/ci/ci_summary_gate.py smoke: full-green → 0; empty(saturation) → 2; in-progress → 2; detect-changes fail → 1.
• uv run pytest tests/ci/test_ci_summary_gate.py → 18 passed.
• uv run pytest tests/ci/test_ci_workflow_resilience.py → 29 passed (incl. the runner-pool-override enforcement, which now correctly skips the ubuntu-latest ci-summary).
• tests/ci/{test_branch_protection_audit,test_ci_summary_gate,test_ci_workflow_resilience}.py → 71 passed.
• ruff format --check + ruff check clean; mypy scripts/ci/ci_summary_gate.py tests/ci/test_ci_summary_gate.py → clean.
• actionlint .github/workflows/ci.yml → 5 findings, byte-identical to the origin/dev baseline (pre-existing shellcheck style/info in untouched run: blocks) — zero new findings. The poller's own shell block is clean.
• pre-commit run --files … → exit 0, 76 hooks, 0 failures.

OCC companion (independent, merged)

omnibase_infra is heavy → verify / occ-preflight requires an Evidence-Source: OCC#<n> companion in onex_change_control. Per operator policy [[feedback_no_self_authored_evidence]], the implementing agent did NOT author it. The independent companion is OCC #3684 ("evidence(OMN-14127): bind infra CI summary poller", merged), wired below via Evidence-Source: OCC#3684. verify / verify is green. occ-preflight / eligibility is blocked by a separate infra-side dep-pin issue (OMN-14130: infra pinned the pre-OMN-13888 omnibase-core==0.46.3, so occ-preflight runs the pre-grandfather validator and rejects the shared-contract append); it clears after OMN-14130 (#2231) merges and this PR is rebased on dev.

Evidence-Ticket: OMN-14127

Evidence-Source: OCC#3684

Summary by CodeRabbit

• New Features

  • CI Summary now evaluates workflow status by polling current job results, improving reliability for long-running runs.
  • Added clearer CI Summary reporting with terminal success, failure, or pending states.

• Bug Fixes

  • Reduced false positives/negatives in CI status checks by handling reruns, skipped gates, and in-progress jobs more accurately.
  • CI Summary now fails closed when required checks are missing or still pending past the deadline.

[coderabbitai]

Warning

Review limit reached

You’ve reached a temporary PR review limit under our Fair Usage Limits Policy.

Your recent review volume is higher than typical usage, so adaptive limits are currently applied.

Next review available in: 21 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: d83af3a2-d458-4e85-b78a-d0a6778ed2b5

📥 Commits

Reviewing files that changed from the base of the PR and between 1ec4807 and 078f61b.

📒 Files selected for processing (6)

• .github/workflows/ci.yml
• config/runner_routing_policy.yaml
• scripts/ci/ci_summary_gate.py
• scripts/ci/detect_test_paths.py
• tests/ci/test_ci_summary_gate.py
• tests/unit/scripts/ci/test_detect_test_paths.py

📝 Walkthrough

Walkthrough

The ci-summary GitHub Actions job is rewritten from a needs-based aggregation to a fail-closed poller running on ubuntu-latest with minimal read permissions, calling a new script scripts/ci/ci_summary_gate.py in a loop against gh api job data until a terminal verdict or 90-minute deadline. The script evaluates strict/skippable gate jobs and a default-deny sweep, with accompanying unit tests.

Changes

CI Summary Fail-Closed Poller

Layer / File(s)	Summary
Gate evaluator core scripts/ci/ci_summary_gate.py	Defines strict/skippable gate job sets, soft allowlist, exit codes, JobState dataclass, dedup_latest(), and evaluate() computing strict gate failures, skippable gate failures, and a default-deny sweep to return success/failure/pending.
Reporting and CLI scripts/ci/ci_summary_gate.py	Adds _report() for human-readable verdict output, _load_jobs() for stdin/file JSON parsing, and main() CLI with --jobs-file/--report-only flags.
Unit tests tests/ci/test_ci_summary_gate.py	New test module covering strict/skippable pass/fail cases, pending scenarios, fail-fast leaf failures, allowlist exclusions, rerun/latest-attempt selection, and docs-only success.
Workflow polling loop .github/workflows/ci.yml	Rewrites the ci-summary job to run on ubuntu-latest, removes the large needs list, adds actions/contents read permissions, extends timeout to 100 minutes, and polls gh api + ci_summary_gate.py for up to 90 minutes before forcing a fail-closed outcome.

Estimated code review effort: 4 (Complex) | ~60 minutes

Possibly related PRs

• OmniNode-ai/omnibase_infra#1819: Both PRs modify the ci-summary job's runs-on runner selection logic in .github/workflows/ci.yml.
• OmniNode-ai/omnibase_infra#2209: Both PRs touch the ci-summary job configuration and its timeout-minutes setting in .github/workflows/ci.yml.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately captures the main change: replacing the CI Summary job with a GitHub-hosted, fail-closed poller to avoid self-hosted runner wedges.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch jonah/omn-14127-ci-summary-poller

---

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.github/workflows/ci.yml:
- Around line 1547-1570: The polling loop in the CI workflow bypasses the
deadline when the gh api fetch fails because the retry path hits continue before
the deadline check. Update the polling logic in the jobs-fetch loop so the
deadline is evaluated on every iteration before attempting gh api, ensuring
persistent API failures still fail closed within the bounded window; keep the
existing terminal-state handling in the ci_summary_gate.py polling flow intact.

In `@scripts/ci/ci_summary_gate.py`:
- Around line 166-175: The deduplication in dedup_latest() is still overwriting
same-name jobs when run_attempt matches, which can hide a failure behind a later
success. Update the JobState selection logic to keep the worst state for each
name within the same attempt rather than blindly replacing the previous entry,
while still ignoring older attempts. Add a regression test in ci_summary_gate.py
covering two leaf jobs with the same name in the same run_attempt where one
fails and one succeeds, and assert the gate fails closed.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: bb3896b1-6cdd-4c00-bfd5-dfd8cbf81716

📥 Commits

Reviewing files that changed from the base of the PR and between 18e703d and 1ec4807.

📒 Files selected for processing (3)

• .github/workflows/ci.yml
• scripts/ci/ci_summary_gate.py
• tests/ci/test_ci_summary_gate.py