Skip to content

fix(OMN-15979): admit Integration Test Removal Gate into EXPECTED_EXTERNAL_CONTEXTS - #2730

Merged
jonahgabriel merged 2 commits into
devfrom
jonah/omn-15979-integration-test-removal-gate-wiring
Aug 16, 2026
Merged

jonahgabriel merged 2 commits into
devfrom
jonah/omn-15979-integration-test-removal-gate-wiring

Conversation

@jonahgabriel

@jonahgabriel jonahgabriel commented Aug 12, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Admits the "Integration Test Removal Gate" job (OMN-8732,
.github/workflows/integration-test-check.yml, job check-test-removal) into
EXPECTED_EXTERNAL_CONTEXTS in scripts/ci/ci_summary_gate.py — the OMN-15496
mechanism (PR #2569) that makes a cross-workflow check-run fail-closed against
CI Summary, omnibase_infra dev's sole required branch-protection context.

Why: this gate's own workflow header says "No override mechanism — consolidation
requires a replacement test to be added in the same PR", but before this PR it was
reachable by neither branch protection nor the CI Summary poller. PR #2720 (head
ce4e88f8) merged 2026-08-11T04:47:58Z with this context reporting failure.

Disposition (WAVE1-03 item, ticket OMN-15979)

Measured the same way OMN-15496 seeded its original 17: merge-time report rate +
green rate over the last 16 merged omnibase_infra dev PRs.

Disposition per the ticket's own branching rule: consistently green / root-causeable
when red → add to EXPECTED_EXTERNAL_CONTEXTS, fail-closed.

RED proof (TDD)

TestIntegrationTestRemovalGateExternalContext was written and run RED first
(verified by stashing the tuple-entry edit and re-running — 5 of 9 new tests failed
for the expected reasons: context not admitted, no-wedge replay found zero blocks,
the real #2720 payload resolved SUCCESS instead of FAILURE, the falsification-control
length assertion, and the synthetic-red flip). Restoring the fix turned all 9 GREEN.
Includes:

  • test_synthetic_red_flips_a_clean_pr_to_failure — a synthetic red on PR fix(OMN-15846): cross-DB disposition for 083/096/097 #2719's
    genuinely all-green real payload flips CI Summary's verdict SUCCESS → FAILURE.
  • test_red_at_merge_blocks_the_real_pr — replays PR fix(OMN-15860): install pydantic before the OMN-15676 config-paths assertion step #2720's real, unedited
    merge-time payload and proves it would now be blocked end-to-end (FAILURE, context
    named in the report) instead of merging clean.
  • test_falsification_control_tuple_entry_is_load_bearing — dropping only this
    context from the tuple greens the same real-red payload, proving the FAILURE above
    isn't coming from an unrelated context.
  • test_no_wedge_replay_over_this_window / test_every_sampled_pr_reports_the_full_expected_set
    — the admitted tuple doesn't wedge any of the 15 healthy PRs in the window.

uv run pytest tests/ci/test_ci_summary_gate.py — 59 passed.
uv run pytest tests/ci/ — 906 passed.

Seams

Boundary touched: the EXPECTED_EXTERNAL_CONTEXTS tuple is consumed by
ci_summary_gate.py's evaluate()/evaluate_external_contexts(), invoked from
.github/workflows/ci.yml's ci-summary job. No signature/schema change — this PR
only grows a tuple of literal GitHub check-run display-name strings the poller
already resolves generically from commits/{sha}/check-runs. No new field, no new
dotted-path, no DB/topic/contract surface touched.

Known pre-existing, unrelated local-hook failure (documented, not fixed here)

The local pre-push governed impacted-test selector (scripts/hooks/prepush_smart_tests.sh,
OMN-13973) buckets any scripts/ci/*.py change to run all of tests/ci/ — which swept
in tests/ci/test_env_parity.py::test_k8s_bound_keys_are_bound_in_compose, already red
on unmodified origin/dev (positive-controlled: same failure, same head, clean
canonical clone). Root cause: KAFKA_CONSUMER_GROUP / PROJECTION_RUNNER_HEALTH_PORT
are bound in the new onex-dev k8s writer Deployments (OMN-15800, landed
2026-08-11T09:55Z) but not yet in docker/docker-compose.infra.yml — a different,
already-in-flight workstream (projection-writer fleet), out of this ticket's scope
("do exactly the disposition, nothing broader"). This exact test/root-cause was
independently adjudicated today, on a different PR (#2694, merged
2026-08-12T15:35:19Z, merge commit 52d8b601c) as "pre-existing, not caused by this
diff, does not block merge" — Env Parity runs its own separate workflow, outside
ci.yml's job graph, and is absent from STRICT_GATE_JOBS/SKIPPABLE_GATE_JOBS/
EXPECTED_EXTERNAL_CONTEXTS, so it does not gate CI Summary. Pushed with
SKIP=prepush-smart-tests (the standard pre-commit-framework per-hook skip, not
--no-verify — every other local hook, including mypy, ONEX Architecture Layer Validation, and Deploy-scope DoD parity, ran and passed) rather than blocking this
ticket's fix on an unrelated, already-adjudicated gap. Did not add any
K8S_ONLY_KEYS/COMPOSE_PARITY_DEBT_KEYS classification — that would be an
allowlist addition outside this ticket's disposition.

Ticket: OMN-15979

Summary by CodeRabbit

  • New Features

    • Added the Integration Test Removal Gate to CI Summary validation.
    • CI summaries now include this check when evaluating merge readiness.
  • Bug Fixes

    • Prevented CI summaries from passing when the integration test removal check is missing, incomplete, or unsuccessful.
    • Ensured failed integration test removal checks correctly block approval until resolved.

Evidence-Ticket: OMN-15979
Evidence-Source: OCC#6407

…ERNAL_CONTEXTS

The "Integration Test Removal Gate" job (OMN-8732) hard-blocks a PR that
deletes a tests/integration/*.py file without a replacement and its own
workflow header says "No override mechanism" -- but before this change it
was absent from all three omnibase_infra dev enforcement surfaces: branch
protection (required_status_checks = exactly ["CI Summary"]),
EXPECTED_EXTERNAL_CONTEXTS, and MEASURED_NOT_ENFORCED_CONTEXTS. PR #2720
(head ce4e88f) merged 2026-08-11T04:47:58Z with this context reporting
`failure`.

Measured 16/16 present, 15/16 green over two independent 16-PR windows:
the original OMN-15496 seed window 2026-07-29T23:04Z -> 2026-07-30T14:54Z
(#2546...#2567, backfilled into omn15496_merge_time_external_check_runs.json
-- 16/16 green there) and the current window 2026-08-09T23:11Z ->
2026-08-11T04:47:58Z (#2705...#2720,
tests/ci/fixtures/omn15979_merge_time_external_check_runs.json -- 15/16
green). The one red, #2720, is root-caused: its job (id 93668844708)
recorded ZERO steps -- a self-hosted-runner dispatch failure -- and the
PR's diff touched only .github/workflows/build-and-push-runtime.yml (no
tests/integration files), so the gate's own substantive check logic was
never at risk of a legitimate red.

TDD: TestIntegrationTestRemovalGateExternalContext added RED (verified by
stashing the tuple entry and re-running -- 5 failures for the right
reasons) before the fix went GREEN. Includes a synthetic-red regression
(test_synthetic_red_flips_a_clean_pr_to_failure) proving a red check-run
on this context flips CI Summary's verdict, and a live-replay of PR
#2720's real payload proving it would now be blocked end-to-end.

Ticket: OMN-15979
@coderabbitai

coderabbitai Bot commented Aug 12, 2026 •

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 8313fba9-ce77-4b09-b2a8-25be71f87a7c

📥 Commits

Reviewing files that changed from the base of the PR and between 52d8b60 and 1e7014f.

📒 Files selected for processing (4)
  • scripts/ci/ci_summary_gate.py
  • tests/ci/fixtures/omn15496_merge_time_external_check_runs.json
  • tests/ci/fixtures/omn15979_merge_time_external_check_runs.json
  • tests/ci/test_ci_summary_gate.py

📝 Walkthrough

Walkthrough

The CI Summary gate now requires the external Integration Test Removal Gate context. Merge-time fixtures include the context, and tests verify successful, failing, missing, and synthetic verdict behavior across separate sample windows.

Changes

Integration Test Removal Gate

Layer / File(s) Summary
Register gate and update merge-time data
scripts/ci/ci_summary_gate.py, tests/ci/fixtures/omn15496_merge_time_external_check_runs.json
The gate now requires Integration Test Removal Gate. The fixture adds successful records for all 16 sampled pull requests.
Validate gate enforcement
tests/ci/test_ci_summary_gate.py
Tests verify context admission, complete fixture coverage, the real failing pull request, synthetic failure transitions, falsification, and disjoint sample windows.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Mergeability Score: ⚪ Minimal · up to 1e701

This localized change admits the Integration Test Removal Gate into CI Summary evaluation and adds regression coverage; no actionable merge-blocking risk remains beyond normal checks and review.

Possibly related PRs

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely identifies the main change: admitting the Integration Test Removal Gate into EXPECTED_EXTERNAL_CONTEXTS.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch jonah/omn-15979-integration-test-removal-gate-wiring

Comment @coderabbitai help to get the list of available commands.

@github-actions

github-actions Bot commented Aug 12, 2026 •

Copy link
Copy Markdown
Contributor

✅ Hostile Reviewer — PASSED

Blocking findings (critical): 0
Total findings: 0
Models succeeded: qwen3-review,qwen3-review-b


Gate semantics (pilot phase)

Verdict Meaning Blocks merge?
passed No critical findings No
blocked CRITICAL findings found Yes
degraded All models unavailable (infra) No (pilot)

Powered by omniintelligence.review_pairing.cli_review — node-based adversarial review via HandlerLlmCliSubprocess (OMN-8468/OMN-8524)

jonahgabriel pushed a commit to OmniNode-ai/onex_change_control that referenced this pull request Aug 13, 2026
#6407)

* evidence: OCC companion pass 1 for OmniNode-ai/omnibase_infra#2730

* evidence: OCC companion self-bind for #6407

---------

Co-authored-by: node-occ-companion-effect <occ-companion-effect@omninode.ai>
@jonahgabriel
jonahgabriel merged commit 42a1f7e into dev Aug 16, 2026
142 of 150 checks passed
@jonahgabriel
jonahgabriel deleted the jonah/omn-15979-integration-test-removal-gate-wiring branch August 16, 2026 14:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant