Repository navigation
ci(OMN-14172): roll out integration silent-skip guard to omnibase_infra - #2242
Conversation
Enforcement-not-detection rollout of the OMN-14172 silent-skip false-green guard (omnimarket canary #1652, MERGED) to omnibase_infra — CI gate + pre-commit hook ship in the same PR (Operating Rule #5): - scripts/ci/check_integration_skips.py + integration_skip_guard.yaml, calibrated for infra's real Postgres-absence skip vocabulary (grepped from tests/); allowlists Kafka/Consul/Vault/Qdrant/live/catalog skips - new `integration-guard` CI job: provisions postgres:16-alpine (mirrors migration-integration), applies all migrations, exports OMNIBASE_INFRA_DB_URL + POSTGRES_* env, runs the curated Postgres-only proofs with --junitxml, then enforces check_integration_skips.py (fail-closed) - wired BLOCKING via ci_summary_gate.py SKIPPABLE_GATE_JOBS (the CI Summary umbrella required context); NO new branch-protection required context is registered here — deferred to operator/Codex once green on real PRs - integration-skip-guard pre-commit hook (--selftest, no DB needed) - tests/ci/test_check_integration_skips.py: case-(a) PASS + case-(b) RED regression proof plus full-vocabulary coverage OCC companion owed: omnibase_infra verify/receipt-gate needs Evidence-Source: OCC#<n> (Codex authors).
|
Warning Review limit reachedYou’ve reached a temporary PR review limit under our Fair Usage Limits Policy. Next review available in: 29 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (3)
📝 WalkthroughWalkthroughAdds a CI "Integration Silent-Skip Guard" that parses JUnit XML to detect false-green silent skips of provisioned services, backed by a new config YAML and Python script with self-tests. Wires the guard into the CI workflow, pre-commit hooks, and CI summary gate; adds unit tests and a minor Postgres test fixture change. ChangesIntegration Silent-Skip Guard
Estimated code review effort: 3 (Moderate) | ~25 minutes Sequence Diagram(s)sequenceDiagram
participant CIWorkflow as CI Workflow
participant Postgres as Postgres Service
participant IntegrationTests as Curated Integration Tests
participant GuardScript as check_integration_skips.py
participant CISummary as CI Summary Gate
CIWorkflow->>Postgres: provision + run migrations
CIWorkflow->>IntegrationTests: run curated Postgres tests, produce JUnit XML
IntegrationTests-->>CIWorkflow: JUnit report
CIWorkflow->>GuardScript: run_gate(junit report, config)
GuardScript->>GuardScript: parse_junit, classify_skip, evaluate
GuardScript-->>CIWorkflow: exit code 0/1/2 + JUnit artifact
CIWorkflow->>CISummary: report job conclusion
CISummary->>CISummary: check SKIPPABLE_GATE_JOBS for guard job
Possibly related PRs
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
|
Temporarily closing/reopening to refresh stale OCC-preflight pull_request payloads after central Evidence-Source OCC#3746 merged; head SHA unchanged. |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.pre-commit-config.yaml:
- Around line 825-826: The `files` pattern in the pre-commit config has
unintended leading spaces in the alternation branches, so only the first path
matches correctly. Update the regex in the `files` entry to remove the spaces
after each `|` while keeping the same targets, so the hook matches
`scripts/ci/check_integration_skips.py`,
`scripts/ci/integration_skip_guard.yaml`,
`tests/ci/test_check_integration_skips.py`, `.github/workflows/ci.yml`,
`tests/integration/runtime/db/test_postgres_repository_runtime_integration.py`,
and `tests/integration/handlers/test_registration_storage_postgres_uuid_cast.py`
as intended.
In `@scripts/ci/integration_skip_guard.yaml`:
- Around line 34-41: The integration-guard workflow selection is missing the
second curated Postgres proof, so it only runs the registration storage test.
Update the curated test selection in the ci workflow to stay in sync with
scripts/ci/integration_skip_guard.yaml and include
test_postgres_repository_runtime_integration alongside
test_registration_storage_postgres_uuid_cast, using the existing
integration-guard step as the reference point.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: d59bc6c2-1247-4f81-8327-7f5c3609ce6d
📒 Files selected for processing (7)
.github/workflows/ci.yml.pre-commit-config.yamlscripts/ci/check_integration_skips.pyscripts/ci/ci_summary_gate.pyscripts/ci/integration_skip_guard.yamltests/ci/test_check_integration_skips.pytests/integration/handlers/test_registration_storage_postgres_uuid_cast.py
1 participant
OMN-14172 — integration silent-skip guard rollout to omnibase_infra
Rolls out the integration silent-skip false-green guard (canary landed in
omnimarket #1652, MERGED) to the next queue repo, omnibase_infra. This is
enforcement-not-detection: the CI gate + pre-commit hook ship in the same PR
(Operating Rule #5).
What a silent-skip false-green is
An
@pytest.mark.integrationreal-DB test that self-skips because Postgres looksabsent turns the required Tests job green while the real-DB assertion never
ran. In omnibase_infra the analogous real-DB proof is the OMN-9041 UUID-cast
regression guard (
test_registration_storage_postgres_uuid_cast.py) — aproduction crash a silent Postgres-absent skip would have hidden.
Changes
scripts/ci/check_integration_skips.py— fail-closed stdlib+yaml gate copiedfrom the canary; infra-adapted
--selftestfixtures (case-a PASS / case-b RED)use infra's real curated test IDs + verbatim skip strings.
scripts/ci/integration_skip_guard.yaml— calibrated for omnibase_infra(see calibration evidence below).
.github/workflows/ci.yml— newintegration-guardjob: provisionspostgres:16-alpine(mirrorsmigration-integration), applies all migrations(
scripts/run-migrations.py), exportsOMNIBASE_INFRA_DB_URL+POSTGRES_*,runs the curated Postgres-only proofs with
--junitxml, then enforcescheck_integration_skips.py(fail-closed).scripts/ci/ci_summary_gate.py— added the job's display name toSKIPPABLE_GATE_JOBSso it BLOCKS via the existingCI Summaryumbrella required context (same tier as
migration-integration; thedefault-deny sweep also fails CI Summary on any non-success/skipped result).
.pre-commit-config.yaml—integration-skip-guardhook (--selftest, no DB).tests/ci/test_check_integration_skips.py— case-a PASS + case-b REDregression proofs plus full Postgres-absence-vocabulary coverage.
Calibration (grep evidence)
The canonical
PostgresConfig.from_env()(tests/helpers/util_postgres.py)reads only
OMNIBASE_INFRA_DB_URL; other conftests readPOSTGRES_HOST/PORT/PASSWORD/USER/DATABASE. The job exports both conventions.required_services.postgres.missing_skip_patternscovers the full Postgres-absenceskip vocabulary grepped from
tests/, e.g.:PostgreSQL integration tests skipped—test_postgres_repository_runtime_integration.py:124PostgreSQL not available—test_registration_storage_postgres_uuid_cast.py:76,dlq/conftest.py:212,snapshot/...:35/87PostgreSQL not configured/not reachable—ledger/conftest.py:61,injection_effectiveness/conftest.py:78/86Database not configured/Database not reachable—runtime/test_projector_shell_database.py:145/157No database URL configured—verification/test_registration_contract_verify.py:50Explicitly allowlisted (not provisioned by this job, never a false-green):
Kafka/Redpanda broker, Consul, Vault, Qdrant, Valkey, live-LLM SLO probes,
Docker-in-Docker, and the catalog-data skip
postgres manifest not found in catalog(test_catalog_extra_networks.py:80) which names "postgres" but is nota service absence.
Curated executor set (
curated_test_paths, mirrors the omnimarket 2-proofcanary): both are Postgres-only and self-contained against a freshly-migrated DB
—
test_postgres_repository_runtime_integration.pycreates its own table;test_registration_storage_postgres_uuid_cast.pyuses the migration-createdregistration_projectionstable.Local proof
check_integration_skips.py --selftest→ SELFTEST PASSED (case-a PASS,case-b RED, 3 violations).
(both curated proofs silently skip) → gate RED (exit 1).
tests/ci/test_check_integration_skips.py→ 12 passed;test_ci_summary_gate.py+test_ci_workflow_resilience.py→ 64 passed(ci.yml + gate-list changes don't break workflow-structure tests).
ruff format(unchanged) +ruff check(clean) +mypy(3 files, clean) +pre-commit run --files <changed>→ all Passed (SPDX, yamlfmt, new hook).Deferred to operator/Codex (do NOT do in this PR)
Required-context registration is DEFERRED. No new branch-protection
required status context is registered here — the job blocks via the existing
CI Summaryumbrella. The operator/Codex should register/confirm once the jobproves green on real PRs. (
scripts/audit-branch-protection.sh --repo omnibase_infra --dry-runalready reports a pre-existing Check-B FAIL forCI Summary/Handler Contract Compliance/verify / verify/call-reject-skip-tokencontexts not appearing on the last-5 commits — this PRdoes not touch branch protection and does not add to that set.)
Evidence-Source: OCC#3748
Evidence-Ticket: OMN-14172
Central OCC evidence merged in onex_change_control#3748 for the refreshed omnibase_infra rollout head and superseded legacy whole-file receipts with per-entry bindings.
Closes OMN-14172
Summary by CodeRabbit
New Features
Bug Fixes