Repository navigation
fix(OMN-17427): skip apt when PG16 tools present in CI - #3551
Merged
Merged
Conversation
The unbounded sudo apt-get update stalled on omnimarket#3435 (split 12) and #3512 (split 16), causing the shard to hit the 90-minute job timeout and be cancelled. Because the job was cancelled, pytest never ran, no .coverage.N file was uploaded, and the Coverage Sweep Gate failed closed with ARTIFACT_MISSING, which also reds CI Summary. This change skips apt when the runner image already has PG16 initdb, pg_ctl and psql at version 16 in .github/workflows/ci.yml and .github/workflows/nightly-full-suite.yml. When apt does run, it uses Acquire::Retries 3 and 30-second http/https timeouts, and the step has timeout-minutes: 10. The fail-closed checks are unchanged. No coverage threshold or baseline changed. tests/ci/test_omn18693_native_postgres_ci_wiring.py now pins the step timeout and the conditional apt in all three steps.
Adds requirement req-pg16-provision-bounded with acceptance AC18, one tests evidence_requirement, and dod_evidence item dod-omn17427-ac18-pg16-provision-bounded bound to AC18 in contracts/OMN-17427.yaml. The bound check runs uv run pytest tests/ci/test_omn18693_native_postgres_ci_wiring.py -q, which fails at the merge base (3 failed) and passes on this branch. Purpose: repo-evidence dod-verify needs this pull request to bind evidence of its own.
✅ Architectural Review — PASSEDErrors: 0 What this checks
No architectural violations found. Static architectural lint — no model inference (OMN-14176). |
✅ Hostile Reviewer — PASSEDBlocking findings (critical/error): 0 Gate semantics
Powered by omniintelligence.review_pairing.cli_review — node-based adversarial review via HandlerLlmCliSubprocess (OMN-8524) |
Contributor
Author
|
Lab: host=h202 | lane=cause-market-coverage-0545 | head=c6afcb4a6872e5131e92f5f715e65a7856248744 | cmd=uv run pytest tests/ci/test_omn18693_native_postgres_ci_wiring.py tests/ci/test_coverage_census_cutover_wiring.py -q | 12 passed; the PG16 wiring test gives 3 failed against dev f61f904's workflows; bash -n clean on all three provision steps; the preinstalled-PG16 branch exits 0 against stub binaries without calling apt |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Ticket: OMN-17427
Summary
The Coverage Sweep Gate and CI Summary are red on omnimarket#3435, #3472 and #3512. The failure is not due to missing coverage or a moved baseline; the shards never ran pytest. The root cause is that the "Provision PostgreSQL 16 server tools for native migration proofs" step in the CI and nightly workflows runs an unbounded
sudo apt-get updateeven though the runner image already ships PostgreSQL 16; when a mirror fetch stalls, the shard sits there until the 90-minute job timeout cancels it. This results in anARTIFACT_MISSINGfailure for the coverage data file.Cause
On the green dev shard (push run 37729507227 at dev 149d334, Tests Split 3/20), the same provision step took 5 seconds and logged "postgresql-16 is already the newest version (16.15-1.pgdg24.04+2)". This confirms the ubuntu-latest runner image already ships PG16. However, the previous workflow logic did not check for preinstalled binaries and unconditionally executed the apt round trip. On PR runs, this apt step stalled:
sudo apt-get update(last line fetching noble-security InRelease at 18:19:14Z), then "The operation was canceled." at 19:48:13Z. The Coverage Sweep Gate failed with "aggregate FAIL-CLOSED [ARTIFACT_MISSING] split 12 metadata present but its coverage data file .coverage.12 is absent". Splits 3, 4, 5, 6, 9, 13 and 16 were also cancelled.Change
In
.github/workflows/ci.yml(jobs test and integration-guard) and.github/workflows/nightly-full-suite.yml(job shard), the provision step now checks for preinstalled PG16initdb,pg_ctlandpsqlat version 16. It runs apt only when they are absent. When apt does run, it writes/etc/apt/apt.conf.d/80-ci-network-timeoutswithAcquire::Retries 3and 30-second http/https timeouts. The step hastimeout-minutes: 10. The fail-closed checks after the step (three binaries executable, initdb reports 16, PATH export) are unchanged.tests/ci/test_omn18693_native_postgres_ci_wiring.pynow asserts the step timeout and the conditional apt for all three steps.contracts/OMN-17427.yamlgains AC18 anddod_evidenceitemdod-omn17427-ac18-pg16-provision-boundedbound to that test.Evidence
On lab host h202:
tests/ci/test_omn18693_native_postgres_ci_wiring.pygives 3 failed against dev f61f904's workflows and passes on this branch.tests/ci/test_omn18693_native_postgres_ci_wiring.pyplustests/ci/test_coverage_census_cutover_wiring.py: 12 passed.bash -npasses on all three step scripts.Not the cause
On #3542, #3548 and #3550, the Coverage Sweep Gate refuses because Repo Evidence Dependency failed, so Detect Changes and every test shard were skipped. That is a per-PR evidence gap from repo-evidence / dod-verify, not this step.