Repository navigation
fix(OMN-16367): evict per-run compose proof images at source and gate the invariant (retention wave 1) - #3121
Merged
jonahgabriel merged 1 commit intoSep 1, 2026
Conversation
… the invariant Seven 'docker compose up --build' steps mint per-run images (project name embeds run_id/attempt) that nothing ever removed: tagged (never dangling), uniquely named (never enumerable in a keep list), tagged latest (hard-kept by the disk GC's first guard). ~2.2 TB of orphans and the fleet-wide ENOSPC red on .201 trace to exactly this (2026-08-21 audit; plan rev 2 in omni_home docs/plans/2026-08-21-201-ci-image-retention-permanent-fix-plan.md). Wave 1 of the plan (items 1+2+9), workflow-only — zero host GC change: - item 1: all seven teardowns now carry '--rmi local' (the plan counted six; legacy-rds-fixture-proof.yml postdates it — the seventh proof workflow its own item 9 predicted). Never '--rmi all', which would remove shared pulled bases (postgres:16-alpine). - item 2: every built service in the four proof compose files stamps org.omninode.retention=disposable + run-id + workflow build labels; the pulled postgres in domain-adapter-proof takes no label. - item 9: tests/ci/test_compose_proof_teardown_policy.py enforces both invariants (plus the --rmi-all ban and lane-compose label ban) as a CI step in the compose-required-env-coverage job and a pre-commit hook, same PR, with the compose-image-retention-ok annotation escape. Refs OMN-16367
Contributor
|
| Verdict | Meaning | Blocks merge? |
|---|---|---|
passed |
No critical findings | No |
blocked |
CRITICAL findings found | Yes |
degraded |
All models unavailable (infra) | No (pilot) |
Powered by omniintelligence.review_pairing.cli_review — multi-model adversarial review (OMN-8468/OMN-8524)
jonahgabriel
pushed a commit
to OmniNode-ai/onex_change_control
that referenced
this pull request
Sep 1, 2026
#7968) * evidence: OCC companion pass 1 for OmniNode-ai/omnibase_infra#3121 * evidence: OCC companion self-bind for #7968 --------- Co-authored-by: node-occ-companion-effect <occ-companion-effect@omninode.ai>
jonahgabriel
deleted the
jonah/omn-16367-retention-wave1-eviction-at-source
branch
September 1, 2026 13:50
jonahgabriel
added a commit
that referenced
this pull request
Sep 2, 2026
…gate Adds glm-review (z.ai GLM Coding Plan) as a third, independent-hardware adversarial reviewer alongside the two local qwen3-review keys, and fixes the finder-never-gate wiring: the shipped hostile-reviewer.yml verdict parser read merged_findings/total_input_findings, fields ModelMultiReviewResult never had, so every finding was silently discarded and "blocked" was dead code. - scripts/ci/post_hostile_review_threads.py: publishes findings as resolvable PR review threads (dedupe by fingerprint, diff-anchored where possible, capped with a truncation banner, REQUEST_CHANGES posture on critical/major findings). - scripts/ci/check_hostile_review_threads.py: the new deterministic merge surface -- fails while unresolved hostile-reviewer-marker threads exist, ignores outdated/foreign threads, fail-closed on any GraphQL fetch error. - hostile-reviewer.yml: wires glm-review into the model roster, adds a scheme-aware preflight reachability probe, posts findings via the new poster, and adds the "Hostile Review Thread Gate" job (not yet a required context -- OMN-17492 canary). - Incident replays (tests/incident_replays/registry.yaml): verbatim captured bytes from the real omnibase_infra PR #3121 review (10 findings the old parser reported as 0) and the real omnibase_core PR #1604 thread state (the OMN-16823 false-red CodeRabbit-prose incident), each pinned by sha256 and driving the real guard code. Companion leg (omniintelligence PR #875, glm-review model registry entry) is already merged to dev. OMN-17492 Claude-Session: https://claude.ai/code/session_01TdrxtHYgVT1oNUYis2brcQ
jonahgabriel
added a commit
that referenced
this pull request
Sep 2, 2026
…gate (#3135) Adds glm-review (z.ai GLM Coding Plan) as a third, independent-hardware adversarial reviewer alongside the two local qwen3-review keys, and fixes the finder-never-gate wiring: the shipped hostile-reviewer.yml verdict parser read merged_findings/total_input_findings, fields ModelMultiReviewResult never had, so every finding was silently discarded and "blocked" was dead code. - scripts/ci/post_hostile_review_threads.py: publishes findings as resolvable PR review threads (dedupe by fingerprint, diff-anchored where possible, capped with a truncation banner, REQUEST_CHANGES posture on critical/major findings). - scripts/ci/check_hostile_review_threads.py: the new deterministic merge surface -- fails while unresolved hostile-reviewer-marker threads exist, ignores outdated/foreign threads, fail-closed on any GraphQL fetch error. - hostile-reviewer.yml: wires glm-review into the model roster, adds a scheme-aware preflight reachability probe, posts findings via the new poster, and adds the "Hostile Review Thread Gate" job (not yet a required context -- OMN-17492 canary). - Incident replays (tests/incident_replays/registry.yaml): verbatim captured bytes from the real omnibase_infra PR #3121 review (10 findings the old parser reported as 0) and the real omnibase_core PR #1604 thread state (the OMN-16823 false-red CodeRabbit-prose incident), each pinned by sha256 and driving the real guard code. Companion leg (omniintelligence PR #875, glm-review model registry entry) is already merged to dev. OMN-17492 Claude-Session: https://claude.ai/code/session_01TdrxtHYgVT1oNUYis2brcQ
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
OMN-16367 — wave 1 of the .201 CI image-retention permanent fix
/dataon.201repeatedly fills to 100% and reds the whole 88-runner fleet plus every lane on the host. The 2026-08-21 audit traced it: sevendocker compose ... up --buildCI steps mint a per-run image set (project name embedsrun_id/attempt) that nothing ever removes — tagged (never dangling, soimage prunemisses them), uniquely named (never enumerable in a static keep list), and taggedlatest(unconditionally kept by the disk GC's first guard). ~2,986 images / ~2.2 TB at audit time; re-measured today at +53 orphan images in 16 minutes of normal CI traffic.This is wave 1 (items 1+2+9) of the rev-2 plan (
omni_home/docs/plans/2026-08-21-201-ci-image-retention-permanent-fix-plan.md), landed together so no unguarded window exists. Workflow-only: zero host GC change, zero.201mutation — takes effect through CI on merge.Item 1 — eviction at source
All seven per-run teardowns now run
down --volumes --remove-orphans --rmi local:application-acl-postgres16-proof.ymlapplication-acl-proof-<run>-<attempt>ci.yml× 4omn15361-{domain,adapter,acl,legacy}-<run>-<attempt>docker-build.ymlomn15421-<run>-<attempt>legacy-rds-fixture-proof.ymlomn15422-<run>-<attempt>The plan counted six;
legacy-rds-fixture-proof.ymlpostdates it — literally the "seventh proof workflow" its own item 9 predicted.--rmi local(never--rmi all) is verified correct for all four compose files: it evicts exactly the per-run built images and leaves the shared pulledpostgres:16-alpinealone (plan §2.7).Item 2 — durable ownership label
Every built service in the four proof compose files stamps
org.omninode.retention: disposable+org.omninode.run-id+org.omninode.workflowunderbuild.labels— the ownership fact the wave-2 GC classifier will key on (a name shape is not an ownership claim). The pulledpostgresindomain-adapter-prooftakes no label. The pinned.proof-dependenciescheckouts can't receive labels retroactively; their images are still evicted by item 1, and the label gap there is exactly what the plan's prefix bridge (item 3, wave 2) covers.Item 9 — the gate, in the same wave
tests/ci/test_compose_proof_teardown_policy.py: everyup --buildstep must have a later same-job teardown bound to the same compose file withif: always(),--volumes, and--rmi local;--rmi allnever appears; every built service in a referenced live-tree compose file carries the disposable label; runtime-lane compose files (prod/stability-test/judge/infra) never carry it. Escape hatch:# compose-image-retention-ok: <reason>annotation (theno-raw-prod-bypassconvention). Wired as a ci.yml step (compose-required-env-coverage job) and a pre-commit hook in this same PR. The plan's four failure fixtures are in the test and each provably bites.dod_evidence
--rmi allecho string in the new ci.yml step) — proof it bites on this tree./data55% used, avail_kb 1,744,544,940, 1381 images, 1293 matching disposable patterns, 0 labeled. Read-only 15-min sampler armed on.201(recorded on the ticket before install).Refs: OMN-16367
Evidence-Ticket: OMN-16367
Evidence-Source: OCC#7968