Repository navigation
fix(OMN-16989): lift h201's omnibase_infra denial — the 15 host-coupled failures were a gate-runner container artifact - #3041
Conversation
…e-red classes behind it
OMN-16989 recorded 15 host-coupled `omnibase_infra` failures on `.201` and set
`repos_denied=omnibase_infra` on `h201`. Every one of the 15 was measured inside
the `.201` GATE-RUNNER CONTAINER via `scripts/ci/run_on_gate_runner.sh` -- a
different execution environment from the one the table routes work to, which is
the `.201` HOST over the OMN-16991 remote leg.
Re-measured on the host over the real leg, receipts in
`.onex_state/prepush_distribution/receipts.jsonl`:
tests/unit/ + tests/integration/chains/ 24930 collected, pytest_exit=0, 695s
tests/ci/ (the tree the 15 came from) 1892 collected, pytest_exit=0, 122s
both host_mode=authorizing, chosen_host=omninode-pc. Zero host-coupled failures.
The denial was pinning a verdict from an environment nothing dispatches to.
Four real defects surfaced on the way and are fixed, not denied around:
1. The remote wrapper's PATH prefix was macOS-only by construction.
`/opt/homebrew/bin` has no meaning on a Linux row and h201 is the fleet's only
one. Its measured non-interactive PATH omits `~/.local/bin`, where BOTH `uv`
and `shellcheck` live -- already covered by the existing entries, so this
caused no red on h201, but the list is now Linux-complete. The added entries
go AFTER every measured one, so they can only add resolution.
2. The transplanted tree has no remote-tracking refs. `git bundle create <b> HEAD`
carries one ref, and this suite subprocesses the hook, which resolves
`${PREPUSH_BASE_REF:-origin/dev}` first. On h201 the whole
`test_prepush_hook_host_identity_guard.py` behavioral proof reduced to
`base ref 'origin/dev' could not be resolved`. The dispatcher now hands the
wrapper the base ref and sha and the wrapper `update-ref`s it in. BASE_SHA is
a merge-base, so it is always an ancestor of HEAD and already in the bundle.
Unresolvable => skipped silently: it may only add resolution, never refuse.
3. `check_deploy_scope_dod.resolve_omni_home` walks EVERY parent of the repo
root, so on the transplant it reached `/data/omninode` and imported an
omniclaude clone pinned at omniclaude#1600 (the host's current clone, #1963,
is not an ancestor). Four tests failed on `has no attribute
'parse_evidence_metadata'` -- a red about another repo's clone freshness on
whichever host ran the suite, with nothing saying so. Same
cross-repo-hidden-input shape OMN-16989 named on the env-parity tests.
`load_canonical_validator` now asserts the surface it calls and raises a NAMED
staleness error; the fixture skips with that reason. The hook still
hard-errors, but with a remediation instead of an AttributeError five frames
down.
4. `test_release_identity_passes_when_version_ahead` asserted an AMBIENT
repo-state precondition ("someone bumped the version in this PR") rather than
stating it. Between a release tag landing and the post-release bump landing
(OMN-13912, PR #3033, itself blocked on a stale runner-image identity lock) it
is red on EVERY developer clone and ONLY there: `actions/checkout` does not
fetch tags, so in CI `git tag --list` is empty, the gate is exempt and the
test passes; locally the tags exist and it fails. Measured 2026-08-30 -- dev
at 0.38.14 with `v0.38.14` cut at 13:24Z -- it refused every local governed
pre-push in the repo while being invisible in CI. The test now names the
precondition and skips with it. The GATE is untouched: `check-release-identity`
(pre-commit) and the CI gate both still run the real script against the real
diff.
Table + tests, per the runbook's promotion procedure:
- `scripts/hooks/prepush_hosts.tsv`: h201 `repos_denied` -> `-`, note carries the
measured numbers.
- `test_201_is_denied_for_omnibase_infra_until_omn16989_closes` ->
`test_201_denies_no_repo_since_omn16989_closed`.
- `test_a_repo_denied_host_is_never_chosen` now drives a SYNTHETIC table (the
`_SYNTHETIC_TABLE` precedent set when h105 left shadow), so the placement RULE
is no longer pinned to whichever repo the lab happens to deny today -- pinning
it made a capacity-policy edit look like a mechanism regression, which is
exactly the failure this run hit.
- `test_no_row_denies_a_repo_today_so_the_rule_needs_a_synthetic_fixture` guards
that fixture choice: the moment a real row denies a repo again, it fails and
says the live table may be pinned again.
Not fixed here: `pyproject.toml` version-vs-published is a release-cut condition
owned by OMN-13912; this diff touches neither `src/**` nor `pyproject.toml`, so
the release-identity gate itself does not apply to it.
|
Warning Review limit reachedNext included review available in 17 minutes. View limit detailsLimit details: You’ve used the included review currently available. Your 138 included PR review attempts over the past 7 days set your current allowance at 1 review per hour. Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. Review configuration: ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: ⛔ Files ignored due to path filters (1)
📒 Files selected for processing (6)
Comment |
…ibase_infra#3041 (#7733) * evidence(OMN-16989): author OCC companion for OmniNode-ai/omnibase_infra#3041 OCC companion by node_pr_lifecycle_fix_effect (OMN-13317 F1 / OMN-13990 / OMN-14285). Product PR head 18c669be63c65941cf7cdd9ec2c27f1108a20c29. * evidence(OMN-16989): self-bind OCC#7733 + rebind contract_sha256 --------- Co-authored-by: omnimarket-bot <bot@omninode.ai>
|
| Verdict | Meaning | Blocks merge? |
|---|---|---|
passed |
No critical findings | No |
blocked |
CRITICAL findings found | Yes |
degraded |
All models unavailable (infra) | No (pilot) |
Powered by omniintelligence.review_pairing.cli_review — multi-model adversarial review (OMN-8468/OMN-8524)
OMN-16989 — the 15 "host-coupled" failures were a container artifact;
h201is green on the hostscripts/hooks/prepush_hosts.tsvcarriedrepos_denied=omnibase_infraonh201because OMN-16989 measured 15 host-coupled failures there. Every one of the 15
was measured inside the
.201gate-runner CONTAINER, driven byscripts/ci/run_on_gate_runner.sh. That is not the environment this table routeswork to — the table routes to the
.201host over the OMN-16991 remote leg(bundle transplant into a scratch workroot,
uv syncin a fresh tree, thewrapper's developer-shell PATH, and the committed table as the identity source
instead of an env override the pytest child then scrubs).
Re-measured over the REAL leg, not by hand
Driven through the shipped dispatcher's own functions (
pick_capacity_host→prepush_remote_run, extracted verbatim by line range), with placement forced toh201via the picker's own deterministicPREPUSH_SLOT_OVERRIDE_MAPsurface — nobypass knob was added,
PREPUSH_FULL_SUITEwas never set.pytest_exittests/unit/+tests/integration/chains/(the heavy escalation argv)tests/ci/(the tree the 15 came from)Both
host_mode=authorizing,chosen_host=omninode-pc,chosen_label=h201.Receipt on this PR's exact head:
{"ts":"2026-08-30T16:48:08Z","repo":"omnibase_infra","head_sha":"18c669be63c65941cf7cdd9ec2c27f1108a20c29", "chosen_host":"omninode-pc","chosen_label":"h201","host_mode":"authorizing", "selection_paths":"tests/unit/ tests/integration/chains/ ","pytest_exit":0, "collected":24939,"duration_s":695, "suite_log_sha256":"35e94ecfe3183e9b78838f5ceed313f644e22d0a433378ca72f2b12fb9c881de"} {"ts":"2026-08-30T16:24:23Z","repo":"omnibase_infra","head_sha":"18c669be63c65941cf7cdd9ec2c27f1108a20c29", "chosen_host":"omninode-pc","chosen_label":"h201","host_mode":"authorizing", "selection_paths":"tests/ci/ ","pytest_exit":0, "collected":1892,"duration_s":130, "suite_log_sha256":"20e49c2acdac9741e48ce594eea2202c8b27a0799d42a50538b754ac17cc0faa"}The 15, re-classified against the real leg
omninode-pcis a first-class row, so the ".200or refuse" encoding is already parametrized. One of the four still failed — onbase ref 'origin/dev' could not be resolved, which is about the transplant, not identity (fix 2 below).omninode_infrais not a sibling of a transplanted tree.TestEntrypointOrphanReap, the threeTestImportEnvironmentIsolation, the fourTestRedeployStartCLIandtest_cache_effectiveness_metricsall pass on the 32-core host; the 4-CPU container cap that produced the timing/TimeoutExpiredfailures is not in play.The denial was pinning a verdict from an environment nothing dispatches to.
Four real defects surfaced on the way — fixed, not denied around
The remote wrapper's PATH prefix was macOS-only by construction.
/opt/homebrew/binhas no meaning on a Linux row andh201is the fleet's onlyone. Measured non-interactively 2026-08-30,
.201's ssh PATH is/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin—
~/.local/binis absent and bothuvandshellchecklive there, so theexisting
$(dirname "$UV")/~/.local/binentries already covered it (zerotool-missing reds in the full run). The Linux analogues are appended after
every measured entry, so they can only add resolution.
The transplanted tree has no remote-tracking refs.
git bundle create <b> HEADcarries one ref, and this suite subprocesses this hook, which resolves
${PREPUSH_BASE_REF:-origin/dev}first. The dispatcher now hands the wrapper thebase ref and its sha and the wrapper
update-refs it after checkout.BASE_SHAis a merge-base, so it is always an ancestor of HEAD and already in the bundle.
Unresolvable ⇒ skipped silently: it may only add resolution, never refuse a run.
resolve_omni_homewalks every parent of the repo root. On the transplant itreached
/data/omninodeand imported an omniclaude clone pinned atomniclaude#1600 — the host's current clone (fix(OMN-13062): migration-gate vacuity fix — sentinel discipline, wait-for-postgres, skip-manifest, sync-check nonzero (retro A-10) #1963) lives at
/data/omninode/omni_homeand is not an ancestor. Four tests failed onmodule '_deploy_gate_canonical_validator' has no attribute 'parse_evidence_metadata': a red about another repo's clone freshness onwhichever host ran the suite, with nothing saying so — the same cross-repo
hidden-input shape OMN-16989 named on the env-parity tests.
load_canonical_validatornow asserts the surface it calls and raises a namedstaleness error; the fixture skips with that reason attached. The hook itself
still hard-errors, but with a remediation instead of an
AttributeErrorfiveframes down.
test_release_identity_passes_when_version_aheadasserted an ambient repo state.Between a release tag landing and the post-release bump landing (OMN-13912,
chore(OMN-13912): post-release dev version bump to 0.38.15 #3033 — itself blocked on a stale runner-image identity lock) it is red on
every developer clone and only there:
actions/checkoutdoes not fetch tags,so in CI
git tag --listis empty, the gate is exempt and the test passes;locally the tags exist and it fails. Measured 2026-08-30 — dev at
0.38.14withv0.38.14cut at 13:24Z — it refused every local governed pre-push in the repowhile being invisible in CI. The test now names the precondition and skips
with it. The gate is untouched:
check-release-identity(pre-commit) and the CIgate both still run the real script against the real diff.
Table + tests, per the runbook's promotion procedure
scripts/hooks/prepush_hosts.tsv:h201repos_denied→-, note carries themeasured numbers.
test_201_is_denied_for_omnibase_infra_until_omn16989_closes→test_201_denies_no_repo_since_omn16989_closed.test_a_repo_denied_host_is_never_chosennow drives a synthetic table (the_SYNTHETIC_TABLEprecedent set whenh105left shadow). Pinning the placementRULE to whichever repo the lab happens to deny today made a capacity-policy edit
look like a mechanism regression — which is exactly the failure this run hit.
test_no_row_denies_a_repo_today_so_the_rule_needs_a_synthetic_fixtureguards thatfixture choice: the moment a real row denies a repo again it fails and says the
live table may be pinned again.
docs/runbooks/lab-prepush-host-table.mdrecords the lift, the container-vs-hostdistinction, and both transplant classes as host prerequisites.
Gate posture
Nothing here lets the gate accept less work.
h201moving fromrepo-deniedto aplacement candidate can only add an execution target; every verdict still comes
from a real
uv run pyteston anauthorizingrow of the committed table, readback through a MARKER bound to
{head_sha, argv_sha}. No newPREPUSH_*knob, no--no-verify, nocore.hooksPathoverride, no bypass token of the reject-skip-token family, noPREPUSH_FULL_SUITE, noENABLE_SMART_TESTS=off, no override grant consumed.Local evidence
tests/unit/scripts/test_prepush_host_table.py: 65 → 69 tests, all green.tests/ci/test_check_deploy_scope_dod.py: 6 → 8, all green.test_prepush_host_table,test_check_deploy_scope_dod,test_prepush_hook_host_identity_guard,test_pytest_full_suite_host_guard; 109 green across the remainingtest_prepush*files.Closes OMN-16989.
Evidence-Ticket: OMN-16989
Evidence-Source: OCC#7733
Not fixed here — filed instead
tests/unit/scripts/test_heavy_lock.py::test_a_wedged_holder_is_distinguishable_from_a_working_onefalse-reds on a loaded.200:_group_is_gonegives async SIGKILL delivery a fixed 10 s budget, and the leader'sreturncodeis already-9when it trips. It is green on.201(inside the 24,939 run above) and passes isolated on.200in 10.08 s; it failed once in this PR's own 20-minute selection at ~1.8x load and passed on the re-run (4140 passed, 15 skipped). Same defect class as this ticket, different host, and it is OMN-16995's surface — filed as OMN-17239 rather than absorbed here.Verification
4140 passed, 15 skippedon the narrow impacted selection (scripts/ci/tests/ scripts/tests/ tests/ci/ tests/scripts/ tests/unit/scripts/). The selection is not whole-suite-equivalent, so it ran locally and the remote leg was not engaged.pre-commit run --all-files: every hook Passed, tree clean.origin/dev@5367b2882, which picks up OMN-17161'sh101promotion and OMN-13912's 0.38.15 bump. Theprepush_hosts.tsvmerge is clean and every capacity row now carriesrepos_denied=-.--no-verify, noPREPUSH_*override, noENABLE_SMART_TESTS, no skip token, no override grant consumed. Runtime lanes (dev/stability/prod/judgecompose projects) were never contacted — all work is filesystem plus a scratch workroot at/data/omninode/onex-prepush.