Repository navigation
feat(OMN-19543): a .200 verify-class CI runner so the dev-200 lane can be verified (AC5) - #4145
Conversation
…n be verified Declares the host in config/runner_fleet.yaml (arm64, verify class, always-on per operator approval), adds its compose file modelled on the .202 lane-pinned runner (docker socket + host-gateway alias only, no cloud credential mount), adds the host-200 leg to the arm64-verify-runner-proof matrix with its credential-check step skipped for the same reason, and covers the new runner with unit/integration/ci tests mirroring the .202 coverage. Does not create a runner registration token or register the runner live: an org registration token can only be minted by an operator (gh api orgs/OmniNode-ai/actions/runners/registration-token), so this PR prepares the config/compose/verification only. Standing it up is a separate operator step (see this PR's body and the compose file's own header for the exact recipe). Onex-Lane: prune-schedule-and-runner-83
jonahgabriel
changed the title
|
No OCC evidence companion was minted for this PR. this PR is already bound to OCC#11348; its evidence companion exists and nothing needs authoring To clear this: Nothing to do — the companion already exists. Reported by |
#11348) * evidence: OCC companion pass 1 for OmniNode-ai/omnibase_infra#4145 * evidence: OCC companion self-bind for #11348 --------- Co-authored-by: node-occ-companion-effect <occ-companion-effect@omninode.ai>
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
There was a problem hiding this comment.
Hostile Reviewer — adversarial findings (OMN-17492)
Models succeeded: qwen3-review, qwen3-review-b, glm-review
Models failed: none
New finding threads: 6
Deduped (already posted on this PR): 0
Below quorum (one model only, reported not threaded): 13
Nit-level findings suppressed: 3
The model is the FINDER, never the gate: merge is gated only by the
deterministic Hostile Review Thread Gate, which blocks while
hostile-reviewer threads are unresolved. Resolve each thread after
addressing (or rejecting, with a reply) its finding.
Below quorum: 13 finding(s) raised by one model only (OMN-18479)
These are reported and NOT dropped, but they get no thread and do not block: a single model's finding no other model reproduced is not evidence enough to stop a merge. Read them; act on them if they are right.
- [CRITICAL]
docker/docker-compose.runners-omninode-studio-runner.yml(qwen3-review) — Unconditional docker socket mount grants full host control | The compose file mounts /var/run/docker.sock into the runner container. The header comment claims this is a 'read use here' (revision label - [MAJOR]
docker/docker-compose.runners-omninode-studio-runner.yml(qwen3-review) — Missing test for RUNNER_TOKEN empty default | The compose file uses${STUDIO_RUNNER_TOKEN:-}which defaults to an empty string if the environment variable is not set. This means the runner will star - [MAJOR]
.github/workflows/arm64-verify-runner-proof.yml(qwen3-review) — Workflow skip condition is fragile and not tested | The workflow addsif: matrix.host != 'host-200'to skip the credential assertion step. This is a string comparison against a matrix value. If the - [MAJOR]
docker/docker-compose.runners-omninode-studio-runner.yml, header comments and volumes(glm-review) — Docker socket mount mischaracterized as a read-only surface | The compose file and comments repeatedly assert the docker socket is 'a read use here (revision labels, docker exec health and psql reads) - [MAJOR]
docker/docker-compose.runners-omninode-studio-runner.yml, environment section(glm-review) — RUNNER_TOKEN default contradicts the file's stated fail-closed contract | The header argues for per-host compose files because 'each host keeps its own fail-closed variable contract', yet RUNNER_TOKEN - [MAJOR]
.github/workflows/arm64-verify-runner-proof.yml, credential assertion step(glm-review) — host-200 leg of the proof workflow does not prove the credentials assertion | The workflow's stated subject includes asserting lab credentials are readable, and the compose header describes the host-2 - [MINOR]
docker/docker-compose.runners-omninode-studio-runner.yml(qwen3-review) — CPU and memory limits may be insufficient for verify jobs | The runner is limited to 1.0 CPU and 2GB memory. Verify jobs that read docker state and execute health checks may require more resources, es - [MINOR]
tests/ci/test_studio_verify_runner_omn19657.py(qwen3-review) — Test for host-200 uniqueness is incomplete | The testtest_studio_verify_runner_is_the_only_runner_labelled_host_200only checks compose files in thedocker/directory. It does not check for other - [MINOR]
tests/ci/test_studio_verify_runner_omn19657.py(qwen3-review) — Test for forbidden labels is incomplete | The test checks that the runner does not carryomnibase-ci,omnibase-deploy, oromnipc2-customerlabels. However, it does not check for other labels tha - [MINOR]
docker/docker-compose.runners-omninode-studio-runner.yml, volumes(glm-review) — Unused creds named volume contradicts the deliberate-absence inventory | The header enumerates what is deliberately absent ('no operator env file and no lab credentials directory') yet the service mou - [MINOR]
tests/ci/test_studio_verify_runner_omn19657.py, test_studio_verify_runner_reads_the_lane_and_nothing_more(glm-review) — Credential-absence test checks volumes only, not environment or other mounts | test_studio_verify_runner_reads_the_lane_and_nothing_more asserts operator.env, OMNI_HOME, lab-credentials, and .omnibase - [MINOR]
tests/ci/test_studio_verify_runner_omn19657.py, test_the_dev_200_route_labels_have_a_runner_to_land_on(glm-review) — Positive-control seam test is unreachable by skip design | test_the_dev_200_route_labels_have_a_runner_to_land_on skips when the routing table or its verify block is absent, which is the current state - [MINOR]
tests/ci/test_studio_verify_runner_omn19657.py, _labels and test_studio_verify_runner_is_the_only_runner_labelled_host_200(glm-review) — Cross-file label scan assumes mapping-form environment in all runner files | _labels calls env.get("RUNNER_LABELS", ""), which requires environment to be a mapping. test_studio_verify_runner_is_the_on
Findings demoted from threads (anchor rejected)
-
[MAJOR] hostile-reviewer (qwen3-review)
LANE_PROBE_HOST enables lateral movement to host loopback | The environment variable LANE_PROBE_HOST is set to host.docker.internal, which resolves to the host's loopback interface via the host-gateway alias. This allows any job running on this runner to probe and interact with all services bound to 127.0.0.1 on the host, including the 'always-on gpt-oss-120b planner (:8130)' mentioned in the comments. Combined with the docker socket, this creates a path for a compromised job to interact with sensitive host
Resolve this thread when addressed — the
Hostile Review Thread Gateblocks while hostile-reviewer threads are unresolved (OMN-17492). -
[MAJOR] hostile-reviewer (qwen3-review)
Test assertion on cpus value is fragile and likely wrong | The test asserts
runner.get("cpus") == "1.0". In YAML, the value"1.0"is a string, butcpus: "1.0"in the compose file is also a string. However, YAML parsers may interpret1.0as a float if not quoted. The compose file hascpus: "1.0"(quoted), so it is a string. But the test compares against the string"1.0". This is correct only if the YAML parser preserves the string type. More critically, the test asserts an exact string match, whiResolve this thread when addressed — the
Hostile Review Thread Gateblocks while hostile-reviewer threads are unresolved (OMN-17492). -
[MAJOR] hostile-reviewer (qwen3-review-b)
Unconditional docker socket bind on a host serving a production planner | The compose file mounts /var/run/docker.sock into the runner container. The header explicitly states this host also serves the 'always-on gpt-oss-120b planner (:8130) and the dev-200 compose lane'. The docker socket grants full control over the host's container runtime, including the ability to start privileged containers that mount the host root filesystem. While the header claims the socket is a 'read use', the socket itself provide
Resolve this thread when addressed — the
Hostile Review Thread Gateblocks while hostile-reviewer threads are unresolved (OMN-17492). -
[MAJOR] hostile-reviewer (qwen3-review-b)
Test for exclusive host-200 label is brittle against future fleet expansion | The test
test_studio_verify_runner_is_the_only_runner_labelled_host_200asserts that exactly one runner carries thehost-200label. This is a strong invariant that will break if a second runner is ever added to the host-200 lane (e.g., for capacity or redundancy). The test will fail, forcing a code change to update the expected list. This couples the test to a specific operational state rather than a logical invariant. The logResolve this thread when addressed — the
Hostile Review Thread Gateblocks while hostile-reviewer threads are unresolved (OMN-17492). -
[MINOR] hostile-reviewer (qwen3-review)
No documentation for STUDIO_RUNNER_TOKEN minting process | The comment says 'STUDIO_RUNNER_TOKEN=<org registration token, minted by an operator; see OMN-19657 return note -- this lane does not mint one>'. This is vague. There is no documentation for how to mint the token, who is authorized to mint it, or how to rotate it. An operator may not know how to obtain the token, leading to a blocked setup. | Evidence: STUDIO_RUNNER_TOKEN=<org registration token, minted by an operator; see OMN-19657 return note -- t
Resolve this thread when addressed — the
Hostile Review Thread Gateblocks while hostile-reviewer threads are unresolved (OMN-17492). -
[MINOR] hostile-reviewer (qwen3-review-b)
Missing test for RUNNER_TOKEN environment variable default | The compose file uses
RUNNER_TOKEN: ${STUDIO_RUNNER_TOKEN:-}. The test suite does not verify that this variable is correctly interpolated or that the service fails gracefully if the token is not provided. The header mentions the token is 'minted by an operator', but there is no test to ensure the compose file is fail-closed if the token is missing, which is a security best practice for runner registration. | Evidence: RUNNER_TOKEN: ${STUDIO_RUNNResolve this thread when addressed — the
Hostile Review Thread Gateblocks while hostile-reviewer threads are unresolved (OMN-17492).
✅ Hostile Reviewer — REVIEWEDCritical findings: 0
Semantics (OMN-17492 — the model finds, thread resolution gates)
Powered by omniintelligence.review_pairing.cli_review — multi-model adversarial review: qwen3-review, qwen3-review-b, glm-review (OMN-8468/OMN-8524/OMN-17492) |
…seline The PR's new tests/ci/test_studio_verify_runner_omn19657.py carries one deliberate, documented skip (test_the_dev_200_route_labels_have_a_runner_to_land_on, the .200 analogue of the existing .202 OMN-19507 baseline entry: the routing table declares no dev-200 verify block until a sibling PR lands one). The Skip Count Ratchet (OMN-18776) gate correctly refused the undeclared growth (+1, 587 vs baseline 586). This adds the node id to config/skip_count_baseline.yaml with its provenance and bumps max_skips to 587, following the same pattern as the existing OMN-19507 entry.
|
OCC autobind rebound this PR's evidence-source stamp line to The receipt gate's own eligibility validator, run against the change-control tree the gate pins for Reported by |
There was a problem hiding this comment.
Hostile Reviewer — adversarial findings (OMN-17492)
Models succeeded: qwen3-review, qwen3-review-b, glm-review
Models failed: none
New finding threads: 8
Deduped (already posted on this PR): 0
Below quorum (one model only, reported not threaded): 11
Nit-level findings suppressed: 2
The model is the FINDER, never the gate: merge is gated only by the
deterministic Hostile Review Thread Gate, which blocks while
hostile-reviewer threads are unresolved. Resolve each thread after
addressing (or rejecting, with a reply) its finding.
Below quorum: 11 finding(s) raised by one model only (OMN-18479)
These are reported and NOT dropped, but they get no thread and do not block: a single model's finding no other model reproduced is not evidence enough to stop a merge. Read them; act on them if they are right.
- [MAJOR]
tests/ci/test_studio_verify_runner_omn19657.py(qwen3-review) — Test asserts string equality on 'cpus' which is a float in YAML | The testtest_studio_verify_runner_reads_the_lane_and_nothing_moreassertsrunner.get("cpus") == "1.0". However, in the compose fi - [MAJOR]
docker/docker-compose.runners-omninode-studio-runner.yml(qwen3-review-b) — host.docker.internal:host-gateway may not resolve to host loopback on Docker Desktop | The compose file maps host.docker.internal to host-gateway, and the header claims this forwards to the host's loo - [MAJOR]
docker/docker-compose.runners-omninode-studio-runner.yml, header comment and services.omninode-studio-runner-1.volumes(glm-review) — Docker socket mount grants root-equivalent host access to any job on this runner | The new compose file mounts /var/run/docker.sock into the runner container while describing the use as 'a read use he - [MAJOR]
config/skip_count_baseline.yaml, OMN-19543 waiver entry and trailing node_ids append(glm-review) — Baseline increment has no enforcement tying its removal to the sibling PR | The ratchet ceiling is raised from 586 to 587 with the stated intent that 'the skip ends in the same change that gives dev-2 - [MAJOR]
.github/workflows/arm64-verify-runner-proof.yml, credential assertion step(glm-review) — Workflow proof for host-200 silently drops the credential assertion without a compensating check | The workflow adds host-200 to the matrix and guards the credential assertion with if: matrix.host != - [MINOR]
docker/docker-compose.runners-omninode-studio-runner.yml(qwen3-review) — Missing test for RUNNER_TOKEN default empty string behavior | The compose file setsRUNNER_TOKEN: ${STUDIO_RUNNER_TOKEN:-}. IfSTUDIO_RUNNER_TOKENis not set, the runner will start with an empty t - [MINOR]
tests/ci/test_studio_verify_runner_omn19657.py(qwen3-review-b) — Skip baseline test depends on a file that may not exist in the repo | test_the_dev_200_route_labels_have_a_runner_to_land_on checks for config/deploy_lane_routing.yaml and skips if it is absent or lac - [MINOR]
tests/ci/test_studio_verify_runner_omn19657.py::test_the_dev_200_route_labels_have_a_runner_to_land_on(glm-review) — Positive-control test skips permanently if the seam never lands | test_the_dev_200_route_labels_have_a_runner_to_land_on skips when the routing file or verify block is absent. Combined with the raised - [MINOR]
docker/docker-compose.runners-omninode-studio-runner.yml, environment.RUNNER_TOKEN(glm-review) — Empty RUNNER_TOKEN produces an unbounded restart loop with restart: unless-stopped | RUNNER_TOKEN: ${STUDIO_RUNNER_TOKEN:-} defaults to the empty string. With restart: unless-stopped, a stand-up that - [MINOR]
docker/docker-compose.runners-omninode-studio-runner.yml, STANDINGS IT UP section and environment(glm-review) — Registration token readable via docker inspect on a shared work host | The runner registration token is passed as a container environment variable, so it persists in the container config and is readab - [MINOR]
tests/unit/observability/runner_health/test_runner_fleet_hosts.py(glm-review) — Fleet reconciliation test hardcodes expected host tuples in a parallel structure | test_runner_fleet_hosts.py extends a hardcoded tuple of (host, arch, count) that must be manually kept in lockstep wi
Findings demoted from threads (anchor rejected)
-
[MAJOR] hostile-reviewer (qwen3-review)
Docker socket mount grants full host control to untrusted job code | The compose file mounts
/var/run/docker.sockinto the runner container. While the comment claims this is a 'read use', the Docker socket provides full control over the host's Docker daemon, including the ability to start privileged containers that can escape the container and gain root on the host. This is a well-known security risk. The comment 'the docker socket is a read use here' is misleading; the socket itself does not distinguishResolve this thread when addressed — the
Hostile Review Thread Gateblocks while hostile-reviewer threads are unresolved (OMN-17492). -
[MAJOR] hostile-reviewer (qwen3-review-b)
Docker socket bind grants full host control to runner jobs | The compose file mounts /var/run/docker.sock into the runner container. The header comment asserts this is a 'read use here' and that the runner carries 'no write surface into a compose lane.' This is factually incorrect: possession of the Docker socket is equivalent to root on the host. Any job scheduled to this runner can create privileged containers, mount host filesystems, or exfiltrate data from the always-on planner and dev-200 lane. The 're
Resolve this thread when addressed — the
Hostile Review Thread Gateblocks while hostile-reviewer threads are unresolved (OMN-17492). -
[MINOR] hostile-reviewer (qwen3-review)
CPU and memory limits may be too low for verify jobs | The runner is capped at 1 CPU and 2GB memory. Verify jobs that read Docker state, run health checks, or execute psql queries may exceed these limits, especially if the dev-200 lane is under load. The comment says 'Capped like the .202 lane-pinned runner', but it is unclear whether .202's limits are appropriate for the workload. If verify jobs are resource-intensive, this cap could cause timeouts or OOM kills, leading to flaky CI. | Evidence: cpus: "1.0"
Resolve this thread when addressed — the
Hostile Review Thread Gateblocks while hostile-reviewer threads are unresolved (OMN-17492). -
[MINOR] hostile-reviewer (qwen3-review)
Test for exclusive host-200 label is brittle | The test
test_studio_verify_runner_is_the_only_runner_labelled_host_200globs alldocker-compose.runners*.ymlfiles and checks that only one service has thehost-200label. This is a good invariant, but it is brittle: if a new runner file is added that also useshost-200(e.g., for a different purpose), the test will fail. The test does not distinguish between 'this is the only verify runner for host-200' and 'no other runner should ever use host-200'.Resolve this thread when addressed — the
Hostile Review Thread Gateblocks while hostile-reviewer threads are unresolved (OMN-17492). -
[MINOR] hostile-reviewer (qwen3-review)
Lane-pinned runner design may not scale to multiple lanes | The design pins a runner to a specific lane (dev-200) by mounting its Docker socket and using
host.docker.internal. This is a reasonable pattern for a single lane, but it does not scale to multiple lanes on the same host. If a future requirement is to run verify jobs for multiple lanes on the same host, this design would require a separate runner for each lane, leading to resource duplication. The comment acknowledges this is 'the same reason hosResolve this thread when addressed — the
Hostile Review Thread Gateblocks while hostile-reviewer threads are unresolved (OMN-17492). -
[MINOR] hostile-reviewer (qwen3-review-b)
Test for exclusive host-200 label only checks compose files, not runner_fleet.yaml | test_studio_verify_runner_is_the_only_runner_labelled_host_200 globs docker-compose.runners*.yml files to find carriers of the host-200 label. However, runner_fleet.yaml declares hosts with classes and runner_name_prefix, and the actual runner labels are set at registration time via the RUNNER_LABELS environment variable in the compose file. The test verifies the compose file, which is correct, but it does not cross-check t
Resolve this thread when addressed — the
Hostile Review Thread Gateblocks while hostile-reviewer threads are unresolved (OMN-17492). -
[MINOR] hostile-reviewer (qwen3-review-b)
Runner group 'omnibase-deploy' on a verify-class runner is a label/role mismatch | The compose file sets RUNNER_GROUP: omnibase-deploy, but the runner is explicitly in the verify class and is not intended for deploy jobs. The header states the runner is 'deliberately NOT labelled omnibase-ci' and carries no deploy duties. Placing a verify-class runner in the omnibase-deploy runner group means GitHub Actions will consider it eligible for any job that targets the omnibase-deploy group, unless the group is pur
Resolve this thread when addressed — the
Hostile Review Thread Gateblocks while hostile-reviewer threads are unresolved (OMN-17492). -
[MINOR] hostile-reviewer (qwen3-review-b)
CPU and memory caps may be insufficient for verify jobs on M2 Ultra | The runner is capped at 1.0 CPU and 2g memory. The host is an M2 Ultra Mac Studio, which has significantly more CPU and memory. The caps are justified by the need to coexist with the always-on planner and dev-200 lane. However, verify jobs that run test suites or build artifacts may exceed these limits, causing OOM kills or CPU throttling that manifests as flaky test failures. The caps are the same as the .202 runner, but the .202 host is
Resolve this thread when addressed — the
Hostile Review Thread Gateblocks while hostile-reviewer threads are unresolved (OMN-17492).
1 participant
OMN-19543 AC5: a .200 verify-class CI runner so the dev-200 lane can be verified
Ticket: OMN-19543 (AC5)
Epic: OMN-19500
(Not OMN-19657 -- that ticket's own gate/probe covers the separate omnimarket dead-letter-prune scheduling PR, #2919. This PR was opened under that number by a shared CLAIM row covering both pieces of work; retitled to the ticket whose acceptance criteria this PR actually falsifies.)
Summary
Adds a new always-on verify runner on host
.200(Mac Studio M2 Ultra) to the runner fleet. This runner supports the dev-200 deploy lane being built in parallel PRs (#4118, #4128) by providing a dedicated arm64 host for per-merge verify jobs. The runner is declared in the fleet config, a new Docker Compose file, and the arm64 proof workflow. It is intentionally minimal: no lab kubeconfig, no operator env file, and no OMNI_HOME clone bind.Changes
config/runner_fleet.yaml.200:stickybeatz-studio.tail75df5e.ts.netarm64omninode-studio-runner1[verify]pmsetor LaunchDaemon keep-awake configuration is required; this host is the always-on primary work box.docker/docker-compose.runners-omninode-studio-runner.ymlomninode-studio-runner-1, modelled on the existing.202lane-pinned verify runner (docker-compose.runners-omnipc2-verify-runner.yml).host.docker.internalRUNNER_LABELS:self-hosted,omnibase-verify,host-200,linux,arm64,arch-arm641.02g.202runner shape.docker compose up, same shape as.202's) and the readback commands..github/workflows/arm64-verify-runner-proof.ymlhost-200matrix leg. This is required because the host is arm64 and the multi-host contract testtest_every_arm64_verify_host_has_a_leg_in_the_proof_workflowenforces it.host-200), since this runner mounts no kubeconfig by design — the same reasonhost-202(amd64, so out of this arm64-only workflow) carries none either.tests/unit/observability/runner_health/test_runner_fleet_hosts.py.200host to the pinnedEXPECTED_HOSTStuple.tests/ci/test_studio_verify_runner_omn19657.pytests/ci/test_omnipc2_verify_runner_omn19507.pyfor the.202runner: labels, architecture labels, exclusivity of thehost-200label, absence of forbidden mounts, CPU/memory caps, nogroup_add.Why
The operator approved a
.200CI runner. This approval is recorded in theOPERATOR-CONSENTrow indocs/tracking/ROLLING_WORK_LEDGER.md(line 4842,2026-09-25T18:27:30Z), answering an open question first raised in aTERMINALrow on the same file (2026-09-25T10:29:02Z): "may a verify runner be registered on .200". This change provides the runner infrastructure so the dev-200 deploy lane (OMN-19543) has a host that can execute its per-merge verify job.What this PR does not do
.200. Standing up the runner is a separate, manual operator step, documented in the new compose file's own header comment..201only, per operator approval.Operator steps (to stand up the runner once this PR lands)
gh api -X POST orgs/OmniNode-ai/actions/runners/registration-token, org-admin scope required). It is valid for 1 hour..200(stickybeatz-studio, this Mac):RUNNER_DIR=$HOME/.omnibase/runners, rsync the runner artifacts and this compose file under it, build/load theomninode-runner:latestimage, then:STUDIO_RUNNER_TOKENenv var at the moment of thedocker compose up, the same wayMINI_RUNNER_TOKEN/AIR_RUNNER_TOKEN/OMNIPC2_VERIFY_RUNNER_TOKENare passed for the other per-host runners: never written to a file in this repo, never printed, never committed.gh api orgs/OmniNode-ai/actions/runners --paginate --jq '.runners[] | select(.name=="omninode-studio-runner-1")'— expectstatus: "online"and labelsself-hosted, omnibase-verify, host-200, linux, arm64, arch-arm64. Optionallyworkflow_dispatch.github/workflows/arm64-verify-runner-proof.ymlfor the four-assertion proof (arch, toolchain, docker socket; the credentials step is skipped for this host by design).Tests
uv run pyteston the touched unit, integration, and CI test files: all green (206 passed, 1 skipped — the skip is the seam test for the dev-200 routing table's ownverify:block, which lands in a sibling PR).test_runner_fleet_hosts.py,test_runner_fleet_config.py)test_runner_fleet_multihost_contract.py,test_runner_fleet_config_contract.py)test_studio_verify_runner_omn19657.py)deploy-runners.sh --host=targeting tests (test_deploy_runners_host_targeting.py) unaffected, confirmed still greenpre-commit run --files <changed>: all passed (compose validated withdocker compose config -q, workflow and fleet YAML parsed, ruff/mypy/SPDX clean).PR body drafted via
onex delegate(run id13337610-4f34-42c1-86bf-8bfbf05b69cf, quality_gate_passed=true, score=1.0); facts checked against the diff and hand-edited for the operator-steps section, which was written by hand.Evidence-Ticket: OMN-19543
Evidence-Source: OCC#11260