Repository navigation
fix(OMN-14900): container-recreate-durable safe.directory + private runner OMNI_HOME write targets for release-train scripts - #2393
Conversation
…ry on all runtime_build git invocations
Five release-train-lab.yml runs (2026-07-21T03:59-04:19Z) died on uid-mismatch
against the SHARED /data/omninode/omni_home clones bind-mounted into
omninode-deploy-runner: dubious ownership (128), FETCH_HEAD EACCES (255),
'dev' already checked out in a host worktree (128). The live relief was
exec-applied container state + an uncommitted compose hand-edit -- dropped by
any --force-recreate. This makes the fix committed and recreate-durable:
- docker-compose.runners.yml (deploy runner only): PRIVATE OMNI_HOME via
DEPLOY_RUNNER_OMNI_HOME (fail-fast :? interpolation), identical
host:container bind path (docker-outside-of-docker requirement), SHARED
${OMNI_HOME} mount REMOVED (shared-clone writes structurally impossible);
committed root-phase chown init wrapper; reconciled to live reality:
repo-scoped GITHUB_ORG_URL + explicitly-empty RUNNER_GROUP (repo-scoped
config.sh rejects --runnergroup); GIT_CONFIG_* safe.directory env retained
as defense-in-depth, retargeted at the private clones.
- runners/entrypoint.sh: --runnergroup now conditional on non-empty
RUNNER_GROUP (no-colon default so explicit-empty survives) -- prevents
bricked re-registration after recreate.
- NEW ensure_runner_clones.sh: idempotent provisioning of the 5 private
clones (public repos, anonymous https; base URL overridable for tests),
euid-operability assertion, called from all three entry scripts.
- refresh_stability_lane.sh: git_clone() safe.directory wrapper ported from
refresh_dev_lane.sh; every bare git -C replaced; ensure call added.
- refresh_dev_lane.sh: ensure call added (wrapper already present).
- cut_release_train_tag.sh: scoped wrapper; execute-mode ensure + fetch
before tag-name mint; GitHub tag ref now created via gh api
(repos/.../git/refs) with the workflow token -- private clones carry no
push credentials (release-train-lab.yml cut-tag job gets
permissions: contents: write + GH_TOKEN).
- stage_workspace.sh, deploy_source_ref.py, check_sibling_lock_pins.py:
safe.directory scoping on every git invocation (RT-1 write path no longer
depends on non-durable container env GIT_CONFIG_* pass-through).
- Seeded-failure tests (tests/unit/scripts/
test_runner_private_omni_home_safe_directory.py): 13 of 20 RED at
24ba852 pre-fix, 20 GREEN post-fix -- static no-bare-git-C guards,
GIT_TEST_ASSUME_DIFFERENT_OWNER behavioral RED/GREEN, compose
config-as-data recreate-durability assertions, ensure-script behavioral
suite on file:// fixtures.
- Runbook + workflow header refreshed; recreate procedure documented
(operator-gated; live container untouched by this PR).
Sibling tag resolution decision (Q2): externally-cut lab tags fail LOUDLY at
RT-1 sibling checkout (private sibling clones cannot resolve them);
runner-cut tags resolve via local tags persisted on the private bind.
Refs OMN-14900 (resume condition 1). Context is referenced by PR number only
(release-train trigger infra#2388; runner listener-liveness infra#2194;
sibling-clone force/mixed-ref follow-up filed off infra#2388) because
deploy-gate treats a bare ticket citation as an evidence binding: this PR's
deploy evidence binds OMN-14900 alone, and its OCC companion is authored in
Codex's lane.
|
Warning Review limit reachedYou’ve reached a temporary PR review limit under our Fair Usage Limits Policy. Next review available in: 40 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (13)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
|
| Verdict | Meaning | Blocks merge? |
|---|---|---|
passed |
No critical findings | No |
blocked |
CRITICAL findings found | Yes |
degraded |
All models unavailable (infra) | No (pilot) |
Powered by omniintelligence.review_pairing.cli_review — node-based adversarial review via HandlerLlmCliSubprocess (OMN-8468/OMN-8524)
1 participant
Ticket
OMN-14900 — release-train-lab runner write-path hardening. This PR makes the 2026-07-21 exec-applied live relief container-recreate-durable: every fix lives in committed scripts/compose/workflow config, none in mutable container state.
Mode-1 evidence — five red
release-train-lab.ymlruns, 2026-07-21All five runs failed on uid-mismatch writes against the SHARED
/data/omninode/omni_homeclones bind-mounted intoomninode-deploy-runner:Failure modes observed across the five runs:
fatal: detected dubious ownership(exit 128),FETCH_HEADEACCES (exit 255), and'dev' is already checked outin a host worktree (exit 128). The live relief was an exec-appliedgit config --global safe.directoryinside the running container plus an uncommitted compose hand-edit — state that any--force-recreatesilently drops. That non-durability is the exact failure mode this PR eliminates.Seam / write-target table (file-by-file)
docker/docker-compose.runners.ymlOMNI_HOMEbind (all git write targets)DEPLOY_RUNNER_OMNI_HOME(fail-fast:?interpolation); identical host:container bind path (docker-outside-of-docker requirement); SHARED${OMNI_HOME}mount REMOVED — shared-clone writes are now structurally impossible; committed root-phase chown init wrapper; reconciled to live reality: repo-scopedGITHUB_ORG_URL+ explicitly-emptyRUNNER_GROUP;GIT_CONFIG_*safe.directory env retained as defense-in-depth, retargeted at the private clonesdocker/runners/entrypoint.shconfig.sh)--runnergroupnow conditional on non-emptyRUNNER_GROUP(no-colon default so explicit-empty survives) — prevents bricked re-registration after recreate (repo-scopedconfig.shrejects--runnergroup)scripts/runtime_build/ensure_runner_clones.sh(NEW)$OMNI_HOME/<repo>inside the runner)scripts/runtime_build/refresh_stability_lane.shgit_clone()safe.directory wrapper ported fromrefresh_dev_lane.sh; every baregit -Creplaced; ensure call addedscripts/runtime_build/refresh_dev_lane.shscripts/runtime_build/cut_release_train_tag.shgh api repos/.../git/refswith the workflow token — private clones carry no push credentials.github/workflows/release-train-lab.ymlpermissions: contents: write+GH_TOKENfor the gh-api tag-ref creation; header refreshedscripts/runtime_build/stage_workspace.shGIT_CONFIG_*pass-through)scripts/runtime_build/deploy_source_ref.pyscripts/runtime_build/check_sibling_lock_pins.pydocs/runbooks/release-train-lab.mdtests/unit/scripts/test_runner_private_omni_home_safe_directory.py(NEW)tests/scripts/test_deploy_runtime_build_context.pyDesign decision (Q2, sibling tag resolution): externally-cut lab tags fail LOUDLY at RT-1 sibling checkout (private sibling clones cannot resolve them); runner-cut tags resolve via local tags persisted on the private bind.
Seeded-failure tests
tests/unit/scripts/test_runner_private_omni_home_safe_directory.py: 13 of 20 RED at pre-fix base24ba852d, 20 GREEN post-fix. Coverage: static no-bare-git -Cguards across the runtime_build scripts,GIT_TEST_ASSUME_DIFFERENT_OWNERbehavioral RED/GREEN (real dubious-ownership reproduction, not a surrogate), compose config-as-data recreate-durability assertions (the fix must survive--force-recreateby construction), and an ensure-script behavioral suite onfile://fixtures.D1 — runner recreate is operator-gated (this PR enables it, does not perform it)
The live
omninode-deploy-runnercontainer is untouched by this PR. Applying the fix requires a container recreate withDEPLOY_RUNNER_OMNI_HOMEset — an operator-gated follow-up whose procedure is documented indocs/runbooks/release-train-lab.md. This PR is the prerequisite that makes that recreate safe and durable instead of state-destroying.OCC companion
The OCC evidence companion is Codex's lane — this PR authors no evidence for itself (implementer must not author their own evidence). The pre-push deploy-scope DoD hook returned
NOTICE_COMPANION_UNMERGEDfordocker/docker-compose.runners.yml; the hosteddeploy-gate/ Receipt-Gate are expected RED (companion-shape) until the companion lands and bindsEvidence-Ticket/Evidence-Source.Do not auto-merge. Merges are Codex's.
Local gates (author-run)
tests/ci/ tests/unit/scripts/→ 1661 passed, 1 skipped.NOTICE_COMPANION_UNMERGED(expected — companion unmerged).Evidence-Ticket: OMN-14900
Evidence-Source: OCC#4619
Evidence-Commit: 3039a089904f970ef3ad89419f39e9b918e7bab1