From f18eee68f0933eb3206f909c4d8079e952647bb3 Mon Sep 17 00:00:00 2001 From: jonahgabriel Date: Mon, 17 Aug 2026 02:29:55 -0400 Subject: [PATCH] fix(OMN-16114): extend C2 git-mirror rewrite to sibling actions/checkout steps The C2 git mirror only ever accelerated two things: the primary GITHUB_REPOSITORY checkout (workspace pre-seed, keyed off GITHUB_REPOSITORY) and uv's git-dependency fetches (a fetch-only insteadOf for the .git-suffixed URL form). Neither mechanism ever looked at a step's `repository:` input, so sibling `actions/checkout` steps -- e.g. dispatcher-route-coverage.yml's "Checkout omnimarket (sibling)" -- went straight to github.com with zero acceleration even when the target repo was already mirrored. Confirmed root cause of ~27% of dispatcher-route-coverage runs timing out at the 30-minute job budget (RPC 408 / GnuTLS -110 / empty-reply, retried unbounded until the timeout killed the job); ~13 workflow files carry the same unmirrored sibling-checkout pattern. Adds wire_sibling_checkout_mirror_rewrite (C2c) to docker/runners/runner-job-started.sh: it reads the current job's own workflow YAML (via GITHUB_WORKFLOW_REF + GITHUB_JOB, fetched at GITHUB_SHA from the already-pre-seeded own-repo workspace) to discover every (repo, ref) pair a sibling actions/checkout step in THIS job will request, proves the mirror can serve every one of them (by-SHA probe for exact-SHA pins, ls-remote for branch/tag literals), and only then installs a fetch-only insteadOf for that repo. A GHA expression ref, a missing ref, or any unservable pin disqualifies the whole repo for the job -- conjunctive gating, since insteadOf rewrites the URL, not one ref, so a partially-servable repo (e.g. ci.yml's application-database-domain-enforcement job checks out omnibase_infra twice at two different pinned SHAs) must not be redirected at all. own_repo is excluded unconditionally, even under a self-referential `repository:` entry, because it shares the primary checkout's URL form and the primary fetches a possibly seconds-old GITHUB_SHA the mirror may not have yet. Refactors wire_uv_git_mirror_rewrite (C2b) to share a new accumulator (_c2_rewrite_add_pair / _c2_rewrite_flush) with the new function: both mechanisms discover fetch-only insteadOf pairs into one GIT_CONFIG_COUNT namespace and flush once, so the two GITHUB_ENV writes can no longer clobber each other's KEY_i/VALUE_i entries. Also factors the GITHUB_SHA delta-fetch (previously inline in C2b) into a shared, idempotent _c2_ensure_head_sha_fetched helper reused by both. Fail-open throughout, matching every existing mirror mechanism: missing GITHUB_ENV, missing git, unseeded workspace, unreadable workflow file, no sibling checkouts in this job's block, or any probe miss all fall through to today's behavior (github.com, unmirrored). No workflow file changes -- this is a mechanism-only fix reachable by all ~13 affected workflows without per-workflow patches. Verified empirically (not assumed) that insteadOf has no server-side fallback: a sibling checkout whose exact SHA is absent from the mirror is never rewritten, because if it were, actions/checkout would hard-fail rather than falling through to github.com. New test file tests/scripts/test_runner_job_started_mirror_rewrite.py exercises this and seven other scenarios (branch-ref present/absent, exact-SHA present/absent, conjunctive gating across two pins of one repo, a dynamic GHA-expression ref, own-repo exclusion, and the shared-accumulator merge) against a real local `git daemon`, matching the production daemon's own invocation shape -- no mocking of git. 9/10 pass locally; the 10th (full-script subprocess run) is gated on GNU `realpath -m` exactly like the existing debris-cleanup test file and runs for real on the Ubuntu 22.04 CI runners. Host-side note: runner-job-started.sh is bind-mounted by inode from /home/jonah/.omnibase/runners/docker/runners/runner-job-started.sh on the runner host. This PR is repo-side only -- the host copy still needs the documented in-place `cat > ...` update (never mv/install, see docker/runners/README-c2b-uv-git-mirror.md's inode-splitting trap) after merge to take effect on the fleet. No other host change is required: this reuses the existing mirror set, refresh timer, and per-mirror allowFilter/allowAnySHA1InWant config as-is. OMN-16114 --- docker/runners/runner-job-started.sh | 350 +++++++- .../test_runner_job_started_mirror_rewrite.py | 825 ++++++++++++++++++ 2 files changed, 1138 insertions(+), 37 deletions(-) create mode 100644 tests/scripts/test_runner_job_started_mirror_rewrite.py diff --git a/docker/runners/runner-job-started.sh b/docker/runners/runner-job-started.sh index a045226c28..05d9331518 100755 --- a/docker/runners/runner-job-started.sh +++ b/docker/runners/runner-job-started.sh @@ -223,6 +223,73 @@ seed_workspace_from_mirror() { # treated as a miss and simply forgoes the speedup. _C2B_REWRITE_REPOS="${OMNI_GIT_MIRROR_REWRITE_REPOS:-onex_change_control}" +# --------------------------------------------------------------------------- +# Shared rewrite accumulator (OMN-16063 C2b + OMN-16114 C2c) +# --------------------------------------------------------------------------- +# +# GIT_CONFIG_COUNT is a single flat index namespace in the job's environment. +# The two mechanisms below (the uv git-dependency rewrite, and the OMN-16114 +# sibling actions/checkout rewrite further down) each discover their own set +# of fetch-only insteadOf pairs to install. If each wrote its own +# `GIT_CONFIG_COUNT=N` line to GITHUB_ENV independently, the SECOND write +# would win -- GITHUB_ENV lines just set env vars, a later same-named write +# replaces rather than merges, silently dropping the first mechanism's +# GIT_CONFIG_KEY_0/VALUE_0 entries even though both lines are present in the +# file. Both mechanisms append to this shared accumulator instead; exactly +# one flush, after both have run, writes the combined set. +declare -a _C2_REWRITE_ENV_LINES=() +_C2_REWRITE_COUNT=0 + +# Adds one fetch-only redirect: fetches of $2 (upstream) resolve to $1 +# (mirror); pushes to $2 stay pinned at $2 via an identity pushInsteadOf +# (git resolves pushInsteadOf before insteadOf, so this pins the push path +# back on github.com without touching the fetch redirect -- see the "Push is +# pinned back on github.com" note in docker/runners/README-c2b-uv-git-mirror.md +# for the field verification this relies on). +_c2_rewrite_add_pair() { + local mirror_url="$1" upstream_url="$2" + _C2_REWRITE_ENV_LINES+=("GIT_CONFIG_KEY_${_C2_REWRITE_COUNT}=url.${mirror_url}.insteadOf") + _C2_REWRITE_ENV_LINES+=("GIT_CONFIG_VALUE_${_C2_REWRITE_COUNT}=${upstream_url}") + _C2_REWRITE_COUNT=$((_C2_REWRITE_COUNT + 1)) + _C2_REWRITE_ENV_LINES+=("GIT_CONFIG_KEY_${_C2_REWRITE_COUNT}=url.${upstream_url}.pushInsteadOf") + _C2_REWRITE_ENV_LINES+=("GIT_CONFIG_VALUE_${_C2_REWRITE_COUNT}=${upstream_url}") + _C2_REWRITE_COUNT=$((_C2_REWRITE_COUNT + 1)) +} + +_c2_rewrite_flush() { + if [[ "${_C2_REWRITE_COUNT}" -eq 0 ]]; then + return 0 + fi + if [[ -z "${GITHUB_ENV:-}" || ! -w "${GITHUB_ENV}" ]]; then + echo "[c2-mirror-rewrite] GITHUB_ENV unwritable; ${_C2_REWRITE_COUNT} discovered rewrite pair(s) not applied (fail-open)." + return 0 + fi + { + printf '%s\n' "${_C2_REWRITE_ENV_LINES[@]}" + echo "GIT_CONFIG_COUNT=${_C2_REWRITE_COUNT}" + } >>"${GITHUB_ENV}" 2>/dev/null || { + echo "[c2-mirror-rewrite] GITHUB_ENV write failed; job unaffected (fail-open)." + return 0 + } + return 0 +} + +# Fetches GITHUB_SHA into the pre-seeded own-repo workspace, once, so both +# mechanisms below can read files (uv.lock, workflow YAML) EXACTLY as they +# exist at the commit this job builds -- not the mirror's default-branch tip, +# which a PR may have bumped away from. Idempotent: a second call with the +# object already present is a fast no-op fetch, not a re-clone. +_c2_head_sha_fetched=0 +_c2_ensure_head_sha_fetched() { + local workspace_dir="$1" infra_mirror="$2" + if [[ "${_c2_head_sha_fetched}" -eq 1 ]]; then + return 0 + fi + _c2_head_sha_fetched=1 + [[ -n "${GITHUB_SHA:-}" ]] || return 0 + timeout 60 git -C "${workspace_dir}" fetch --quiet "${infra_mirror}" "${GITHUB_SHA}" 2>/dev/null || true +} + wire_uv_git_mirror_rewrite() { local workspace_dir="$1" @@ -247,11 +314,10 @@ wire_uv_git_mirror_rewrite() { # default-branch tip: a PR that bumps the pin would otherwise be verified # against the wrong SHA. The pre-seed left full default-branch history in # the workspace, so fetching GITHUB_SHA is a small delta, not a clone. + _c2_ensure_head_sha_fetched "${workspace_dir}" "${infra_mirror}" local lock_blob="" if [[ -n "${GITHUB_SHA:-}" ]]; then - if timeout 60 git -C "${workspace_dir}" fetch --quiet "${infra_mirror}" "${GITHUB_SHA}" 2>/dev/null; then - lock_blob="$(git -C "${workspace_dir}" cat-file -p "${GITHUB_SHA}:uv.lock" 2>/dev/null || true)" - fi + lock_blob="$(git -C "${workspace_dir}" cat-file -p "${GITHUB_SHA}:uv.lock" 2>/dev/null || true)" fi if [[ -z "${lock_blob}" ]]; then # Deliberately NOT falling back to the seeded default-branch uv.lock. @@ -263,8 +329,7 @@ wire_uv_git_mirror_rewrite() { return 0 fi - local repo pin rewrite_count=0 - local -a env_lines=() + local repo pin for repo in ${_C2B_REWRITE_REPOS}; do # Never rewrite the job's own repository -- that is checkout's remote. if [[ "${repo}" == "${own_repo}" ]]; then @@ -307,46 +372,253 @@ wire_uv_git_mirror_rewrite() { continue fi - # Only the `.git` form -- see the scoping note above. + # Only the `.git` form -- see the scoping note above. FETCH-ONLY: see + # _c2_rewrite_add_pair for why an identity pushInsteadOf is also + # installed (`insteadOf` on its own ALSO rewrites the push URL -- this + # is documented git behaviour, not an edge case; verified in + # docker/runners/README-c2b-uv-git-mirror.md that an unpinned push + # dies with "access denied or repository not exported" against the + # daemon, which deliberately serves no receive-pack). local upstream_url="https://github.com/${GITHUB_REPOSITORY%%/*}/${repo}.git" - env_lines+=("GIT_CONFIG_KEY_${rewrite_count}=url.${repo_mirror}.insteadOf") - env_lines+=("GIT_CONFIG_VALUE_${rewrite_count}=${upstream_url}") - rewrite_count=$((rewrite_count + 1)) - - # Make the redirect FETCH-ONLY. `insteadOf` on its own ALSO rewrites the - # push URL -- this is documented git behaviour, not an edge case, and it - # was verified here: with only the insteadOf above installed, - # `git remote -v` reported - # origin git://172.18.0.1:9418/onex_change_control.git (push) - # and a push died with "access denied or repository not exported" - # because the daemon deliberately serves no receive-pack. Any job that - # pushed to the `.git`-suffixed URL would have failed CLOSED -- exactly - # the class of outcome the rest of this function is gated to prevent. - # - # git resolves a push URL against pushInsteadOf rules FIRST and only - # falls back to insteadOf when none match, so an IDENTITY pushInsteadOf - # (upstream -> itself) pins pushes back on github.com while leaving the - # fetch redirect fully intact. Verified in a throwaway container off - # omninode-runner:latest: fetch -> git://172.18.0.1:9418/..., push -> - # https://github.com/..., and the by-SHA fetch uv performs still rc=0. - env_lines+=("GIT_CONFIG_KEY_${rewrite_count}=url.${upstream_url}.pushInsteadOf") - env_lines+=("GIT_CONFIG_VALUE_${rewrite_count}=${upstream_url}") - rewrite_count=$((rewrite_count + 1)) - + _c2_rewrite_add_pair "${repo_mirror}" "${upstream_url}" echo "[c2-mirror-rewrite] ${repo} pin ${pin:0:12} present on mirror; uv git fetch -> ${repo_mirror} (fetch-only; push and actions/checkout stay on github.com)." done - if [[ "${rewrite_count}" -eq 0 ]]; then + return 0 +} + +# --------------------------------------------------------------------------- +# OMN-16114 C2c -- extend the fetch-only rewrite to sibling actions/checkout +# steps +# --------------------------------------------------------------------------- +# +# THE GAP THIS CLOSES. C2's pre-seed (seed_workspace_from_mirror) only warms +# the workspace for the job's OWN repository -- it is keyed off +# GITHUB_REPOSITORY alone and never looks at a step's `repository:` input. +# C2b's rewrite above only rewrites the `.git`-suffixed URL form uv's git +# dependencies use. Neither mechanism ever touches a SIBLING `actions/checkout` +# step -- a second checkout in the same job with an explicit `repository:` +# different from GITHUB_REPOSITORY, e.g. dispatcher-route-coverage.yml's +# "Checkout omnimarket (sibling)" step. Those go straight to github.com with +# zero acceleration even when the target repo is already mirrored. Confirmed +# root cause of ~27% of `dispatcher-route-coverage` job runs timing out at the +# 30-minute budget (RPC 408 / GnuTLS -110 / empty-reply, retried unbounded +# until the job timeout kills it) -- see OMN-16114. +# +# WHY THIS CANNOT BE A SINGLE UNCONDITIONAL insteadOf. Unlike C2b, which +# rewrites ONE well-known dependency with a pin resolvable from uv.lock, +# sibling checkouts are numerous, per-workflow, and fetch a mix of branch +# names (`ref: dev`), historical exact SHAs, and refs computed by an earlier +# step in the SAME job (unresolvable here, before any step has run). +# `insteadOf` has NO server fallback (verified in +# docker/runners/README-c2b-uv-git-mirror.md): once a URL is rewritten, a +# fetch the mirror cannot serve fails outright -- git does not then retry the +# un-rewritten URL. A blanket per-repo rewrite would therefore convert "mirror +# is stale for this one ref" into a hard failure, worse than the flakiness +# this component exists to remove. So this function does not trust the mirror +# by default -- it discovers exactly which (repo, ref) pairs THIS job's steps +# will request and proves the mirror can serve every one of them first. +# +# DISCOVERY MECHANISM. GITHUB_WORKFLOW_REF (a standard Actions env var, e.g. +# "OmniNode-ai/omnibase_infra/.github/workflows/ci.yml@refs/heads/dev") names +# the exact workflow file for this run; GITHUB_JOB names the exact job. Both +# are read from the OWN repo's checkout at GITHUB_SHA (the delta fetch shared +# with C2b via _c2_ensure_head_sha_fetched, so a PR that edits its own +# workflow file is scanned as it exists on THIS commit, not the mirror's +# stale default-branch copy). The job's YAML block is scanned for +# `repository: OmniNode-ai/` / `ref: ` pairs with plain text +# matching, not a YAML parser -- this repo's workflow files use a regular, +# predictable indentation shape, and a text scan avoids adding a new +# interpreter dependency to a hook that runs before every job on the fleet. +# +# CONJUNCTIVE PER-REPO GATING. `insteadOf` operates on the URL, not on a +# specific ref -- once a repo's rewrite is installed, it applies to EVERY +# fetch against that URL for the rest of the job, including ones this +# function never saw. So a repo is only made rewrite-eligible when EVERY +# (repo, ref) pair discovered for it in this job's block is individually +# proven servable: +# - a 40-hex-char ref is treated as an exact SHA and probed exactly like +# C2b's uv pin (`fetch --depth=1 --filter=tree:0`). This is the empirical +# answer to "does an exact-SHA fetch fall through to origin if the mirror +# lacks it" the ticket asked to verify, not assume: it does not -- so +# this must be proven true before any rewrite is installed. A miss here +# disqualifies the WHOLE repo for this job, not just that one occurrence +# (ci.yml's application-database-domain-enforcement job checks out +# omnibase_infra twice at two different pinned SHAs; both must be +# servable or neither checkout is redirected). +# - any other literal (branch/tag name, e.g. `dev`) is probed with +# `git ls-remote --exit-code`, which the 120s-refreshed mirror +# (config/runner_fleet.yaml git_mirror.refresh_interval_seconds) almost +# always answers. +# - a ref containing a GitHub Actions expression (`${{ ... }}`) or with no +# `ref:` line at all (defaults to the target repo's default branch, +# unknowable here) is unresolvable and disqualifies the repo outright -- +# never installed on a guess. +# +# NEVER THE JOB'S OWN REPOSITORY. `own_repo` is excluded unconditionally, by +# construction, even where it appears under an explicit `repository:` key +# (ci.yml's `.proof-dependencies` job checks out omnibase_infra-the-repo +# twice more, at older pinned SHAs, from an omnibase_infra job). Both the +# primary checkout and these self-referential sibling checkouts share the +# identical no-`.git`-suffix URL form actions/checkout always computes +# (`${GITHUB_SERVER_URL}/${owner}/${repo}`), so a rewrite keyed on that URL +# cannot tell them apart. The primary checkout fetches GITHUB_SHA -- the PR's +# own, possibly seconds-old head commit -- which the 120s-refresh mirror can +# easily not have yet; redirecting it with no fallback would break every job +# on every fresh push. Losing acceleration on the rare same-repo sibling +# checkout is the accepted, deliberate cost of that safety margin. +_C2C_SIBLING_PROBE_TIMEOUT="${OMNI_GIT_MIRROR_CHECKOUT_PROBE_TIMEOUT:-20}" + +wire_sibling_checkout_mirror_rewrite() { + local workspace_dir="$1" + + # `if`, not an AND-list: same `set -e` reasoning as every kill switch + # above -- a false AND-list would return non-zero and kill the hook. + if [[ "${OMNI_GIT_MIRROR_CHECKOUT_REWRITE_DISABLE:-0}" == "1" ]]; then + return 0 + fi + if [[ "${OMNI_GIT_MIRROR_DISABLE:-0}" == "1" ]]; then return 0 fi + if [[ -z "${GITHUB_ENV:-}" || ! -w "${GITHUB_ENV}" ]]; then + return 0 + fi + command -v git >/dev/null 2>&1 || return 0 + [[ -d "${workspace_dir}/.git" ]] || return 0 + [[ -n "${GITHUB_JOB:-}" ]] || return 0 + + local workflow_path="${GITHUB_WORKFLOW_REF:-}" + workflow_path="${workflow_path%%@*}" + case "${workflow_path}" in + "${GITHUB_REPOSITORY:-}"/*) + workflow_path="${workflow_path#"${GITHUB_REPOSITORY}"/}" + ;; + *) + return 0 + ;; + esac - { - printf '%s\n' "${env_lines[@]}" - echo "GIT_CONFIG_COUNT=${rewrite_count}" - } >>"${GITHUB_ENV}" 2>/dev/null || { - echo "[c2-mirror-rewrite] GITHUB_ENV write failed; job unaffected (fail-open)." + local own_repo="${GITHUB_REPOSITORY##*/}" + local infra_mirror="git://${_C2_MIRROR_HOST}:${_C2_MIRROR_PORT}/${own_repo}.git" + + _c2_ensure_head_sha_fetched "${workspace_dir}" "${infra_mirror}" + local workflow_blob="" + if [[ -n "${GITHUB_SHA:-}" ]]; then + workflow_blob="$(git -C "${workspace_dir}" cat-file -p "${GITHUB_SHA}:${workflow_path}" 2>/dev/null || true)" + fi + if [[ -z "${workflow_blob}" ]]; then + echo "[c2-mirror-rewrite] could not read ${workflow_path} at ${GITHUB_SHA:-}; no sibling checkout rewrite this job (fail-open)." return 0 - } + fi + + # Isolate this job's own block: a top-level (2-space-indented) key + # matching GITHUB_JOB up to the next top-level key. Workflow files with + # many jobs (ci.yml has 40+) would otherwise pay a probe round-trip per + # job on every single run, most of which have no sibling checkout at all. + local job_block + job_block="$(printf '%s\n' "${workflow_blob}" | awk -v job="${GITHUB_JOB}" ' + $0 ~ "^ " job ":[[:space:]]*$" { injob=1; next } + injob && /^ [A-Za-z0-9_.-]+:[[:space:]]*$/ { injob=0 } + injob { print } + ')" + [[ -n "${job_block}" ]] || return 0 + + # (repo, ref) pairs in encounter order, TSV. A `repository:` line starts + # a pending pair; the next `ref:` line completes it; hitting the next + # step boundary (`- name:` / `- uses:`) with no `ref:` seen completes it + # with an empty ref (unresolved -- defaults to the repo's default branch). + local pins + pins="$(printf '%s\n' "${job_block}" | awk ' + /repository:[[:space:]]*OmniNode-ai\// { + repo = $0 + sub(/.*OmniNode-ai\//, "", repo) + gsub(/[[:space:]]*#.*/, "", repo) + gsub(/[[:space:]]+$/, "", repo) + pending_repo = repo + next + } + pending_repo != "" && /^[[:space:]]*ref:/ { + r = $0 + sub(/^[[:space:]]*ref:[[:space:]]*/, "", r) + gsub(/[[:space:]]*#.*$/, "", r) + gsub(/[[:space:]]+$/, "", r) + print pending_repo "\t" r + pending_repo = "" + next + } + pending_repo != "" && /^[[:space:]]*-[[:space:]]*(name|uses):/ { + print pending_repo "\t" + pending_repo = "" + } + END { + if (pending_repo != "") print pending_repo "\t" + } + ')" + [[ -n "${pins}" ]] || return 0 + + local pins_file + pins_file="$(mktemp)" + printf '%s\n' "${pins}" >"${pins_file}" + + local repo + for repo in $(cut -f1 "${pins_file}" | sort -u); do + if [[ "${repo}" == "${own_repo}" ]]; then + continue + fi + + local repo_mirror="git://${_C2_MIRROR_HOST}:${_C2_MIRROR_PORT}/${repo}.git" + local all_ok=1 ref + while IFS= read -r ref; do + if [[ -z "${ref}" ]]; then + all_ok=0 + break + fi + case "${ref}" in + *'${{'*) + all_ok=0 + break + ;; + esac + + if [[ "${ref}" =~ ^[0-9a-f]{40}$ ]]; then + # Exact SHA -- same by-SHA servability probe C2b uses for the + # uv pin (see the long comment above this function for why + # this specific check exists). + local probe_dir probe_ok=0 + probe_dir="$(mktemp -d)" + if git -C "${probe_dir}" init --quiet 2>/dev/null \ + && timeout "${_C2C_SIBLING_PROBE_TIMEOUT}" git -C "${probe_dir}" fetch --quiet --depth=1 --filter=tree:0 \ + "${repo_mirror}" "${ref}" >/dev/null 2>&1; then + probe_ok=1 + fi + rm -rf "${probe_dir}" + if [[ "${probe_ok}" -ne 1 ]]; then + echo "[c2-mirror-rewrite] ${repo}@${ref:0:12} (exact SHA) not served by ${repo_mirror}; leaving ${repo} sibling checkout(s) on github.com (fail-open)." + all_ok=0 + break + fi + else + # Branch/tag literal -- confirm the mirror currently + # advertises it. + if ! timeout "${_C2C_SIBLING_PROBE_TIMEOUT}" git ls-remote --exit-code "${repo_mirror}" "${ref}" >/dev/null 2>&1; then + echo "[c2-mirror-rewrite] ${repo}@${ref} not advertised by ${repo_mirror}; leaving ${repo} sibling checkout(s) on github.com (fail-open)." + all_ok=0 + break + fi + fi + done < <(awk -F'\t' -v r="${repo}" '$1==r{print $2}' "${pins_file}") + + if [[ "${all_ok}" -eq 1 ]]; then + # No `.git` suffix -- see the scoping note in the header comment + # above this function; this is the exact URL form actions/checkout + # computes (`${GITHUB_SERVER_URL}/${owner}/${repo}`). + local upstream_url="${GITHUB_SERVER_URL:-https://github.com}/${GITHUB_REPOSITORY%%/*}/${repo}" + _c2_rewrite_add_pair "${repo_mirror}" "${upstream_url}" + echo "[c2-mirror-rewrite] ${repo}: every sibling-checkout ref for job ${GITHUB_JOB} served by ${repo_mirror}; actions/checkout -> mirror (fetch-only)." + fi + done + + rm -f "${pins_file}" return 0 } @@ -544,6 +816,8 @@ if rm -rf -- "${canonical_workspace}" 2>"${err_file}"; then mkdir -p -- "${canonical_workspace}" seed_workspace_from_mirror "${canonical_workspace}" wire_uv_git_mirror_rewrite "${canonical_workspace}" || true + wire_sibling_checkout_mirror_rewrite "${canonical_workspace}" || true + _c2_rewrite_flush || true exit 0 fi @@ -569,6 +843,8 @@ elif sudo -n /bin/rm -rf -- "${canonical_workspace}" 2>"${err_file}"; then mkdir -p -- "${canonical_workspace}" seed_workspace_from_mirror "${canonical_workspace}" wire_uv_git_mirror_rewrite "${canonical_workspace}" || true + wire_sibling_checkout_mirror_rewrite "${canonical_workspace}" || true + _c2_rewrite_flush || true exit 0 else echo "[runner-job-started] ERROR: scoped sudo fallback also failed:" >&2 diff --git a/tests/scripts/test_runner_job_started_mirror_rewrite.py b/tests/scripts/test_runner_job_started_mirror_rewrite.py new file mode 100644 index 0000000000..d156d686ea --- /dev/null +++ b/tests/scripts/test_runner_job_started_mirror_rewrite.py @@ -0,0 +1,825 @@ +# SPDX-FileCopyrightText: 2025 OmniNode.ai Inc. +# SPDX-License-Identifier: MIT + +"""Regression coverage for the C2/C2b/C2c fetch-only mirror rewrite mechanisms +in `docker/runners/runner-job-started.sh` (OMN-16063, OMN-16114). + +OMN-16114 extends the existing uv git-dependency rewrite +(`wire_uv_git_mirror_rewrite`, C2b) to sibling `actions/checkout` steps +(`wire_sibling_checkout_mirror_rewrite`, C2c): a second checkout in the same +job with an explicit `repository:` different from the job's own repo, e.g. +dispatcher-route-coverage.yml's "Checkout omnimarket (sibling)" step. Those +previously went straight to github.com with zero mirror acceleration and were +the confirmed root cause of ~27% of `dispatcher-route-coverage` runs timing +out at the 30-minute job budget (see OMN-16114). + +`insteadOf` has no server-side fallback: once a URL is rewritten, a fetch the +mirror cannot serve fails outright rather than retrying the un-rewritten URL +(documented in docker/runners/README-c2b-uv-git-mirror.md, re-verified for +the sibling-checkout case by test_exact_sha_absent_from_mirror_fails_open_not_a_silent_fallthrough +below). Every test here proves either (a) the rewrite is installed only when +the mirror can actually serve every ref the discovered job would request, or +(b) it is correctly withheld (fail-open, job runs on github.com exactly as it +does today) when that cannot be proven. + +TEST STRATEGY. Two tiers: + + 1. Fast, portable, function-level tests (the bulk of this file) that source + only the function-definitions PREFIX of the hook script (everything + before the `wire_pypi_cache || true` main-body marker) and call + `wire_uv_git_mirror_rewrite` / `wire_sibling_checkout_mirror_rewrite` / + `_c2_rewrite_flush` directly against a real local `git daemon`. These + never touch the workspace-reset logic at the bottom of the script, so + they are NOT gated on GNU `realpath -m` and run on Linux CI, `.200`, and + a bare macOS dev box alike. + 2. One end-to-end subprocess test that runs the REAL top-level script + exactly as production invokes it, proving the two new call sites are + actually wired into the success path. This one DOES exercise the + GNU-`realpath -m`-gated workspace-reset logic and is skipped on a BSD + `realpath` host (same skip guard as + test_runner_job_started_root_owned_debris.py) -- it runs for real on + Linux CI and is not the surface these fail-open/gating tests depend on. + +All mirror "hosts" here are a real `git daemon --export-all` serving local +bare repos over 127.0.0.1, mirroring exactly how the production daemon is +started (docker/runners/systemd/omninode-git-mirror-daemon.service): no +mocking of git itself, so the negative-probe assertions (missing SHA, missing +branch) exercise the real git wire protocol, not a stand-in. +""" + +from __future__ import annotations + +import os +import shutil +import socket +import subprocess +import tempfile +import time +from collections.abc import Iterator +from pathlib import Path + +import pytest + +REPO_ROOT = Path(__file__).resolve().parents[2] +HOOK_SCRIPT = REPO_ROOT / "docker" / "runners" / "runner-job-started.sh" +_MAIN_BODY_MARKER = "\nwire_pypi_cache || true\n" + +pytestmark = [pytest.mark.unit] + + +def _has_gnu_realpath_m() -> bool: + try: + result = subprocess.run( + ["realpath", "-m", "."], + capture_output=True, + text=True, + timeout=5, + check=False, + ) + except OSError: + return False + return result.returncode == 0 + + +def _functions_only_script() -> str: + """Everything in the hook script before the main body starts running. + + Sourcing this (instead of the whole file) lets tests call the mirror + functions directly without the script's own `exit 0`/`exit 1` at the + bottom terminating the test process, and without needing GNU + `realpath -m` (only the main body's workspace-confinement check uses it). + """ + text = HOOK_SCRIPT.read_text() + idx = text.index(_MAIN_BODY_MARKER) + assert idx > 0, ( + "expected marker 'wire_pypi_cache || true' not found in " + f"{HOOK_SCRIPT} -- the function-defs/main-body boundary moved; " + "update _MAIN_BODY_MARKER" + ) + return text[:idx] + + +def _find_free_port() -> int: + with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s: + s.bind(("127.0.0.1", 0)) + return int(s.getsockname()[1]) + + +def _run_git(*args: str, cwd: Path, env: dict[str, str] | None = None) -> str: + result = subprocess.run( + ["git", *args], + cwd=cwd, + capture_output=True, + text=True, + timeout=15, + check=True, + env=env, + ) + return result.stdout.strip() + + +def _make_mirror_repo( + mirror_root: Path, + repo: str, + *, + branch: str = "dev", + files: dict[str, str] | None = None, +) -> str: + """Creates or updates `${mirror_root}/${repo}.git`, a bare repo with one + new commit on `branch`, serving-config applied exactly like + `apply_mirror_serving_config` in git-mirror-refresh.sh (allowFilter / + allowAnySHA1InWant -- without these the by-SHA probe this component + depends on cannot succeed against a real daemon). Returns the commit SHA. + + Safe to call more than once for the same `repo` (e.g. to seed a second + commit that references the first commit's SHA): each call uses its own + scratch work dir and either clones a fresh bare repo or pushes into the + existing one. + """ + work = Path(tempfile.mkdtemp(prefix=f"{repo}-work-", dir=mirror_root)) + env = { + **os.environ, + "GIT_AUTHOR_NAME": "test", + "GIT_AUTHOR_EMAIL": "test@example.invalid", + "GIT_COMMITTER_NAME": "test", + "GIT_COMMITTER_EMAIL": "test@example.invalid", + } + _run_git("init", "--quiet", f"--initial-branch={branch}", cwd=work, env=env) + for name, content in (files or {"README.md": f"{repo}\n"}).items(): + path = work / name + path.parent.mkdir(parents=True, exist_ok=True) + path.write_text(content) + _run_git("add", name, cwd=work, env=env) + _run_git("commit", "--quiet", "-m", "seed", cwd=work, env=env) + sha = _run_git("rev-parse", "HEAD", cwd=work, env=env) + + bare = mirror_root / f"{repo}.git" + if bare.exists(): + _run_git( + "push", + "--quiet", + "--force", + str(bare), + f"{branch}:{branch}", + cwd=work, + env=env, + ) + else: + _run_git( + "clone", "--quiet", "--bare", str(work), str(bare), cwd=mirror_root, env=env + ) + _run_git("config", "uploadpack.allowFilter", "true", cwd=bare, env=env) + _run_git("config", "uploadpack.allowAnySHA1InWant", "true", cwd=bare, env=env) + return sha + + +@pytest.fixture +def mirror_daemon(tmp_path: Path) -> Iterator[tuple[str, int, Path]]: + """A real `git daemon --export-all`, matching + omninode-git-mirror-daemon.service's invocation shape, serving + `tmp_path/mirror` on an ephemeral 127.0.0.1 port. Repos are added to the + returned root by the individual tests via `_make_mirror_repo`.""" + if shutil.which("git-daemon") is None and shutil.which("git") is None: + pytest.skip("git not available") + + mirror_root = tmp_path / "mirror" + mirror_root.mkdir() + port = _find_free_port() + + proc = subprocess.Popen( + [ + "git", + "daemon", + "--verbose", + "--reuseaddr", + "--export-all", + f"--base-path={mirror_root}", + "--listen=127.0.0.1", + f"--port={port}", + str(mirror_root), + ], + stdout=subprocess.PIPE, + stderr=subprocess.STDOUT, + text=True, + ) + try: + # Readiness: the daemon logs "Ready to rock" once listening, but + # polling the actual port is a more direct readiness signal and + # avoids depending on a specific log string across git versions. + deadline = time.monotonic() + 10 + ready = False + while time.monotonic() < deadline: + with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s: + s.settimeout(0.2) + if s.connect_ex(("127.0.0.1", port)) == 0: + ready = True + break + time.sleep(0.1) + if not ready: + proc.terminate() + pytest.fail(f"git daemon did not start listening on 127.0.0.1:{port}") + yield "127.0.0.1", port, mirror_root + finally: + proc.terminate() + try: + proc.wait(timeout=5) + except subprocess.TimeoutExpired: + proc.kill() + + +def _base_env( + tmp_path: Path, + host: str, + port: int, + *, + own_repo: str = "omnibase_infra", + github_sha: str, +) -> dict[str, str]: + github_env = tmp_path / "github_env" + github_env.write_text("") + return { + **os.environ, + "OMNI_GIT_MIRROR_HOST": host, + "OMNI_GIT_MIRROR_PORT": str(port), + "GITHUB_ENV": str(github_env), + "GITHUB_REPOSITORY": f"OmniNode-ai/{own_repo}", + "GITHUB_SERVER_URL": "https://github.com", + "GITHUB_SHA": github_sha, + "GITHUB_JOB": "test-job", + "GITHUB_WORKFLOW_REF": f"OmniNode-ai/{own_repo}/.github/workflows/test.yml@refs/heads/dev", + # Isolate from the uv-dependency mechanism unless a test opts in. + "OMNI_GIT_MIRROR_REWRITE_DISABLE": "1", + } + + +def _github_env_pairs(github_env_path: Path) -> tuple[int, list[tuple[str, str]]]: + """Parses the flushed GIT_CONFIG_COUNT / GIT_CONFIG_KEY_i / VALUE_i lines + into (count, [(key_string, value_string), ...]).""" + keys: dict[int, str] = {} + values: dict[int, str] = {} + count = 0 + for line in github_env_path.read_text().splitlines(): + if not line or "=" not in line: + continue + name, _, val = line.partition("=") + if name == "GIT_CONFIG_COUNT": + count = int(val) + elif name.startswith("GIT_CONFIG_KEY_"): + keys[int(name.removeprefix("GIT_CONFIG_KEY_"))] = val + elif name.startswith("GIT_CONFIG_VALUE_"): + values[int(name.removeprefix("GIT_CONFIG_VALUE_"))] = val + pairs = [(keys[i], values[i]) for i in sorted(keys)] + return count, pairs + + +def _run_functions( + *, + workspace_dir: Path, + env: dict[str, str], + calls: str, + tmp_path: Path, +) -> subprocess.CompletedProcess[str]: + """Sources the function-defs prefix of the real hook script, then runs + `calls` (e.g. 'wire_sibling_checkout_mirror_rewrite "$1"; _c2_rewrite_flush') + against it, with `workspace_dir` as $1.""" + driver = tmp_path / "driver.sh" + driver.write_text(_functions_only_script() + "\n" + calls + "\n") + return subprocess.run( + ["bash", str(driver), str(workspace_dir)], + env=env, + capture_output=True, + text=True, + timeout=30, + check=False, + ) + + +def _seed_own_repo_workspace( + tmp_path: Path, host: str, port: int, own_repo: str, workflow_content: str +) -> tuple[Path, str]: + """Builds the own-repo mirror (containing the workflow file at a known + commit) and an empty local workspace git repo pointed at it, matching + what `seed_workspace_from_mirror` would have left behind: a `.git` dir + that can delta-fetch GITHUB_SHA from the own-repo mirror.""" + mirror_root = tmp_path / "mirror" + mirror_root.mkdir(exist_ok=True) + sha = _make_mirror_repo( + mirror_root, + own_repo, + branch="dev", + files={".github/workflows/test.yml": workflow_content}, + ) + workspace = tmp_path / "workspace" + workspace.mkdir() + _run_git("init", "--quiet", cwd=workspace) + return workspace, sha + + +# --------------------------------------------------------------------------- +# wire_sibling_checkout_mirror_rewrite (OMN-16114 C2c) +# --------------------------------------------------------------------------- + + +def test_branch_ref_present_on_mirror_installs_rewrite(tmp_path: Path) -> None: + """The dominant real-world case: dispatcher-route-coverage.yml's + `repository: OmniNode-ai/omnimarket` / `ref: dev` sibling checkout. When + the mirror has that branch, the fetch-only rewrite must be installed.""" + host, port = "127.0.0.1", _find_free_port() + mirror_root = tmp_path / "mirror" + mirror_root.mkdir() + + proc = subprocess.Popen( + [ + "git", + "daemon", + "--reuseaddr", + "--export-all", + f"--base-path={mirror_root}", + f"--listen={host}", + f"--port={port}", + str(mirror_root), + ], + stdout=subprocess.DEVNULL, + stderr=subprocess.DEVNULL, + ) + try: + deadline = time.monotonic() + 10 + while time.monotonic() < deadline: + with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s: + s.settimeout(0.2) + if s.connect_ex((host, port)) == 0: + break + time.sleep(0.1) + + workflow = ( + "jobs:\n" + " test-job:\n" + " steps:\n" + " - name: Checkout omnimarket (sibling)\n" + " uses: actions/checkout@pin\n" + " with:\n" + " repository: OmniNode-ai/omnimarket\n" + " ref: dev\n" + " path: omnimarket\n" + ) + workspace, sha = _seed_own_repo_workspace( + tmp_path, host, port, "omnibase_infra", workflow + ) + _make_mirror_repo(mirror_root, "omnimarket", branch="dev") + + env = _base_env(tmp_path, host, port, github_sha=sha) + result = _run_functions( + workspace_dir=workspace, + env=env, + calls='wire_sibling_checkout_mirror_rewrite "$1"; _c2_rewrite_flush', + tmp_path=tmp_path, + ) + assert result.returncode == 0, result.stderr + + count, pairs = _github_env_pairs(Path(env["GITHUB_ENV"])) + assert count == 2, f"expected one fetch+push pair; got {result.stderr}" + fetch_key = f"url.git://{host}:{port}/omnimarket.git.insteadOf" + assert (fetch_key, "https://github.com/OmniNode-ai/omnimarket") in pairs + push_key = "url.https://github.com/OmniNode-ai/omnimarket.pushInsteadOf" + assert (push_key, "https://github.com/OmniNode-ai/omnimarket") in pairs + finally: + proc.terminate() + proc.wait(timeout=5) + + +def test_branch_ref_absent_from_mirror_fails_open( + mirror_daemon: tuple[str, int, Path], tmp_path: Path +) -> None: + """The mirror is reachable but has never heard of this repo (not in + git_mirror.repos, or not yet mirrored) -- no rewrite, job stays on + github.com exactly as it does today.""" + host, port, _mirror_root = mirror_daemon + workflow = ( + "jobs:\n" + " test-job:\n" + " steps:\n" + " - name: Checkout unmirrored sibling\n" + " uses: actions/checkout@pin\n" + " with:\n" + " repository: OmniNode-ai/never-mirrored\n" + " ref: dev\n" + ) + workspace, sha = _seed_own_repo_workspace( + tmp_path, host, port, "omnibase_infra", workflow + ) + # Deliberately do NOT create a "never-mirrored.git" bare repo on the daemon. + + env = _base_env(tmp_path, host, port, github_sha=sha) + result = _run_functions( + workspace_dir=workspace, + env=env, + calls='wire_sibling_checkout_mirror_rewrite "$1"; _c2_rewrite_flush', + tmp_path=tmp_path, + ) + assert result.returncode == 0, result.stderr + count, pairs = _github_env_pairs(Path(env["GITHUB_ENV"])) + assert count == 0 + assert pairs == [] + assert "not advertised by" in result.stdout + result.stderr + + +def test_exact_sha_present_on_mirror_installs_rewrite( + mirror_daemon: tuple[str, int, Path], tmp_path: Path +) -> None: + host, port, mirror_root = mirror_daemon + pinned_sha = _make_mirror_repo(mirror_root, "onex_change_control", branch="dev") + workflow = ( + "jobs:\n" + " test-job:\n" + " steps:\n" + " - name: Checkout OCC exact pin\n" + " uses: actions/checkout@pin\n" + " with:\n" + " repository: OmniNode-ai/onex_change_control\n" + f" ref: {pinned_sha}\n" + ) + workspace, sha = _seed_own_repo_workspace( + tmp_path, host, port, "omnibase_infra", workflow + ) + + env = _base_env(tmp_path, host, port, github_sha=sha) + result = _run_functions( + workspace_dir=workspace, + env=env, + calls='wire_sibling_checkout_mirror_rewrite "$1"; _c2_rewrite_flush', + tmp_path=tmp_path, + ) + assert result.returncode == 0, result.stderr + count, pairs = _github_env_pairs(Path(env["GITHUB_ENV"])) + assert count == 2, result.stderr + fetch_key = f"url.git://{host}:{port}/onex_change_control.git.insteadOf" + assert (fetch_key, "https://github.com/OmniNode-ai/onex_change_control") in pairs + + +def test_exact_sha_absent_from_mirror_fails_open_not_a_silent_fallthrough( + mirror_daemon: tuple[str, int, Path], tmp_path: Path +) -> None: + """OMN-16114 explicitly asked this to be VERIFIED, not assumed: does an + exact-SHA checkout of a mirrored repo "fall through to origin" when the + mirror lacks that one commit? It does not (insteadOf has no server-side + fallback -- see README-c2b-uv-git-mirror.md). This test proves the + MECHANISM's own defense against that fact: a SHA the mirror cannot serve + must never receive a rewrite in the first place, because if it did, the + checkout step would hard-fail with no recovery.""" + host, port, mirror_root = mirror_daemon + _make_mirror_repo(mirror_root, "onex_change_control", branch="dev") + missing_sha = "f" * 40 # syntactically valid, guaranteed absent from a fresh repo + workflow = ( + "jobs:\n" + " test-job:\n" + " steps:\n" + " - name: Checkout OCC exact pin\n" + " uses: actions/checkout@pin\n" + " with:\n" + " repository: OmniNode-ai/onex_change_control\n" + f" ref: {missing_sha}\n" + ) + workspace, sha = _seed_own_repo_workspace( + tmp_path, host, port, "omnibase_infra", workflow + ) + + env = _base_env(tmp_path, host, port, github_sha=sha) + result = _run_functions( + workspace_dir=workspace, + env=env, + calls='wire_sibling_checkout_mirror_rewrite "$1"; _c2_rewrite_flush', + tmp_path=tmp_path, + ) + assert result.returncode == 0, result.stderr + count, pairs = _github_env_pairs(Path(env["GITHUB_ENV"])) + assert count == 0, ( + "a SHA absent from the mirror must never receive a rewrite -- " + f"insteadOf has no fallback, this would hard-fail the checkout step. got: {pairs}" + ) + assert "not served by" in result.stdout + result.stderr + + +def test_conjunctive_gating_one_missing_sha_disqualifies_whole_repo( + mirror_daemon: tuple[str, int, Path], tmp_path: Path +) -> None: + """ci.yml's application-database-domain-enforcement job checks out + omnibase_infra TWICE in the same job at two different pinned SHAs + (.proof-dependencies/legacy-fixture, .proof-dependencies/domain-adapter). + `insteadOf` rewrites the whole URL, not one ref -- so if only ONE of the + two pins is servable, installing the rewrite would still redirect BOTH + checkouts, hard-failing the unservable one. Neither may be rewritten + unless the mirror can serve every pin discovered for that repo.""" + host, port, mirror_root = mirror_daemon + present_sha = _make_mirror_repo(mirror_root, "omnibase_core", branch="dev") + absent_sha = "a" * 40 + workflow = ( + "jobs:\n" + " test-job:\n" + " steps:\n" + " - name: Checkout dependency A\n" + " uses: actions/checkout@pin\n" + " with:\n" + " repository: OmniNode-ai/omnibase_core\n" + f" ref: {present_sha}\n" + " path: dep-a\n" + " - name: Checkout dependency B\n" + " uses: actions/checkout@pin\n" + " with:\n" + " repository: OmniNode-ai/omnibase_core\n" + f" ref: {absent_sha}\n" + " path: dep-b\n" + ) + workspace, sha = _seed_own_repo_workspace( + tmp_path, host, port, "omnibase_infra", workflow + ) + + env = _base_env(tmp_path, host, port, github_sha=sha) + result = _run_functions( + workspace_dir=workspace, + env=env, + calls='wire_sibling_checkout_mirror_rewrite "$1"; _c2_rewrite_flush', + tmp_path=tmp_path, + ) + assert result.returncode == 0, result.stderr + count, _pairs = _github_env_pairs(Path(env["GITHUB_ENV"])) + assert count == 0, ( + "one unservable pin must disqualify the whole repo, not just that occurrence" + ) + + +def test_dynamic_gha_expression_ref_disqualifies_repo( + mirror_daemon: tuple[str, int, Path], tmp_path: Path +) -> None: + """ci.yml's application-database-domain-enforcement job resolves + omnimarket's ref via an earlier step's output + (`${{ steps.resolve-omnimarket-ref... }}`). That ref is unknowable at + job-start scan time -- must never be treated as servable on a guess.""" + host, port, mirror_root = mirror_daemon + _make_mirror_repo(mirror_root, "omnimarket", branch="dev") + workflow = ( + "jobs:\n" + " test-job:\n" + " steps:\n" + " - name: Checkout exact typed registry dependency\n" + " uses: actions/checkout@pin\n" + " with:\n" + " repository: OmniNode-ai/omnimarket\n" + " ref: ${{ steps.resolve-omnimarket-ref.outputs.ref }}\n" + ) + workspace, sha = _seed_own_repo_workspace( + tmp_path, host, port, "omnibase_infra", workflow + ) + + env = _base_env(tmp_path, host, port, github_sha=sha) + result = _run_functions( + workspace_dir=workspace, + env=env, + calls='wire_sibling_checkout_mirror_rewrite "$1"; _c2_rewrite_flush', + tmp_path=tmp_path, + ) + assert result.returncode == 0, result.stderr + count, _pairs = _github_env_pairs(Path(env["GITHUB_ENV"])) + assert count == 0 + + +def test_own_repository_never_rewritten_even_if_self_referenced( + mirror_daemon: tuple[str, int, Path], tmp_path: Path +) -> None: + """ci.yml's application-database-domain-enforcement job also checks out + omnibase_infra-the-repo (its OWN repo) twice more at older pinned SHAs, + from a job running inside omnibase_infra. The primary checkout uses the + identical no-.git-suffix URL form; a rewrite keyed on that URL cannot + tell the two apart, and the primary checkout fetches GITHUB_SHA (a + possibly seconds-old head commit the mirror may not have caught up to + yet) with no fallback. own_repo must be excluded unconditionally, + regardless of whether its pins would otherwise probe clean.""" + host, port, mirror_root = mirror_daemon + own_repo = "omnibase_infra" + # Build the own-repo mirror directly (not via _seed_own_repo_workspace, + # since that also writes the workflow file at the SAME commit this + # occurrence pins to -- proving even a servable, present pin of own_repo + # is still excluded). + workflow = ( + "jobs:\n" + " test-job:\n" + " steps:\n" + " - name: Checkout self at an older pin\n" + " uses: actions/checkout@pin\n" + " with:\n" + " repository: OmniNode-ai/omnibase_infra\n" + " ref: SELF_SHA_PLACEHOLDER\n" + " path: .proof-dependencies/legacy-fixture\n" + ) + mirror_root.mkdir(exist_ok=True) + sha = _make_mirror_repo( + mirror_root, + own_repo, + branch="dev", + files={".github/workflows/test.yml": workflow}, + ) + # The pin references a real, present, servable commit on own_repo's own + # mirror (its own seed commit) -- if own_repo exclusion were broken, this + # occurrence would probe clean and a rewrite would be installed. + workflow_final = workflow.replace("SELF_SHA_PLACEHOLDER", sha) + sha2 = _make_mirror_repo( + mirror_root, + own_repo, + branch="dev", + files={".github/workflows/test.yml": workflow_final}, + ) + workspace = tmp_path / "workspace" + workspace.mkdir() + _run_git("init", "--quiet", cwd=workspace) + + env = _base_env(tmp_path, host, port, own_repo=own_repo, github_sha=sha2) + result = _run_functions( + workspace_dir=workspace, + env=env, + calls='wire_sibling_checkout_mirror_rewrite "$1"; _c2_rewrite_flush', + tmp_path=tmp_path, + ) + assert result.returncode == 0, result.stderr + count, _pairs = _github_env_pairs(Path(env["GITHUB_ENV"])) + assert count == 0, "own_repo must never be a sibling-checkout rewrite target" + + +def test_kill_switch_disables_without_breaking_the_job( + mirror_daemon: tuple[str, int, Path], tmp_path: Path +) -> None: + host, port, mirror_root = mirror_daemon + _make_mirror_repo(mirror_root, "omnimarket", branch="dev") + workflow = ( + "jobs:\n" + " test-job:\n" + " steps:\n" + " - name: Checkout omnimarket (sibling)\n" + " uses: actions/checkout@pin\n" + " with:\n" + " repository: OmniNode-ai/omnimarket\n" + " ref: dev\n" + ) + workspace, sha = _seed_own_repo_workspace( + tmp_path, host, port, "omnibase_infra", workflow + ) + + env = _base_env(tmp_path, host, port, github_sha=sha) + env["OMNI_GIT_MIRROR_CHECKOUT_REWRITE_DISABLE"] = "1" + result = _run_functions( + workspace_dir=workspace, + env=env, + calls='wire_sibling_checkout_mirror_rewrite "$1"; _c2_rewrite_flush', + tmp_path=tmp_path, + ) + assert result.returncode == 0, result.stderr + count, _pairs = _github_env_pairs(Path(env["GITHUB_ENV"])) + assert count == 0 + + +# --------------------------------------------------------------------------- +# Shared accumulator (both mechanisms flush through one GIT_CONFIG_COUNT) +# --------------------------------------------------------------------------- + + +def test_shared_accumulator_merges_uv_and_sibling_rewrites_without_clobbering( + mirror_daemon: tuple[str, int, Path], tmp_path: Path +) -> None: + """GIT_CONFIG_COUNT is a single flat namespace. If wire_uv_git_mirror_rewrite + and wire_sibling_checkout_mirror_rewrite each wrote their own + GIT_CONFIG_COUNT independently, the second write would silently drop the + first mechanism's KEY_0/VALUE_0 -- this is the regression the shared + _c2_rewrite_add_pair/_c2_rewrite_flush accumulator exists to prevent.""" + host, port, mirror_root = mirror_daemon + occ_sha = _make_mirror_repo(mirror_root, "onex_change_control", branch="dev") + _make_mirror_repo(mirror_root, "omnimarket", branch="dev") + + workflow = ( + "jobs:\n" + " test-job:\n" + " steps:\n" + " - name: Checkout omnimarket (sibling)\n" + " uses: actions/checkout@pin\n" + " with:\n" + " repository: OmniNode-ai/omnimarket\n" + " ref: dev\n" + ) + uv_lock = f"""[[package]] +name = "onex-change-control" +source = {{ git = "https://github.com/OmniNode-ai/onex_change_control.git?rev={occ_sha}#{occ_sha}" }} +""" + mirror_root.mkdir(exist_ok=True) + sha = _make_mirror_repo( + mirror_root, + "omnibase_infra", + branch="dev", + files={ + ".github/workflows/test.yml": workflow, + "uv.lock": uv_lock, + }, + ) + workspace = tmp_path / "workspace" + workspace.mkdir() + _run_git("init", "--quiet", cwd=workspace) + + env = _base_env(tmp_path, host, port, github_sha=sha) + del env["OMNI_GIT_MIRROR_REWRITE_DISABLE"] # opt the uv mechanism back in + result = _run_functions( + workspace_dir=workspace, + env=env, + calls=( + 'wire_uv_git_mirror_rewrite "$1"; ' + 'wire_sibling_checkout_mirror_rewrite "$1"; ' + "_c2_rewrite_flush" + ), + tmp_path=tmp_path, + ) + assert result.returncode == 0, result.stderr + count, pairs = _github_env_pairs(Path(env["GITHUB_ENV"])) + assert count == 4, ( + f"expected 2 pairs (uv + sibling) = 4 entries, got {count}: {result.stderr}" + ) + occ_fetch_key = f"url.git://{host}:{port}/onex_change_control.git.insteadOf" + market_fetch_key = f"url.git://{host}:{port}/omnimarket.git.insteadOf" + keys = [k for k, _ in pairs] + assert occ_fetch_key in keys, ( + "uv-mechanism entry was clobbered by the sibling-checkout flush" + ) + assert market_fetch_key in keys, ( + "sibling-mechanism entry was clobbered by the uv flush" + ) + + +# --------------------------------------------------------------------------- +# End-to-end wiring: the real top-level script actually calls both functions +# --------------------------------------------------------------------------- + + +def _has_realpath_gate() -> bool: + return _has_gnu_realpath_m() + + +@pytest.mark.skipif( + not _has_realpath_gate(), + reason="full-script run needs GNU `realpath -m` (Ubuntu 22.04 runner image " + "and Linux CI; absent on BSD/macOS) -- same gate as " + "test_runner_job_started_root_owned_debris.py", +) +def test_full_script_success_path_installs_sibling_checkout_rewrite( + mirror_daemon: tuple[str, int, Path], tmp_path: Path +) -> None: + """Runs the REAL script (not the function-defs prefix) end to end, + proving the two new call sites (wire_sibling_checkout_mirror_rewrite + + _c2_rewrite_flush) are actually reached on the normal success path, after + the workspace-reset logic this file's sibling test covers.""" + host, port, mirror_root = mirror_daemon + _make_mirror_repo(mirror_root, "omnimarket", branch="dev") + workflow = ( + "jobs:\n" + " test-job:\n" + " steps:\n" + " - name: Checkout omnimarket (sibling)\n" + " uses: actions/checkout@pin\n" + " with:\n" + " repository: OmniNode-ai/omnimarket\n" + " ref: dev\n" + ) + mirror_root.mkdir(exist_ok=True) + sha = _make_mirror_repo( + mirror_root, + "omnibase_infra", + branch="dev", + files={".github/workflows/test.yml": workflow}, + ) + + runner_home = tmp_path / "actions-runner" + workspace = runner_home / "_work" / "omnibase_infra" / "omnibase_infra" + workspace.mkdir(parents=True) + + github_env = tmp_path / "github_env" + github_env.write_text("") + env = { + **os.environ, + "RUNNER_HOME": str(runner_home), + "GITHUB_WORKSPACE": str(workspace), + "OMNI_GIT_MIRROR_HOST": host, + "OMNI_GIT_MIRROR_PORT": str(port), + "GITHUB_ENV": str(github_env), + "GITHUB_REPOSITORY": "OmniNode-ai/omnibase_infra", + "GITHUB_SERVER_URL": "https://github.com", + "GITHUB_SHA": sha, + "GITHUB_JOB": "test-job", + "GITHUB_WORKFLOW_REF": "OmniNode-ai/omnibase_infra/.github/workflows/test.yml@refs/heads/dev", + "OMNI_GIT_MIRROR_REWRITE_DISABLE": "1", + } + result = subprocess.run( + ["bash", str(HOOK_SCRIPT)], + env=env, + capture_output=True, + text=True, + timeout=60, + check=False, + ) + assert result.returncode == 0, result.stderr + count, pairs = _github_env_pairs(github_env) + assert count == 2, result.stderr + fetch_key = f"url.git://{host}:{port}/omnimarket.git.insteadOf" + assert (fetch_key, "https://github.com/OmniNode-ai/omnimarket") in pairs