diff --git a/CHANGELOG.md b/CHANGELOG.md index fec38bd4db..30eafe8250 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -23,6 +23,7 @@ this file. The format follows Keep a Changelog, and versioned releases follow Semantic Versioning where the repository publishes a release. ## [Unreleased] +- **Catch scheduler target-list drift before it silently fails an hourly heartbeat.** `hourly-review-repair.yml`'s per-cron `target_repository` matrix and the `OPENCODE_REPOSITORY_DISPATCH_TARGETS` repository variable (which gates `ALLOWED_TARGET_REPOSITORIES` in `pr-review-merge-scheduler.yml`/`pr-review-fix-scheduler.yml`) are two independently hand-maintained lists with no structural link -- three repositories (`governance-risk-compliance`, `nonnest2`, `quarantine-sandbox-runtime`) were added to the hourly matrix without a corresponding variable update, so their hourly heartbeat failed closed with "target repository is not allowlisted" until each was found and fixed the same day. Added `scripts/ci/opencode_repository_dispatch_targets.json`, a hand-maintained mirror of the variable's live value, and a new contract test (`test_every_hourly_caller_target_is_in_the_dispatch_targets_mirror`) asserting every hourly-caller target is present in it, so a future PR that repeats the omission fails at review time instead of at the next silent hourly failure. See `docs/doctoring/scheduler-target-list-drift-20260902.md`. - **Fix a stale `test_strix_quick_gate.sh` assertion left broken by the `#1630` scheduler-cadence lengthening.** `pr-review-merge-scheduler.yml`'s repository-local heartbeat was changed from a quarter-hourly `cron: "*/30 * * * *"` to an hourly diff --git a/docs/doctoring/scheduler-target-list-drift-20260902.md b/docs/doctoring/scheduler-target-list-drift-20260902.md new file mode 100644 index 0000000000..095f9d2193 --- /dev/null +++ b/docs/doctoring/scheduler-target-list-drift-20260902.md @@ -0,0 +1,89 @@ +# Doctoring record: scheduler target-list drift (2026-09-02) + +## Incident + +`hourly-review-repair.yml`'s per-cron `target_repository` matrix and the +`OPENCODE_REPOSITORY_DISPATCH_TARGETS` repository variable (which gates +`ALLOWED_TARGET_REPOSITORIES` in `pr-review-merge-scheduler.yml` / +`pr-review-fix-scheduler.yml`, and the agent-mention dispatch allowlist) are +two independently hand-maintained lists of "repositories legitimately +targetable by an OpenCode-driven dispatch." They have no structural link: +adding a repository to one does not add it to the other. + +This caused three real, silent failures, all discovered and fixed the same +day: + +- `governance-risk-compliance` — added to the hourly matrix (run + `.github/actions/runs/33524178483/job/99910668839`, 2026-09-01) before the + variable was updated; every hourly heartbeat failed with `##[error]Scheduler + target repository is not allowlisted: ContextualWisdomLab/governance-risk-compliance.` + A prior fix attempt (commit `7bf98d0`) hardcoded the repository name + directly into both scheduler workflows as a "temporary propagation bridge" + instead of fixing the variable — this violated this repo's own thin-caller + convention (`CLAUDE.md`: "Product hourly callers stay thin. Do not + hard-code ... into `pr-review-fix-scheduler.yml`") and broke + `test_no_target_repository_is_hard_coded_in_the_shared_scheduler` on `main`. + Fixed properly in `contextual-orchestrator#1028`'s sibling PR here + (`fix(scheduler): admit governance-risk-compliance via the org variable, not + a hardcode`, #1743): added the repository to the variable directly, removed + the hardcode. +- `nonnest2` and `quarantine-sandbox-runtime` — found by diffing the hourly + matrix's target list against the live variable's value while scoping this + fix: both were present in the hourly matrix (present since the original + 18-file-to-1 consolidation, ADR-0021) but absent from the variable, + meaning their hourly heartbeat had been failing closed the same way, + undetected because the queue backlog this session was separately + investigating (a hard 60-concurrent-job org plan limit, confirmed via the + GitHub Actions Settings UI) meant these runs weren't being watched + individually. Fixed the same way: added both to the variable. + +## Root cause + +Not a logic bug in either scheduler — `target_allowed` fails closed exactly +as designed when a target isn't in the allowlist, which is correct behavior +for an *actually* unauthorized target. The defect is that there is no +mechanism keeping the two lists in sync, and no test catching a PR that adds +a repository to one list without the other. + +## Fix + +- `scripts/ci/opencode_repository_dispatch_targets.json` — a new, + hand-maintained mirror of `OPENCODE_REPOSITORY_DISPATCH_TARGETS`'s live + value (there is no API to commit a repository variable's value to source + control, so this file is deliberately a mirror, not a generator — whoever + updates the live variable updates this file in the same PR, per the file's + own header comment). +- `tests/test_hourly_review_repair_callers.py::test_every_hourly_caller_target_is_in_the_dispatch_targets_mirror` — + asserts every `target_repository` in `hourly-review-repair.yml`'s + `_EXPECTED_TARGETS` (the existing, already-tested canonical model of the + workflow's `case` statement) is present in the mirror. A future PR that + adds a repository to the hourly matrix without also updating the mirror + (and, by the mirror's own documented discipline, the live variable) now + fails this test at review time instead of failing the next hourly + heartbeat silently. + +## What this does not do + +This does not verify the mirror file's contents actually match the live +variable's *current* value — that would require a network call to the +GitHub API at test time, which this repo's offline `pytest tests` suite +deliberately does not do (see `pyproject.toml`'s `pythonpath` setup; every +other contract test in this module is a pure file-content assertion). A +mismatch between the mirror and the live variable (e.g. someone runs `gh +variable set` without updating this file, or vice versa) is not caught by +this test — only a mismatch between the *workflow matrix* and the mirror is. +Closing that remaining gap (verifying the mirror against the live variable) +needs either a step in an existing regularly-running workflow or a documented +manual verification command, and was deliberately left out of this fix to +keep it a pure test addition with zero production-workflow risk; see the +open item below. + +## Follow-up (not done here, deliberately out of scope for this fix) + +Add a live-verification step (in an existing workflow, not a new one, per +this session's org-culture reasoning: prefer a loud contract-test-style +failure a human must resolve with an explicit commit over an +auto-mutating workflow that "magically" fixes drift) that fetches +`OPENCODE_REPOSITORY_DISPATCH_TARGETS`'s live value and fails loudly if it +diverges from `scripts/ci/opencode_repository_dispatch_targets.json`. Left +open pending a decision on which existing workflow should host that step. diff --git a/scripts/ci/opencode_repository_dispatch_targets.json b/scripts/ci/opencode_repository_dispatch_targets.json new file mode 100644 index 0000000000..dd82dd1fd0 --- /dev/null +++ b/scripts/ci/opencode_repository_dispatch_targets.json @@ -0,0 +1,58 @@ +{ + "$comment": "Mirrors the live ContextualWisdomLab/.github repository variable OPENCODE_REPOSITORY_DISPATCH_TARGETS, which gates ALLOWED_TARGET_REPOSITORIES in pr-review-merge-scheduler.yml/pr-review-fix-scheduler.yml and the agent-mention dispatch allowlist. There is no API to commit an org/repo variable's value to source control, so this file is a hand-maintained mirror -- update it AND run `gh variable set OPENCODE_REPOSITORY_DISPATCH_TARGETS --repo ContextualWisdomLab/.github` in the same PR whenever a repository is added. tests/test_hourly_review_repair_callers.py::test_every_hourly_caller_target_is_in_the_dispatch_targets_mirror locks every repository hourly-review-repair.yml dispatches to as a subset of this list -- see docs/doctoring/scheduler-target-list-drift-20260902.md for the incident history (governance-risk-compliance, nonnest2, quarantine-sandbox-runtime all silently failed their hourly heartbeat because this sync was missed) that this file and test exist to catch before it recurs.", + "targets": [ + "ContextualWisdomLab/.github", + "ContextualWisdomLab/ContextualWisdomLab.github.io", + "ContextualWisdomLab/DiagramWeave", + "ContextualWisdomLab/EgressWeave", + "ContextualWisdomLab/EmbedRelay", + "ContextualWisdomLab/IRT-bibliography-set", + "ContextualWisdomLab/LineageWeave", + "ContextualWisdomLab/OriginWeave", + "ContextualWisdomLab/Orgmetra", + "ContextualWisdomLab/RankWeave", + "ContextualWisdomLab/TEPP", + "ContextualWisdomLab/ThreadWeave", + "ContextualWisdomLab/aFIPC", + "ContextualWisdomLab/accounting-information-platform", + "ContextualWisdomLab/appguardrail", + "ContextualWisdomLab/bandscope", + "ContextualWisdomLab/ccube-jco-potential-customer", + "ContextualWisdomLab/clearfolio", + "ContextualWisdomLab/codec-carver", + "ContextualWisdomLab/context-graph-contracts", + "ContextualWisdomLab/contextual-orchestrator", + "ContextualWisdomLab/disksage", + "ContextualWisdomLab/enterprise-architecture-core", + "ContextualWisdomLab/fast-mlsirm", + "ContextualWisdomLab/feelanet-adfs", + "ContextualWisdomLab/four-pillars", + "ContextualWisdomLab/governance-risk-compliance", + "ContextualWisdomLab/gyeot", + "ContextualWisdomLab/hyosung-itx-slogan-brief", + "ContextualWisdomLab/inkspan", + "ContextualWisdomLab/kaefa", + "ContextualWisdomLab/keyverse", + "ContextualWisdomLab/learning-management-platform", + "ContextualWisdomLab/life-os", + "ContextualWisdomLab/linux-cluster-ops", + "ContextualWisdomLab/macos_utility_packs", + "ContextualWisdomLab/metering-billing-platform", + "ContextualWisdomLab/mhtml-etl-gateway", + "ContextualWisdomLab/mightyETL", + "ContextualWisdomLab/naruon", + "ContextualWisdomLab/newsdom-api", + "ContextualWisdomLab/noema", + "ContextualWisdomLab/nonnest2", + "ContextualWisdomLab/pg-erd-cloud", + "ContextualWisdomLab/pg-llm-batch", + "ContextualWisdomLab/psychometrics-commons", + "ContextualWisdomLab/quarantine-sandbox-runtime", + "ContextualWisdomLab/saju-caldav", + "ContextualWisdomLab/scopeweave", + "ContextualWisdomLab/semantic-data-portal", + "ContextualWisdomLab/wardnet", + "ContextualWisdomLab/xtrm-lead-pi-outbound", + "ContextualWisdomLab/xtrmLLMBatchPython" + ] +} diff --git a/tests/test_hourly_review_repair_callers.py b/tests/test_hourly_review_repair_callers.py index eccf7630f2..df0f501eb6 100644 --- a/tests/test_hourly_review_repair_callers.py +++ b/tests/test_hourly_review_repair_callers.py @@ -18,6 +18,7 @@ _CALLER = Path(".github/workflows/hourly-review-repair.yml") _REUSABLE_SCHEDULER = Path(".github/workflows/pr-review-fix-scheduler.yml") +_DISPATCH_TARGETS_MIRROR = Path("scripts/ci/opencode_repository_dispatch_targets.json") _FORMER_CALLERS = ( "accounting-information-platform-hourly-review-repair.yml", @@ -431,3 +432,46 @@ def test_resolve_unreviewed_conflicts_is_explicit_and_matches_the_default() -> N 1 ].split("retry_hours:", maxsplit=1)[0] assert "default: true" in policy_block + + +def test_every_hourly_caller_target_is_in_the_dispatch_targets_mirror() -> None: + """Every hourly-caller repository must also be a registered dispatch target. + + `pr-review-merge-scheduler.yml`/`pr-review-fix-scheduler.yml` validate + every dispatch's target repository against the live + ``OPENCODE_REPOSITORY_DISPATCH_TARGETS`` repository variable ( + ``ALLOWED_TARGET_REPOSITORIES``); ``hourly-review-repair.yml``'s own + per-cron ``target_repository`` matrix is a second, independently + hand-maintained list with no structural link to the variable. Three + repositories (governance-risk-compliance, then nonnest2 and + quarantine-sandbox-runtime, all discovered 2026-09-02) were added to the + hourly matrix without a corresponding update to the variable, so every + one of their hourly heartbeats failed closed with "target repository is + not allowlisted" until caught -- see + ``docs/doctoring/scheduler-target-list-drift-20260902.md``. This test + cannot see the live variable's actual value (no API commits it to + source control), so it checks the hourly matrix against + ``scripts/ci/opencode_repository_dispatch_targets.json``, a + hand-maintained mirror of that variable's contents -- catching the + "added to the workflow matrix, forgot the mirror (and, by the update + discipline the mirror's own header documents, forgot the live + variable)" mistake at PR-review time instead of at the next silent + hourly failure. + """ + mirror = json.loads(_DISPATCH_TARGETS_MIRROR.read_text(encoding="utf-8")) + mirrored_targets = set(mirror["targets"]) + + hourly_caller_targets = { + target["target_repository"] + for targets in _EXPECTED_TARGETS.values() + for target in targets + } + + missing = hourly_caller_targets - mirrored_targets + assert not missing, ( + "hourly-review-repair.yml dispatches to a repository absent from " + f"{_DISPATCH_TARGETS_MIRROR}: {sorted(missing)}. Add it to the mirror's " + "\"targets\" list AND run `gh variable set OPENCODE_REPOSITORY_DISPATCH_TARGETS " + "--repo ContextualWisdomLab/.github` with the updated value in the same PR, " + "or the next hourly heartbeat for this repository will fail closed." + )