Repository navigation
Run validate-macos on every PR and require it on main - #226
Merged
seathatflowsinourveins merged 6 commits intoSep 25, 2026
Merged
Conversation
…gate bootstrap-* separately validate-macos already runs this repository's full test suite on macos-15, but adoption-bootstrap.yml's pull_request trigger carried a paths: filter, so it never ran at all on most PRs -- PR #219's macOS-only credential-guard failures (/var and /tmp are OS symlinks there) would not have blocked merge under any current required check. Drop the paths: filter from pull_request entirely, add a lightweight `changes` job (harden-runner, contents: read, plain git diff, no new third-party action) that keeps bootstrap-linux/bootstrap-macos/bootstrap-macos-brew path-gated on pull_request the way GitHub's own paths: filter already gates them on push, and require validate-macos in .github/main-ruleset.json (strict_required_status_checks_policy stays false). Adds tests/test_workflow_hardening.py::AdoptionBootstrapMacosRequiredTests, updates tests/test_adoption_bootstrap_macos.py's now-stale pull_request-paths assertions, and records the decision, alternatives, cost and overturn conditions in docs/decisions/2026-09-22-github-automation-closure.md. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
CI's pinned actionlint 1.7.12 runs shellcheck on run: scripts and failed on SC2254 at the path-pattern case match. Glob matching there is the intent: each pattern is a path glob, and case's * also matches /, which gives the push paths: list's ** semantics. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
evidence.json was rebuilt from main's copy with this branch's six changed entries refreshed. #224 renamed the fixture this branch still registered under its old path. The docs auto-merged; both changes are present. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…e changes job, NUL-safe diff
Independent review found that needs: changes plus a bare if: (no status function) applies
an implicit success() to bootstrap-linux/bootstrap-macos/bootstrap-macos-brew, so once
changes itself is skipped (it only runs on pull_request), all three jobs were skipped too
on every push, schedule and workflow_dispatch run -- confirmed live by the coordinator's
dispatch run 36085789483 (changes skipped, all three bootstrap-* jobs completed as
skipped, validate-macos ran normally). Fix: if: ${{ !cancelled() && (github.event_name !=
'pull_request' || needs.changes.outputs.bootstrap != 'false') }}, following the
security-scan.yml precedent; != 'false' also makes a failed/cancelled changes job run
everything (fail safe). The changes job itself now fails safe on a missing payload SHA or
a failed git diff (echoes bootstrap=true and exits 0 instead of exiting 1 and letting
needs: skip the dependents), and switches to git diff -z with a NUL-delimited read so a
quoted non-ASCII filename still matches a PATTERNS glob. Also: SC2254-annotated case glob
comment corrected to describe over-matching (not exact paths: semantics), stale workflow
comments and decision-record claims corrected (PR #219 did run and fail validate-macos,
just wasn't required; "fourth job" -> "fifth"; measured cost 8.2-13.4 min, median 10.9 over
36 runs; decision record restructured to flat Evidence/Alternatives/Decision/Cost/Overturn/
Measured; added a rollout note about already-open PRs needing a new push after the ruleset
is applied), and tests/test_workflow_hardening.py strengthened to require a status function
and != 'false' (proven to fail against the pre-fix condition text) plus fail-safe/NUL-safety
assertions on the changes job's script.
Verified: actionlint 1.7.12 (with shellcheck 0.11.0) and zizmor --offline --no-config
--no-ignores --persona regular --strict-collection both report 0 findings on
.github/workflows. tests.test_workflow_hardening, test_adoption_bootstrap_macos,
test_adoption_bootstrap, test_adoption_docs_consistency, test_workflow_security_coverage,
test_workflow_security and the full python3 -m unittest suite (4720 tests) all pass. All
six validators (validate.py, validate_catalogs.py, landscape.py, evidence_manifest.py
--check, verdict_review_gate.py --base origin/main, trading_gates.py --check) pass.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Run 36086815267 at 8ec88c1: changes skipped (it runs only on pull_request); the bootstrap-linux, bootstrap-macos, bootstrap-macos-brew and validate-macos jobs all ran and succeeded. The before-fix run 36085789483 had skipped the three bootstrap jobs. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…s in the record - shell: bash runs as bash -eo pipefail, so the script now runs set +e and guards mktemp with the fail-safe bootstrap=true write. The test asserts both. - Workflow comment and decision record: only push is path-filtered; schedule and workflow_dispatch run every job. - #219: validate-macos failed on earlier heads (72a15b8, 1bd4e2f) and passed at the merged head 04c867b. It was not a red run beside the merge. From the independent delta review of 8ec88c1/c19cb31d (verdict ready). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
seathatflowsinourveins
deleted the
claude/require-validate-macos-20260925
branch
September 25, 2026 03:02
seathatflowsinourveins
pushed a commit
that referenced
this pull request
Sep 25, 2026
…y branch evidence.json was rebuilt from main's copy. This branch's convergence-record entry and its ten sim-capacity file registrations were re-applied. The CI lock path this branch still carried was not re-registered, because #233 renamed it. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This was referenced Sep 25, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
This makes
validate-macos(macos-15, the full test suite on macOS) safe to require onmain, and adds it to the ruleset target.github/main-ruleset.json.Why. On #219, the new credential tests failed only on macOS: 5 failures and 13 errors, because
/varand/tmpare OS symlinks there and the no-follow guard refused them.validate-macoswas not a required check, so nothing would have blocked that merge.Why not just add it to the ruleset.
validate-macoslives inadoption-bootstrap.yml, whosepull_requesttrigger had apaths:filter. On a PR that touches none of those paths, the job never runs. A required check that never reports leaves the PR waiting forever, and it also means a macOS break in unlisted code went untested.Change:
adoption-bootstrap.ymldrops thepull_requestpath filter and adds achangesjob. The job runs on ubuntu, with harden-runner first, and uses plaingit diff --name-onlybetween the base and head SHAs. It adds no new third-party action.bootstrap-linux,bootstrap-macosandbootstrap-macos-brewkeep their path gating throughneeds: changesplusif:. A job skipped byif:reports success.validate-macosruns on every PR. The push, schedule and dispatch triggers and their paths are unchanged, and PR concurrency with cancel-in-progress is added..github/main-ruleset.jsonadds{"context": "validate-macos", "integration_id": 15368}.strict_required_status_checks_policystaysfalse: its recorded overturn condition, amainfailure traced to merge skew, has not occurred as of 2026-09-25.tests/test_workflow_hardening.pygains a target test that requires validate-macos and keeps strict off.AdoptionBootstrapMacosRequiredTestschecks four things: there is no PR path filter;validate-macoscannot be skipped byiforneeds; the bootstrap jobs stay path-gated; and thechangespattern list equalspush'spaths:list.tests/test_adoption_bootstrap_macos.pystale trigger assertions are fixed.docs/github-automation.mdgets the required-checks list and the "a required check must run on every PR" note. The decision record gains the dated section "validate-macos required (2026-09-25)", covering evidence, alternatives, cost (about 7–12 min per PR, free on a public repo) and overturn conditions.After merge, the coordinator applies
.github/main-ruleset.jsonto live ruleset 23739774 and verifies that the live and target rules match.Checks
python3 -m unittestsuite ran 4717 tests, all OK (534 host-expected skips).test_workflow_hardeningran 46 OK.Review note: Codex CLI is at its usage limit until 2026-09-30, so this PR has an independent Claude review only.
🤖 Generated with Claude Code