Skip to content

cmux-next fast gates: web lint, crash ratchet, defaults stamp and prepush gate fail on the PR merged with base - #23264

Merged
teamleaderleo merged 6 commits into
feat-cmux-nextfrom
claude/cmux-next-fast-gates
Oct 21, 2026
Merged

teamleaderleo merged 6 commits into
feat-cmux-nextfrom
claude/cmux-next-fast-gates

Conversation

@teamleaderleo

@teamleaderleo teamleaderleo commented Oct 21, 2026 •

Copy link
Copy Markdown
Collaborator

Why

Four base reds landed on feat-cmux-next on 2026-10-21. Each came through a check that does not fail a pull request:

  • Two came through webviews lint/format and switch-fetches. On a pull request those checks only warn (ci-web.yml react-apps-check, CMUX_NEXT_STYLE_GUARDS=warn). They fail only on push (cmux-next-web.yml "web lint (fast)", scripts/web_lint_prepush.py).
  • One came from a PR that was self-merged past its own red defaults-snapshot.
  • One came from direct pushes that tripped the crash ratchet.

feat-cmux-next has no required checks. Its ruleset blocks only deletion and force-push.

What

.github/workflows/cmux-next-fast-gates.yml ("cmux-next fast gates", Linux) has three jobs, so pull request code never runs next to a write token:

  • resolve names the head and BASE without running any checked-out code: git rev-parse on the merge ref, or on dispatch the GitHub API. A dispatch refuses a head from a fork, a closed pull request or a base other than feat-cmux-next before anything is fetched.
  • fast gates (needs: resolve, contents: read only) runs the PR's code: four push-time gates in fail mode on BASE..merge, listed below.
  • post (statuses: write only) checks nothing out. It writes the status from resolve's head and base and the gates job's result, never from anything the gates job wrote.

The four gates:

  1. scripts/web_lint_prepush.py --base BASE --head MERGE: the same diff-scoped lint that web lint (fast) runs on push. Bun, Node and node_modules are installed only when the diff touches a web package the script would lint. Otherwise the gate is skipped.
  2. The crash ratchet. It calls scripts/ci/crash_ratchet_compare.py the way cmux-next-guards.yml does: on the merge and on BASE's exported tree.
  3. scripts/ci/defaults_guard.py check-stamp --base BASE, the same call as in cmux-next-guards.yml.
  4. scripts/cmux-next/prepush-gate.py --base BASE --head MERGE, which fails only on violations the PR adds. Its pinned tools are cached by lint-tools.lock.

The step logic is in scripts/ci/cmux_next_fast_gates.py:

  • prepare --head SHA --base SHA finds the merge on pull_request, or makes it on dispatch, fails unless that merge is exactly BASE + HEAD as resolved (a head that moved after resolve is refused), and writes merge and web.
  • run runs every gate, even after one fails, writes a summary table and exits 1 if any gate failed.

Both scripts already accepted --base/--head, so neither needed new flags. The new test is also wired into the tooling-tests job of cmux-next-guards.yml.

The push-attribution part of the original plan was dropped: it would have posted comments on teammates' PRs. Attribution moves to base-red-watch.

Contract

Other PRs depend on this contract: next-autoland in leo-workspace and the hq merge hook. Each will enforce it in its own PR.

  • Triggers:
    • pull_request into feat-cmux-next with no paths filter, which uses the default merge ref.
    • workflow_dispatch with input pr, validated as digits only. A dispatch fetches refs/pull/<pr>/head and the current feat-cmux-next tip and runs git merge --no-edit itself (--no-ff). A conflict fails with a message naming the PR and the conflicting files.
  • In both cases BASE is the first parent of the tested merge, and HEAD_SHA is the PR head.
  • No pull_request_target and no workflow_run trigger.
  • concurrency: fast-gates-${{ github.event.pull_request.number || inputs.pr }} with cancel-in-progress: true.
  • Result: a commit status on HEAD_SHA:
    • context cmux-next/fast-gates
    • state success or failure
    • description base <40-hex BASE>
    • target_url set to the run URL
  • Permissions: contents: read for the workflow; resolve adds pull-requests: read; fast gates has contents: read only; post has statuses: write only.
  • Fork PRs: no status is posted, and the gates still pass or fail. A dispatch for a fork PR fails in resolve.
  • PR-controlled values reach shell only through env.
  • FRESH rule, judged by mergers:
    • The newest cmux-next/fast-gates status on the head is success, and its base B is an ancestor of the current tip T.
    • Either B == T, or GET compare/B...T has ahead_by <= 250 and none of its files is among the PR's files (GET pulls/N/files).
    • Otherwise the status is stale. The merger dispatches cmux-next-fast-gates.yml with pr=N on ref feat-cmux-next and judges again after that run.

Tests

tests/test_cmux_next_fast_gates.py has two parts:

  • Workflow shape:
    • name, the three jobs and their runner (the runner expression is the same as cmux-next-guards.yml's)
    • resolve runs no repository script and checks out only on pull_request; a dispatch refuses a fork head, a closed PR or another base there; fast gates needs it and takes its head and base
    • post has one step, no checkout, and takes head and base from resolve and only the result from fast gates, which has no outputs
    • no paths filter, and no pull_request_target or workflow_run trigger
    • the dispatch input is validated as digits
    • no ${{ }} inside run, in any job
    • concurrency
    • minimal permissions per job (only post can write)
    • no step or job is continue-on-error, and nothing uses CMUX_NEXT_STYLE_GUARDS
    • all four gate scripts are present
    • the status context, description, target URL and fork skip
    • Bun is installed only for web changes
    • the test is wired into tooling-tests
  • Script behavior on temporary git repositories (fake gate scripts record their argv):
    • a dispatch merges the PR head into a shallow clone's tip, with BASE as the first parent
    • a conflict fails and names the PR and BASE
    • a head that moved after resolve, a base not on feat-cmux-next, and a short sha are refused
    • a non-digit PR is refused
    • the pull_request merge ref is read, and a mismatched head or base is refused
    • a web change sets web=true
    • every gate is called with BASE and the merge, and the ratchet gets BASE's tree
    • any one red gate fails the run, and the rest still run
    • with no web change only web lint is skipped
    • a bad base runs nothing

Red (test commit 7ce13bd, before the workflow and script existed):

$ python3 tests/test_cmux_next_fast_gates.py
Ran 20 tests in 9.704s
FAILED (failures=12, errors=11)

Green (3fe7900, and still green at dc7b214):

$ python3 tests/test_cmux_next_fast_gates.py
Ran 20 tests in 11.159s
OK

Review fix (chief-fix-3 BLOCK at dc7b214: dispatch ran fork-PR code in the job that posts). Red on the test commit c018308 with the old workflow and script, green on 8ccd8ee:

$ python3 tests/test_cmux_next_fast_gates.py
Ran 25 tests in 14.292s
FAILED (failures=19, errors=4)

$ python3 tests/test_cmux_next_fast_gates.py
Ran 25 tests in 17.143s
OK

Changelog

none

Observed / Not run

Observed:

  • actionlint on cmux-next-fast-gates.yml and cmux-next-guards.yml is clean (again at 8ccd8ee).

  • At 8ccd8ee these still pass:
    tests/test_ci_workflow_guards_are_wired.py, test_ci_workflow_permissions_are_job_scoped.py, test_ci_actionlint_covers_every_workflow.py, test_ci_workflow_path_filter_parity.py and test_ci_repo_variable_defaults.py.

  • In scripts/ci/guards-local.sh --group ci, the only reds are unrelated to this change:

    • shellcheck is not installed on huge-blue
    • "canonical guard profile" refuses an uncommitted tree
    • nightly-next-train's environment secrets, which are red on base
    • a verify_local environment test
  • python3 scripts/cmux-next/prepush-gate.py: no new violations.

  • A real cmux_next_fast_gates.py run --base <feat-cmux-next tip> --web false on this branch on huge-blue passed: crash ratchet 63.2s, defaults stamp 0.3s, prepush gate 3.3s, web lint skipped.

  • On this PR, the workflow's first run (39578318736, head 3fe7900) passed all four gates, but the status POST through curl -f got a 403 with no reason shown. The token had Statuses: write. dc7b214 posts with gh api instead, so a refusal prints GitHub's message.

  • Run 39578793945 on head dc7b214 passed and posted cmux-next/fast-gates = success, description base efaab9b377161282c7a53d797d073ce259b00dfb, with the target URL set to the run.

    • The job took 70s and the run 2m29s, including the queue.
    • Crash ratchet 28.5s, defaults stamp 0.1s, prepush gate 0.1s, web lint skipped.
  • The first 403 is now explained. Run 39580378636 on head 8ccd8ee: resolve and fast gates passed (base 66a1a07), and post got gh: API rate limit exceeded for installation (HTTP 403) from the Actions token, the same installation 403 that is red on base (high-signal review link and others). It is infra, not this change, and it is not rerun.

Not run:

  • Web lint, because this PR touches no web path.
  • The dispatch path on GitHub, including resolve's API reads and the fork refusal. It can only be dispatched on feat-cmux-next after this merges, so only the temp-repo and shape tests cover it.
  • Swift, cargo and app builds (not relevant).

Agent session: huge-blue

🤖 Generated with Claude Code

teamleaderleo and others added 2 commits October 21, 2026 09:28
…push gate fail on the PR merged with base

A pull_request and workflow_dispatch (pr=N) workflow runs the four push-time
gates in fail mode on BASE..merge and posts cmux-next/fast-gates with
"base <sha>" on the PR head, so mergers can judge freshness against the tip.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?harness=claude&session=01a15c6d-a1d2-7fc2-9458-92c6bdcb7b4d&host=huge-blue
@teamleaderleo teamleaderleo added the risk:normal Risk tier (docs/ci/risk-tiers.md) label Oct 21, 2026
@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

Owned by land-and-ci on huge-blue · session

@blacksmith-sh

blacksmith-sh Bot commented Oct 21, 2026 •

Copy link
Copy Markdown

Found 5 test failures on Blacksmith runners:

Failures

Test View Logs
/tmp/tmp64c11oqr/summary-missing.json (live):/summary-missing.json (live): View Logs
/tmp/tmph23ot7nx/summary-fail.json (live):/summary-fail.json (live): View Logs
convention violations found (ERROR lines above)./
convention violations found (ERROR lines above).
View Logs
test_a_file_a_test_reads_selects_that_test (main.PullRequestTiers.test_a_file_a_tes
t_reads_selects_that_test)/
test_a_file_a_test_reads_selects_that_test (main.PullRequestTiers.test_a_file_a_tes
t_reads_selects_that_test)
View Logs
test_every_depth_fetch_checks_for_a_shallow_checkout (main.NoShallowDevFetchTests.t
est_every_depth_fetch_checks_for_a_shallow_checkout)/
test_every_depth_fetch_checks_for_a_shallow_checkout (main.NoShallowDevFetchTests.t
est_every_depth_fetch_checks_for_a_shallow_checkout)
View Logs

Fix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.

@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

Bead cx-xtehh created for this PR by the cmux coordinator.

@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

Review (subagent, correctness first): BLOCK at dc7b214

Defect: the dispatch path lets code from a fork PR decide what gets posted, with a statuses: write token. The body says "Fork PRs: no status is posted", but this path breaks that.

.github/workflows/cmux-next-fast-gates.yml:115 posts on every workflow_dispatch. Neither the workflow nor scripts/ci/cmux_next_fast_gates.py:96-100 checks that PR N comes from this repository or targets feat-cmux-next. PR code runs in the same job before the status step: web_needed (lines 70-78, called at 129) imports scripts/web_lint_prepush.py from the merged tree, and the gate scripts run from the merge too.

Failure scenario: a fork PR changes scripts/web_lint_prepush.py, and a merger dispatches pr=N, which is what the contract says to do for a stale status. The PR's code can then:

  • append head_sha=<another PR's head> and base=<tip> to $GITHUB_OUTPUT. The last value for a key wins, so the status lands on another commit with a base that looks fresh.
  • prepend a fake gh to $GITHUB_PATH. The status step then runs it with GH_TOKEN (statuses: write) and it can post success on any SHA.
  • or simply make a gate exit 0.

It also runs the fork's code on vars.LINUX_RUNNER (line 39).

Suggested fix:

  • On dispatch, before any PR code runs, check through the API that head.repo.full_name == github.repository and base.ref == feat-cmux-next, and refuse otherwise.
  • Resolve head_sha and base in a trusted step that runs before the merge.
  • Post the status from a separate job (needs: fast-gates) that uses only those trusted values.

Everything else holds:

  • pull_request path: the status goes on the verified HEAD^2 (it is refused unless it equals pull_request.head.sha). success needs both the prepare and gates steps to succeed. Nothing uses continue-on-error. A cancelled run or a timeout posts nothing.
  • Triggers and tokens: no pull_request_target or workflow_run, persist-credentials: false, and PR text reaches the shell only through env.
  • Gates: all four run, and any failure exits 1.
  • Tests: 20 OK, and they do fail when run swallows a failure.
  • Lint and wiring: actionlint is clean, and the only write is the commit status.
  • Context: cmux-next/fast-gates on the head, as hq#2730 expects. hq#2730 applies the base-freshness rule, so a success left on an older base is caught there.

Nits:

  • A PR can loosen its own gate and pass itself, the same as the push-time jobs.
  • The depth-50 dispatch fetch may merge against a different merge base than GitHub's.
  • Dispatch accepts a PR that targets another base.

Agent session: huge-blue

@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

codex-review: SHIP at dc7b214

  • Reviewed the current diff, its focused tests, and the affected call paths; no correctness bug found.

teamleaderleo and others added 2 commits October 21, 2026 09:51
… gate with read only, post from its own job

A dispatch refuses a fork head, a closed pull request or another base before
anything is fetched; the gates job runs the PR's code with contents: read and
hands post only its result.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?harness=claude&session=01a15c6d-a1d2-7fc2-9458-92c6bdcb7b4d&host=huge-blue
@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

codex-review: SHIP at 8ccd8ee

  • Reviewed the current diff, its focused tests, and the affected call paths; no correctness bug found.

@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

Review (subagent, correctness first): SHIP at 8ccd8ee

The earlier BLOCK (dc7b214) is resolved. PR code can no longer choose what the status job publishes:

  • post: HEAD_SHA and BASE come only from needs.resolve.outputs, and the state only from needs.fast-gates.result. fast-gates declares no outputs, so whatever it writes to GITHUB_OUTPUT, ENV or PATH stays in that job. post has one run step, with no checkout, no uses, no artifacts and no cache.
  • Tokens: statuses: write exists only on post. The gates job has contents: read and no GH_TOKEN.
  • resolve: on dispatch it checks nothing out. It reads the PR from the API and refuses a fork, a deleted head repo, a closed PR or a base other than feat-cmux-next. On pull_request it checks HEAD^2 == head.sha. Both SHAs are regex-checked before output.
  • pull_request path: the status goes on the event head, success only when gates succeed, and there is no continue-on-error. Cancelled runs post nothing, and forks get no post.
  • Tests: 25 OK at head, and 19F/4E against the old code at c018308. Two of three mutations fail a test: post reading a gates output, and the fork check disabled. actionlint is clean.
  • Writes: the only one is POST statuses/$HEAD_SHA.

Nits, none blocking:

  1. No test fails if GH_TOKEN is added to the gates job (harmless today, read-only token). A test that the gates job has no GH_TOKEN and no secrets. would lock it in.
  2. A timed-out gates job posts failure, not nothing. That is safe, but the wording differs.
  3. Dispatch runs on the feat-cmux-next ref, so the gates job's cache saves happen in that branch scope while same-repo PR code runs. A cache miss could be seeded for other PRs' gates. Only writers can do this, and post never reads the cache. Restore-only caches on dispatch would close it.
  4. The dispatch refusal checks are tested as strings, and no test executes that bash.

Agent session: huge-blue

@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

high-signal review link / link is red on 12 of 20 fresh feat-cmux-next PR heads in the last 2:00:00, so it is most likely a base red, not this PR's. The base-reds lane has it; a base merge picks up the fix once it lands.

…shallow

cmux-next CI guard group on 8ccd8ee (job 119156226796): tests/test_ci_no_shallow_dev_fetch.py
flagged cmux_next_fast_gates.py:90 and :94. In a full clone the fetch now takes the refs whole.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?harness=claude&session=01a15c6d-a1d2-7fc2-9458-92c6bdcb7b4d&host=huge-blue
@github-actions

github-actions Bot commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

cmux status (one comment, edited in place; each line is updated by the job that owns it)

Review e991606: HQ feed card · gallery before/after runs · checks

App of this head (app.zip, kept 1 day)

gh workflow run reload-build.yml -R manaflow-ai/cmux --ref feat-cmux-next -f tag=pr23264 -f ref=claude/cmux-next-fast-gates -f platform=macos -f backend_mode=local

red: cmux-next checks (god files, concurrency, crash safety, l10n); base red: the same failure on the base 31c7030; does not block

teamleaderleo added a commit that referenced this pull request Oct 21, 2026
…a shallow checkout

Base red from #23249: tests/test_ci_no_shallow_dev_fetch.py flags base-autofix-settings-exports.sh:60
on every PR's cmux-next CI guard group (seen on #23264, job 119156226796).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?harness=claude&session=01a15c6d-a1d2-7fc2-9458-92c6bdcb7b4d&host=huge-blue
@teamleaderleo
teamleaderleo merged commit 3f206f4 into feat-cmux-next Oct 21, 2026
@teamleaderleo
teamleaderleo deleted the claude/cmux-next-fast-gates branch October 21, 2026 10:59
teamleaderleo added a commit that referenced this pull request Oct 21, 2026
…ng (#23315)

* test: base autofix makes a held repair again on the moved tip, keeps its Mac autofix, and judges settings by their stamp

Red against feat-cmux-next dfb5330: base-autofix-land.sh and base-autofix-repair.sh do not
exist, the autofix job's concurrency group lets a newer push replace the pending autofix that
carries the Mac exports, the push holds settings exports past a settings-area move that is no
stamp input, and defaults_guard.py has no source-stamp.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?harness=claude&session=01a15c6d-a1d2-7fc2-9458-92c6bdcb7b4d&host=huge-blue

* base autofix: make a held repair again on the moved tip, keep the Mac exports' autofix, judge settings by their stamp

- base-autofix-land.sh pushes the repair (base-autofix-push.sh) and, when the push holds a file
  this job makes itself because the tip moved its inputs (the CI target graph, the SDK layers,
  the ratchet, the cloud clients, cargo fmt), checks out the tip, makes the repair again there
  and pushes that, at most three rounds. Held Mac exports still wait for the next run, which
  the failed run keeps in its "since". The repair never sees the push token.
- base-autofix-repair.sh is the Repair step's logic, shared by the first round and the rounds
  on a moved tip (which apply no Mac export or webviews patch made at the older commit).
- The autofix job of a run that carries Mac exports has a concurrency group of its own, so a
  newer push can no longer replace it while it waits behind another run's autofix. Groups are
  per branch.
- The settings exports' inputs are what their stamp hashes (defaults_guard.py source-stamp):
  a settings-area move that leaves the tip's stamp the exports' own, with the same
  regeneration, neither holds them at the push nor makes the Mac job regenerate.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?harness=claude&session=01a15c6d-a1d2-7fc2-9458-92c6bdcb7b4d&host=huge-blue

* test: a push that never got through is no hold, and a held inspector page waits

Red on aa0c3fd (review): the push script's held list survives a failed attempt, so
base-autofix-land.sh remade only the graph and could pass without the settings exports; and
remade() counted the Chief inspector page, which only the webviews patch makes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?harness=claude&session=01a15c6d-a1d2-7fc2-9458-92c6bdcb7b4d&host=huge-blue

* base autofix: only a hold hands on a held list, and the inspector page is never remade on the tip

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?harness=claude&session=01a15c6d-a1d2-7fc2-9458-92c6bdcb7b4d&host=huge-blue

* base autofix: the settings regeneration fetches with --depth only in a shallow checkout

Base red from #23249: tests/test_ci_no_shallow_dev_fetch.py flags base-autofix-settings-exports.sh:60
on every PR's cmux-next CI guard group (seen on #23264, job 119156226796).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?harness=claude&session=01a15c6d-a1d2-7fc2-9458-92c6bdcb7b4d&host=huge-blue

* base autofix: settings fetch expands an empty depth list safely under bash 3.2

Review of #23315: in a full clone the depth array is empty, and "${depth[@]}" under set -u stops on macOS
/bin/bash 3.2. Same guarded expansion as the repo's other depth fetches.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?harness=claude&session=01a15c6d-a1d2-7fc2-9458-92c6bdcb7b4d&host=huge-blue

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

Review (subagent, correctness first): SHIP at e991606

This reviews the delta from 8ccd8ee (SHIP), which is only scripts/ci/cmux_next_fast_gates.py, +6/-1.

  • Shallow check: --is-shallow-repository output is stripped and compared to true, with check=True, so a failed rev-parse stops the run rather than falling back.
  • Full clone: it fetches the same two refspecs as the shallow path (refs/fast-gates/head and refs/fast-gates/base) with no depth. prepare's head, base and merge checks still run on both paths.
  • Guard: its scan gives [90, 94] at 8ccd8ee and [] here. Depth flags now run only after a positive shallow check, so this is a real fix, not a dodge.
  • Tests: test_cmux_next_fast_gates.py is 25/25 (shallow and full clone). The guard's repo-wide test passes.
  • Scope: nothing under .github changed, so post still trusts only resolve outputs and needs.fast-gates.result.

Nit: at :100, --unshallow isn't gated on the repo still being shallow. If the deepen steps reach the full history, git's "complete repository" error replaces prepare's clearer message. The run fails either way.

Agent session: huge-blue

@github-actions

github-actions Bot commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

risk:normal Risk tier (docs/ci/risk-tiers.md)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant