Skip to content

base autofix: a queued Mac export job keeps its place (base red: no export since 02:14Z) - #23352

Merged
teamleaderleo merged 6 commits into
feat-cmux-nextfrom
claude/autofix-mac-exports-keep-queue
Oct 21, 2026
Merged

teamleaderleo merged 6 commits into
feat-cmux-nextfrom
claude/autofix-mac-exports-keep-queue

Conversation

@teamleaderleo

@teamleaderleo teamleaderleo commented Oct 21, 2026 •

Copy link
Copy Markdown
Collaborator

Base red, priority zero (coord-deputy). The cmux-next base autofix has not landed a Mac export since 02:14Z. schemas/settings/settings-schema.json on feat-cmux-next still says chief-pet where the source says chief (#22848), so settings/categories.test.ts fails on every webview run.

Why the Mac job never started. From 11:15Z to 11:36Z every run's action exports (Mac) job had a start time and no runner, and was cancelled 3 to 4 minutes later. Runs 39590571817 through 39592618210 are examples. The job's concurrency group (cancel-in-progress: false) never cancels a running job, but it replaces a pending one. With the Mac std runners saturated, each job waited minutes for a runner, and the next push, a few minutes later, replaced it before it started. #23315's fixes for a running Mac job never got the chance to apply.

Fix

  • action-exports has no concurrency group, so a queued job keeps its place.
  • A new step in action export areas, scripts/cmux-next/base-autofix-mac-busy.sh, skips this run's Mac job while another run's Mac job is queued or running. That job regenerates the settings exports on the newest tip when it starts (base autofix: regenerate the settings exports on the newest tip, so they land #23249).
  • A run that skipped its Mac job fails at the end ("Mac exports deferred to the run already making them"). The next run diffs from the last successful run, so its changes stay in the next run's range.
  • A failed API read starts the Mac job: two Mac jobs beat none. Two runs checking at the same moment can also both start one.

Also fixes my own base red from #23315: tests/test_cmux_next_pr_autofix_settings.py SettingsOnlyArtifactLeavesTheActionFollowers. Three base subtests fail with exit 127 in cmux-next checks at 92c12bc (seen on #23330). #23315 moved the base Repair step into scripts/cmux-next/base-autofix-repair.sh, which reads EXPORTS_DIR from the step env. The test ran the step's script without that file or env. That job was already red on base for other reasons, which hid this on #23315.

Test

Red commits first:

  • 51e1cea fixes the test itself. It was red on base: python3 tests/test_cmux_next_pr_autofix_settings.py, FAILED (failures=3), base-autofix-repair.sh: No such file or directory.
  • 926833d adds the queue tests. python3 tests/test_cmux_next_base_autofix_finishes.py against base's workflow gives FAILED (failures=8):
    • the Mac job has no group;
    • a run skips its Mac job while one is in flight;
    • a deferred run fails;
    • base-autofix-mac-busy.sh with a gh stand-in: another run's queued, waiting or in-progress Mac job counts; its own, a completed one or none doesn't; an unreadable API starts the job.

Green at the head: both files OK (27 and 6 tests), and actionlint on the workflow is clean.

Observed / Not run

  • Observed: the tests above, red then green, on huge-blue. The job timings above come from the Actions API.
  • Not run: a real feat-cmux-next run of the new workflow. After this merges I'll do one glaeda-gh wait on the first run whose Mac job finishes. Then I'll check that settings-schema.json says chief on base and that settings/categories passes, and report both.
  • tests/test_ci_no_shallow_dev_fetch.py fails in this sparse checkout only (a file not checked out).

Changelog

none

Agent session: huge-blue

🤖 Generated with Claude Code


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.


Summary by cubic

Fixes the base autofix pipeline so a queued Mac export job keeps its place instead of being replaced by the next push, and a deferred run passes its changes to the next run instead of losing them.

Written for commit a7ab126. Summary will update on new commits.

View guided diff Turn on auto-fix

teamleaderleo and others added 3 commits October 21, 2026 11:39
…h with its step env

#23315 moved the base Repair step into scripts/cmux-next/base-autofix-repair.sh, which reads EXPORTS_DIR from
the step env. SettingsOnlyArtifactLeavesTheActionFollowers ran the step's script without that file or env, so
its three base subtests failed with 127 in cmux-next checks on feat-cmux-next (92c12bc).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?harness=claude&session=01a15c6d-a1d2-7fc2-9458-92c6bdcb7b4d&host=huge-blue
… while one is in flight

2026-10-21 11:15-11:36Z: every base autofix run's Mac job waited for a saturated runner and the next push
replaced it while pending, so none started and no export landed after 02:14Z.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?harness=claude&session=01a15c6d-a1d2-7fc2-9458-92c6bdcb7b4d&host=huge-blue
…s own while one is in flight

The action-exports job's concurrency group (cancel-in-progress false) still replaced a pending job. With the Mac
runners saturated every job waited minutes for a runner, and the next push replaced it before it started: from
11:15Z to 11:36Z on 2026-10-21 every run's Mac job was cancelled without a runner, and no export landed after
02:14Z (settings-schema.json still says chief-pet; settings/categories red).

The job has no group now. A Linux step (base-autofix-mac-busy.sh) skips it while another run's Mac job is queued
or running; that job regenerates the settings exports on the newest tip when it starts. A run that skipped fails
at the end, so the next run's since covers its changes. A failed read starts the job: two Mac jobs beat none.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?harness=claude&session=01a15c6d-a1d2-7fc2-9458-92c6bdcb7b4d&host=huge-blue
@teamleaderleo teamleaderleo added the risk:normal Risk tier (docs/ci/risk-tiers.md) label Oct 21, 2026
@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

Owned by land-and-ci on huge-blue · session

@github-actions

github-actions Bot commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

cmux status (one comment, edited in place; each line is updated by the job that owns it)

Review a7ab126: HQ feed card · gallery before/after runs · checks

App of this head (app.zip, kept 1 day)

gh workflow run reload-build.yml -R manaflow-ai/cmux --ref feat-cmux-next -f tag=pr23352 -f ref=claude/autofix-mac-exports-keep-queue -f platform=macos -f backend_mode=local

web and size checks: green

@blacksmith-sh

blacksmith-sh Bot commented Oct 21, 2026

Copy link
Copy Markdown

Found 2 test failures on Blacksmith runners:

Failures

Test View Logs
/tmp/tmpi7fg91lm/summary-missing.json (live):/summary-missing.json (live): View Logs
/tmp/tmpm_d71v3s/summary-fail.json (live):/summary-fail.json (live): View Logs

Fix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.

teamleaderleo and others added 3 commits October 21, 2026 11:46
…over it

A run that left its Mac exports to another run's queued or running Mac job
failed so the next run's since still covered its changes. With the Mac runners
saturated that is most runs, and base-red-watch blamed each red one on whoever
pushed. Now it ends green with a 'Mac exports deferred' job naming the run that
carries the exports, and base-autofix-since.sh (both change lookups) returns the
newest successful run without that job.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?harness=claude&session=01a15c6d-a1d2-7fc2-9458-92c6bdcb7b4d&host=huge-blue
@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

Bead cx-dgsy5 created for this PR by the cmux coordinator.

@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

Review (subagent, correctness first): SHIP at a7ab126

Covers the whole PR at this head. An earlier full review passed 34936c1. The delta review covers 34936c1..a7ab126: d47a3d2 (red tests), 364891d (a deferred run ends green with a "Mac exports deferred" job; base-autofix-since.sh passes over such runs in both change lookups) and a7ab126 (registers tests/test_crash_class_fast_gate.py, lane linux-guard).

  • No change can be skipped for good. While R1's Mac job is in flight, later runs diff from the success before R1, so they see Mac areas and defer. When R1 succeeds, since moves to h1. When R1 fails, since stays and the next run makes its own exports.
  • Deferred runs push no Mac exports (--unverified-exports, no --generated).
  • since.sh fallbacks only widen the set of repairs. Re-runs work as intended with the jobs endpoint's filter=latest.
  • Registry: right lane (cmux-next-guards.yml:323 runs the file), no duplicate, and the validator on the merge tree reports nothing for it.
  • The test files pass and actionlint is clean. The new tests fail on 34936c1.

Nits: at the head alone (before #23331), the registry validator flags the file as missing, but CI checks the merge ref. A cancelled 2-minute deferred job makes its run red, which is rare and loses no changes.

@teamleaderleo
teamleaderleo merged commit e625806 into feat-cmux-next Oct 21, 2026
@teamleaderleo
teamleaderleo deleted the claude/autofix-mac-exports-keep-queue branch October 21, 2026 11:55
teamleaderleo pushed a commit that referenced this pull request Oct 21, 2026
…te; braces in strings don't end a class

Base registered tests/test_crash_class_fast_gate.py (#23352) after the
pre-commit hook had registered it on this branch, so keep base's entry only.
Review: declares_nonisolated_deinit counted a "}" in a string literal; it now
scans main_actor_scope.strip(line), and the header names the rule.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?harness=claude&session=01a15ba2-4ce2-7e02-8fa2-5cd34f7648b1&host=cosmic-gray
@github-actions

github-actions Bot commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

teamleaderleo added a commit that referenced this pull request Oct 21, 2026
…o nonisolated deinit (the #23050 Release crash) (#23361)

* ci(next): a crash-class test for a generic main-actor class with no nonisolated deinit (red)

#23050's FirstMouseHostingView, a generic NSHostingView subclass in a
uiSwiftSettings target, had an implicit isolated deinit that crashed the
Release compile's optimizer (fixed in #23233). This fixture test expects
check-concurrency.sh to refuse that shape; the rule comes next.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?harness=claude&session=01a15ba2-4ce2-7e02-8fa2-5cd34f7648b1&host=cosmic-gray

* ci(next): check-concurrency refuses a generic main-actor class with no nonisolated deinit

In a uiSwiftSettings target a generic class's implicit deinit is isolated, and
the Release compile's optimizer crashes on it (EarlyPerfInliner, swift-frontend
6.3, macOS 14 target): #23050's FirstMouseHostingView broke every Release
compile until #23233, the same crash #20408 fixed before. The crash-class rule
now requires `nonisolated deinit` there (a reviewed `// concurrency-allow:`
escapes it). Non-UI targets stay out of scope.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?harness=claude&session=01a15ba2-4ce2-7e02-8fa2-5cd34f7648b1&host=cosmic-gray

* Merge feat-cmux-next; one registry entry for test_crash_class_fast_gate; braces in strings don't end a class

Base registered tests/test_crash_class_fast_gate.py (#23352) after the
pre-commit hook had registered it on this branch, so keep base's entry only.
Review: declares_nonisolated_deinit counted a "}" in a string literal; it now
scans main_actor_scope.strip(line), and the header names the rule.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?harness=claude&session=01a15ba2-4ce2-7e02-8fa2-5cd34f7648b1&host=cosmic-gray

* ci(next): the generic-class rule ignores braces in strings; the header names it

Review: declares_nonisolated_deinit counted a "}" in a string literal and ended
the class before its deinit. Scan main_actor_scope.strip(line), as the scope
scan does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?harness=claude&session=01a15ba2-4ce2-7e02-8fa2-5cd34f7648b1&host=cosmic-gray

---------

Co-authored-by: Leo Li <leoli.4u@gmail.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
teamleaderleo added a commit that referenced this pull request Oct 21, 2026
… action export since 02:14Z) (#23453)

* test: base autofix makes the action exports on the newest tip and judges them from there

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?harness=claude&session=01a15c6d-a1d2-7fc2-9458-92c6bdcb7b4d&host=huge-blue

* base autofix: make the action exports on the newest tip and judge them from there

No action export landed after 02:14Z on 2026-10-21 (ActionSurfaceParityTests.reportIsFresh red on
base). Once #23352 stopped the Mac job being replaced, it finished green, but base-autofix-push.sh
judged the exports from the run's own commit: the job queues for up to half an hour and the actions
area moves every few minutes, so every run held them (runs 39607727081, 39609533631).

- scripts/cmux-next/base-autofix-action-exports.sh (the Mac step): starts on the newest tip when the
  actions area moved while the job queued, regenerates there again while it keeps moving (3 in
  all), prints "made at: SHA", and leaves the checkout at the run's commit for the settings step.
- base-autofix-push.sh judges the action exports from ACTIONS_MADE_AT, as it does the settings
  exports: they land whole as made there when nothing after it moved the actions area or wrote
  one of them. Their own inputs are the actions area only, so the shortcuts and app-host areas no
  longer hold them (they still hold what is built from them).
- base-autofix-land.sh hands ACTIONS_MADE_AT to the push; the Mac job gets 90 minutes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?harness=claude&session=01a15c6d-a1d2-7fc2-9458-92c6bdcb7b4d&host=huge-blue

* test: files built from the action exports are made again on the tip; a failed checkout of the tip fails the Mac step

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?harness=claude&session=01a15c6d-a1d2-7fc2-9458-92c6bdcb7b4d&host=huge-blue

* base autofix: make the docs shortcuts, app-host files and snapshot again on the tip

The docs shortcuts (the "keyboard-shortcuts docs freshness" base red), the app-host files and the
defaults snapshot are built from the action exports at the run's commit, so a CmuxNextActions move
during the run held them even when the exports made after it landed, and the next run held them
the same way. base-autofix-land.sh now makes them again on the tip when the tip's action exports
are known current: none was held and nothing after ACTIONS_MADE_AT moved their inputs (the exports
this run landed are no move). The snapshot also needs the tip's settings stamp current.
base-autofix-repair.sh builds them from the tip's committed exports when told so.

Review fix: base-autofix-action-exports.sh's check_out runs under `||`, where set -e is off, so a
failed checkout of the tip went on and named exports made at the old commit as made at the tip.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?harness=claude&session=01a15c6d-a1d2-7fc2-9458-92c6bdcb7b4d&host=huge-blue

* test: the snapshot waits when the tip's settings stamp is stale, judged by a guard that finds its repository like the real one

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?harness=claude&session=01a15c6d-a1d2-7fc2-9458-92c6bdcb7b4d&host=huge-blue

* base autofix: judge the tip's settings stamp with the tip's own guard in its checkout

Review fix: base-autofix-land.sh ran a copy of defaults_guard.py from a temp directory, but the guard
finds the repository from its own path; there it read no stamp, noted it and passed, so a stale
settings stamp never held the defaults snapshot. The round now checks the tip out first, runs the
tip's guard there, and also needs the tip to have a stamp.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?harness=claude&session=01a15c6d-a1d2-7fc2-9458-92c6bdcb7b4d&host=huge-blue

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

risk:normal Risk tier (docs/ci/risk-tiers.md)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant