ci(test_reds): read display-named Swift Testing reds and a zero-width-space suite end (port from main) by teamleaderleo · Pull Request #35892 · manaflow-ai/cmux

teamleaderleo · 2026-11-04T22:04:20Z

Summary

feat-cmux-next's scripts/ci/test_reds.py is the base-red classifier copy the landers run for every base. main fixed two Swift Testing parsing bugs in its own copy (#35723, #35609) that never reached this one:

@Test("A capped plan …") prints ✘ Test "A capped plan …" recorded an issue, with no (). The pattern needed the parentheses, so a job whose only reds were display-named tests had no failing tests and was compared by its error lines instead.
Swift Testing can print a zero-width space (U+200B) before an event line after interleaved output. The anchored suite patterns missed ✔ Suite "x" passed, so a passing suite read as cut short.

This ports both, with main's tests. It also fixes two review findings from #35893, which brings main to this copy:

A run rollup (ci-status) whose check-run stayed failed after its job was re-run green: the run was rescanned and its child reds could be excused. The exact job's conclusion is now checked for run rollups too, as it already was for routed ones.
Where the evidence reader raises RateLimited (main's main_fix_evidence.py), the classifier exits 3, which gh-merge-green reports as a rate limit. This copy's reader has no such class, so nothing changes here. After it lands, main can carry this file byte for byte (the main PR adds a parity check).

Testing

python3 tests/test_test_reds.py: on the base, the two new tests fail (test_display_named_swift_testing_reds_are_compared_by_name, test_a_suite_end_after_a_zero_width_space_is_whole); on the head, 117 tests pass. tests/test_base_red_excuse.py: the two new tests (test_a_red_ci_status_check_whose_job_was_retried_green_blocks, test_a_rate_limited_read_exits_3) fail on the test-only commit and pass at the head (196). tests/test_base_red_excuse.py and tests/test_ci_guard_check_isolation.py also pass.

Observed / Not run

Observed: the commands above, red on the test-only commit and green at the head.
Not run: no live PR re-judged with this copy.

Changelog

none

Agent session: agent session

🤖 Generated with Claude Code

…o-width-space suite end Both read wrong on feat-cmux-next's copy, which the landers run for every base; main already has the fixes (#35723, #35609). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Agent-Session: https://cmux.com/deeplink/agent?id=aid-8707001042ea634e8771

… zero-width space Ports main's #35723 and #35609 to feat-cmux-next's copy, the canonical base-red classifier the landers run for every base, so main's copy can be the same bytes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Agent-Session: https://cmux.com/deeplink/agent?id=aid-8707001042ea634e8771

github-actions · 2026-11-04T22:04:38Z

cmux status _{(one comment, edited in place; each line is updated by the job that owns it)}

Review f183487: HQ feed card · gallery before/after runs · checks

App of this head (app.zip, kept 1 day)

gh workflow run reload-build.yml -R manaflow-ai/cmux --ref feat-cmux-next -f tag=pr35892 -f ref=fm3h/test-reds-swift-display-names -f platform=macos -f backend_mode=local

Shipped in:

cmux NEXT nightly 1.0.0-nightly.4198271655001 (2a1e13d383)

teamleaderleo · 2026-11-04T22:04:40Z

Owned by an agent session · session

teamleaderleo · 2026-11-04T22:11:22Z

Review (codex, router-review): SHIP at 4b079b4

Reviewer: codex gpt-6-astra high, fast tier.

Follow-ups: 1. They land as checklist items in this PR's one follow-up tracking issue in this repo (Follow-ups: #N ..., label follow-up; add to an open one before filing, never one issue per finding), linking this receipt; the lane that opened the PR fixes them forward.

SHIP at 4b079b4

Visual verdict: unchanged by this diff

Focused test suites pass (117, 194, and 8 tests).

[P2] [follow-up] [category:correctness] [finding:F1] [outcome:pending] [scripts/ci/test_reds.py:65] The quoted-name regex cannot parse display names containing escaped or embedded quotes. Such a failing test is omitted from failed, so classify can return None for a red job instead of comparing it. Use an escaped-string-aware matcher and add a regression test.

Follow-ups:

F1: support quoted display names containing escaped quotes.

cmux-bot

cmux-bot approves 4b079b4: an independent review passed this exact head (router-review receipt by teamleaderleo). A push needs a new review; a later BLOCK at this head dismisses this approval.

…run's reds, and a rate limit exiting 1 Review findings on #35893 (F1, F2). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Agent-Session: https://cmux.com/deeplink/agent?id=aid-8707001042ea634e8771

…ate-limited read A ci-status check-run can stay failed after its job was re-run green; the run was then rescanned and its child reds could be excused. Check the exact job's conclusion for run rollups as for routed ones. Where the evidence reader raises RateLimited (main's), exit 3 so gh-merge-green reports a rate limit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Agent-Session: https://cmux.com/deeplink/agent?id=aid-8707001042ea634e8771

cursor · 2026-11-04T22:22:34Z

Bugbot needs on-demand usage enabled

Bugbot uses usage-based billing for this team and requires on-demand usage to be enabled.

A team admin can enable on-demand usage in the Cursor dashboard.

…ed; exit 3 on a rate limit Review findings F1 and F2. The classifier stays feat-cmux-next's bytes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Agent-Session: https://cmux.com/deeplink/agent?id=aid-8707001042ea634e8771

teamleaderleo · 2026-11-04T22:46:21Z

Review (codex, router-review): SHIP at 113b580

Reviewer: codex gpt-6.1-sol xhigh, fast tier.

Follow-ups: 3. They land as checklist items in this PR's one follow-up tracking issue in this repo (Follow-ups: #N ..., label follow-up; add to an open one before filing, never one issue per finding), linking this receipt; the lane that opened the PR fixes them forward.

SHIP at 113b580

Visual verdict: no images; Leo tries the build

[P1] [follow-up] [category:correctness] [finding:F1] [outcome:pending] scripts/ci/test_reds.py:65 keys display-named failures only by quoted text. Duplicate display names across suites can make an unrelated head failure appear shared with a base failure and be excused. Include suite/source identity or canonical test names.
[P2] [follow-up] [category:correctness] [finding:F2] [outcome:pending] The same regex stops at embedded or escaped quotes, so display names such as "handles \"quoted\" input" are omitted from failed_names. Parse escaped quotes and add coverage.
[P2] [follow-up] [category:reliability] [finding:F3] [outcome:pending] base_red_excuse.py:1397 only catches RateLimited if it reaches main; load_rollups/derive_rollups catch and wrap Refused first. A rate-limited workflow read for supported external repos can therefore still exit 1 instead of 3.

Follow-ups:

F1: Qualify display names before comparison.
F2: Support escaped quotes.
F3: Preserve RateLimited through intermediate exception handlers.

… and an older red listed before a newer green Review findings on #35893. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Agent-Session: https://cmux.com/deeplink/agent?id=aid-8707001042ea634e8771

… them; a pull_request_target run is never base evidence Review findings on #35893 (F1, F2). main's older copy excluded both PR events. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Agent-Session: https://cmux.com/deeplink/agent?id=aid-8707001042ea634e8771

cursor · 2026-11-04T22:47:51Z

Bugbot needs on-demand usage enabled

Bugbot uses usage-based billing for this team and requires on-demand usage to be enabled.

A team admin can enable on-demand usage in the Cursor dashboard.

…t_target is never base evidence Review findings. The classifier stays feat-cmux-next's bytes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Agent-Session: https://cmux.com/deeplink/agent?id=aid-8707001042ea634e8771

teamleaderleo · 2026-11-04T23:09:31Z

Review (codex, router-review): SHIP at f183487

Reviewer: codex gpt-6.1-sol xhigh, fast tier.

Follow-ups: 2. They land as checklist items in this PR's one follow-up tracking issue in this repo (Follow-ups: #N ..., label follow-up; add to an open one before filing, never one issue per finding), linking this receipt; the lane that opened the PR fixes them forward.

SHIP at f183487

Visual verdict: unchanged by this diff

Targeted classifier and base-red tests pass.

[P2] [follow-up] [category:correctness] [finding:F1] [outcome:pending] scripts/ci/test_reds.py:65 fails to parse display names containing escaped quotes (\"), so that test’s red is omitted and cannot be compared. Use an escaped-string-aware pattern and add coverage.
[P2] [follow-up] [category:correctness] [finding:F2] [outcome:pending] scripts/ci/base_red_excuse.py:1337 sorts only by created_at. GitHub timestamps have second-level precision; equal timestamps preserve API order, so an older red run can precede a newer green run and incorrectly provide base evidence. Tie-break by run ID.

Follow-ups:

[finding:F1] Handle escaped quotes in Swift display names.
[finding:F2] Add a run-ID tie-breaker to base-run ordering.

cmux-bot

cmux-bot approves f183487: an independent review passed this exact head (router-review receipt by teamleaderleo). A push needs a new review; a later BLOCK at this head dismisses this approval.

teamleaderleo and others added 2 commits November 4, 2026 21:55

teamleaderleo mentioned this pull request Nov 4, 2026

ci(base-red): main runs feat-cmux-next's base-red classifier; a parity check keeps them the same #35893

Open

github-actions Bot added the risk:low Risk tier (docs/ci/risk-tiers.md) label Nov 4, 2026

cmux-bot Bot approved these changes Nov 4, 2026

View reviewed changes

teamleaderleo and others added 2 commits November 4, 2026 22:22

test(base-red): reproduce a retried-green ci-status job excusing its …

da45a74

…run's reds, and a rate limit exiting 1 Review findings on #35893 (F1, F2). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Agent-Session: https://cmux.com/deeplink/agent?id=aid-8707001042ea634e8771

teamleaderleo temporarily deployed to ci-routing-agent November 4, 2026 22:22 — with GitHub Actions Inactive

github-actions Bot added risk:normal Risk tier (docs/ci/risk-tiers.md) and removed risk:low Risk tier (docs/ci/risk-tiers.md) labels Nov 4, 2026

teamleaderleo mentioned this pull request Nov 4, 2026

base-red classifier: lost base runners as no verdict; escaped quotes in Swift Testing display names #35902

Open

teamleaderleo and others added 2 commits November 4, 2026 22:47

test(base-red): reproduce a pull_request_target run as base evidence,…

37fb5b2

… and an older red listed before a newer green Review findings on #35893. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Agent-Session: https://cmux.com/deeplink/agent?id=aid-8707001042ea634e8771

teamleaderleo temporarily deployed to ci-routing-agent November 4, 2026 22:47 — with GitHub Actions Inactive

teamleaderleo added the ready-to-land Reviewed at this head; next-autoland merges it into feat-cmux-next when its only reds are base reds label Nov 4, 2026

cmux-bot Bot approved these changes Nov 4, 2026

View reviewed changes

cmux-bot Bot merged commit d2a578b into feat-cmux-next Nov 4, 2026
200 checks passed

cmux-bot Bot deleted the fm3h/test-reds-swift-display-names branch November 4, 2026 23:19

Navigation Menu

Repository navigation

ci(test_reds): read display-named Swift Testing reds and a zero-width-space suite end (port from main) - #35892

ci(test_reds): read display-named Swift Testing reds and a zero-width-space suite end (port from main)#35892
cmux-bot[bot] merged 6 commits into
feat-cmux-nextfrom
fm3h/test-reds-swift-display-names

teamleaderleo commented Nov 4, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Nov 4, 2026 •

edited

Loading

Uh oh!

teamleaderleo commented Nov 4, 2026

Uh oh!

teamleaderleo commented Nov 4, 2026

Uh oh!

cmux-bot Bot left a comment

Uh oh!

cursor Bot commented Nov 4, 2026

Uh oh!

teamleaderleo commented Nov 4, 2026

Uh oh!

cursor Bot commented Nov 4, 2026

Uh oh!

teamleaderleo commented Nov 4, 2026

Uh oh!

cmux-bot Bot left a comment

Uh oh!

Uh oh!

This branch was previously deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

teamleaderleo commented Nov 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Testing

Observed / Not run

Changelog

Uh oh!

github-actions Bot commented Nov 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

teamleaderleo commented Nov 4, 2026

Uh oh!

teamleaderleo commented Nov 4, 2026

Uh oh!

cmux-bot Bot left a comment

Choose a reason for hiding this comment

Uh oh!

cursor Bot commented Nov 4, 2026

Bugbot needs on-demand usage enabled

Uh oh!

teamleaderleo commented Nov 4, 2026

Uh oh!

cursor Bot commented Nov 4, 2026

Bugbot needs on-demand usage enabled

Uh oh!

teamleaderleo commented Nov 4, 2026

Uh oh!

cmux-bot Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

This branch was previously deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

teamleaderleo commented Nov 4, 2026 •

edited

Loading

github-actions Bot commented Nov 4, 2026 •

edited

Loading