ci: passing guard tests print no ::error annotations

[teamleaderleo]

PR #15160 run 36420353579 showed Error: No ci-ui-tests.yml run titled 'UI tests for CI run 100 attempt 1' appeared... and Error: https://x/900 ended success without running the UI tests on its run page. Those came from test fixtures in "Validate focused test launcher", which passed. The runner parses any output line starting with :: as a workflow command, so a script under test that annotates on purpose turned into a red annotation.

An audit of all 171 Python guard tests in ci-guards.yml and ci-fast-guards.yml found six that leak on a pass: test_ci_ui_tests_dispatch, test_ci_pr_media, test_ci_late_placement, test_ci_main_regression_bisect, test_ci_prune_pr_media and test_ios_upload_batching. test_ci_helper_prebuild_lifecycle does too, going by its output here, and so does test_ci_parallel_artifact_transport, which prints seven ::warning:: lines from ci-artifact-transport.yml. test_ci_merge_main still prints ::group:: folds, which are not annotations and are left alone. Each now runs unittest.main(buffer=True), which captures a test's stdout and stderr and prints them only when that test fails. A passing run prints nothing, and a failure still shows its full output.

The regression test, tests/test_ci_guard_tests_emit_no_workflow_commands.py, runs those modules with GITHUB_ACTIONS=true and fails if a passing module prints a line the runner would parse as a command. It is wired into the same step. A module that fails in that environment is reported as skipped instead of passing silently.

Red (commit 1, 17bc4f7): python3 tests/test_ci_guard_tests_emit_no_workflow_commands.py gave FAILED (failures=5); helper_prebuild_lifecycle fails locally under the sandbox, so it was skipped. Green (commit 2): OK in 4.4 s. Each of the six modules still passes on its own.

Changelog

none

— Radish g1 🌿 (run_worker_20260928_22d4c630)

🤖 Generated with Claude Code

Summary by CodeRabbit

• Tests
  • Added a CI check to detect GitHub workflow commands in output from passing guard tests.
  • Updated test output handling to buffer results for clearer CI reporting.

[coderabbitai]

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Repository: manaflow-ai/cmux/.coderabbit.yaml

Review profile: ASSERTIVE

Plan: Advanced

Run ID: 9338cd41-161e-4dfb-be21-eca2587799e9

📥 Commits

Reviewing files that changed from the base of the PR and between 31a59ab and 016d7a2.

📒 Files selected for processing (11)

• .github/workflows/ci-guards.yml
• tests/test-execution.toml
• tests/test_ci_guard_tests_emit_no_workflow_commands.py
• tests/test_ci_helper_prebuild_lifecycle.py
• tests/test_ci_late_placement.py
• tests/test_ci_main_regression_bisect.py
• tests/test_ci_parallel_artifact_transport.py
• tests/test_ci_pr_media.py
• tests/test_ci_prune_pr_media.py
• tests/test_ci_ui_tests_dispatch.py
• tests/test_ios_upload_batching.py

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 0 remain after this review.

---

📝 Walkthrough

Walkthrough

The pull request adds a unittest that checks output from selected test modules for GitHub workflow commands. It enables output buffering in eight test entry points and adds the guard test to the CI launcher and linux-guard registry.

Changes

Workflow Command Output Guard

Layer / File(s)	Summary
Check test output for workflow commands tests/test_ci_guard_tests_emit_no_workflow_commands.py	The new test runs eight modules concurrently with GitHub Actions enabled. It captures their combined output, checks successful runs for workflow-command lines, and reports up to ten matches per module.
Buffer module output and run the guard tests/test_ci_*.py, tests/test_ios_upload_batching.py, .github/workflows/ci-guards.yml, tests/test-execution.toml	Eight test entry points enable output buffering. The CI workflow launcher and linux-guard registry include the new guard test.

Priority: ⬇️ Low

Estimated code review effort: 3 (Moderate) | ~20 minutes

Change: Bug fix

Merge Risk: ⚪ Minimal · up to 016d7

The guard and its CI wiring are mergeable after normal checks; no unresolved failure was established.

🚥 Pre-merge checks | ✅ 24 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 50.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 2 functions across 9 files. (2 skipped: 2…	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (24 passed)

Check name	Status	Explanation
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Cmux Cloud Persistent Session And Early Input	✅ Passed	PASS. The PR changes only CI guard registration, a workflow-command regression test, and unittest output buffering in existing test modules. The authoritative diff introduces no Cloud terminal creat…
Cmux Swift Actor Isolation	✅ Passed	The pull request changes only .github/workflows/ci-guards.yml and Python/TOML files under tests/. The authoritative diff contains no Swift files and no production Swift changes, so the Swift actor…
Cmux Swift Blocking Runtime	✅ Passed	The reviewed diff changes only Python tests, TOML, and a GitHub Actions workflow. It contains no Swift or Objective-C production changes and introduces no runtime blocking or timing synchronization co…
Cmux Browser Automation Off-Main	✅ Passed	The custom check is not applicable. The pull request changes only CI workflow, test registry, and Python test files. It changes no Swift browser automation targets, socket-worker routing, WebKit/AppKi…
Cmux Expensive Synchronous Load	✅ Passed	PASS: The pull request changes only CI workflow and Python test files. The authoritative diff contains no Swift paths and no production Swift changes, so the expensive synchronous load condition does …
Cmux Cache Substitution Correctness	✅ Passed	PASS: The PR changes only Python test modules plus CI YAML/TOML. The diff contains no production Swift, TypeScript, or JavaScript changes, and it does not alter a persistence, history, undo, or snapsh…
Cmux No Hacky Sleeps	✅ Passed	PASS: The pull request changes only Python test modules and CI/test registration files. It adds a subprocess timeout in test-only scaffolding, but it adds no production TypeScript, JavaScript, shell, …
Cmux Algorithmic Complexity	✅ Passed	PASS. The PR changes only CI configuration and Python test modules; it does not change production Swift, TypeScript, JavaScript, shell, or runtime code. The new concurrent loop processes a fixed tuple…
Cmux Swift Concurrency	✅ Passed	The pull request changes only CI workflow and Python/TOML test files. The review diff contains no Swift paths or Swift code, so it does not introduce or expand any legacy Swift concurrency pattern.
Cmux Swift @Concurrent	✅ Passed	The pull request changes only CI workflow, TOML, and Python test files. The authoritative diff contains no Swift files, so the Swift @concurrent check is not applicable.
Cmux Swift Package Boundaries	✅ Passed	PASS: The authoritative diff changes only GitHub workflow configuration, TOML test registration, and Python test files. It contains no Swift source files, Package.swift files, or production Swift chan…
Cmux Swiftpm Lockfiles	✅ Passed	PASS: The PR changes only CI/test files. The workflow change adds one Python test invocation and does not alter SwiftPM dependencies or Xcode package references. No Package.swift, Package.resolved, .g…
Cmux Swift Logging	✅ Passed	The pull request changes only CI configuration and Python test files; it contains no Swift changes. Therefore the Swift logging check is not applicable, and it introduces no prohibited production Swif…
Cmux User-Facing Error Privacy	✅ Passed	PASS. The PR changes only CI workflow registration, test execution metadata, and test modules. The added error text and captured command output are developer-facing test diagnostics, and the rule expl…
Cmux Full Internationalization	✅ Passed	The pull request changes only CI workflow registration, test execution metadata, and Python test modules. The added and modified text is test documentation, assertions, skip messages, and CI diagnosti…
Cmux Swiftui State Layout	✅ Passed	PASS: The pull request changes only Python, TOML, and YAML files. The authoritative diff contains no SwiftUI or Swift source changes, so it does not introduce or expand any state, layout, list-row sto…
Cmux Architecture Rethink	✅ Passed	PASS: The authoritative pull-request diff changes only Python, TOML, and YAML files. It contains no Swift architectural changes, so the Swift architectural rethink failure conditions do not apply.
Cmux Swift Auxiliary Window Close Shortcuts	✅ Passed	PASS: The pull request changes only CI configuration and Python test files. The authoritative diff contains no Swift changes and no standalone cmux-owned window code, so the auxiliary-window close-sho…
Cmux Source Artifacts	✅ Passed	PASS. The diff contains only hand-written CI workflow/config changes and Python test source. The new test uses a runtime TemporaryFile, but it does not add a temp folder or generated artifact to sourc…
Cmux No Test Or Debug Seam In Production Source	✅ Passed	PASS: The pull-request diff changes only CI configuration, TOML, and Python test files. It contains no Swift file under a production Sources/ path, so it cannot introduce the specified production te…
Title check	✅ Passed	The title clearly describes the primary change: preventing passing guard tests from emitting GitHub workflow error annotations.
Description check	✅ Passed	The description explains the problem, implementation, regression test, test results, and changelog status. It does not use the template's explicit Summary and Testing headings and omits the checklist,…

Full details: Docstring Coverage

Explanation

Docstring coverage is 50.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 2 functions across 9 files. (2 skipped: 2 unsupported.)

• Fix all pre-merge checks with AI

✨ Finishing Touches 💡 1

📝 Generate docstrings 💡

• Commit to this branch
• Create a new PR

🧪 Generate unit tests (beta)

• Commit to this branch
• Create a new PR

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

All contributors have signed the CLA ✍️ ✅
_{Posted by the CLA Assistant Lite bot.}

[github-actions]

CI failure attribution

CI passes on 016d7a28d9 (run 36444760774 attempt 1).

Written by scripts/ci/classify_failures.py (ci-failure-attribution.yml); signatures are its SIGNATURES table. A machine verdict is the runner's fault, not this PR's.

[cursor]

Bugbot is paused — on-demand spend limit reached

Bugbot uses usage-based billing for this team and has hit its on-demand spend limit.

A team admin can raise the spend limit in the Cursor dashboard, or wait for the next billing cycle to continue.

[github-actions]

Merge receipt for 016d7a28d9: every check was green at merge (11 verified; 13 skipped by policy). Full suite runs on main after merge.