Skip to content

ci: fold obsolete focused gates into timed app-host shards - #13415

Closed
teamleaderleo wants to merge 7 commits into
mainfrom
ci-6134-fold-truthful-focused-gates
Closed

teamleaderleo wants to merge 7 commits into
mainfrom
ci-6134-fold-truthful-focused-gates

Conversation

@teamleaderleo

@teamleaderleo teamleaderleo commented Sep 21, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

The broad app-host classifier now fails on:

  • ordinary XCTest assertion failures;
  • Swift Testing failures;
  • missing/zero-test summaries;
  • any app-host crash/restart or timeout marker.

Four shard-5 focused gates still existed solely to work around the old failure-accounting bug from #7471. This PR returns those suites to the normal timing-weighted shard batch:

  • CloudNotificationDismissParityTests
  • AgentRestoreLiveOwnerAdmissionTests
  • GhosttyOptionAsAltModsTests
  • CLISSHSessionAttachAnchorTests

The dedicated workflow steps and focused-exclusion entries are removed together, so every suite still has exactly one owner.

Measurement

Hosted run 35589300329 attempt 1:

  • Cloud notification parity: 15s
  • restore admission: 11s
  • Option/Alt: 10s
  • SSH attach anchor: 10s

Those four dedicated launches cost 46s on shard 5. The run-wide app-host launch audit found 45 xcodebuild test invocations with ~306s total start→first-test overhead (6.3s median / 10.7s p90).

Expected effect:

  • app-host xcodebuild starts: 45 → 41
  • focused full-suite selectors: 60 → 56
  • broad timing-weighted selectors: 2782 → 2786
  • total semantic selector ownership: unchanged (2842)
  • shard-5 focused reservation: 245s → 199s, using the measured 46s removal; the returned suite costs are then accounted by the timing packer.

Isolation kept

This PR deliberately leaves process-global/destructive/crash-prone gates focused: system-proxy state, portal/window registries, numeric locale, renderer-memory measurement, Pi Feed global state, RemoteTmux mirror topology/crash suites, socket/file-store state, and other explicitly isolated workloads.

No retry, parallel-testing, or classifier behavior changes.

Related: #6134, #7471, #12180, #13095.

Validation

  • red-first sharding guard requires these suites to have broad ownership
  • existing strict-suite/workflow equality guard still proves every selector is owned exactly once
  • current classifier tests prove assertion/crash/timeout failures remain red
  • full hosted macOS validation requested

Summary by cubic

Folds four obsolete focused test gates back into the normal timing-weighted shard batch now that the broad app-host classifier catches assertion, Swift Testing, zero-test-summary, and crash/timeout failures.

  • Removes the dedicated shard-5 workflow steps and focused-exclusion entries for CloudNotificationDismissParityTests, AgentRestoreLiveOwnerAdmissionTests, GhosttyOptionAsAltModsTests, and CLISSHSessionAttachAnchorTests.
  • Lowers shard-5 reserved wall time from 245s to 199s; the returned suite costs are accounted by the timing packer.
  • Adds a CI guard that fails if these suites regain dedicated focused ownership or drop out of broad shard discovery; discovery is validated against the repository root.
  • Leaves process-global, destructive, and crash-prone gates focused; no changes to retry, parallel-testing, or classifier behavior.

Written for commit 2c5d245. Summary will update on new commits.

Review in cubic

@coderabbitai

coderabbitai Bot commented Sep 21, 2026 •

Copy link
Copy Markdown

Warning

Review limit reached

Next included review available in 45 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 10 included reviews currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Repository: manaflow-ai/cmux/.coderabbit.yaml

Review profile: ASSERTIVE

Plan: Advanced

Run ID: ac6f9637-4da2-4907-8596-0168ba23f1b8

📥 Commits

Reviewing files that changed from the base of the PR and between 93768d9 and 2c5d245.

📒 Files selected for processing (3)
  • .github/workflows/ci.yml
  • scripts/ci/cmux_unit_test_shard.py
  • tests/test_ci_cmux_unit_test_shard.py

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

Copy link
Copy Markdown
Contributor

All contributors have signed the CLA ✍️ ✅
Posted by the CLA Assistant Lite bot.

@teamleaderleo teamleaderleo added the full-ci EXPENSIVE: full macOS tests/builds; overrides selective PR routing. Not needed for normal checks. label Sep 21, 2026 — with ChatGPT Codex Connector
@teamleaderleo
teamleaderleo enabled auto-merge (squash) September 21, 2026 12:35
@greptile-apps

greptile-apps Bot commented Sep 21, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 5/5

The PR appears safe to merge; the latest correction passes the repository root expected by broad test discovery.

Summary

This PR moves four regression suites from dedicated focused CI launches into the normal timing-weighted app-host shards.

  • Removes the four focused workflow steps and their focused-selector exclusions.
  • Reduces shard 5 reserved wall time from 245 to 199 seconds.
  • Adds a guard proving the folded suites remain discoverable through broad sharding and lack focused ownership.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  A[Discover app-host test suites] --> B{Focused selector?}
  B -- Yes --> C[Dedicated focused gate]
  B -- No --> D[Timing-weighted broad shard]
  E[Four folded regression suites] --> D
  F[Ownership guard] --> G[Verify broad discovery]
  F --> H[Verify no focused owner]
Loading

Reviews (3) · Last reviewed commit: "test(ci): discover folded suites from re..."

Comment thread tests/test_ci_cmux_unit_test_shard.py Outdated
Comment thread tests/test_ci_cmux_unit_test_shard.py Outdated

Copy link
Copy Markdown
Collaborator Author

Superseded by #13426, which folds the broader set of obsolete focused gates after the same truthful failure-accounting work.

auto-merge was automatically disabled September 22, 2026 02:18

Pull request was closed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

full-ci EXPENSIVE: full macOS tests/builds; overrides selective PR routing. Not needed for normal checks.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant