Skip to content

test(cmux): add rapid spawn/kill IOSurface fixture - #4

Merged
EtanHey merged 1 commit into
mainfrom
feat/p2e-rapid-spawn-kill-fixture
Apr 27, 2026
Merged

EtanHey merged 1 commit into
mainfrom
feat/p2e-rapid-spawn-kill-fixture

Conversation

@EtanHey

@EtanHey EtanHey commented Apr 27, 2026 •

Copy link
Copy Markdown
Owner

Summary

  • Add tests/fixtures/rapid_spawn_kill.sh to rapidly launch, sample, terminate, and relaunch cmux app processes with no inter-iteration settle time.
  • Add RapidSpawnKillFixtureTests to run the fixture under Apple leaks --atExit and enforce a default VM: IOSurface <= 50 MB budget.
  • Document fixture usage and environment overrides in tests/fixtures/README.md.

TDD / Verification

  • RED: targeted XCTest failed before the fixture existed: Expected executable fixture at .../tests/fixtures/rapid_spawn_kill.sh.
  • GREEN: shellcheck tests/fixtures/rapid_spawn_kill.sh.
  • GREEN: git diff --check and git diff --cached --check.
  • GREEN: PATH="/opt/homebrew/opt/zig@0.15/bin:$PATH" xcodebuild -quiet -project GhosttyTabs.xcodeproj -scheme cmux-unit -configuration Debug -destination 'platform=macOS,arch=arm64' -derivedDataPath /tmp/cmux-p2e-bash-derived -only-testing:cmuxTests/RapidSpawnKillFixtureTests/testRapidSpawnKillFixtureKeepsIOSurfaceFootprintUnderBudget test.
  • Leaks evidence extracted from xcresult attachment: three samples at 12.60 MB; final VM: IOSurface = 12.60 MB.

Notes

  • CodeRabbit local review failed before producing findings due EAGAIN from posix_spawn '/opt/homebrew/bin/git' while trying to diff unrelated node_modules/undici* paths.
  • One fresh DerivedData rerun hit No space left on device; temporary cmux DerivedData directories created during verification were removed and the final pass used the resolved DerivedData cache.

Note

Medium Risk
Adds a new integration-style XCTest that shells out to leaks/vmmap and repeatedly spawns/terminates the app, which can be timing- and environment-sensitive in CI.

Overview
Adds an IOSurface regression harness that rapidly spawns and kills the cmux app, samples IOSurface footprint via vmmap, and reports a peak VM: IOSurface = <N> MB value (tests/fixtures/rapid_spawn_kill.sh).

Introduces RapidSpawnKillFixtureTests (wired into the cmuxTests Xcode target) to run the fixture under leaks --atExit, attach the combined output to the test result, and fail if the measured IOSurface footprint exceeds a configurable default budget (50 MB). Documentation for running and configuring the fixture is added under tests/fixtures/README.md.

Reviewed by Cursor Bugbot for commit 78a38b0. Bugbot is set up for automated code reviews on this repo. Configure here.

Summary by CodeRabbit

  • Tests

    • Added a new test suite for validating memory stability during rapid process spawn/kill cycles, including timeout and memory threshold assertions.
  • Documentation

    • Added documentation for the rapid spawn/kill stress testing fixture, including configuration options and usage examples.

@coderabbitai

coderabbitai Bot commented Apr 27, 2026 •

Copy link
Copy Markdown

Caution

Review failed

Pull request was closed or merged during review

📝 Walkthrough

Walkthrough

This change introduces a new stress test fixture rapid_spawn_kill.sh that repeatedly spawns and terminates a cmux application process while monitoring IOSurface memory usage through vmmap. A corresponding XCTest (RapidSpawnKillFixtureTests) runs the fixture under /usr/bin/leaks and validates that IOSurface memory stays within configured thresholds. The Xcode project configuration is updated to wire the new test into the build pipeline, and documentation is provided explaining the fixture's behavior and environment variables.

Changes

Cohort / File(s) Summary
Xcode Project Configuration
GhosttyTabs.xcodeproj/project.pbxproj
Added test file reference, build file entry, and registration in cmuxTests target's Sources build phase.
Test Implementation
cmuxTests/RapidSpawnKillFixtureTests.swift
New XCTest class that executes the rapid_spawn_kill.sh fixture via leaks, resolves cmux app bundle path from environment variables and Xcode build directories, parses IOSurface memory output, and asserts it remains within CMUX_RAPID_SPAWN_KILL_IOSURFACE_LIMIT_MB threshold (default 50 MB).
Fixture Script & Documentation
tests/fixtures/rapid_spawn_kill.sh, tests/fixtures/README.md
New bash fixture script that iteratively spawns/kills cmux processes, polls for readiness via vmmap IOSurface sampling, tracks peak memory, and provides summary output; documentation explains usage, environment variables, and integration with the XCTest.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Poem

🐰 Hop, spawn, and kill with glee!
Memory leaks we'll never see,
IOSurface bounds held tight,
Stress tests running through the night,
Swift assertions shining bright!

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly and concisely summarizes the main change: adding a rapid spawn/kill IOSurface fixture for testing.
Description check ✅ Passed The description covers all key template sections (Summary, Testing/Verification) with comprehensive details, though the Demo Video and Checklist sections are not fully completed.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/p2e-rapid-spawn-kill-fixture

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@EtanHey
EtanHey force-pushed the feat/p2e-rapid-spawn-kill-fixture branch from 4a6e4ea to 78a38b0 Compare April 27, 2026 17:19
@EtanHey
EtanHey merged commit ab68116 into main Apr 27, 2026
4 of 10 checks passed
@EtanHey
EtanHey deleted the feat/p2e-rapid-spawn-kill-fixture branch April 27, 2026 17:23

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 2 potential issues.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 78a38b0. Configure here.

environment: [
"CMUX_RAPID_SPAWN_KILL_APP_PATH": appURL.path,
"CMUX_RAPID_SPAWN_KILL_ITERATIONS": "3",
"CMUX_RAPID_SPAWN_KILL_FORCE_WINDOW": "1",

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unused force-window env var has no effect

Medium Severity

The CMUX_RAPID_SPAWN_KILL_FORCE_WINDOW environment variable is set by the test to force IOSurface allocation, but the rapid_spawn_kill.sh fixture and the cmux app don't consume it. This makes the variable inert, and the test's assertion for IOSurface allocation relies on default cmux behavior, which could be misleading or flaky.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 78a38b0. Configure here.

local executable
executable="$(resolve_executable)"
mkdir -p "$TMP_ROOT"
trap 'rm -rf "$TMP_ROOT"' EXIT

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

User-supplied tmpdir wiped recursively on exit

Low Severity

When CMUX_RAPID_SPAWN_KILL_TMPDIR is provided, the script does mkdir -p on the supplied path and then unconditionally rm -rfs it on EXIT. Without $$ or another uniqueness suffix, any pre-existing contents of that directory (including data from concurrent runs sharing the same path) are destroyed. The default value is safe because it embeds the PID, but the user-overridable form is not, and the README only labels it as a "scratch directory" without warning about destructive cleanup.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 78a38b0. Configure here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant