journeys tui-kit: tk-storm, the L1 CLI storm in compiled Rust (held schedule, rate verdict) by lawrencecchen · Pull Request #31764 · manaflow-ai/cmux

lawrencecchen · 2026-11-02T03:56:40Z

Summary

This is item 2 of the compiled-load plan (STATUS 4801; cmux-ad 03:40Z). The TYPING UNDER LOAD L1 CLI storm is now tk-storm, in Rust. This PR is stacked on #31748 (tk-gradient); its own change is the top 2 commits, 1c3338e and e7cd7c4 (review fixes).

tk-storm: N worker threads share one schedule: call n is due at start + n / rate.
- They run load_profiles.py's L1 mix with the same weights and argv shapes: list, tab create and close (each worker closes only its own tabs), terminal write, screen read, notify --desktop false, workspace status, progress and log, and snapshot.get on the app socket.
- Each call is one calls.jsonl line.
- Each CLI call is killed after 30 s.
- The rate window ends at the stop, and only calls that started inside it count.
- The summary has achieved_per_s, ok_per_s, asked_per_s, verdict and leftover_tabs, which the Python teardown closes. The verdict is NOT RUN when the calls that worked are below 95 % of the asked rate.
load_profiles L1: --engine auto (the default) runs tk-storm when it is built. Python then does only the set-up, the wait and the teardown. The Python storm, and every profile with an asked rate, now also gives a verdict. LOAD-DONE prints the verdict and the engine.
Items from the journeys tui-kit: tk-gradient, the 120 fps gradient in compiled Rust (byte-identical, rate verdict) #31748 review:
- tk-gradient waits on a full non-blocking stdout (poll), so it no longer stops half way.
- Its events carry darwin_bg.
- build-bins passes --target-dir, and fails when a program is missing after the build.
- heavy --build-bins always builds (incrementally).
- The Python bytes field counts UTF-8 bytes.
- --duration and --frames refuse negative values, and --size refuses 0.
- The pace_probe docstring is first in its function again.

Testing

Observed / Not run

Observed (cmux-orchestrator-1):

test_load_profiles.py: 15 tests OK. The new test_rust_storm_holds_the_rate is skipped here (no Rust build on this VM). The new test_python_engine_verdict passes.
test_tuikit.py: 36 tests OK, 2 skipped.
rustfmt --check: clean.
prepush-gate.py: no new violations.

Not run:

The crate is not compiled. The fleet self-test compiles it after this lands, and runs the Rust tests with CMUX_TUIKIT_REQUIRE_RUST=1. One of them checks that tk-storm holds 60 calls/s with no failed call against the stand-in CLI, whose grammar check refuses any wrong argv. The same test checks that the log has every counted call, that no second holds more than 1.3x the rate, and that at least 12 of the 13 kinds ran.
Against a real tagged build: the CLI call latency sets the reachable rate (about 8 workers ÷ the call latency). Above that, the run says NOT RUN.

Changelog

none

…(byte-identical, rate verdict) cmux-ad 03:40Z: every load that must hold a rate becomes compiled code, and a run below 95 % of its target is NOT RUN. Plan in STATUS 4801, item 1. - scripts/journeys/tui-kit/rust/: its own workspace (std only, no dependencies; not part of cmux-tui and not built by CI), so it builds from a copy of the kit: `tuikit.py build-bins` = cargo build --release --offline --manifest-path .../rust/Cargo.toml. tk-gradient draws tuikit.py's gradient frames byte for byte for the same style. tuikit.py passes gradient_style(seed) on the command line, so the seed stays the Python kit's. It paces with spin, hybrid or sleep like Pacer, and logs the same events. Its end event has achieved_fps, target_fps and a verdict ("ok", or "NOT RUN" below 95 %). - tuikit.py gradient --engine auto (default) execs tk-gradient when it is built, else runs Python. The Python engine's end event now has the same achieved_fps, target_fps and verdict. - heavy_profile build records gradient_engine. --build-bins builds the programs first. - pace_probe adds rust-spin, rust-hybrid and rust-sleep. tui-kit-selftest.sh builds the programs, runs `cargo test -- --ignored` (the crate's tests are ignored in a plain cargo test), then test_tuikit.py with CMUX_TUIKIT_REQUIRE_RUST=1, so the parity and rate tests fail and never skip. Its TUI-KIT-PACE line is rust-spin. Tests: test_tuikit.py: 36 OK on cmux-orchestrator-1. The 2 Rust tests skip here, because I build no Rust on this VM. New tests: same bytes as Python for 5 seeds, sizes and cells; holds 120 fps (achieved >= 95 %, p95 interval < 12.5 ms); a Python run below the floor is NOT RUN. test_load_profiles.py: 13 OK. rustfmt --check passes. Not compiled yet: the first compile is the macOS self-test after this lands.

…chedule, rate verdict); #31748 review items This is item 2 of the compiled-load plan (STATUS 4801; cmux-ad 03:40Z). - rust/src/storm.rs and bin/tk-storm. N worker threads share one schedule: call n is due at start + n / rate. They run load_profiles.py's L1 mix with the same weights and argv shapes: list, tab create and close (each worker closes only its own tabs), terminal write, screen read, notify --desktop false, workspace status, progress and log, and snapshot.get on the app socket. Each call is a calls.jsonl line. The summary has achieved_per_s, ok_per_s, asked_per_s, verdict (NOT RUN below 95 %) and leftover_tabs, which the Python teardown closes. - load_profiles L1 --engine auto (the default) runs tk-storm when it is built, and Python only for set-up, the wait and teardown. The Python storm, and every profile with an asked rate, now also gives a verdict. LOAD-DONE prints the verdict and the engine. - Items from the #31748 review: - tk-gradient waits on a full non-blocking stdout (poll) and does not stop half way. - Its events carry darwin_bg. - build-bins passes --target-dir and fails when a program is missing after the build. - heavy --build-bins always builds (incremental). - The Python bytes field counts UTF-8 bytes. - --duration and --frames refuse negative values, and --size refuses 0. - The pace_probe docstring is first in its function again. Stacked on #31748 (tk-gradient). Tests on cmux-orchestrator-1: test_load_profiles.py 15 OK and test_tuikit.py 36 OK. The Rust tests skip here, because I build no Rust on this VM; the fleet self-test runs them with CMUX_TUIKIT_REQUIRE_RUST=1 (tk-storm holds 50 calls/s against the stand-in CLI, whose grammar check refuses any wrong argv). rustfmt --check passes. Not compiled yet: it compiles in the macOS self-test after it lands.

github-actions · 2026-11-02T03:56:52Z

cmux status _{(one comment, edited in place; each line is updated by the job that owns it)}

Review 27dd27a: HQ feed card · gallery before/after runs · checks

App of this head (app.zip, kept 1 day)

gh workflow run reload-build.yml -R manaflow-ai/cmux --ref feat-cmux-next -f tag=pr31764 -f ref=cp/tui-kit-rust-storm -f platform=macos -f backend_mode=local

Shipped in:

cmux NEXT nightly 1.0.0-nightly.4140120384601 (64b2566442)
cmux NEXT nightly 1.0.0-nightly.4140182202501 (5a56d58c64)
cmux NEXT nightly 1.0.0-nightly.4140259429701 (49aee507d9)
cmux NEXT nightly 1.0.0-nightly.4140296966501 (9e1c48fa22)
cmux NEXT nightly 1.0.0-nightly.4140454331101 (6996915557)

The feat-cmux-next push run https://github.com/manaflow-ai/cmux/actions/runs/41399258629 at 00ae302 failed: cmux-next checks (god files, concurrency, crash safety, l10n).
Those jobs last passed at 54efdc7. This pull request is one of 2 merged since: #31764, #31851.
If the failure is in your change, fix forward on feat-cmux-next. Pull requests run only the tiers their paths reach (docs/ci/cmux-next-tiers.md); label a batch PR full-ci to run them all before merging.

blacksmith-sh · 2026-11-02T04:04:15Z

Found 6 test failures on Blacksmith runners:

Failures

Test	View Logs
`main.DeveloperIdAppTests.test_download_prints_the_active_profile_only/` `test_download_prints_the_active_profile_only`	View Logs
`stubs: /System/Library/Frameworks/Security.framework/Versions/A/Security has no stub/` `Security has no stub`	View Logs
`stubs: CoreFoundation.tbd: install-name /System/Library/Frameworks/` `CoreFoundation.framework/Versions/A/CoreFoundation, missing 1: ['_CFRetain']/` `CoreFoundation, missing 1: ['_CFRetain']`	View Logs
`stubs: CoreFoundation.tbd: install-name /System/Library/Frameworks/` `CoreFoundation.framework/Versions/A/CoreFoundation, missing 1: ['_log2']/` `CoreFoundation, missing 1: ['_log2']`	View Logs
`test_every_compile_job_names_its_warm_sources (main.ThisTree.test_every_compile_job` `_names_its_warm_sources)/` `test_every_compile_job_names_its_warm_sources (main.ThisTree.test_every_compile_job` `_names_its_warm_sources)`	View Logs
`test_the_repository_highlights_validate (main.DigestTests.test_the_repository_highl` `ights_validate)/` `test_the_repository_highlights_validate (main.DigestTests.test_the_repository_highl` `ights_validate)`	View Logs

^{Need help on this PR? Tag @codesmith-bot with what you need.}

…ict, call timeout, early exit) Fixes from the review of 1c3338e (SHIP, with medium items): - The rate window ends at the stop (the first quit a worker sees, or the duration), not after the calls in flight drained. Only calls that started in the window count; the summary also gives drained_after_stop. - The verdict counts the calls that worked (ok_per_s >= 95 % of the asked rate), in tk-storm and in the Python Calls summary: calls that fail fast are no load. - Each CLI call is killed after 30 s (load_profiles.py's timeout). A helper thread reads the output, and the wait is a channel receive, so the wait adds no time to a call. - L1 sees when tk-storm ends by itself (it records a refusal and does not wait out the duration), checks the storm workspace's ids before the start, and shows tk-storm's stderr when there is no summary. - A close that fails puts the tab in leftover_tabs for the teardown; the worker makes new tabs. - The rust storm test now checks that the log has every counted call (at most one in flight per worker after the stop), that no second holds more than 1.3x the rate, and that at least 12 of the 13 kinds ran, at 60 calls/s. Tests: test_load_profiles.py 15 OK and test_tuikit.py 36 OK on cmux-orchestrator-1. The Rust tests skip here; I build no Rust on this VM.

cursor · 2026-11-02T04:06:51Z

Bugbot needs on-demand usage enabled

Bugbot uses usage-based billing for this team and requires on-demand usage to be enabled.

A team admin can enable on-demand usage in the Cursor dashboard.

lawrencecchen · 2026-11-02T04:09:28Z

Review (subagent, correctness first): SHIP at e7cd7c4

I checked compiling by reading only: no Rust is built here. The first review at 1c3338e gave SHIP with 4 medium items, and all are fixed:

the rate window ends at the stop;
the verdict uses ok_per_s;
each CLI call has a 30 s kill;
Python sees an early exit of tk-storm.
The low items are fixed too. rustfmt is clean. test_load_profiles.py ran 15 OK and test_tuikit.py 36 OK in a clean copy.
Not blocking: quit_at is the time a worker sees the flag, not the signal time; stdout is read before stderr, so more than 64 KB on stderr waits for the 30 s kill; watch the 60 calls/s test on the fleet; the test does not check that every created tab was closed; by_kind also counts the drained calls.

…rust-storm # Conflicts: # scripts/journeys/tui-kit/heavy_profile.py # scripts/journeys/tui-kit/rust/Cargo.toml # scripts/journeys/tui-kit/rust/src/bin/tk-gradient.rs # scripts/journeys/tui-kit/rust/src/lib.rs # scripts/journeys/tui-kit/rust/src/sys.rs # scripts/journeys/tui-kit/tuikit.py

… above the run_variant docstring

lawrencecchen · 2026-11-02T06:01:37Z

Review (subagent, correctness first): SHIP at 27dd27a

I reviewed the PR again after the merge of origin/feat-cmux-next. The net diff against the merged feat parent 4e9b96d is the reviewed change 8bda9e9..e7cd7c4, line for line (11 files, +850 -23). In heavy_profile.py, the 3-way merge kept #31706's STOP/check_stop. The stale engine line in pace_probe.py is gone (27dd27a). There are no conflict markers. test_tuikit.py ran 36 OK and test_load_profiles.py 16 OK in a clean copy. rustfmt is clean.

…t (uptime clock, rt pacing) (#31784) * journeys tui-kit: tk-gradient, the 120 fps gradient in compiled Rust (byte-identical, rate verdict) cmux-ad 03:40Z: every load that must hold a rate becomes compiled code, and a run below 95 % of its target is NOT RUN. Plan in STATUS 4801, item 1. - scripts/journeys/tui-kit/rust/: its own workspace (std only, no dependencies; not part of cmux-tui and not built by CI), so it builds from a copy of the kit: `tuikit.py build-bins` = cargo build --release --offline --manifest-path .../rust/Cargo.toml. tk-gradient draws tuikit.py's gradient frames byte for byte for the same style. tuikit.py passes gradient_style(seed) on the command line, so the seed stays the Python kit's. It paces with spin, hybrid or sleep like Pacer, and logs the same events. Its end event has achieved_fps, target_fps and a verdict ("ok", or "NOT RUN" below 95 %). - tuikit.py gradient --engine auto (default) execs tk-gradient when it is built, else runs Python. The Python engine's end event now has the same achieved_fps, target_fps and verdict. - heavy_profile build records gradient_engine. --build-bins builds the programs first. - pace_probe adds rust-spin, rust-hybrid and rust-sleep. tui-kit-selftest.sh builds the programs, runs `cargo test -- --ignored` (the crate's tests are ignored in a plain cargo test), then test_tuikit.py with CMUX_TUIKIT_REQUIRE_RUST=1, so the parity and rate tests fail and never skip. Its TUI-KIT-PACE line is rust-spin. Tests: test_tuikit.py: 36 OK on cmux-orchestrator-1. The 2 Rust tests skip here, because I build no Rust on this VM. New tests: same bytes as Python for 5 seeds, sizes and cells; holds 120 fps (achieved >= 95 %, p95 interval < 12.5 ms); a Python run below the floor is NOT RUN. test_load_profiles.py: 13 OK. rustfmt --check passes. Not compiled yet: the first compile is the macOS self-test after this lands. * journeys tui-kit: tk-storm, the L1 CLI storm in compiled Rust (held schedule, rate verdict); #31748 review items This is item 2 of the compiled-load plan (STATUS 4801; cmux-ad 03:40Z). - rust/src/storm.rs and bin/tk-storm. N worker threads share one schedule: call n is due at start + n / rate. They run load_profiles.py's L1 mix with the same weights and argv shapes: list, tab create and close (each worker closes only its own tabs), terminal write, screen read, notify --desktop false, workspace status, progress and log, and snapshot.get on the app socket. Each call is a calls.jsonl line. The summary has achieved_per_s, ok_per_s, asked_per_s, verdict (NOT RUN below 95 %) and leftover_tabs, which the Python teardown closes. - load_profiles L1 --engine auto (the default) runs tk-storm when it is built, and Python only for set-up, the wait and teardown. The Python storm, and every profile with an asked rate, now also gives a verdict. LOAD-DONE prints the verdict and the engine. - Items from the #31748 review: - tk-gradient waits on a full non-blocking stdout (poll) and does not stop half way. - Its events carry darwin_bg. - build-bins passes --target-dir and fails when a program is missing after the build. - heavy --build-bins always builds (incremental). - The Python bytes field counts UTF-8 bytes. - --duration and --frames refuse negative values, and --size refuses 0. - The pace_probe docstring is first in its function again. Stacked on #31748 (tk-gradient). Tests on cmux-orchestrator-1: test_load_profiles.py 15 OK and test_tuikit.py 36 OK. The Rust tests skip here, because I build no Rust on this VM; the fleet self-test runs them with CMUX_TUIKIT_REQUIRE_RUST=1 (tk-storm holds 50 calls/s against the stand-in CLI, whose grammar check refuses any wrong argv). rustfmt --check passes. Not compiled yet: it compiles in the macOS self-test after it lands. * journeys tui-kit storm: review fixes (load-window rate, ok-based verdict, call timeout, early exit) Fixes from the review of 1c3338e (SHIP, with medium items): - The rate window ends at the stop (the first quit a worker sees, or the duration), not after the calls in flight drained. Only calls that started in the window count; the summary also gives drained_after_stop. - The verdict counts the calls that worked (ok_per_s >= 95 % of the asked rate), in tk-storm and in the Python Calls summary: calls that fail fast are no load. - Each CLI call is killed after 30 s (load_profiles.py's timeout). A helper thread reads the output, and the wait is a channel receive, so the wait adds no time to a call. - L1 sees when tk-storm ends by itself (it records a refusal and does not wait out the duration), checks the storm workspace's ids before the start, and shows tk-storm's stderr when there is no summary. - A close that fails puts the tab in leftover_tabs for the teardown; the worker makes new tabs. - The rust storm test now checks that the log has every counted call (at most one in flight per worker after the stop), that no second holds more than 1.3x the rate, and that at least 12 of the 13 kinds ran, at 60 calls/s. Tests: test_load_profiles.py 15 OK and test_tuikit.py 36 OK on cmux-orchestrator-1. The Rust tests skip here; I build no Rust on this VM. * journeys tui-kit: tk-beacon, the SWITCH FRAMES beacon in compiled Rust (uptime clock, rate verdict); rt pacing This is item 4 of the compiled-load plan (STATUS 4801). It answers the SWITCH owner's request in the item after 4794 for the beacon spec in cp/switch-frames-kit scripts/journeys/ext_switch.py (ansi, rows). - rust/src/beacon.rs and bin/tk-beacon. The three rows match ext_switch.py: Y, ws[6], pane[4], parity, R; then Y, gray(clock) bits 17..9, parity, R; then Y, bits 8..0, parity, R. Magenta is 1 and cyan is 0, and each block is --cell-cols columns. The clock is CLOCK_UPTIME_RAW (the clock of the recorder's frames.json) in 2 ms units, 18 bits, as the SWITCH owner asked: ext_switch.py reads time.time_ns. Each frame is one DEC 2026 update. The label row is "WS n PANE m frame k". --fill gives the moving shade. The end event has achieved_hz, target_hz and verdict (NOT RUN below 95 %). `tuikit.py beacon` execs it; there is no Python engine. - New pace mode rt (macOS): a time-constraint thread (thread_policy_set: period 1/hz, computation 0.75 ms, constraint 1.5 ms) that waits with mach_wait_until, so there is no timer coalescing and no spinning. The beacon defaults to rt, because 18 beacons at 240 Hz that spin would be the load under test. The gradient keeps spin, which is proven on a mini; pace_probe adds rust-rt to compare the two. The start event says whether the kernel accepted the real-time policy. Stacked on #31764. Tests: test_tuikit.py 37 OK on cmux-orchestrator-1. 3 Rust tests skip here, because I build no Rust on this VM. The new BeaconRust test decodes the rows from the pty: ws and pane ids, parity, and a clock that advances, at 240 Hz or more than 95 %. test_load_profiles.py 15 OK. rustfmt --check passes. The first compile is the macOS self-test after this lands. * journeys tui-kit: drop the stale engine line the merge with feat kept above the run_variant docstring --------- Co-authored-by: cmux-bot[bot] <348531757+cmux-bot[bot]@users.noreply.github.com>

lawrencecchen added 2 commits November 2, 2026 03:43

lawrencecchen added the risk:low Risk tier (docs/ci/risk-tiers.md) label Nov 2, 2026

github-actions Bot added risk:normal Risk tier (docs/ci/risk-tiers.md) and removed risk:low Risk tier (docs/ci/risk-tiers.md) labels Nov 2, 2026

lawrencecchen temporarily deployed to ci-routing-agent November 2, 2026 04:01 — with GitHub Actions Inactive

lawrencecchen temporarily deployed to ci-routing-agent November 2, 2026 04:08 — with GitHub Actions Inactive

lawrencecchen added the ready-to-land Reviewed at this head; next-autoland merges it into feat-cmux-next when its only reds are base reds label Nov 2, 2026

lawrencecchen mentioned this pull request Nov 2, 2026

journeys tui-kit: tk-beacon, the SWITCH FRAMES beacon in compiled Rust (uptime clock, rt pacing) #31784

Merged

lawrencecchen removed the ready-to-land Reviewed at this head; next-autoland merges it into feat-cmux-next when its only reds are base reds label Nov 2, 2026

lawrencecchen temporarily deployed to ci-routing-agent November 2, 2026 05:57 — with GitHub Actions Inactive

journeys tui-kit: drop the stale engine line the merge with feat kept…

27dd27a

… above the run_variant docstring

lawrencecchen temporarily deployed to ci-routing-agent November 2, 2026 05:58 — with GitHub Actions Inactive

lawrencecchen temporarily deployed to ci-routing-agent November 2, 2026 06:01 — with GitHub Actions Inactive

lawrencecchen added the ready-to-land Reviewed at this head; next-autoland merges it into feat-cmux-next when its only reds are base reds label Nov 2, 2026

lawrencecchen merged commit f1eb8e8 into feat-cmux-next Nov 2, 2026
130 of 134 checks passed

lawrencecchen deleted the cp/tui-kit-rust-storm branch November 2, 2026 06:18

github-actions Bot mentioned this pull request Nov 2, 2026

fix(restore): a terminal's last frame stays through a slow attach, the quit and a daemon restart (dogfood port) #31851

Merged

Navigation Menu

Repository navigation

journeys tui-kit: tk-storm, the L1 CLI storm in compiled Rust (held schedule, rate verdict) - #31764

journeys tui-kit: tk-storm, the L1 CLI storm in compiled Rust (held schedule, rate verdict)#31764
lawrencecchen merged 5 commits into
feat-cmux-nextfrom
cp/tui-kit-rust-storm

lawrencecchen commented Nov 2, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Nov 2, 2026 •

edited

Loading

Uh oh!

blacksmith-sh Bot commented Nov 2, 2026 •

edited

Loading

Uh oh!

cursor Bot commented Nov 2, 2026

Uh oh!

lawrencecchen commented Nov 2, 2026

Uh oh!

lawrencecchen commented Nov 2, 2026

Uh oh!

Uh oh!

This branch was previously deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

lawrencecchen commented Nov 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Testing

Observed / Not run

Changelog

Uh oh!

github-actions Bot commented Nov 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

blacksmith-sh Bot commented Nov 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Failures

Uh oh!

cursor Bot commented Nov 2, 2026

Bugbot needs on-demand usage enabled

Uh oh!

lawrencecchen commented Nov 2, 2026

Uh oh!

lawrencecchen commented Nov 2, 2026

Uh oh!

Uh oh!

This branch was previously deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

lawrencecchen commented Nov 2, 2026 •

edited

Loading

github-actions Bot commented Nov 2, 2026 •

edited

Loading

blacksmith-sh Bot commented Nov 2, 2026 •

edited

Loading