Skip to content

journeys tui-kit: tk-beacon, the SWITCH FRAMES beacon in compiled Rust (uptime clock, rt pacing) - #31784

Merged
lawrencecchen merged 9 commits into
feat-cmux-nextfrom
cp/tui-kit-rust-beacon
Nov 2, 2026
Merged

lawrencecchen merged 9 commits into
feat-cmux-nextfrom
cp/tui-kit-rust-beacon

Conversation

@lawrencecchen

@lawrencecchen lawrencecchen commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

Summary

This is item 4 of the compiled-load plan (STATUS 4801). It answers the SWITCH FRAMES owner's request for a compiled beacon. The spec and decoder are in cp/switch-frames-kit scripts/journeys/ext_switch.py (ansi, rows). This PR is stacked on #31764; its own change is the top commit, 43989b9.

  • tk-beacon: it draws the same three rows as ext_switch.py:
    • row 0: Y, ws[6], pane[4], parity, R;
    • row 1: Y, gray(clock) bits 17..9, parity, R;
    • row 2: Y, gray(clock) bits 8..0, parity, R.
    • Magenta is 1 and cyan is 0. Each block is --cell-cols columns wide.
    • The clock is CLOCK_UPTIME_RAW, in 2 ms units, 18 bits. The SWITCH owner asked for this clock because the recorder's frames.json uses it. The Python beacon reads time.time_ns.
    • Each frame is one DEC 2026 update. The label row is WS n PANE m frame k, and --fill adds the moving shade.
    • The end event has achieved_hz, target_hz and verdict (NOT RUN below 95 %).
    • tuikit.py beacon --ws N --pane M execs it. There is no Python engine.
  • New pace mode rt (macOS): the thread becomes a time-constraint thread (thread_policy_set: period 1/hz, computation 0.75 ms, constraint 1.5 ms) and waits with mach_wait_until. This avoids timer coalescing, and it does not spin.
    • The beacon defaults to rt. A SWITCH profile runs 18 beacons at 240 Hz, and 18 spinning cores would become the load under test.
    • The gradient keeps spin, which is proven on a mini.
    • pace_probe adds a rust-rt variant to compare the two.
    • The start event records whether the kernel accepted the real-time policy (realtime).

Testing

Observed / Not run

Observed (cmux-orchestrator-1):

  • test_tuikit.py: 37 tests OK, 3 skipped. The skipped tests are the Rust ones, because no Rust is built on this VM. The new BeaconRust test decodes the rows from a real pty: the ws and pane ids, the parity, and a clock that advances, at no less than 95 % of 240 Hz.
  • test_load_profiles.py: 15 tests OK.
  • rustfmt --check: clean.
  • prepush-gate.py: no new violations.

Not run:

  • Nothing here is compiled yet. The macOS self-test after the merge compiles it and runs the parity, decode and rate tests with CMUX_TUIKIT_REQUIRE_RUST=1.
  • Whether rt holds 240 Hz without spinning on a mini is unknown. The rust-rt pace_probe row and each run's verdict will say.

Changelog

none


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.


Summary by cubic

Adds tk-beacon, a compiled Rust binary that draws the SWITCH FRAMES beacon spec (ext_switch.py ansi/rows) on CLOCK_UPTIME_RAW in 2 ms units, so the beacon matches the clock the recorder's frames.json uses. tuikit.py beacon execs it; there is no Python engine.

Adds a new rt pace mode on macOS: a time-constraint thread (thread_policy_set with period 1/hz, computation 0.75 ms, constraint 1.5 ms) that waits via mach_wait_until, avoiding timer coalescing without spinning. The beacon defaults to rt because 18 spinning beacons at 240 Hz would be the load under test; both the beacon and tk-gradient accept it, while the gradient keeps spin as its default. pace_probe adds a rust-rt variant to compare against spin. Runs below 95 % of the asked rate now record verdict NOT RUN across the gradients, storms, and beacons.

Migration

  • Run tuikit.py build-bins once per Mac to compile tk-gradient, tk-storm, and tk-beacon (std only, no dependencies, not built by CI).

Written for commit 7958050. Summary will update on new commits.

View guided diff Turn on auto-fix

…(byte-identical, rate verdict)

cmux-ad 03:40Z: every load that must hold a rate becomes compiled code, and a run below 95 % of its
target is NOT RUN. Plan in STATUS 4801, item 1.

- scripts/journeys/tui-kit/rust/: its own workspace (std only, no dependencies; not part of cmux-tui
  and not built by CI), so it builds from a copy of the kit:
  `tuikit.py build-bins` = cargo build --release --offline --manifest-path .../rust/Cargo.toml.
  tk-gradient draws tuikit.py's gradient frames byte for byte for the same style. tuikit.py passes
  gradient_style(seed) on the command line, so the seed stays the Python kit's. It paces with spin,
  hybrid or sleep like Pacer, and logs the same events. Its end event has achieved_fps, target_fps
  and a verdict ("ok", or "NOT RUN" below 95 %).
- tuikit.py gradient --engine auto (default) execs tk-gradient when it is built, else runs Python.
  The Python engine's end event now has the same achieved_fps, target_fps and verdict.
- heavy_profile build records gradient_engine. --build-bins builds the programs first.
- pace_probe adds rust-spin, rust-hybrid and rust-sleep. tui-kit-selftest.sh builds the programs,
  runs `cargo test -- --ignored` (the crate's tests are ignored in a plain cargo test), then
  test_tuikit.py with CMUX_TUIKIT_REQUIRE_RUST=1, so the parity and rate tests fail and never skip.
  Its TUI-KIT-PACE line is rust-spin.

Tests: test_tuikit.py: 36 OK on cmux-orchestrator-1. The 2 Rust tests skip here, because I build
no Rust on this VM. New tests: same bytes as Python for 5 seeds, sizes and cells; holds 120 fps
(achieved >= 95 %, p95 interval < 12.5 ms); a Python run below the floor is NOT RUN.
test_load_profiles.py: 13 OK. rustfmt --check passes. Not compiled yet: the first compile is the
macOS self-test after this lands.
…chedule, rate verdict); #31748 review items

This is item 2 of the compiled-load plan (STATUS 4801; cmux-ad 03:40Z).

- rust/src/storm.rs and bin/tk-storm. N worker threads share one schedule: call n is due at
  start + n / rate. They run load_profiles.py's L1 mix with the same weights and argv shapes:
  list, tab create and close (each worker closes only its own tabs), terminal write, screen read,
  notify --desktop false, workspace status, progress and log, and snapshot.get on the app socket.
  Each call is a calls.jsonl line. The summary has achieved_per_s, ok_per_s, asked_per_s, verdict
  (NOT RUN below 95 %) and leftover_tabs, which the Python teardown closes.
- load_profiles L1 --engine auto (the default) runs tk-storm when it is built, and Python only for
  set-up, the wait and teardown. The Python storm, and every profile with an asked rate, now also
  gives a verdict. LOAD-DONE prints the verdict and the engine.
- Items from the #31748 review:
  - tk-gradient waits on a full non-blocking stdout (poll) and does not stop half way.
  - Its events carry darwin_bg.
  - build-bins passes --target-dir and fails when a program is missing after the build.
  - heavy --build-bins always builds (incremental).
  - The Python bytes field counts UTF-8 bytes.
  - --duration and --frames refuse negative values, and --size refuses 0.
  - The pace_probe docstring is first in its function again.

Stacked on #31748 (tk-gradient). Tests on cmux-orchestrator-1: test_load_profiles.py 15 OK and
test_tuikit.py 36 OK. The Rust tests skip here, because I build no Rust on this VM; the fleet self-test
runs them with CMUX_TUIKIT_REQUIRE_RUST=1 (tk-storm holds 50 calls/s against the stand-in CLI,
whose grammar check refuses any wrong argv). rustfmt --check passes. Not compiled yet: it compiles in
the macOS self-test after it lands.
…ict, call timeout, early exit)

Fixes from the review of 1c3338e (SHIP, with medium items):
- The rate window ends at the stop (the first quit a worker sees, or the duration), not after the
  calls in flight drained. Only calls that started in the window count; the summary also gives
  drained_after_stop.
- The verdict counts the calls that worked (ok_per_s >= 95 % of the asked rate), in tk-storm and in
  the Python Calls summary: calls that fail fast are no load.
- Each CLI call is killed after 30 s (load_profiles.py's timeout). A helper thread reads the output,
  and the wait is a channel receive, so the wait adds no time to a call.
- L1 sees when tk-storm ends by itself (it records a refusal and does not wait out the duration),
  checks the storm workspace's ids before the start, and shows tk-storm's stderr when there is no
  summary.
- A close that fails puts the tab in leftover_tabs for the teardown; the worker makes new tabs.
- The rust storm test now checks that the log has every counted call (at most one in flight per
  worker after the stop), that no second holds more than 1.3x the rate, and that at least 12 of the
  13 kinds ran, at 60 calls/s.

Tests: test_load_profiles.py 15 OK and test_tuikit.py 36 OK on cmux-orchestrator-1. The Rust tests
skip here; I build no Rust on this VM.
…t (uptime clock, rate verdict); rt pacing

This is item 4 of the compiled-load plan (STATUS 4801). It answers the SWITCH owner's request in the item
after 4794 for the beacon spec in cp/switch-frames-kit scripts/journeys/ext_switch.py (ansi, rows).

- rust/src/beacon.rs and bin/tk-beacon. The three rows match ext_switch.py: Y, ws[6], pane[4], parity,
  R; then Y, gray(clock) bits 17..9, parity, R; then Y, bits 8..0, parity, R. Magenta is 1 and cyan
  is 0, and each block is --cell-cols columns. The clock is CLOCK_UPTIME_RAW (the clock of the
  recorder's frames.json) in 2 ms units, 18 bits, as the SWITCH owner asked: ext_switch.py reads
  time.time_ns. Each frame is one DEC 2026 update. The label row is "WS n  PANE m  frame k".
  --fill gives the moving shade. The end event has achieved_hz, target_hz and verdict (NOT RUN
  below 95 %). `tuikit.py beacon` execs it; there is no Python engine.
- New pace mode rt (macOS): a time-constraint thread (thread_policy_set: period 1/hz, computation
  0.75 ms, constraint 1.5 ms) that waits with mach_wait_until, so there is no timer coalescing and
  no spinning. The beacon defaults to rt, because 18 beacons at 240 Hz that spin would be the load
  under test. The gradient keeps spin, which is proven on a mini; pace_probe adds rust-rt to compare
  the two. The start event says whether the kernel accepted the real-time policy.

Stacked on #31764. Tests: test_tuikit.py 37 OK on cmux-orchestrator-1. 3 Rust tests skip here,
because I build no Rust on this VM. The new BeaconRust test decodes the rows from the pty: ws and
pane ids, parity, and a clock that advances, at 240 Hz or more than 95 %. test_load_profiles.py
15 OK. rustfmt --check passes. The first compile is the macOS self-test after this lands.
@lawrencecchen lawrencecchen added the risk:low Risk tier (docs/ci/risk-tiers.md) label Nov 2, 2026
@github-actions github-actions Bot added risk:normal Risk tier (docs/ci/risk-tiers.md) and removed risk:low Risk tier (docs/ci/risk-tiers.md) labels Nov 2, 2026
@lawrencecchen

Copy link
Copy Markdown
Contributor Author

Review (subagent, correctness first): SHIP at 43989b9

No blocker. I checked compiling by reading only. The mach FFI types and the time-constraint struct layout are right. Parity with ext_switch.py is exact, except for the clock change on purpose. judge_switch.py fits a clock offset for each pane, so the clock domain is safe. The BeaconRust decoder is correct.
Not blocking, for a follow-up:

  • The shared Pacer catches up on late frames in a burst until it is 0.25 s behind, while ext_switch.py resets its schedule. A beacon option to reset, or a count of late frames, would make short stalls visible.
  • 18 real-time beacons outrank cmux's threads. The rate data on the mini decides whether rt or spin is the default.
  • The Python engines take --pace rt but sleep.
  • The rt help text is missing.
    Note: the fleet cannot run the self-test at a branch head. The worker refuses a measure script that is not on feat-cmux-next, so the first compile runs after the merge.

@lawrencecchen lawrencecchen added the ready-to-land Reviewed at this head; next-autoland merges it into feat-cmux-next when its only reds are base reds label Nov 2, 2026
…rust-storm

# Conflicts:
#	scripts/journeys/tui-kit/heavy_profile.py
#	scripts/journeys/tui-kit/rust/Cargo.toml
#	scripts/journeys/tui-kit/rust/src/bin/tk-gradient.rs
#	scripts/journeys/tui-kit/rust/src/lib.rs
#	scripts/journeys/tui-kit/rust/src/sys.rs
#	scripts/journeys/tui-kit/tuikit.py
@lawrencecchen lawrencecchen removed the ready-to-land Reviewed at this head; next-autoland merges it into feat-cmux-next when its only reds are base reds label Nov 2, 2026
@github-actions

github-actions Bot commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

cmux status (one comment, edited in place; each line is updated by the job that owns it)

Review 26e52fa: HQ feed card · gallery before/after runs · checks

App of this head (app.zip, kept 1 day)

gh workflow run reload-build.yml -R manaflow-ai/cmux --ref feat-cmux-next -f tag=pr31784 -f ref=cp/tui-kit-rust-beacon -f platform=macos -f backend_mode=local

Shipped in:

@lawrencecchen

Copy link
Copy Markdown
Contributor Author

Review (subagent, correctness first): SHIP at cccc81a

This is the re-review after the merge of the updated #31764 branch (which has origin/feat-cmux-next in it). The net diff against the merged feat parent 4e9b96d matches the reviewed 8bda9e9..43989b9 line for line: 16 files, +1402 -28. There are no conflict markers and no duplicate lines from the merge. In a clean copy, test_tuikit.py ran 37 tests OK and test_load_profiles.py 16 OK. The Rust paths skip until the fleet build runs.

@lawrencecchen lawrencecchen added the ready-to-land Reviewed at this head; next-autoland merges it into feat-cmux-next when its only reds are base reds label Nov 2, 2026
@blacksmith-sh

blacksmith-sh Bot commented Nov 2, 2026 •

Copy link
Copy Markdown

Found 2 test failures on Blacksmith runners:

Failures

Test View Logs
test_every_compile_job_names_its_warm_sources (main.ThisTree.test_every_compile_job
_names_its_warm_sources)/
test_every_compile_job_names_its_warm_sources (main.ThisTree.test_every_compile_job
_names_its_warm_sources)
View Logs
test_the_repository_highlights_validate (main.DigestTests.test_the_repository_highl
ights_validate)/
test_the_repository_highlights_validate (main.DigestTests.test_the_repository_highl
ights_validate)
View Logs

Fix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.

# Conflicts:
#	scripts/journeys/tui-kit/rust/Cargo.toml
#	scripts/journeys/tui-kit/rust/src/bin/tk-gradient.rs
#	scripts/journeys/tui-kit/rust/src/sys.rs
#	scripts/journeys/tui-kit/tuikit.py
@lawrencecchen lawrencecchen removed the ready-to-land Reviewed at this head; next-autoland merges it into feat-cmux-next when its only reds are base reds label Nov 2, 2026
@lawrencecchen

Copy link
Copy Markdown
Contributor Author

Review (subagent, correctness first): SHIP at 26e52fa

This review is after the merge of origin/feat-cmux-next 05415d3, which contains the #31764 squash. The net diff against feat equals the tk-beacon commit 43989b9 line for line: 11 files, +553 -6. The merge left no conflict markers and no duplicate bins or functions, and it undoes nothing from the storm squash. In a clean copy, test_tuikit.py ran 37 tests OK and test_load_profiles.py ran 16 tests OK.

@lawrencecchen lawrencecchen added the ready-to-land Reviewed at this head; next-autoland merges it into feat-cmux-next when its only reds are base reds label Nov 2, 2026
@cursor

cursor Bot commented Nov 2, 2026

Copy link
Copy Markdown

Bugbot needs on-demand usage enabled

Bugbot uses usage-based billing for this team and requires on-demand usage to be enabled.

A team admin can enable on-demand usage in the Cursor dashboard.

@cmux-bot
cmux-bot Bot temporarily deployed to ci-routing-agent November 2, 2026 06:38 Inactive
@lawrencecchen
lawrencecchen merged commit b93ee63 into feat-cmux-next Nov 2, 2026
115 of 118 checks passed
@lawrencecchen
lawrencecchen deleted the cp/tui-kit-rust-beacon branch November 2, 2026 06:46
lawrencecchen added a commit that referenced this pull request Nov 5, 2026
…ild; tui-kit beacon clippy) (#36608)

* test(optchat-chief): the undo after a reconnect's delete waits for that delete's summaries to rebuild

Red on every full rust-workspaces run since #36057 (3deee62): the undo
raced the editor's rebuild worker and got Editor::undo's documented 409 'its
summaries are being rebuilt; try again in a moment'. The Memory Inspector and the
memory_edit tests wait (wait_rebuilt) before an undo; this test now does too.

* tui-kit: beacon::frame takes the run's Beacon (ws, pane, cell columns), so clippy's too_many_arguments passes

cargo clippy -D warnings in scripts/journeys/tui-kit/rust failed (frame took 8
arguments, from #31784); it shows on every full rust-workspaces run (#36427's
run changed .github/rust-workspaces.toml). The three values that name a beacon
for its whole run are one struct; tk-beacon builds it at the call.

* tui-kit: tk-beacon parses its command line into a named Options, so clippy's type_complexity passes

This branch was previously deployed

1 inactive deployment
ci-routing-agent — 79580505 Deployed Nov 2, 2026 by cmux-bot[bot] via cmux-tui tree preflight #99804
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-to-land Reviewed at this head; next-autoland merges it into feat-cmux-next when its only reds are base reds risk:normal Risk tier (docs/ci/risk-tiers.md)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant