[teamleaderleo]

Leo asked:

"how are we doing the file/folder syncage? ... that makes it take a super long time... what if it's not on GitHub?"

and approved: "2. Compress what's sent and stop encoding it as text."

Leo-ask: ask-6e29ca080a

What changes

Daemon (cmux-tui-core). A new capability, fs-write-deflate-v1, is advertised only where fs-v1 is (fs_ops::advertised). It is host-dependent, so it is kept out of build_capabilities like fs-v1. Under it, fs.write takes "encoding":"deflate" with decoded_size on both the line op and the write stream. The wire bytes (the line's bytes_base64, or the stream's size raw bytes) are one raw deflate stream (RFC 1951):

• The free-space check and the 12 MiB single-write limit apply to decoded_size.
• Decoding is streamed into the same PendingWrite and stops at decoded_size. Decoding to more or fewer bytes, or bytes that are not deflate, is params.invalid and leaves nothing at the path, not even the temporary file. A small payload can't fill the disk.
• The write stream also takes permissions, as the line op already did.
• Atomicity is unchanged (temp file, fsync, rename). spec/commands.md (fs-v1) documents it. The contract fixture is unchanged: it records no stream and no new field.

Client (link sync push, single-file copy, seed bundle upload). The client asks identify once per connection. When the machine lists fs-write-deflate-v1, it deflates each part at level 3, but only if the file's first 64 KiB shrink by more than a tenth. Packs, images and archives go as they are. It also checks that each compressed part is actually smaller. Without the capability, every request is byte-for-byte the old base64 request, so old machine images keep working. Unchanged:

• progress callbacks
• IO_DEADLINE and the answer budget
• 8 MiB parts with append and rename
• the resume manifest (SHA-256, permissions, revision)
• the Mac-side skip rules

Round trips. The benchmark showed the push is bound by latency, not bandwidth: 490 MB at 8 MB/s needs about 61 s, but the push took 495 s. Answering each file's "what is in the way" check from the cached folder listing cut that to 308 s. Review found that this lets a symlink the machine swaps in mid-push be written through (#34362's protection), so that change was taken back out of this PR. It returns with a no-follow write in #34999.

Also fixes the stale seed.rs header: "Not wired into start-chat yet." has been false since start-chat's copy started calling seed_cli::push.

What this does not do yet

"Stop encoding it as text" over the raw write stream is not reachable on a Cloud copy today. The stream is routed only on the link's remote entry (route_first_line). The remote connect --local-socket bridge carries bounded JSON lines to the daemon's local socket, whose line path refuses "stream":true. Carrying the stream needs changes in cmux-remote: #34482. The bigger remaining win is sending files over several connections: #34483. Both are measured with the harness below.

Backward compatibility

• New client to an old machine: no capability, so plain base64 exactly as before (a_machine_without_the_capability_gets_plain_base64).
• Old client to a new machine: no encoding field, so the daemon writes the bytes as before.
• The new fields are accepted only together. decoded_size without encoding, or any encoding other than deflate, is refused.

Tests

• fs_ops::encoding unit tests: round trip, a decode past decoded_size refused (64 MiB of zeros declared as 1 MiB), short or foreign bytes refused, the line limit applied to decoded_size.
• fs_wire_tests, end to end through route_first_line: a 40 MiB file over one deflate stream with 0755 kept; a decompressed-size overrun leaves no file or temporary; a line write decodes, keeps 0600, and refuses an overrun.
• remote.rs: the old-machine fallback sends plain base64; with the capability, a file larger than one part decodes from its parts and random bytes go uncompressed.
• tests/link_push_bench.rs: a_push_to_a_cloud_daemon_crosses_compressed_and_lands_exact, against a real Cloud-shaped daemon (exact bytes, 0755, wire bytes below the tree size).
• Unchanged suites pass: link_sync (18), link_access (5), cloud_move_journey (3), sync_cli lib (21), fs_ lib (32).
• Clippy -D warnings on cmux-tui-core and cmux-tui is clean in the changed code. Two base reds in cli/automation* (manual_is_multiple_of, type_complexity) are untouched by this PR.

Performance

The same harness on both sides: link_push_bench, a real headless daemon with the Cloud identity, pushed through a unix-socket proxy at 8 MB/s each way with 30 ms one-way latency. The fixture is 3,190 files (365 MB, a 172 MB .git: 3,000 source-like text files packed plus 160 MiB of random blobs left loose). "Before" ran on the commit with only the harness, 3294d76.

metric	before	after	delta	harness or run link
push wall time	494.8 s	489.4 s	-5.4 s (-1.1%)	link_push_bench, fleet Linux job b081eb17 (before) / run 41786488153 (after, at e9478e9)
bytes up on the wire	490.7 MB	461.0 MB	-29.7 MB (-6.1%)	same
bytes down on the wire	0.81 MB	0.82 MB	+0.01 MB	same

Measured but not shipped here: with the per-file fs.stat answered from the cached listing, the same harness gave 308.1 s / 460.6 MB (-37.7% wall). That change is #34999. The fixture is mostly incompressible random blobs, so compression's byte gain is small here; source-heavy trees gain more (the 200-file text test crosses in well under the tree size). The packs change (#34537, merged into this branch by update-branch) is not part of either number.

Reuse:

cargo test -p cmux-tui --test link_push_bench -- --ignored --nocapture link_push_bench
# env: CMUX_BENCH_FILES=3000 CMUX_BENCH_BLOB_MB=160 CMUX_BENCH_RATE_MBPS=8 CMUX_BENCH_LATENCY_MS=30

The diff is about 950 lines with tests and the benchmark harness, which another PR shares (the shipped code is about 330 lines). It is kept as one PR because the capability, its decoder and its client are one protocol change, and the benchmark's before number needed the harness on its own commit.

Agent session: agent session

🤖 Generated with Claude Code

---

^{Open in [code]smith · Need help on this PR? Tag @codesmith-bot with what you need.}

---

Summary by cubic

Speeds up Cloud copies by compressing fs.write bytes through a new fs-write-deflate-v1 capability, while machines or clients without it get plain base64 exactly as before.

• The fs.write line op and write streams accept "encoding":"deflate" with decoded_size; wire bytes are one raw deflate stream, decoded as they arrive. Decoding to more or fewer bytes, or to non-deflate bytes, is params.invalid and leaves nothing at the path; the free-space check and write limit apply to decoded_size. Write streams also take permissions.
• The client (link sync push, single-file copy, seed bundle upload) deflates parts at level 3 only when the machine advertises the capability and the file's first 64 KiB shrinks by more than a tenth. Without the capability or the fields, every request is byte-for-byte the old base64 one.
• Reverted the cached-listing change: the "is a folder in the way?" check is again a fresh fs.stat right before each write, so base's symlink protection stays current; reusing the listing needs a no-follow write first.
• Raw write streams still travel as base64 lines on Cloud copies; stream routing and multi-connection copies are follow-ups.
• Fixes the stale seed.rs header, which claimed the copy path was not wired into start-chat.

^{Written for commit c6cac5f. Summary will update on new commits.}

[cursor]

Bugbot needs on-demand usage enabled

Bugbot uses usage-based billing for this team and requires on-demand usage to be enabled.

A team admin can enable on-demand usage in the Cursor dashboard.

[teamleaderleo]

Owned by an agent session · session

[github-actions]

cmux status _{(one comment, edited in place; each line is updated by the job that owns it)}

Review e9478e9: HQ feed card · gallery before/after runs · checks

App of this head (app.zip, kept 1 day)

gh workflow run reload-build.yml -R manaflow-ai/cmux --ref feat-cmux-next -f tag=pr34538 -f ref=cloud-copy-wire -f platform=macos -f backend_mode=local

Shipped in:

• cmux NEXT nightly 1.0.0-nightly.4182499138501 (e0aa712b3b)
• cmux NEXT nightly 1.0.0-nightly.4184087390301 (08052d331e)
• cmux NEXT nightly 1.0.0-nightly.4184224384601 (a3abfc4160)
• cmux NEXT nightly 1.0.0-nightly.4184409870901 (78ee0e108e)

[teamleaderleo]

Review (codex gpt-6.1-sol xhigh, router-review): SHIP at 6600293

Visual verdict: unchanged by this diff

No correctness, regression, security, or test-breakage findings identified from the changed protocol and sync paths.

Follow-ups: none from the review. Filed from this work: #34482 (raw byte stream over the Cloud link's local socket), #34483 (parallel or pipelined file writes).

Reds at this head that are not this PR's: cmux-tui Windows check (cli.rs:1206 workflow::HELP without cfg(unix)) and clippy (changed cmux-tui packages) (cli/automation.rs:98 type_complexity, cli/automation/teleport.rs:130 manual_is_multiple_of). Both are on base in files this PR doesn't touch, and #34476 and #34469 fix them.

[cursor]

Bugbot needs on-demand usage enabled

Bugbot uses usage-based billing for this team and requires on-demand usage to be enabled.

A team admin can enable on-demand usage in the Cursor dashboard.

[blacksmith-sh]

Found 6 test failures on Blacksmith runners:

Failures

Test	View Logs
/tmp/tmpxn1i3cdm/summary-missing.json (live):/summary-missing.json (live):	View Logs
/tmp/tmpyorpt75g/summary-fail.json (live):/summary-fail.json (live):	View Logs
[new_workspaces] debug.new_workspace timed 0 of 50 new workspace rows/ [new_workspaces] debug.new_workspace timed 0 of 50 new workspace rows	View Logs
[new_workspaces] the window draws no frames/[new_workspaces] the window draws no frames	View Logs
[tabs] debug.tab_switch timed 0 of 40 switches/ [tabs] debug.tab_switch timed 0 of 40 switches	View Logs
[tabs] the window draws no frames/[tabs] the window draws no frames	View Logs

^{Need help on this PR? Tag @codesmith-bot with what you need.}

[teamleaderleo]

Review (codex gpt-6.1-sol xhigh, router-review): SHIP at 82631ce

Visual verdict: no images; unchanged UI.

The only delta since the previous SHIP at 6600293 is rustfmt. Two [P2] [follow-up] findings, filed: #34691 (require the deflate stream's end before commit) and #34692 (fall back to fs.stat when a destination folder's listing is fs.too_large).

The reds are base reds in files this PR doesn't touch (Windows cli.rs workflow::HELP; clippy in cli/automation*), fixed by #34476 / #34469. cargo fmt --all -- --check is clean at this head.

[cursor]

Bugbot needs on-demand usage enabled

Bugbot uses usage-based billing for this team and requires on-demand usage to be enabled.

A team admin can enable on-demand usage in the Cursor dashboard.

[teamleaderleo]

Review (codex gpt-6-astra high, router-review): BLOCK at 6d1d75c

BLOCK at 6d1d75c
Visual verdict: no images; Leo tries the build

1. [P1] [blocking] [category:correctness] [finding:F1] [outcome:pending] Preserve the symlink check before writing — sync_cli.rs:594. The cached listing now supplies the symlink protection, potentially minutes before a later file is written. If the machine replaces b.txt with a symlink while an earlier sibling uploads, the cached kind permits the write; PendingWrite::begin follows that symlink and overwrites its target inside the daemon’s roots. Base checks fs.stat after the earlier upload and skips this file. Retain a fresh check or enforce no-follow semantics in the write operation, and cover this sequence.

2. [P1] [blocking] [category:tests] [finding:F2] [outcome:pending] Update the upload fake for capability negotiation — sync_cli.rs:324. This now sends identify, but access_cli/uploads_tests.rs:90 panics on that command. Consequently, a_sparse_file_over_1_gib_uploads_in_bounded_parts loses its connection and fails before uploading. Have the fake answer identify with an empty capability list.

Source review completed; Rust tests could not run because Cargo is unavailable.

Follow-ups: none

[cursor]

Bugbot needs on-demand usage enabled

Bugbot uses usage-based billing for this team and requires on-demand usage to be enabled.

A team admin can enable on-demand usage in the Cursor dashboard.

A newer verdict blocks c6cac5f (no independent SHIP at the head).

[teamleaderleo]

Review (codex, router-review): SHIP at 20fff43

Reviewer: codex gpt-6-astra high, fast tier.

SHIP at 20fff43

Visual verdict: unchanged by this diff

Reviewed the deflate decoder, atomic stream cleanup, capability negotiation, multipart writes, permissions, and compatibility paths. Focused tests could not run because cargo is unavailable in the review environment.

Follow-ups: none

[lawrencecchen]

Lead: after this PR merged, cargo clippy --all-targets -- -D warnings on feat fails: error: very complex type used. Consider factoring parts into type definitions at cmux-tui/crates/cmux-tui/tests/link_push_bench.rs:59:12 (queue: Mutex<(VecDeque<(Instant, Vec<u8>)>, bool)>). Seen in fleet step 89e53e3123c4e6bbe358b96e. Please add a type alias for the queue (no allow). Open PRs #35149 and #35138 also touch this file; one fix is enough.

[lawrencecchen]

Base red from this PR on owned Macs: server::fs_wire::tests::a_deflate_write_stream_over_its_decoded_size_leaves_no_file panics with BrokenPipe at fs_wire_tests.rs:266 (write_all(...).unwrap() in routed). It failed 3 of 3 at feat 8dd1c86 (fleet steps 066333febf5f8cb210cf6608, b7fd734b42b25f26ee1b45de, 2f2ec4b7cf2767d635aa8f80) and at d422b94 (e7dda1f85c179d8af561ebfb). The daemon refuses at decoded_size and closes; the ~64 KiB wire does not fit a macOS socketpair buffer (~8 KiB), so the rest of the body hits a closed socket. Linux buffers it all, so hosted CI stayed green. Fix (test helper only, no product change): #35306. teamleaderleo, for your information; tell me if you want the daemon to drain the declared bytes before it answers instead.