Skip to content

acpmux deps: timing record per archive; count the install after a restore - #35237

Merged
teamleaderleo merged 4 commits into
feat-cmux-nextfrom
lane/dep-cache-meta
Nov 4, 2026
Merged

teamleaderleo merged 4 commits into
feat-cmux-nextfrom
lane/dep-cache-meta

Conversation

@teamleaderleo

@teamleaderleo teamleaderleo commented Nov 4, 2026 •

Copy link
Copy Markdown
Collaborator

Follow-up to #35159 (the per-machine restore-or-install choice). That choice needs a cold install measured on this machine, so a new Cloud machine always restored, even where its archive is slower to fetch than the install it replaces (bun on a fast link).

This stores a small timing record next to each archive when it is first uploaded: the uploader's cold install time and the archive size, under its own derived key in the same ecosystem. A machine with no local record fetches it (one small request) and decides with Stats::choose_from. If there is no record, it restores, as before.

  • remote.rs: Meta, meta_key, fetch_meta (capped at 4 KiB), store_meta.
  • warm.rs: the lookup when there is no local cold record. The background machine deps upload gets a new --cold-ms flag and stores the record after an archive is created, and so does --upload-now.
  • stats.rs: choose_from, plus ACPMUX_DEPS_LOCAL_STATS=off so the bench can act as a new machine.
  • detect.rs: KEY_VERSION becomes v2, so archives from before the timing record don't restore without one.
  • deps-bench.sh: a third round per ecosystem, <eco>Fresh, that runs with local stats off.

The expected restore now also counts the install after it. The bench showed Cargo restoring on a fast link (1.15 s fetch + 0.13 s unpack + 1.67 s cargo fetch --offline = 2.98 s) and losing to its 2.75 s cold fetch, because the choice ignored the offline fetch. Stats::after_restore_ms uses this machine's measured share per ecosystem (learned after each restore) or a default share of the cold install (Cargo 0.65, npm/pnpm 0.5, uv 0.3, yarn 0.1, bun 0.05). Mid-download give-up (too_slow) counts it too.

bench=deps-slow failed on the runner kernel ("Specified qdisc kind is unknown": no netem). The prepare step now tries modprobe sch_netem. If netem is still missing, it installs scripts/cmux-next/delay-proxy.py, a local CONNECT proxy that delays each chunk 75 ms each way. The bench routes every tool's HTTPS through it (HTTPS_PROXY), and the evidence records link and proxyHosts.

Cost: on a machine with no local record, the first warm of a key makes one extra small GET.

Risk: normal. No secrets are involved: the record holds two numbers.

Agent session: agent session

🤖 Generated with Claude Code

@github-actions github-actions Bot added the risk:normal Risk tier (docs/ci/risk-tiers.md) label Nov 4, 2026
@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

Owned by an agent session · session

@github-actions

github-actions Bot commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

cmux status (one comment, edited in place; each line is updated by the job that owns it)

Review d57e786: HQ feed card · gallery before/after runs · checks

App of this head (app.zip, kept 1 day)

gh workflow run reload-build.yml -R manaflow-ai/cmux --ref feat-cmux-next -f tag=pr35237 -f ref=lane/dep-cache-meta -f platform=macos -f backend_mode=local

web and size checks: green

Shipped in:

teamleaderleo and others added 3 commits November 4, 2026 02:30
…ooses too

The uploader stores what it measured (its cold install, the archive's size)
as a small entry of its own (remote::meta_key). A machine with no cold
install of its own for that key reads it (one small request) and installs
cold when the restore is not expected to beat it. Keys move to v2 (zstd -12
--long=27 archives, each with its record). The bench gains a fresh-machine
round (ACPMUX_DEPS_LOCAL_STATS=off).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?id=aid-fb75b7af9902231028aa
…ut netem

The choice compared a cold install with only the download and unpack, so
Cargo restored on a fast link and lost to its own 1.7 s offline fetch.
The expected restore now adds the install after it: as measured on this
machine per ecosystem, or a default share of the cold install.

bench=deps-slow fell over on a kernel without netem; it now routes every
tool's HTTPS through a local proxy that adds the same 150 ms.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?id=aid-fb75b7af9902231028aa
@teamleaderleo teamleaderleo changed the title acpmux deps: timing record beside each archive so new machines choose restore vs install acpmux deps: timing record per archive; count the install after a restore Nov 4, 2026
@blacksmith-sh

blacksmith-sh Bot commented Nov 4, 2026 •

Copy link
Copy Markdown

Found 9 test failures on Blacksmith runners:

Failures

Test View Logs
/tmp/tmpazevk4aj/summary-missing.json (live):/summary-missing.json (live): View Logs
/tmp/tmpxjufpfx3/summary-fail.json (live):/summary-fail.json (live): View Logs
[new_workspaces] debug.new_workspace timed 0 of 50 new workspace rows/
[new_workspaces] debug.new_workspace timed 0 of 50 new workspace rows
View Logs
[new_workspaces] the window draws no frames/[new_workspaces] the window draws no frames View Logs
[tabs] debug.tab_switch timed 0 of 40 switches/
[tabs] debug.tab_switch timed 0 of 40 switches
View Logs
[tabs] the window draws no frames/[tabs] the window draws no frames View Logs
nightly cmux-tui resolve/nightly cmux-tui resolve View Logs
test_the_committed_contract_is_the_workflows (main.RollupsFromNeeds.test_the_commit
ted_contract_is_the_workflows)/
test_the_committed_contract_is_the_workflows (main.RollupsFromNeeds.test_the_commit
ted_contract_is_the_workflows)
View Logs
the short nightly decide gate must use the pooled Linux route with a Blacksmith fallbac
k/
the short nightly decide gate must use the pooled Linux route with a Blacksmith fallbac
k
View Logs

Fix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.

@teamleaderleo
teamleaderleo merged commit 26439b6 into feat-cmux-next Nov 4, 2026
170 of 175 checks passed
@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

Review (codex gpt-6-astra high, router-review): SHIP at d57e786

Visual verdict: unchanged by this diff

  • [P2] [follow-up] [category:reliability] [finding:F1] [outcome:pending] cmux-tui/crates/acpmux/src/deps/remote.rs:228 does not enforce the advertised 4 KiB metadata cap before downloading. Remote::fetch only invokes give_up after 300 ms, so a fast or malformed hub response can write an arbitrarily large metadata entry to the temporary file.
  • [P2] [follow-up] [category:performance] [finding:F2] [outcome:pending] cmux-tui/crates/acpmux/src/deps/warm.rs:164-183,271-284 fetches metadata on every warm for a machine without a local cold record. A successful metadata-based restore records warm_ms and archive_bytes, but never persists that metadata as a known record, contradicting the stated one-extra-GET cost.

Follow-ups:

@teamleaderleo
teamleaderleo deleted the lane/dep-cache-meta branch November 4, 2026 11:21
teamleaderleo added a commit that referenced this pull request Nov 4, 2026
Each push that touches .github/, tests/ or scripts/ci/ judges the guard
profile at its tip, and 14 steps were red there. This clears nine (fork routing and the Blacksmith failover also need #35717):

- full clones: commandRules.ts runs no git; it lists `git clone`'s --depth
  as a flag to skip when it names a shown command (#35137).
- send-path caps: the dependency-cache timing record bound (#35237) and the
  local replica's file size bound (larger files read from the machine)
  refuse nothing a person sends; listed as safe.
- fleet rule lints: a sample machine name in a colour test fixture, and a
  plan table row naming SecItemCopyMatching in prose; listed as safe.
- pinned Xcode: drop the exemption for nightly.yml's launch-smoke-nightly,
  which no longer exists.
- macOS job class: nightly.yml's republish-nightly-next (#35588) only
  downloads and re-uploads a verified build: portable-checks.
- label events: the production cmux-vm jobs are gated first on
  github.ref == refs/heads/feat-cmux-next, which a pull_request event never
  has (its ref is refs/pull/N/merge); the guard now reads that, for
  pull_request only and only when no top-level || follows.
- fork routing: tail-production and sweep-orphans-production take the fork
  pull-request branch before vars.LINUX_RUNNER, like their siblings.
- supply chain: the jobs that mint the cmux-bot token (swift-test,
  generated-files, push-attribution, #35123) run in ci-routing-agent off
  pull requests, as macos-placement and compile-status do; tail-production
  reads its Cloudflare secrets in cmux-vm-production.
- upload retention: the production journey's step table keeps 3 days.


Agent-Session: https://cmux.com/deeplink/agent?id=aid-d750cf10922c785efa64

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

This branch was previously deployed

3 inactive deployments
artifacts — d57e7865 Deployed Nov 4, 2026 by teamleaderleo via publish tree-addressed cmux-tui #111219
sentry-dsyms — d57e7865 Deployed Nov 4, 2026 by teamleaderleo via upload cmux-tui dSYMs to Sentry #111219
ci-routing-agent — d57e7865 Deployed Nov 4, 2026 by teamleaderleo via cmux-tui tree preflight #111219
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

risk:normal Risk tier (docs/ci/risk-tiers.md)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant