Skip to content

perf: cap default tokio worker threads at 16 - #11029

Merged
jdx merged 2 commits into
mainfrom
perf/tokio-worker-cap
Jul 16, 2026
Merged

jdx merged 2 commits into
mainfrom
perf/tokio-worker-cap

Conversation

@jdx

@jdx jdx commented Jul 15, 2026 •

Copy link
Copy Markdown
Owner

Summary

Linux leg of the startup-perf work (#11025, #11027). Benchmarked the shipped v2026.7.7 binary on a 32-core Arch Linux box against asdf 0.20 and profiled where the time goes.

The Linux-specific finding: startup is system-time dominated (6ms sys for mise version, ~2200 minor faults), and a big contributor scales with core count — the tokio runtime is built with worker_threads = nprocs.max(8), and tokio spawns every worker eagerly at build. On a 32-core machine that's 32 threads (clone + stack + TLS faults each) before any work happens, on every invocation including the per-prompt hook-env fast path. Async I/O doesn't need a worker per core; blocking work runs on tokio's separate on-demand pool.

This caps the default at 16 (floor of 8 unchanged). Machines with ≤16 cores see no behavior change. Explicit MISE_JOBS still raises it without limit.

Benchmarks (32-core Linux, hyperfine -N, warm, isolated env; worker count simulated via MISE_JOBS on the release binary)

workers mise version hook-env fast path (per prompt) sys time
32 (old default) 6.3 ms 4.1 ms 6.1 ms
16 (new default) 5.8 ms ~3.4 ms 4.5 ms
8 5.6 ms 3.3 ms 4.0 ms

Parallel install of multiple tools verified working with the cap.

Other Linux findings (no PR needed)

  • glibc's dynamic loader is not the problem: LD_DEBUG=statistics shows relocation processing of the 125k RELA entries takes only ~0.15ms (~84% of loader time but tiny in absolute terms).
  • -z pack-relative-relocs (RELR) is not shippable: binaries would fail to load on glibc < 2.36, and the release pipeline explicitly validates Amazon Linux 2 (glibc 2.26) compatibility. It would only save ~0.15ms + ~3MB anyway.
  • The remaining gap to asdf (~1.1ms total) is first-touch page faults across the large text segment — that's the PGO/BOLT territory (BOLT is Linux-only), for which jdx/aube's benchmarks/pgo.bash pipeline is a ready-made template.

🤖 Generated with Claude Code


Note

Low Risk
Startup-only default thread cap with unchanged explicit MISE_JOBS override; machines with ≤16 cores are unaffected.

Overview
Reduces CLI startup cost on many-core machines by changing how the default Tokio multi-thread worker count is chosen in main.

Previously the runtime used available_parallelism() with a floor of 8, so a 32-core host eagerly spawned 32 workers at build time (each paying clone/stack/TLS cost on every invocation, including hook-env). The default is now min(nprocs, 16) with the same floor of 8; hosts with ≤16 cores behave as before. An explicit MISE_JOBS still sets the count with no cap.

Inline comments document why async I/O does not need one worker per core and that blocking work uses Tokio’s separate pool.

Reviewed by Cursor Bugbot for commit 7301b72. Bugbot is set up for automated code reviews on this repo. Configure here.

Summary by CodeRabbit

  • Performance
    • Improved default resource usage by limiting background task workers to a maximum of 16 while maintaining a minimum of 8.
    • Explicit worker-count settings continue to be honored.

Tokio spawns every worker thread eagerly when the runtime is built, so
worker count is pure startup cost paid by every invocation — clone +
stack + TLS pages per thread before any work happens. The default was
nprocs with a floor of 8, which scales that cost linearly with core
count for no benefit: async I/O doesn't need a worker per core, and
blocking work runs on tokio's separate on-demand pool.

Measured on a 32-core Linux box (Arch, x86_64, hyperfine -N, warm,
v2026.7.7 release binary, simulated via MISE_JOBS):

- mise version: 6.3ms (32 workers) -> 5.6ms (8) / 5.8ms (16)
- hook-env early-exit fast path (per shell prompt): 4.1ms -> 3.3ms
- system time: 6.1ms -> 4.0ms

Machines with <=16 cores are unaffected. An explicit MISE_JOBS still
raises the worker count without limit.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Jul 15, 2026 •

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

The Tokio multi-thread runtime now caps its default worker count at 16 when MISE_JOBS is unset, while preserving the minimum of 8 workers.

Changes

Runtime defaults

Layer / File(s) Summary
Cap default worker count
src/main.rs
The default Tokio worker count uses the smaller of detected CPU parallelism and 16, then enforces a minimum of 8.

Estimated code review effort: 1 (Trivial) | ~5 minutes

Poem

A rabbit hops through cores so bright,
Capping workers at sixteen just right.
Eight remain when clouds are gray,
Tokio keeps the load at bay.
“Bounded speed!” the bunny sings,
While steady threads do useful things.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely summarizes the main change: capping default Tokio worker threads at 16.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@greptile-apps

greptile-apps Bot commented Jul 15, 2026 •

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR reduces startup overhead on many-core Linux machines by capping the default Tokio worker thread count at 16 (was nprocs, minimum 8). Tokio eagerly spawns every worker at runtime build time, so the old default imposed a per-invocation cost proportional to core count even for lightweight paths like hook-env.

  • The new default is min(nprocs, 16).max(8): machines with ≤16 cores are unaffected, and machines with more cores pay for at most 16 eager thread spawns instead of all of them.
  • Explicit MISE_JOBS bypasses the cap entirely (only the floor of 8 applies), preserving the ability to use full parallelism for heavy install workloads.

Confidence Score: 5/5

Safe to merge — it's a single arithmetic expression change with a well-understood effect, backed by benchmark data.

The change is minimal (one expression, well-commented), the floor of 8 is preserved, edge cases like nprocs=0 and MISE_JOBS values below the floor all behave identically to before, and the cap only applies to the default path. Parallel installs were verified in the benchmarks.

No files require special attention.

Important Files Changed

Filename Overview
src/main.rs Caps default Tokio worker threads at 16 (floor 8 unchanged); explicit MISE_JOBS still honored without limit. Logic and edge cases (nprocs=0, MISE_JOBS<8) are handled correctly.

Reviews (2): Last reviewed commit: "style: format tokio worker count express..." | Re-trigger Greptile

@github-actions

github-actions Bot commented Jul 16, 2026 •

Copy link
Copy Markdown
Contributor

Hyperfine Performance

mise x -- echo

Command Mean [ms] Min [ms] Max [ms] Relative
mise-2026.7.7 x -- echo 19.9 ± 1.5 17.4 31.4 1.00
mise x -- echo 21.9 ± 2.2 17.5 41.6 1.10 ± 0.14

mise env

Command Mean [ms] Min [ms] Max [ms] Relative
mise-2026.7.7 env 23.5 ± 1.5 20.3 29.6 1.04 ± 0.09
mise env 22.5 ± 1.4 19.4 29.2 1.00

mise hook-env

Command Mean [ms] Min [ms] Max [ms] Relative
mise-2026.7.7 hook-env 25.2 ± 1.4 21.0 30.8 1.11 ± 0.09
mise hook-env 22.7 ± 1.4 18.9 28.0 1.00
✅ Performance improvement for hook-env is 11%

mise ls

Command Mean [ms] Min [ms] Max [ms] Relative
mise-2026.7.7 ls 20.6 ± 1.6 17.2 28.4 1.04 ± 0.11
mise ls 19.7 ± 1.3 16.2 25.1 1.00

xtasks/test/perf

Command mise-2026.7.7 mise Variance
install (cached) 182ms 183ms +0%
ls (cached) 73ms 70ms +4%
bin-paths (cached) 72ms 73ms -1%
task-ls (cached) 147ms 145ms +1%

@jdx
jdx enabled auto-merge (squash) July 16, 2026 02:53
Entire-Checkpoint: 9e11d637148a
@jdx
jdx merged commit 573eb9d into main Jul 16, 2026
33 of 34 checks passed
@jdx
jdx deleted the perf/tokio-worker-cap branch July 16, 2026 03:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant