perf(bench): read decode-worker placement out of /proc instead of trusting the width label

[justinchuby]

A cross-agent placement report had the default decode pool pinning 16 workers to cpus 0–15 — two workers per physical core on a single 32 MiB L3, half the machine idle — and offered #1729 as the fix.

#1729 is merged (6e8c31ebd, 2026-08-23T01:11:35Z), as is the related width-halving fix #1794 (0652fdd2e). Both are ancestors of origin/main. The report describes a pre-#1729 build.

Rather than argue from the diff, this adds the categorical instrument that settles it.

benches/decode_placement_census.sh

Reads Cpus_allowed_list for every onnx-genai-spmd thread in three configurations. It is a /proc read, not a benchmark — it does not need a quiet host and no number in it is a timing. It still takes the hostlock as a courtesy, since it does spin the pool.

Measured on 0a668d54b, reproduced identically three times:

configuration	spawned workers	pinned CPUs	L3 spread
default, no taskset, no env	15	0,2,4,…,28	8 in L3#0, 7 in L3#1
THREADS=16 under the even mask	15	0,2,4,…,28	8 in L3#0, 7 in L3#1
THREADS=8 under the even mask	7	0,2,…,12	7 in L3#0

One worker per physical core in every case, with the reserved dispatcher CPU (30 at width 16, 14 at width 8) left clear — exactly what decode_affinity::order_pin_targets and reserve_single_group_headroom specify.

The tell was visible in the original report without any re-measurement: its two arms reported 16 and 15 shard participants, and 15 spawned workers plus an inline dispatcher is precisely what current main builds.

What it turned up in my own record

ONNX_GENAI_CPU_DECODE_THREADS=8 confines the process to [0,2,4,6,8,10,12,14] — entirely inside one 32 MiB L3 instance — while width 16 spans both. So t=8 → t=16 on this host doubles cache and memory-controller reach as well as cores. That was not written down anywhere and now is, in both the benchmark record and the ledger.

It is not a confound. acc0_gap_matrix.ort() defaults its pin to native_pin(threads), so both arms get the same CPUs at each width, and both the 1.762x (ORT) and 1.319x (native) scaling figures post-date that fix (4b4dacc7e). It does make the 2.0x ideal a conservative reference for both arms, which leaves the finding understated rather than overstated.

benches/cpu_work_probe.py

A second, independent reason not to read /usr/bin/time -v's Percent of CPU as utilisation here, beyond its being wall-derived: on an SMT host a logical CPU whose sibling is busy is granted a full 100% share while delivering roughly half the work, and no CPU-time instrument can see it — the scheduler really is handing over the CPU; the contention is in hardware, below its view. Only a work-completed probe distinguishes them.

The permanent cpu-0 competitor reported alongside the placement claim did not reproduce: cpu 0 reads cpu_share 0.999–1.000 at 9429/9482/9489 iterations, inside the 8744–9499 band spanned by ten other CPUs, with one transient outlier that two re-probes cleared. On a host shared by several agents, "permanent" was load rather than topology. The instrument point is the durable part and is what the ledger records.

Scope

Docs and bench scripts only — no compiled code changes. shellcheck clean, ruff clean.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 80.96%. Comparing base (203962f) to head (e2190ab).
⚠️ Report is 18 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1937      +/-   ##
==========================================
+ Coverage   80.14%   80.96%   +0.81%     
==========================================
  Files         413      416       +3     
  Lines      201044   205410    +4366     
  Branches   201044   205410    +4366     
==========================================
+ Hits       161132   166306    +5174     
+ Misses      34388    33499     -889     
- Partials     5524     5605      +81     

Flag	Coverage Δ
cli-ort-linux	72.51% <ø> (?)
cli-ort-windows	72.10% <ø> (+0.09%)	⬆️
mlas	85.33% <ø> (?)
offline	81.11% <ø> (+0.75%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.
see 29 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.