[Core] Bound UniProc EngineCore startup threads to available CPUs

[AndreasKaratzas]

In Buildkite CI #91445, an MI355 process with a 12-CPU cgroup quota loaded weights with 192 PyTorch threads. Weight loading took 38 minutes and the job timed out after its accuracy checks passed. The quota-aware budget added by #49919 covered multiprocessing workers, but a UniProc worker runs in EngineCore and did not receive that budget.

For process-managed UniProc, cap startup threads at the smaller of the current PyTorch per-process limit and #49919's available-CPU count divided by the number of local EngineCore processes. Set OMP_NUM_THREADS and #49919's existing vLLM ownership marker only while starting those processes, then restore the parent's settings even if launch fails. An explicit user OMP_NUM_THREADS remains authoritative. The worker keeps #49919's existing transition to one PyTorch thread after weight loading.

This differs from open #45124, which proposes a fixed, tunable four-thread cap inside UniProc for serving. Here the cap is the #49919 affinity/cgroup startup budget, applied before subprocess launch; serving returns to one thread. The launch-only scope is needed because fork inherits the API parent's PyTorch count while spawn reads its OMP setting. Restoring the parent prevents a later launch from mistaking vLLM's generated OMP value for a user override and skipping a fresh CPU budget.

Serving evaluations compare one and four EngineCore threads on a ROCm gfx942 host with 12-CPU server affinity. Both variants started with OMP_NUM_THREADS=4; toggling #49919's existing marker changed only the EngineCore's post-load thread count. These comparisons isolate the existing serving policy rather than exercise the new automatic startup cap.

Model and workload	One EngineCore thread	Four EngineCore threads	Result
Qwen3-0.6B, 96 GSM8K prompts at 8 RPS, two runs each	96/96 completed per run; E2E p50 211/221 ms	96/96 completed per run; E2E p50 224/225 ms	Similar throughput (7.88 RPS each); extracted GSM8K answer accuracy was 14/96 in three runs and 15/96 in one.
Qwen3-0.6B, 96 JSON-schema requests at 8 RPS, two runs each	96/96 schema-valid per run; E2E p50 188/179 ms	96/96 schema-valid per run; E2E p50 195/180 ms	Small difference relative to run variation.
SmolVLM-256M-Instruct, default compilation, four 224x224 images/request, 256 requests/run in A-B-B-A order	256/256 completed per run; mean 5.60 RPS	256/256 completed per run; mean 5.84 RPS	Four threads had a small, order-sensitive 4.3% throughput advantage.
SmolVLM-256M-Instruct, default compilation, one image/request, 256 requests/run	256/256 completed per run; mean 19.07 RPS	256/256 completed per run; mean 18.99 RPS	No stable gain from four threads.

The deterministic four-image SmolVLM request returned the same HTTP 200 answer under both thread counts. A preliminary 64-request eager multimodal comparison showed lower concurrent text-probe latency with four threads (98–105 ms versus 199–202 ms). Two longer eager pairs, each with 256 four-image requests per setting, did not reproduce that gap: probe medians were 150–155 ms with one thread and 141–160 ms with four. Four-image throughput was 5.52/5.63 versus 5.93/5.61 requests/s (one versus four); all 1,024 image requests and 273 probes succeeded. The 96-prompt GSM8K run used a 128-token cap reached by 83 prompts in every run, so its low exact-match score is a consistency check, not an accuracy claim. A single saturated 256-request JSON pair favored one thread (38.90 versus 33.08 RPS; 256/256 schema-valid under each setting), but needs repetition before assigning an effect size.

AI assistance was used for the investigation, implementation, tests, and evaluation analysis.

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[AndreasKaratzas]

/ci run

[github-actions]

✅ Triggered Buildkite CI #91497 for commit d4537e27d4ca.

[AndreasKaratzas]

/ci run

[github-actions]

❌ This PR is 1 commit behind upstream main. Your branch must contain every commit currently on upstream main. No new CI build was started. Merge or rebase onto the latest main, then rerun /ci run. To test this branch at your own risk, use /ci run --allow-stale.

[AndreasKaratzas]

/ci run

[github-actions]

✅ Triggered Buildkite CI #91749 for commit 9c600e5e9aec.

[AndreasKaratzas]

I forgot to activate:

• https://buildkite.com/vllm/ci/builds/91749/list?sid=01a0eb92-387b-40e4-ad1f-ec903bbf8941&tab=output

[njhill]

Thanks @AndreasKaratzas. Thinking about refactoring this a bit but can do it as a follow-on.