[faresobeid]

What

• VLLM_CACHE_ROOT=/tmp/.vllm-cache-<uid> and FLASHINFER_WORKSPACE_BASE=/tmp/.flashinfer-cache-<uid> are added to DEFAULT_INFERENCE_ENV_VARS. TRITON_CACHE_DIR got the same treatment in fix(launch): default TRITON_CACHE_DIR to a node-local path #3877.
• The by-hand copies (/tmp/.vllm-cache, /tmp/.flashinfer-cache) are removed from the four GLM-4.5-Air recipes. Those paths were shared by every user on a node.
• docs/scaling.md gets one paragraph that lists the node-local cache defaults.

The paths are dot-prefixed so the SLURM start-up cleanup (rm -rf /dev/shm/vllm-* /dev/shm/vllm_* /tmp/vllm-* /tmp/vllm_* /tmp/torch-* /tmp/torchelastic_* in the multi_node_rl, multi_node_sft and inference sbatch templates) does not wipe them. These are the same names the recipes used before.

Precedence is the same as for the other launcher defaults: [env_vars] or [inference.env_vars] override them; a value exported in the shell does not.

Why

• Without these, vLLM's AOT/Inductor compile cache lives in ~/.cache/vllm and FlashInfer's JIT cache under $HOME, which is usually the shared filesystem.
• Measured while testing fix(launch): default TRITON_CACHE_DIR to a node-local path #3877 (GLM-4.5-Air TP8, vLLM 0.31): a crashed start left a partial shared vLLM cache. On the next start, 4 TP ranks loaded those artifacts and the other 4 recompiled. The ranks diverged and hung in a collective. So one crash breaks every later start that uses that cache.
• The original scaleswe recipe (exp: scaleswe ablation #2862) moved this cache to /tmp because the NFS cache hung inference startup.
• Cost: the first start on each new node has a cold vLLM cache. GLM-4.5-Air: 465 s cold vs 375 s with a warm vLLM cache, so about 1.5 min once per node.

Overlap

• feat(deepseek_v4): fp8 RL #3627 (garrett361, draft) adds VLLM_CACHE_ROOT/FLASHINFER_WORKSPACE_BASE under /tmp/garrett/ to new DeepSeek-V4 recipes. With this PR those lines can go.

Test plan

• pytest tests/unit/orchestrator tests/unit/test_configs.py tests/unit/utils -m "not gpu": 365 passed (rebased on origin/main 41a8ed0).
• Ran the cleanup globs in a scratch dir that held .vllm-cache-<uid>, .flashinfer-cache-<uid> and, as a control, vllm-cache-<uid>. Only the control matched.
• ruff check and ruff format --check on process.py: clean.
• rl @ examples/advanced/glm-4.5-air/swe.toml --dry-run:
  • The rendered rl.sbatch exports both vars only in the inference block (/tmp/.vllm-cache-<uid>, /tmp/.flashinfer-cache-<uid>).
  • With --inference.env-vars '{"VLLM_CACHE_ROOT": "/scratch/vc"}', the sbatch exports /scratch/vc.

---

Note

Low Risk
Launcher default env vars for inference only; configs can still override, with no auth or training logic changes.

Overview
Inference launches now default vLLM and FlashInfer compile/JIT caches to per-user paths under /tmp (VLLM_CACHE_ROOT, FLASHINFER_WORKSPACE_BASE), matching the existing Triton cache behavior and avoiding shared $HOME or NFS caches that can leave tensor-parallel ranks on mismatched artifacts and hang.

The four GLM-4.5-Air example recipes drop redundant [inference.env_vars] blocks that duplicated those settings. docs/scaling.md documents the launcher defaults and notes overrides via [env_vars] or component env_vars.

^{Reviewed by Cursor Bugbot for commit fe8a34d. Bugbot is set up for automated code reviews on this repo. Configure here.}

[cursor]

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit ac13f03. Configure here.}