Set per-process DeepGEMM cache env

[yueming-yuan]

Summary

• Set a rank-scoped SGLANG_DG_CACHE_DIR for rollout SGLang engines
• Set SGLANG_DG_CACHE_DIR_PER_PROCESS=1 from the DeepSeek V4 launcher to enable per-process DeepGEMM cache isolation
• Keep the TODO note marking this as a hacky rollout-side workaround

Root cause

DeepSeek V4 launches multiple rollout SGLang processes during cold-cache startup. Without per-process DeepGEMM cache isolation, concurrent JIT/cache writes can race in the same cache directory and fail during directory creation.

Validation

• pre-commit run --all-files
• Covered by the DeepSeek V4 16-node disaggregate debug run that reached step 1 with this change in place

Requires the matching SGLang cache-dir support PR: sgl-project/sglang#27604

[gemini-code-assist]

Code Review

This pull request configures DeepGEMM cache directories in miles/ray/rollout/server_group.py by setting the SGLANG_DG_CACHE_DIR and SGLANG_DG_CACHE_DIR_PER_PROCESS environment variables. The reviewer suggests appending the user's UID to the cache directory path to avoid permission conflicts on shared multi-user clusters.

Important

The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.

[gemini-code-assist]

Using a shared path like /tmp/sglang_deep_gemm can cause permission denied errors on shared multi-user clusters if the directory is created by one user and another user subsequently runs a job on the same node. Appending the user's UID to the path prevents these conflicts.

Suggested change

	"SGLANG_DG_CACHE_DIR": f"/tmp/sglang_deep_gemm/{self.worker_type}_rank_{global_rank}",
	"SGLANG_DG_CACHE_DIR": f"/tmp/sglang_deep_gemm_{os.getuid()}/{self.worker_type}_rank_{global_rank}",

[maocheng23]

agree here, we need UID for separation.

[maocheng23]

address the comment but approve to comments

[maocheng23]

agree here, we need UID for separation.