fix(container): make vLLM FlashInfer cubin dirs writable (#12770) - #12976
Conversation
Signed-off-by: Brian Westphal <bwestphal@nvidia.com> Co-authored-by: Anant Sharma <anants@nvidia.com>
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
| # FlashInfer creates package-local cubin symlinks at runtime. Grant group 0 | ||
| # write access so arbitrary OpenShift UIDs can initialize the cubin cache. | ||
| RUN SITE_PACKAGES="$(python3 -c 'import site; print(site.getsitepackages()[0])')" && \ | ||
| CUBINS_DIR="$SITE_PACKAGES/flashinfer_cubin/cubins" && \ | ||
| if [ -d "$CUBINS_DIR" ]; then \ | ||
| find "$CUBINS_DIR" -type d -exec chmod g+rwx {} + ; \ | ||
| fi |
There was a problem hiding this comment.
🔍 Cubin permission fix runs before the vLLM-Omni pip install that may (re)install FlashInfer packages
This chmod layer is placed immediately after the user-creation layer, but the vLLM runtime later runs install_vllm_omni.sh (container/templates/vllm_runtime.Dockerfile:216-227, script at container/deps/vllm/install_vllm_omni.sh). The protected-constraints list pins flashinfer-python but not flashinfer-cubin (container/deps/vllm/protected_packages.txt). If flashinfer_cubin is absent in the base vllm/vllm-openai image and only pulled in by the omni solve — or if the omni solve reinstalls/upgrades it — the if [ -d "$CUBINS_DIR" ] guard silently makes the chmod a no-op, or the freshly-unpacked 755 root-owned directories overwrite the group-writable permissions, so arbitrary OpenShift UIDs still cannot initialize the cubin cache. The sglang template (container/templates/sglang_runtime.Dockerfile:38-42) has no such later FlashInfer-touching install, so copying the placement verbatim into the vLLM template is worth verifying against an actual built image (e.g. ls -ld $(python3 -c 'import site;print(site.getsitepackages()[0])')/flashinfer_cubin/cubins).
Was this helpful? React with 👍 or 👎 to provide feedback.
![dynamo-review-agent[bot]](https://avatars.githubusercontent.com/u/201626793?s=60&v=4){kind=small}
|
🎯 Code Coverage (details) 🔗 Commit SHA: ff8e457 | Docs | Datadog PR Page | Give us feedback! |
3 participants
add #12770 to release
closes: OPS-8074