Repository navigation
Update InferenceX MI355X agentic recipes - #972
Conversation
Use the validated nightly image and disable incompatible AITER shared-expert fusion in the agentic reproduction guide while leaving other MI355X lanes unchanged. Signed-off-by: haic0 <haichzha@amd.com> Co-authored-by: Cursor <cursoragent@cursor.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request updates the DeepSeek-V4-Pro configuration guide to specify a source Docker image and disables the VLLM_ROCM_USE_AITER_FUSION_SHARED_EXPERTS flag (setting it to 0) to avoid initialization failures on the nightly build. The review feedback recommends rephrasing the documentation to avoid a double negative, suggesting 'Set VLLM_ROCM_USE_AITER_FUSION_SHARED_EXPERTS=0 to disable it' instead of 'Disable VLLM_ROCM_USE_AITER_FUSION_SHARED_EXPERTS=0'.
| was not running the configuration the validated manual runs used. The flag is | ||
| mutually exclusive with expert parallelism inside vLLM, which is consistent with | ||
| both arms running EP 1. | ||
| - Disable `VLLM_ROCM_USE_AITER_FUSION_SHARED_EXPERTS=0` for this nightly. |
There was a problem hiding this comment.
The phrasing 'Disable VLLM_ROCM_USE_AITER_FUSION_SHARED_EXPERTS=0' is slightly confusing and reads as a double negative (disabling a setting that is already 0/disabled). It would be clearer to state 'Set VLLM_ROCM_USE_AITER_FUSION_SHARED_EXPERTS=0 to disable it' or 'Disable shared-expert fusion by setting VLLM_ROCM_USE_AITER_FUSION_SHARED_EXPERTS=0'.
- Set VLLM_ROCM_USE_AITER_FUSION_SHARED_EXPERTS=0 to disable it for this nightly.
haic0
changed the title
This branch was successfully deployed
3 participants
Summary
vllm/vllm-openai-rocm:nightly-eed1f3d0c6043bd494424a22443ee198dd56f657.VLLM_ROCM_USE_AITER_FUSION_SHARED_EXPERTSfor that nightly because the mixed MXFP4/FP8 checkpoint fails in the fused shared-expert loading path.Related: SemiAnalysisAI/InferenceX#3071
Test plan
node scripts/build-recipes-api.mjs.git diff --check.