Repository navigation
[DeepSeek][ROCm] V4.1-Flash: enable INT4 QuickReduce on MI355X - #1061
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request updates the metadata and adds a ROCm strategy override for the DeepSeek-V4.1-Flash model. A review comment correctly identifies that the environment variable name used for QuickReduce quantization is incorrect and should be changed from VLLM_ROCM_QUICK_REDUCE_QUANTIZATION to VLLM_ROCM_QUICK_REDUCE_QUANT to prevent the setting from being ignored.
| # INT4 QuickReduce | ||
| VLLM_ROCM_QUICK_REDUCE_QUANTIZATION: "INT4" |
There was a problem hiding this comment.
The environment variable used by vLLM to configure QuickReduce quantization is VLLM_ROCM_QUICK_REDUCE_QUANT, not VLLM_ROCM_QUICK_REDUCE_QUANTIZATION. Using the incorrect variable name will result in the setting being ignored.
# INT4 QuickReduce
VLLM_ROCM_QUICK_REDUCE_QUANT: "INT4"
Signed-off-by: Muhammad Ahmed <Muhammad.Ahmed@amd.com>
6a257f1 to
c310fc8
Compare
|
Thanks @ahmed-bsod Let me verify it first before merging |
|
LGTM, more results can be found https://github.com/SemiAnalysisAI/InferenceX/actions/runs/37080504692/job/111080244536 |
This branch was successfully deployed
2 participants
No description provided.