[ROCm][MoE configs] mi325 mixtral & mi300 qwen_moe #13503
[ROCm][MoE configs] mi325 mixtral & mi300 qwen_moe #13503simon-mo merged 2 commits intovllm-project:mainfrom
Conversation
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
Signed-off-by: Divakar Verma <divakar.verma@amd.com>
|
👋 Hi! Thank you for contributing to the vLLM project. 💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels. Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging. To run CI, PR reviewers can either: Add 🚀 |
|
Im assuming you have run a perf benchmark on this. Also, don't we need OAM versions of these too? |
|
@robertgshaw2-redhat We won't need OAM versions going forward. The following PR will get the correct (& unique) names for AMD GPUs without any trailing "_OAM": #13438 |
Signed-off-by: Louis Ulmer <ulmerlouis@gmail.com>
This PR add the following moe configs: