Skip to content

[Bugfix][Quantization] Respect explicit WNA16 MoE backend - #49065

Closed
aoshen02 wants to merge 1 commit into
vllm-project:mainfrom
aoshen02:codex/wna16-respect-moe-backend
Closed

[Bugfix][Quantization] Respect explicit WNA16 MoE backend#49065
aoshen02 wants to merge 1 commit into
vllm-project:mainfrom
aoshen02:codex/wna16-respect-moe-backend

Conversation

@aoshen02

Copy link
Copy Markdown
Collaborator

Summary

  • honor an explicit moe_backend=triton for compressed-tensors WNA16 MoE
  • keep auto and explicit marlin selecting Marlin when the layer is supported
  • add focused coverage for all three selections

Root cause

The WNA16 selector chose Marlin whenever the layer supported it, before consulting kernel_config.moe_backend. As a result, an explicit Triton request was silently overridden.

Fix

Marlin is selected only when the requested backend is auto or marlin, the layer is supported, and the platform is not ROCm. Other paths continue through the existing WNA16 backend selection.

Validation

  • full pre-commit passed
  • focused test added for auto -> marlin, marlin -> marlin, and triton -> triton
  • the focused pytest command could not run on the host because its system Python does not have pytest installed

AI assistance was used. The submitter reviewed every changed line.

Signed-off-by: aoshen02 <aoshen@inferact.ai>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant