[jinzhen-lin]

This PR add humming moe backend to all dense/moe oracles, and fixed several minor bugs discovered during model testing.

Tested with

• Dense
  • compressed-tensors
    • int4: RedHatAI/Qwen3.5-4B-quantized.w4a16
    • int8: nm-testing/Qwen2.5-VL-3B-Instruct-quantized.w8a8
    • fp8:
      • channelwise: nm-testing/Qwen2-1.5B-Instruct-FP8W8
      • tensorwise: nm-testing/Llama-3.1-8B-Instruct-KV-FP8-Tensor
      • blockwise: nm-testing/Qwen3-0.6B-FP8_BLOCK
  • modelopt:
    • nvfp4: nvidia/Qwen3-8B-NVFP4
    • mxfp8: mgoin/Qwen3-0.6B-MXFP8
  • fp8: Qwen/Qwen3-4B-FP8
  • gptq: Qwen/Qwen2.5-3B-Instruct-GPTQ-Int4
  • awq: Qwen/Qwen1.5-32B-Chat-AWQ
• MoE
  • compressed-tensors
    • nvfp4: RedHatAI/Qwen3.6-35B-A3B-NVFP4
    • mxfp4: doesn't support moe backend selection yet
    • fp8:
      • tensorwise: nm-testing/Qwen3-30B-A3B-Fp8-v1
      • channelwise: nm-testing/Qwen3-30B-A3B-FP8-block
    • int8: RedHatAI/Qwen3-30B-A3B-Instruct-2507-quantized.w8a8
    • wna16: doesn't support moe backend selection yet
    • mxfp8: haven't found a suitable model for testing
  • modelopt
    • nvfp4: nvidia/Qwen3-30B-A3B-NVFP4
    • mxfp8: haven't found a suitable model for testing
  • fp8: Qwen/Qwen3.5-35B-A3B-FP8
  • mxfp4: openai/gpt-oss-20b
  • gptq: Qwen/Qwen3.5-35B-A3B-GPTQ-Int4
  • awq: QuantTrio/Qwen3.6-35B-A3B-AWQ
  • int8: Qwen/Qwen3.5-35B-A3B with --quantization experts_int8

NOTE:

1. When using the humming linear kernel through dense/moe oracles, it will always default to float16/bfloat16 activation unless you specify the activation type via the environment variable VLLM_HUMMING_INPUT_QUANT_CONFIG. In the future, it will default to using the activation format from the checkpoint's config.json.
2. The kernel is disabled by default now. To use it, please add --moe-backend humming for moe kernel and add --linear-backend humming for linear kernel

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[gemini-code-assist]

Code Review

This pull request integrates the Humming MoE backend across various quantization schemes, including FP8, INT8, WNA16, MXFP4, and NVFP4. Key changes include updating the backend oracles to support Humming, implementing weight conversion logic for Humming kernels, and refactoring quantization methods to pass the layer instance to kernel and configuration factories. Additionally, the code now correctly references local_num_experts instead of num_experts in Humming-specific modules. I have no feedback to provide.

[mergify]

Hi @jinzhen-lin, the pre-commit checks have failed. Please run:

uv pip install pre-commit>=4.5.1
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy failing?

mypy is run differently in CI. If the failure is related to this check, please use the following command to run it locally:

# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @jinzhen-lin.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @jinzhen-lin.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork