[MoE Refactor][4/N] Marlin Fp8 Mk

[robertgshaw2-redhat]

SUMMARY:

• convert marlin fp8 to mk in fp8.py

TEST PLAN:

MODEL := "Qwen/Qwen3-Coder-30B-A3B-Instruct-FP8"
PORT := "8000"
GPUS := "1"

launch_fi:
	VLLM_USE_DEEP_GEMM=0 VLLM_USE_FLASHINFER_MOE_FP8=1 chg run --gpus {{GPUS}} -- vllm serve {{MODEL}} --enforce-eager -tp {{GPUS}} --max-model-len 8192 --trust-remote-code --port {{PORT}}

launch_dg:
	VLLM_USE_DEEP_GEMM=1 VLLM_MOE_USE_DEEP_GEMM=1 chg run --gpus {{GPUS}} -- vllm serve {{MODEL}} --enforce-eager -tp {{GPUS}} --max-model-len 8192 --trust-remote-code --port {{PORT}}

launch_triton:
	VLLM_USE_DEEP_GEMM=0 VLLM_MOE_USE_DEEP_GEMM=0 chg run --gpus {{GPUS}} -- vllm serve {{MODEL}} --enforce-eager -tp {{GPUS}} --max-model-len 8192 --trust-remote-code --port {{PORT}}

launch_marlin:
	VLLM_TEST_FORCE_FP8_MARLIN=1 VLLM_USE_DEEP_GEMM=0 chg run --gpus {{GPUS}} -- vllm serve {{MODEL}} --enforce-eager -tp {{GPUS}} --max-model-len 8192 --trust-remote-code --port {{PORT}}

eval LIMIT:
	lm_eval \
		--model local-completions \
		--tasks gsm8k \
		--model_args "model={{MODEL}},base_url=http://localhost:{{PORT}}/v1/completions,num_concurrent=1000,tokenized_requests=False" \
		--limit {{LIMIT}}

TEST RESULT:

local-completions (model=Qwen/Qwen3-Coder-30B-A3B-Instruct-FP8,base_url=http://localhost:8003/v1/completions,num_concurrent=1000,tokenized_requests=False), gen_kwargs: (None), limit: 100.0, num_fewshot: None, batch_size: 1
|Tasks|Version|     Filter     |n-shot|  Metric   |   |Value|   |Stderr|
|-----|------:|----------------|-----:|-----------|---|----:|---|-----:|
|gsm8k|      3|flexible-extract|     5|exact_match|↑  | 0.91|±  |0.0288|
|     |       |strict-match    |     5|exact_match|↑  | 0.90|±  |0.0302|

[gemini-code-assist]

Code Review

This pull request is part of a larger refactoring effort for Mixture-of-Experts (MoE) layers, specifically focusing on integrating the Marlin FP8 kernel into the modular kernel framework. The changes introduce a new quantization configuration function fp8_w8a16_moe_quant_config and wire it up for the Marlin backend in Fp8MoEMethod. While the overall direction of the refactoring is sound, I've found a critical issue in the implementation of the new configuration function that needs to be addressed.

[robertgshaw2-redhat]

opening to run the ci

[mergify]

Hi @robertgshaw2-redhat, the pre-commit checks have failed. Please run:

uv pip install pre-commit
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy or markdownlint failing?

mypy and markdownlint are run differently in CI. If the failure is related to either of these checks, please use the following commands to run them locally:

# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10
# For markdownlint
pre-commit run --hook-stage manual markdownlint

[mergify]

Hi @robertgshaw2-redhat, the pre-commit checks have failed. Please run:

uv pip install pre-commit
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy or markdownlint failing?

mypy and markdownlint are run differently in CI. If the failure is related to either of these checks, please use the following commands to run them locally:

# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10
# For markdownlint
pre-commit run --hook-stage manual markdownlint

[mergify]

Hi @robertgshaw2-redhat, the pre-commit checks have failed. Please run:

uv pip install pre-commit
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy or markdownlint failing?

mypy and markdownlint are run differently in CI. If the failure is related to either of these checks, please use the following commands to run them locally:

# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10
# For markdownlint
pre-commit run --hook-stage manual markdownlint

[mergify]

Hi @robertgshaw2-redhat, the pre-commit checks have failed. Please run:

uv pip install pre-commit
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy or markdownlint failing?

mypy and markdownlint are run differently in CI. If the failure is related to either of these checks, please use the following commands to run them locally:

# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10
# For markdownlint
pre-commit run --hook-stage manual markdownlint

[robertgshaw2-redhat]

unblocked various MoE tests

[mgoin]

Nice