[Revert] Fix performance regression for GLM-4.7-GPTQ decode and MTP acceptance rate

[aabbccddwasd]

Purpose

Fix performance regressions introduced by two recent commits:

• e0327c9db ([Attention][1/n] Remove usage of deprecated seq_lens_cpu and num_computed_tokens_cpu CommonAttentionMetadata properties #31773) - Decode performance regression
  - 654a71fc3 ([torch.compile] Improve Cold Start for MoEs #32805) - MTP (Multi-Token Prediction) acceptance rate regression

Affected Model: GLM-4.7-GPTQ-INT4-INT8MIX

• Model body: int4/int8 mixed quantization
• MTP module (layer 92): manually configured as unquantized BF16 to increase acceptance rate

Test Plan

Performance Impact

The following benchmark data was collected on 4×RTX PRO 6000 BLACKWELL GPUs:

Configuration	Throughput	Delta
v0.13.0 (no regression) + quantized MTP	90 tps	-
v0.13.0 + BF16 MTP	95 tps	baseline
e0327c9 + BF16 MTP	86 tps	-9.5%
654a71f + BF16 MTP	77 tps	-19.0%
After revert + BF16 MTP	95 tps	✅ recovered

Validation

1. Model: GLM-4.7-GPTQ-INT4-INT8MIX with unquantized MTP (BF16)
2. Hardware: 4×RTX PRO 6000 BLACKWELL
3. Kernel selection verified:
   • Quantized layers: Marlin kernels
   • Unquantized MTP: fused_triton kernels (restored)

Test Result

✅ Performance recovered: Throughput restored from 77 tps back to 95 tps
✅ MTP acceptance rate restored: ~30% improvement
✅ Kernel selection corrected: Unquantized MTP now correctly uses fused_triton kernel

Note: Performance data was measured without optimized Triton configuration files. Additional performance gains are expected with proper Triton optimizations.

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
• (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors.

You ask your reviewers to trigger select CI tests on top of fastcheck CI.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

If you have any questions, please reach out to us on Slack at https://slack.vllm.ai.

🚀

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @aabbccddwasd.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[gemini-code-assist]

Code Review

This pull request correctly reverts two previous commits that introduced performance regressions. The changes are consistent across all affected files, removing the problematic code related to on-device attention metadata computation and a temporary workaround for MoE layer compilation. The detailed description, including benchmark data, clearly justifies the revert. This is a solid contribution to restore performance and improve codebase maintainability. I find no issues with the changes.

[mergify]

Hi @aabbccddwasd, the pre-commit checks have failed. Please run:

uv pip install pre-commit
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy or markdownlint failing?

mypy and markdownlint are run differently in CI. If the failure is related to either of these checks, please use the following commands to run them locally:

# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10
# For markdownlint
pre-commit run --hook-stage manual markdownlint

[aabbccddwasd]

Note on MTP Performance Fix

The MTP acceptance rate regression from commit 654a71f (#32805) has been
addressed by PR #33624, which disables the problematic fast_moe_cold_start
optimization when speculative decoding is enabled.

[mergify]

Hi @aabbccddwasd, the pre-commit checks have failed. Please run:

uv pip install pre-commit
pre-commit install
pre-commit run --all-files

Then, commit the changes and push to your branch.

For future commits, pre-commit will run automatically on changed files before each commit.

Tip

Is mypy or markdownlint failing?

mypy and markdownlint are run differently in CI. If the failure is related to either of these checks, please use the following commands to run them locally:

# For mypy (substitute "3.10" with the failing version if needed)
pre-commit run --hook-stage manual mypy-3.10
# For markdownlint
pre-commit run --hook-stage manual markdownlint

[LucasWilkinson]

please restrict the revert to flashinfer.py, we are still planning to deprecate these properties

[aabbccddwasd]

please restrict the revert to flashinfer.py, we are still planning to deprecate these properties

Fixed

[aabbccddwasd]

@LucasWilkinson

[aabbccddwasd]

@LucasWilkinson I'm sorry but my claude code accidentally disabled auto-merge when I try to reslove conflicts. could you help me to merge it?

[voipmonitor]

1. Model: GLM-4.7-GPTQ-INT4-INT8MIX with unquantized MTP (BF16)

I'm curious - why not NVFP4 version on rtx6000 pro?

[aabbccddwasd]

1. Model: GLM-4.7-GPTQ-INT4-INT8MIX with unquantized MTP (BF16)

I'm curious - why not NVFP4 version on rtx6000 pro?

1.bad performance ( around 60tps ), but may improve after #33417, but I don't think it can beat optimized MarlinGPTQ
2.(important) GLM-4.7-GPTQ-INT4-INT8MIX perserved all attention and shared experts in fp8 precision, I belive it may perform better in long content scenarios like code agent