Skip to content

Fix CUTLASS DSL 4.6.0 compatibility - #8

Merged
zyongye merged 1 commit into
vllm-project:devfrom
wzhao18:dev
Jul 20, 2026
Merged

zyongye merged 1 commit into
vllm-project:devfrom
wzhao18:dev

Conversation

@wzhao18

@wzhao18 wzhao18 commented Jul 18, 2026

Copy link
Copy Markdown

vllm bumped cute dsl version to 4.6.0, this PR fixes various compatibility issues introduced by the version bump.

Validated that all unit tests are now passing:

pytest -v -s python/fmha_sm100/cute/test_sparse_atten.py
pytest -v -s python/fmha_sm100/cute/test_fp4_indexer.py
pytest -v -s tests

E2E Minimax:

vllm serve nvidia/MiniMax-M3-NVFP4 \
  --tensor-parallel-size=8 \
  --attention_config.indexer_kv_dtype fp8 \
  --gpu-memory-utilization 0.95 \
  --max-model-len 2304 \
  --kv-cache-dtype fp8 \
  --block-size 128 \
  --language-model-only \
  --max-cudagraph-capture-size 2048 \
  --max-num-batched-tokens 2048 \
  --stream-interval 20 \
  --no-enable-prefix-caching \
  --trust-remote-code \
  --no-enable-flashinfer-autotune

|Tasks|Version|     Filter     |n-shot|  Metric   |   |Value |   |Stderr|
|-----|------:|----------------|-----:|-----------|---|-----:|---|-----:|
|gsm8k|      3|flexible-extract|     5|exact_match|↑  |0.8992|±  |0.0083|
|     |       |strict-match    |     5|exact_match|↑  |0.8984|±  |0.0083|

Co-authored-by: OpenAI Codex <codex@openai.com>
Signed-off-by: wzhao18 <wzhao18.sz@gmail.com>
Comment thread requirements.txt
nvidia-cutlass-dsl>=4.4.1
quack-kernels>=0.2.10
nvidia-cutlass-dsl>=4.6.0
quack-kernels>=0.6.1

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently vllm requires quack-kernels>=0.4.0. In that case do we need to align vllm requirements with MSA requirements?

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can in this pr vllm-project/vllm#49016 to together update.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@arpera Should we pin to an exact version? With quack-kernels>=xxx I believe it will just install the latest version.

@lengrongfu Sounds good. We can update the version there when we update the MSA commit hash. Thanks!

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think there is no need to pin an exact version here since we only need at least 0.6.1. Also, leave please a comment in the files where you update version of quack-kernels to know exactly why we need at least 0.6.1 installed.

@ZJY0516

ZJY0516 commented Jul 19, 2026

Copy link
Copy Markdown
Member

cc @yongye

@zyongye
zyongye merged commit 890aaa1 into vllm-project:dev Jul 20, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants