Skip to content

docs(cookbook): fix Qwen3.8 Flash Next H200 MTP verify with BF16 SSM state #36611

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
zijiexia merged 2 commits into from
Aug 27, 2026
Merged

docs(cookbook): fix Qwen3.8 Flash Next H200 MTP verify with BF16 SSM state #36611

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
f9bf98a
docs(cookbook): use Triton GDN verify on SM90
huangzhilin-hzl Aug 27, 2026
f897085
docs(cookbook): keep existing recipe comments
huangzhilin-hzl Aug 27, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions docs/src/snippets/configs/Qwen/qwen3.8-flash-next.jsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -202,6 +202,7 @@ export const config = {
"--chunked-prefill-size 8192",
"--linear-attn-prefill-backend flashinfer",
"--linear-attn-decode-backend flashinfer",
"--linear-attn-verify-backend triton",

@chatgpt-codex-connector chatgpt-codex-connector Bot Aug 27, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Apply Triton verify when Playground enables MTP on H200

When a user starts from either H200 high-throughput recipe and selects NEXTN/MTP in the Playground, the generic speculative handler adds only the --speculative-* flags, while that base recipe retains FlashInfer decode with BF16 SSM state and no verify override. The generated command therefore still auto-selects FlashInfer verify and hits the SM90 initial_state must be float32 failure this change is meant to prevent; add the Triton override for that customization path as well (for example, by carrying it in all H200 bases or applying it conditionally with the MTP option).

AGENTS.md reference: docs/AGENTS.md:L23-L24

Useful? React with 👍 / 👎.

"--mamba-ssm-dtype bfloat16",
"--speculative-algorithm NEXTN",
"--speculative-num-steps 3",
Expand Down Expand Up @@ -443,6 +444,7 @@ export const config = {
"--chunked-prefill-size 8192",
"--linear-attn-prefill-backend flashinfer",
"--linear-attn-decode-backend flashinfer",
"--linear-attn-verify-backend triton",
"--mamba-ssm-dtype bfloat16",
"--speculative-algorithm NEXTN",
"--speculative-num-steps 3",
Expand Down
Loading