[Docs] GLM-5.3-Flash cookbook: fixed MTP 5/1/6, EP1 + flashinfer_trtllm on Blackwell - #39213
Merged
Merged
Conversation
…lm on Blackwell - Low Latency cells draft from the MTP head at a fixed depth (5 steps, top-k 1, 6 draft tokens) with natural acceptance; the Playground EAGLE preset matches. - Blackwell FP8 cells run EP1 (the sglang default) with the flashinfer_trtllm MoE runner. Hopper keeps EP8 + deep_gemm: trtllm-gen MoE kernels are SM100-only. - The gb300 Low Latency FP8 + TRT-LLM DSA benchmark row carries the 4x GB300 measurement of this command (sglang b3dc038, 2026-09 perf campaign). Rows measured on the earlier command keep their numbers with a TODO, and cells whose command changed without a re-measurement report "Final Verification In Progress". - The encoder-disaggregation language server and PD notes follow the same recipe.
JustinTong0323
requested review from
sogalin,
wisclmy0611 and
zijiexia
as code owners
September 12, 2026 12:53
Contributor
|
Preview deployment for your docs. Learn more about Mintlify Previews.
💡 Tip: Enable Automations to automatically generate PRs for you. |
github-actions
Bot
added
the
documentation
zijiexia
approved these changes
Sep 12, 2026
zijiexia
deleted the
xinyuan/docs-glm53flash-recipe-fixed-mtp-trtllm
branch
September 12, 2026 19:56
mqhc2020
pushed a commit
to mqhc2020/sglang
that referenced
this pull request
Sep 15, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Update the GLM-5.3-Flash cookbook recipe based on a 4x GB300 benchmarking campaign against vLLM (fixed workload: 80 random-token requests, chat 1000/1000 and summarization 8000/1000 at concurrency 1/16/80; evidence archived in the campaign run directory).
Changes, applied per hardware lane:
--speculative-adaptivefrom the Low Latency recipe on all lanes. The adaptive policy caps draft steps to 3 at batch 8-16 and disables drafting at batch >=64; fixed MTP 5/1/6 recovers most of the gap (chat C80 4293 -> 8976 tok/s on stock source).--ep-sizeflag except on Hopper (EP1 is the sglang default,python/sglang/srt/arg_groups/fields/parallel.py).--moe-runner-backenddeep_gemm -> flashinfer_trtllm on Blackwell lanes only (gb300/gb200/b200/b300). trtllm-gen MoE kernels are SM100-only, so H100/H200 keep EP8 + deep_gemm and only drop adaptive drafting. Divisibility checked: moe_intermediate_size 2048 gives 512 (TP4) / 256 (TP8) per partition, both multiples of 128.in-progressand keep earlier numbers on the remaining cells with TODO notes.Validation:
mint validate,mint broken-links, pre-commit, and jsx module checks all pass. GSM8K on 11 campaign lanes using these flags: 90.75-92.72%, all natural stops. A cookbook-protocol (thinking, 32768 max tokens) GSM8K run on the new command is still pending; remaining TODO rows (DCP4, BF16+TileLang, NVFP4 LL, H100 LL, B200/B300 LL) need re-measurement on the new command.CI States
Latest PR Test (Base): ✅ Run #34694893479
Latest PR Test (Extra): ❌ Run #34694893330
Latest PR Test (AMD ROCm 10): ➖ No AMD PR run found for this commit.