GLM-5.3-Flash cookbook: HiCache for LL, fusion-flag drop, EAGLE, default-cell numbers, DCP4 overlay - #36544
Merged
JustinTong0323 merged 11 commits intoAug 27, 2026
Conversation
The DSA draft-pool fix shipped in lmsysorg/sglang:glm-5.3-flash (gf13cb6f6a7.fix1): LL with HiCache L1+L2 starts, packs target 11 + draft 1 layers, and passes GSM8K-20 at 95% with a 100% stop rate. The temporary disable is no longer needed on the release image.
JustinTong0323
requested review from
sogalin,
wisclmy0611 and
zijiexia
as code owners
August 26, 2026 17:39
github-actions
Bot
added
the
documentation
Each measured entry now names the release image build (glm-5.3-flash gf13cb6f6a7.fix1, which carries the HiCache and fusion-gate fixes) alongside the rc2 source cut it was measured at.
The release image carries the fusion-gate fix, so the disable flag is no longer required for correctness (validated: the no-flag serve builds the unfused path and matches the flagged serve). The internal fix1 stamp is not meaningful to public readers; the rc2 source anchor stays.
JustinTong0323
changed the title
All speed rows now come from the current image tree (d6ab04b, the canonical support branch): HT numbers are 3-26% above the earlier rc2 rows across both pairings, LL within 1%. GSM8K 97.35%/97.50% unchanged. The HiCache overhead figures remain rc2-cut measurements and say so.
Notes now name the release-image tree (d6ab04b), quote the refreshed deltas (FP8 1.6% on LL, 2.8-5.9% on HT), drop the fusion-flag disclosure (cells no longer carry it; the gate fix makes it unnecessary), and mark the HiCache overhead figures as rc2-cut measurements.
4 tasks
zijiexia
approved these changes
Aug 27, 2026
Upstream folded NEXTN into EAGLE (the enum no longer carries NEXTN; the spelling survives only as a reserved alias). Generated commands now use the canonical EAGLE spelling; runtime behavior is identical at topk=1.
…surements Re-measured with the published cell commands verbatim (no pinned chunk, caps, or fusion flag): BF16 1,161.22 / 2,660.24 / 4,828.33 and FP8 1,227.07 / 2,738.61 / 4,977.02 output tok/s at concurrency 16 / 64 / 256. Automatic sizing admits 434 running requests on GB300, so the 256 row runs at defaults — the old not-reproducible caveat is dropped. FP8 is 2.9-5.7% above BF16 on the curve.
JustinTong0323
changed the title
Defaults are expected to work; no need to say so.
Review follow-ups: fp8-vs-bf16 delta now 2.9-5.7% everywhere (MDX and notes agree), KV pool absolutes are the default-cell values (12.6M/7.0M), the fp8 stop rate is the d6ab gate (100%), the HT accuracy sentence uses the shared-gate phrasing, and the HiCache L1+L2 figures quote the 16k-geometry measurements (within 4-5% of the non-HiCache rows) instead of the chunk-confounded comparison. LL notes disclose the NEXTN/flag measurement spelling.
…ll-enable # Conflicts: # docs/cookbook/autoregressive/GLM/GLM-5.3-Flash.mdx # docs/src/snippets/configs/zai-org/glm-5.3-flash.jsx
Validated on 4x GB300 (TP4/EP4, adaptive MTP 5/1/6, full decode graph, final weights on the d6ab04b tree): 1,680.61 output tok/s with FP8 + TRT-LLM (native LSE), 1,565.8 with BF16 + TileLang (LSE fix in the release image). The option is disabled off GB300, the gb300 LL cell is Verified for either pairing with DCP4, and the HT DCP4 arm stays unverified (target-only DCP not measured).
JustinTong0323
changed the title
saturn-acc
pushed a commit
to saturn-acc/sglang
that referenced
this pull request
Aug 31, 2026
…ult-cell numbers, DCP4 overlay (sgl-project#36544)
nzr-niu
pushed a commit
to nzr-niu/sglang
that referenced
this pull request
Sep 1, 2026
…ult-cell numbers, DCP4 overlay (sgl-project#36544)
5 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Follow-ups to #36519 driven by measured evidence on the final weights:
--disable-shared-experts-fusion: no longer required — the fusion-gate fix makes the no-flag command build the unfused path at EP>1 (validated: 3/3 probes + GSM8K-20 20/20 with 100% stop rate, no flag).Tests
node docs/scripts/check_cookbook_configs.mjs— OK;mint validate— passed.CI States
Latest PR Test (Base): ✅ Run #33099967517
Latest PR Test (Extra): ✅ Run #33100069159
Latest PR Test (AMD ROCm 7.2): ➖ No AMD PR run found for this commit.