diff --git a/configs/nvidia-master.yaml b/configs/nvidia-master.yaml index 0a6c86df6d..b42b8658c7 100644 --- a/configs/nvidia-master.yaml +++ b/configs/nvidia-master.yaml @@ -9007,9 +9007,10 @@ glm5.2-fp4-b300-sglang-agentic-mtp: # head (num-steps 3, eagle-topk 1, 4 draft tokens = 3 speculative tokens) with # acceptance pinned to the golden AL 2.99 (golden_al_distribution/glm5.2_mtp.yaml, # thinking_on, K=3) through SGLANG_SIMULATE_ACC_*. SGLang v0.5.16 is the first -# release that reads SGLANG_SIMULATE_ACC_TOKEN_MODE. The pinned 2026-09-01 -# nightly includes FlashInfer 0.6.18's BF16 TRTLLM MoE allocation fix, which is -# required by GLM-5.2's unquantized EAGLE draft head at CUDA-graph capture. +# release that reads SGLANG_SIMULATE_ACC_TOKEN_MODE. The pinned 2026-09-08 +# nightly carries forward FlashInfer 0.6.18's BF16 TRTLLM MoE allocation fix +# (first picked up via the 2026-09-01 nightly), which is required by GLM-5.2's +# unquantized EAGLE draft head at CUDA-graph capture. # # Same single arm and concurrency grid as glm5.2-fp4-b300-sglang-agentic-mtp: # cookbook low-latency TP8 with HiCache host-DRAM offload, conc @@ -9017,7 +9018,7 @@ glm5.2-fp4-b300-sglang-agentic-mtp: # memory as well as comparability -- the ~433 GB NVFP4 # checkpoint needs ~54 GB/GPU across 8 B200s and does not fit below 8. glm5.2-fp4-b200-sglang-agentic-mtp: - image: lmsysorg/sglang:nightly-dev-cu13-20260901-07c8f729 + image: lmsysorg/sglang:nightly-dev-cu13-20260908-20ca564b model: nvidia/GLM-5.2-NVFP4 model-prefix: glm5.2 runner: cluster:b200-nscale diff --git a/perf-changelog.yaml b/perf-changelog.yaml index 6b249ea15e..80b1bc543d 100644 --- a/perf-changelog.yaml +++ b/perf-changelog.yaml @@ -6921,3 +6921,12 @@ - "Pick up the latest automatic ROCm DeepSeek-V4 optimizations, including fused mHC post/pre plus RMSNorm, gfx950 C4A top-k dispatch, fused C4 compressor GEMMs, fused SWA q/kv RMSNorm plus q FP8 quantization, and medium-batch cooperative top-k tuning." - "Keep the existing VLLM_ROCM_USE_AITER=1, VLLM_ROCM_USE_AITER_MOE=1, and --moe-backend aiter settings, and explicitly add VLLM_ROCM_USE_AITER_FUSION_SHARED_EXPERTS=1 plus VLLM_ROCM_QUICK_REDUCE_QUANTIZATION=INT4 to both STP and MTP paths. The current checkpoint's shared-expert path does not satisfy the latest vLLM fusion conditions, so that fusion flag self-disables while preserving recipe parity." pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2792 + +- config-keys: + - glm5.2-fp4-b200-sglang-agentic-mtp + scenario-type: + - agentic-coding + description: + - "Update SGLang image from lmsysorg/sglang:nightly-dev-cu13-20260901-07c8f729 (2026-09-01 cu13 dev nightly) to lmsysorg/sglang:nightly-dev-cu13-20260907-30705c00 (2026-09-07 cu13 dev nightly, digest sha256:19b8fa1223cc339c1eae7a5b703f1a8c2543b5b119155bf3d7efaef18f77f007, tag commit sgl-project/sglang@30705c00). Docker Hub last pushed the tag at 2026-09-07T01:43:42Z. The newer nightly carries forward the FlashInfer 0.6.18 BF16 TRTLLM MoE allocation fix that the 2026-09-01 pin was taken for. Engine flags, EAGLE MTP settings, golden acceptance length 2.99, HiCache sizing, and the TP8 conc [1, 4, 8, 12, 16] grid are unchanged; the same tag the B200 Qwen3.5 SGLang AgentX recipes moved to in #2861/#2862 and the new FP8 sibling in #2863 starts on." + - "Re-pin from lmsysorg/sglang:nightly-dev-cu13-20260907-30705c00 to lmsysorg/sglang:nightly-dev-cu13-20260908-20ca564b (2026-09-08 cu13 dev nightly, digest sha256:9a352a35c973a2357372e85f3bcb5388b6b3c46c1329165987260f3b089647dc; Docker Hub last pushed 2026-09-08T01:40:59Z, tag commit sgl-project/sglang@20ca564b). The 2026-09-07 build carries an unguarded kv_index_translator.translate_dcp_read_ids call on the DSA fp8 KV read path that the EAGLE draft backend never binds, so GLM-5.2 MTP runs crash intermittently with AttributeError (observed on the MI355X FP8 sibling in run 34173459478 after 74 minutes of serving). sgl-project/sglang#38318 (merged 2026-09-07T20:03Z) adds the None guard and is six commits behind 20ca564b." + pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2864