Skip to content

[GLM-5.3] Remove experimental DSA metadata optimizations - #38071

Merged
Fridge003 merged 3 commits into
xinyuan/glm-5.3-flash-supportfrom
codex/glm53-remove-experimental-metadata
Sep 4, 2026
Merged

Fridge003 merged 3 commits into
xinyuan/glm-5.3-flash-supportfrom
codex/glm53-remove-experimental-metadata

Conversation

@Fridge003

@Fridge003 Fridge003 commented Sep 4, 2026

Copy link
Copy Markdown
Collaborator

Motivation

Stacked on #36507 (xinyuan/glm-5.3-flash-support, base d769a1d265b8c732b72b946b6ba2a16ae1d85815). Remove the experimental metadata optimizations while retaining GLM-5.3 Flash's No-RoPE, KPool, EAGLE, and TileLang/FA3 fallback support.

This should be added back after clean up.

Modifications

  • Remove the runtime flags and all their feature paths:
    • SGLANG_EXPERIMENTAL_DSA_KPOOL_METADATA_FUSION
    • SGLANG_EXPERIMENTAL_DSA_INGRAPH_VERIFY_METADATA
    • SGLANG_EXPERIMENTAL_DSA_INGRAPH_VERIFY_METADATA_DG_OUT_OF_GRAPH
  • Remove fused KPool metadata generation, draft-step sibling metadata reuse, in-graph verify state/capture/replay, and the out-of-graph DeepGEMM residual helper. KPool decode/verify/draft-extend always use the ordinary pool-aware metadata path.
  • Remove the experimental fused-kernel bounded-scan/launch-preparation changes and preallocated top-k-plan API. Retain the existing non-KPool fused kernels and 64-bit page-table output addressing (the latter prevents large-context offset overflow).
  • Restore ordinary out-of-graph top-k-plan refresh. Preserve per-backend DeepGEMM schedule/KPool refresh, multi-step precompute, EAGLE's verify hook, and static-buffer address stability.
  • Delete the three environment variables outright: no aliases, deprecated-environment entries, or warning-only tests remain.
  • Add a GPU regression test for decode, target-verify, draft-extend-v2 and MTP-precomputed metadata, with actual KPool kernels, DeepGEMM/top-k plans and CUDA graph replay. Checks pool/page boundaries, live tails beyond top-k, request remapping, accept lengths, refreshed schedule/plan contents and stable buffer addresses.

dsa_backend.py is reduced by 480 lines. No changes to the model forward math or No-RoPE/TRTLLM/TileLang/FA3 dispatch paths.

Accuracy Tests

GPU validation at 62f4493387 (before the follow-up deletion of the warning-only registry entries/test), on temporary rx devbox codex-glm53-metadata-trim: 1x NVIDIA H200, driver 580.105.08, PyTorch 2.13.0+cu130.

PYTHONPATH=python python -m pytest -q \
  test/registered/unit/test_environ.py \
  test/registered/unit/layers/attention/test_dsa_kpool_metadata.py \
  test/registered/kernels/test_dsa_kpool_multi_pool.py \
  test/registered/kernels/ops/attention/test_dsa_metadata.py
# 27 passed, 14 subtests passed

PYTHONPATH=python python -m pytest -q \
  test/registered/kernels/ops/attention/test_topk_v2.py -k '8-4096'
# 9 passed, 269 deselected

Additional checks:

  • Ran the new metadata test against unmodified base GLM-5.3-Flash support #36507 on the same H200, with the bare-backend fixture's three former options set to false: both tests/all modes pass. The ordinary-path behavior is retained.
  • Follow-up removal: repository-wide search finds no references to any of the three flags. A stdlib-only import smoke test confirms that setting them neither registers an option nor emits deprecation warnings. The existing environment unit-test file is unchanged relative to GLM-5.3-Flash support #36507.
  • AST comparison confirms No-RoPE/TRTLLM/FA3/TileLang forward methods, the core EAGLE verify hook, KPool init/capture/fallback methods are unchanged. Ordinary precomputed replay is AST-equivalent after inlining the former shared-copy helper, including schedule/top-k/KPool refreshes.
  • Changed-file pre-commit checks and CI registration validation pass.

Full GLM model/GSM8K e2e was not rerun for this removal. No full CI or run-ci was requested; validation was confined to the devbox tests above.

Speed Tests and Profiling

No new speed benchmark. This intentionally removes optional acceleration: workloads that enabled these options lose those optimizations, and reverting the shared bounded-scan/preallocated-plan changes can increase metadata overhead (especially large-context non-KPool workloads). Accuracy thresholds and serving configurations are unchanged.

Checklist

  • Format code with pre-commit.
  • Add focused unit/kernel regression coverage and CI registration.
  • Document the outright removal in this PR; no deprecation compatibility layer.
  • Follow SGLang code style.
  • Full-model accuracy/speed benchmark (not run; focused devbox validation only).

CI States

Latest PR Test (Base): ❌ Run #33928287001
Latest PR Test (Extra): ❌ Run #33928286867
Latest PR Test (AMD ROCm 7.2): ❌ Run #33928287067

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant