Temporarily Remove GLM-5.3 Flash prefill CP support - #37484
Merged
Fridge003 merged 1 commit intoSep 2, 2026
Merged
Conversation
Fridge003
requested review from
1am9trash,
BBuf,
Edwardf0t1,
HaiShaw,
YAMY1234,
Ying1123,
alphabetc1,
ch-wan,
hanming-lu,
hnyls2002,
huangtingwei9988,
hubertlu-tw,
hzh0425,
ispobock,
kkHuang-amd,
merrymercy,
rainj-me,
xiezhq-hermann and
yizhang2077
as code owners
September 1, 2026 22:45
Fridge003
changed the title
Fridge003
requested review from
ByronHsu,
Duyi-Wang,
ShangmingCai and
sogalin
as code owners
September 1, 2026 23:01
Collaborator
Author
|
/rerun-test test/registered/models_e2e/test_glm53_flash_b200.py |
Contributor
|
Results for 🚀 |
Collaborator
Author
|
/rerun-test test/registered/models_e2e/test_glm53_flash_b200.py |
Collaborator
Author
|
/rerun-test test/registered/unit/disaggregation/test_nixl_backend_basic.py |
Collaborator
Author
|
/rerun-test test/registered/unit/mem_cache/test_dsa_layer_shard_utils.py |
Contributor
|
Results for 🚀 |
Contributor
|
Results for 🚀 |
Contributor
|
Results for 🚀 |
Remove the GLM-5.3 hybrid Prefill CP and LayerSplit adaptations, keep non-hybrid DSA LayerSplit and NIXL behavior aligned with main, and drop the remaining KPool CP paths.
Fridge003
force-pushed
the
codex/remove-glm53-prefill-cp
branch
from
September 2, 2026 01:14
6e3559a to
e9a1cd7
Compare
Fridge003
merged commit Sep 2, 2026
515e865
into
xinyuan/glm-5.3-flash-support
80 of 89 checks passed
kpham-sgl
added a commit
that referenced
this pull request
Sep 2, 2026
Follow-up to #36507, companion to #37484 (prefill CP). The DSA backend has the decode-CP merge half but not the partition half: nothing under attention/dsa/ calls get_dcp_lens or update_local_kv_lens_for_dcp, unlike every other DCP-capable MLA backend, and no CI case exercises DSA x DCP. Take the merge half back out of the day-0 PR and re-land the axis whole in a PR based on main. Removed: - dsa_backend: _should_return_dsa_dcp_lse and its three call sites (tilelang decode, tilelang verify, trtllm decode), plus the _forward_tilelang return_lse branch that squeezed LSE for dcp_pack_a2a_send. - tilelang_kernel: the return_lse factory flag on sparse_attention_fwd_kernel_v1/v2 and tilelang_sparse_fwd, the LSE prim_func argument, and the unconditional per-call LSE allocation the always-present signature forced on the non-DCP decode path. The file is now byte-identical to its pre-f040cc72e6 state; the NoPE (tail_dim == 0) kernel selection and the topk % 64 relaxation are untouched. - dcp/comm: the Optional prefix_k_pe accommodation for NoPE models. - eager_runner: the target-verify skip in front of prepare_context_parallel_metadata_for_dcp. That gate was model-agnostic and also changed DeepSeek-V2, Kimi-K2.5, Kimi-K3 and Kimi-Linear under DCP + spec; main's behavior is restored for them. - glm5_next: prepare_context_parallel_metadata_for_dcp, the hook a model defines to opt into DCP. - test_dsa_trtllm_fp8_cp: the two _should_return_dsa_dcp_lse cases. The prefill-CP cases for _should_all_gather_dsa_trtllm_fp8_kv stay, matching what #37484 keeps. dcp/comm.py and eager_runner.py are now byte-identical to main. Main's DSA backend never returned LSE, so DSA x DCP is exactly as unsupported as it is on main -- no regression for GLM-5.2, DeepSeek-V3.2 or any other DSA model. Kept deliberately: the `not get_parallel().dcp_enabled` clause in _ingraph_verify_metadata_eligible (a defensive exclusion on a new feature, not DCP support), and the DCP memory-sizing fixes in pool_configurator / kv_cache_configurator, which are model-agnostic and no-ops at attn_dcp_size == 1. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
kpham-sgl
added a commit
that referenced
this pull request
Sep 2, 2026
Follow-up to #36507, companion to #37484 (prefill CP). The DSA backend has the decode-CP merge half but not the partition half: nothing under attention/dsa/ calls get_dcp_lens or update_local_kv_lens_for_dcp, unlike every other DCP-capable MLA backend, and no CI case exercises DSA x DCP. Take the merge half back out of the day-0 PR and re-land the axis whole in a PR based on main. Removed: - dsa_backend: _should_return_dsa_dcp_lse and its three call sites (tilelang decode, tilelang verify, trtllm decode), plus the _forward_tilelang return_lse branch that squeezed LSE for dcp_pack_a2a_send. - tilelang_kernel: the return_lse factory flag on sparse_attention_fwd_kernel_v1/v2 and tilelang_sparse_fwd, the LSE prim_func argument, and the unconditional per-call LSE allocation the always-present signature forced on the non-DCP decode path. The file is now byte-identical to its pre-f040cc72e6 state; the NoPE (tail_dim == 0) kernel selection and the topk % 64 relaxation are untouched. - dcp/comm: the Optional prefix_k_pe accommodation for NoPE models. - eager_runner: the target-verify skip in front of prepare_context_parallel_metadata_for_dcp. That gate was model-agnostic and also changed DeepSeek-V2, Kimi-K2.5, Kimi-K3 and Kimi-Linear under DCP + spec; main's behavior is restored for them. - glm5_next: prepare_context_parallel_metadata_for_dcp, the hook a model defines to opt into DCP. - test_dsa_trtllm_fp8_cp: the two _should_return_dsa_dcp_lse cases. The prefill-CP cases for _should_all_gather_dsa_trtllm_fp8_kv stay, matching what #37484 keeps. dcp/comm.py and eager_runner.py are now byte-identical to main. Main's DSA backend never returned LSE, so DSA x DCP is exactly as unsupported as it is on main -- no regression for GLM-5.2, DeepSeek-V3.2 or any other DSA model. Kept deliberately: the `not get_parallel().dcp_enabled` clause in _ingraph_verify_metadata_eligible (a defensive exclusion on a new feature, not DCP support), and the DCP memory-sizing fixes in pool_configurator / kv_cache_configurator, which are model-agnostic and no-ops at attn_dcp_size == 1. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #36507.
The removed functionality can be added back in a new PR based on main after the day-0 PR is merged.
Summary
Validation
CI States
Latest PR Test (Base): ❌ Run #33578616468
Latest PR Test (Extra): ❌ Run #33578616438
Latest PR Test (AMD ROCm 7.2): ❌ Run #33578616649