Skip to content

[AMD] Disable DSA dense decode graphs with DP attention - #36079

Open
taking-lying-flat wants to merge 1 commit into
sgl-project:mainfrom
taking-lying-flat:fix-dsa-dual-graph-dp-attention
Open

taking-lying-flat wants to merge 1 commit into
sgl-project:mainfrom
taking-lying-flat:fix-dsa-dual-graph-dp-attention

Conversation

@taking-lying-flat

@taking-lying-flat taking-lying-flat commented Aug 23, 2026

Copy link
Copy Markdown

Summary

  • Do not enable the HIP DSA dense/sparse dual decode graphs when DP attention is active.
  • Keep the existing dual-graph optimization enabled for HIP DSA deployments without DP attention.
  • DP-attention deployments still use decode CUDA graphs; they capture the existing full-indexer graph, which is valid for all KV lengths.

Why

#31324 automatically enabled a dense k-only graph for DSA decode when max_kv_len <= index_topk. As isolated in #36071, replaying that graph on GLM-5.2 with DP attention can produce all-NaN logits on every attention-TP rank. The same topology is clean with eager decode and with the full-indexer graph, including when MTP and overlap scheduling are disabled.

This change applies the narrow correctness guard requested in #36071. It does not attempt to claim or mask a root-cause fix for the HIP k-only graph; that path should be re-enabled with DP attention only after its capture/replay buffer and stream contracts are qualified on the affected hardware.

Scope

  • HIP + DSA + DP attention: use the correct-for-all full-indexer decode graph.
  • HIP + DSA without DP attention: unchanged; retain dense/sparse dual graphs.
  • CUDA and non-DSA models: unchanged.

Fixes #36071

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug][AMD] Custom AR graph registration corrupts DSA DP-attention graph-skew replay

1 participant