Skip to content

Revert "[AMD][ROCm] Keep cos_sin_cache fp32 on HIP for fused QSA indexer kernel (#41282)" - #42541

Merged
HaiShaw merged 1 commit into
sgl-project:mainfrom
michaelzhang-ai:revert/41282-hip-cos-sin-fp32
Oct 5, 2026
Merged

HaiShaw merged 1 commit into
sgl-project:mainfrom
michaelzhang-ai:revert/41282-hip-cos-sin-fp32

Conversation

@michaelzhang-ai

@michaelzhang-ai michaelzhang-ai commented Oct 5, 2026 •

Copy link
Copy Markdown
Collaborator

Motivation

#41282 keeps RotaryEmbedding.cos_sin_cache fp32 on HIP. On HIP, every rope without a fused KV-cache write goes through the sgl_kernel.rotary_embedding fallback, which reads the cache as the query dtype, so CUDA graph capture now fails:

File ".../sglang/srt/layers/rotary_embedding/base.py", line 455, in forward_cuda
File ".../sgl_kernel/elementwise.py", line 414, in rotary_embedding
RuntimeError: expected scalar type BFloat16 but found Float
Exception: Capture cuda graph failed: expected scalar type BFloat16 but found Float

This fails stage-a-test-1-gpu-small-amd (test_basic_sanity.py / test_basic_sanity_eagle3.py) on every AMD PR, so all AMD stage B/C lanes are skipped. Examples: run 37180583674, run 37187809051. #41282's own PR Test (AMD) runs stopped at the label gate, so stage-a never ran on it.

Reverting unblocks AMD CI now. The forward fix that keeps the fp32 cache for the QSA indexer is #42483; it can re-land on top of this.

Modifications

git revert f6fcda8827: base.py and mrope.py are byte-identical to before #41282. Nothing has touched either file since.

Effect on the QSA indexer: _use_fused_prep already requires cos_sin_cache.dtype == torch.float32, so on HIP it goes back to the unfused prep path it used before #41282. No functional break, only the fused prep on HIP is lost until #42483 lands.

Checklist

  • Clean revert, no conflicts
  • CI: stage-a-test-1-gpu-small-amd passes

CI States

Latest PR Test (Base): ⏳ Run #37250822113
Latest PR Test (Extra): ❌ Run #37250832996
Latest PR Test (AMD ROCm 10): ❌ Run #37250822085

@michaelzhang-ai michaelzhang-ai added the run-ci CI: run the baseline test suite on this PR label Oct 5, 2026
@michaelzhang-ai

Copy link
Copy Markdown
Collaborator Author

/rerun-failed-ci

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

run-ci CI: run the baseline test suite on this PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants