Skip to content

[AMD] Fix RoPE cache dtype and diffusion CI failures - #42494

Merged
BBuf merged 4 commits into
sgl-project:mainfrom
BBuf:bbuf/diffusion-rocm-ci-followup-20261004
Oct 4, 2026
Merged

BBuf merged 4 commits into
sgl-project:mainfrom
BBuf:bbuf/diffusion-rocm-ci-followup-20261004

Conversation

@BBuf

@BBuf BBuf commented Oct 4, 2026 •

Copy link
Copy Markdown
Collaborator

ROCm Llama startup fails during graph capture because #41282 keeps the RoPE cache in FP32 while the fallback kernel assumes the query dtype. Let the kernel read FP32 caches and round cos/sin to the query dtype before the existing arithmetic. This preserves the FP32 cache used by QSA without converting the whole cache on every forward.

The diffusion fixes defer quantization imports to break the linear-layer import cycle, enable the existing MiniMax H3 denoising path on HIP, pass the existing HF credential to dual-GPU tests, and install ffmpeg/ffprobe for H3 media validation.

The existing ROCm RoPE test now runs even with AITER enabled, checks FP32-cache outputs against the same-dtype kernel bit for bit, and runs in Stage A so it catches this failure before the longer suites. Correctness, performance and startup thresholds are unchanged.

Validation:

  • Changed-file pre-commit checks passed, including clang-format and test registration.
  • Latest-head AMD Stage A: 9/9 test files passed, including Llama and EAGLE3 (18 tests each, MMLU 0.69 for both) and the ROCm RoPE regression: https://github.com/sgl-project/sglang/actions/runs/37210510566/job/111460697051.
  • AMD kernel unit tests: 7,141 passed, 123 skipped. CUDA kernel unit tests: 13,992 passed, 2,132 skipped. Both tested the merge containing bf11444. Latest-head GitHub lint passed.
  • MI300X cold-L2 diagnostic: 24 cases were bitwise equal to the query-dtype cache path. Each timed call follows a 512 MiB read/write eviction outside event timing. This compares cache specializations in the same candidate, not a full baseline commit. A repeat using identical Q/K addresses and 1,500 interleaved samples per path measured a 1.008 geomean latency ratio (individual medians 0.989–1.036). BF16 shows a small 1–3.6% kernel overhead; this is not a claim of zero performance impact. Diagnostic run: https://github.com/sgl-project/sglang/actions/runs/37212045202/job/111465214461.
  • LTX-2.5 warmup and the remaining diffusion CI results are still under investigation; this PR does not claim all checks have passed.

Follow-up to #42391. The RoPE failure was already present in the base code and is fixed here as part of the ROCm CI follow-up.


CI States

Latest PR Test (Base): ⏳ Run #37210510580
Latest PR Test (Extra): ❌ Run #37210510389
Latest PR Test (AMD ROCm 10): ⏳ Run #37210510566

@github-actions github-actions Bot added amd diffusion SGLang Diffusion labels Oct 4, 2026
@BBuf BBuf added the run-ci CI: run the baseline test suite on this PR label Oct 4, 2026
@BBuf
BBuf marked this pull request as ready for review October 4, 2026 12:10
@BBuf BBuf changed the title [diffusion] Fix ROCm imports and CI media dependencies [AMD] Fix RoPE cache dtype and diffusion CI failures Oct 4, 2026
@BBuf
BBuf merged commit 7a719e9 into sgl-project:main Oct 4, 2026
160 of 184 checks passed
@BBuf
BBuf deleted the bbuf/diffusion-rocm-ci-followup-20261004 branch October 4, 2026 16:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

amd diffusion SGLang Diffusion run-ci CI: run the baseline test suite on this PR sgl-kernel

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant