[ROCm][DSv4.1][Perf] Fuse the inverse RoPE into the sparse decode reduce by Fangzhou-Ai · Pull Request #57435 · vllm-project/vllm

Fangzhou-Ai · 2026-09-17T19:08:40Z

Purpose

The ROCm sparse decode writes the attention output twice. The reduce kernel
combines the split-K partials and stores [T, H, 512], and then
_inverse_rope_gptj_kernel reads that whole tensor back, rotates the trailing
64 rope lanes and writes it out again — a full round trip per layer, for an
epilogue on data the reduce already held in registers.

From a 10-step decode trace on MI355X (TP4, rank 0):

kernel	total	calls	share of kernel time
`_sparse_attn_decode_gfx950_partial_kernel`	4707.3 us	430	3.24%
`_sparse_attn_decode_reduce_kernel`	2214.2 us	430	1.52%
`_inverse_rope_gptj_kernel`	1965.5 us	430	1.35%

430 = 43 layers x 10 steps: every layer pays it, sliding-window and MTP layers
included.

FlashMLA's mega-attention kernel already folds this rotation and the FP8 cast
into its epilogue on SM100 — flash_mla_mega_attn.py describes exactly that —
and the layer interface for it is platform-generic (_alloc_attn_out /
_o_proj, accepts_unnormed_unroped_query). The ROCm Triton path just never
took it.

What changes

The reduce gained an optional inverse-RoPE epilogue. Reshaping the combined row
into adjacent pairs puts the rope lanes in the last 32 of 256 pairs, so giving
the NoPE pairs cos=1 and sin=0 lets one expression cover the whole row with no
branch and no second pass:

even, odd = tl.split(tl.reshape(out, (BLOCK_H, COMB_DIM // 2, 2)))
out = tl.reshape(tl.join(even * cos + odd * sin, odd * cos - even * sin), ...)

Which rows still owe a rotation depends on the batch: the reduce rotates the
decode rows it produces, prefill rows have no such epilogue, and a decode path
that does not fuse (for instance the opt-in HIP kernel in #57282) rotates
nothing. rocm_sparse_attn_decode reports how many leading rows it rotated and
forward_mqa pays the standalone pass for the rest, in place. On a decode-only
step that leaves nothing to do, so the kernel does not launch at all rather
than being replaced by a copy.

That bookkeeping belongs in the eager attention segment specifically, not in
_o_proj. An earlier revision of this PR handed the row count to _o_proj,
which runs inside the compiled region; since vLLM compiles with
skip_all_guards_unsafe, the Python int froze at its trace-time value and rows
ended up rotated zero or two times. GSM8K fell to 0.087 against a 0.902
baseline with fluent output and no error of any kind. The lane arithmetic was
correct throughout, so no kernel-level test caught it -- only the eval did.

Two side effects worth noting: the rotation now happens on the fp32
accumulator rather than the already-rounded bf16 output, dropping a rounding
step, and the NoPE lanes come through bit-exact (x1.0 + 0.0).

Expected effect: ~196 us and 43 launches per step removed, ~1.35% of decode
GPU time on DeepSeek-V4.1-Flash.

Not a duplicate

[ROCm][DSV4][Perf] Use FP8 WO_A output projection #54894 fuses inverse RoPE with group quantization via AITER's
inverse_rope_group_quant — different model (deepseek_v4), different
mechanism, and it keeps the rotation as its own pass. This removes the pass.
The two are compatible in direction: if the V4.1 ROCm path later adopts an
FP8 wo_a, the natural next step is emitting the FP8 cast from this same
epilogue, which is what the SM100 kernel does.
[ROCm] Add head-fused HIP sparse MLA decode kernel for gfx950 #57282 adds an opt-in HIP decode kernel for a specific batch range and falls
through to this Triton path otherwise; the two compose, and the rotated-row
count above is what keeps them honest.
[ROCm][DSv4.1][Perf] One-pass dequantize and a gather-sized grid for the compressed K cache #56720 and [ROCm][DSv4.1] Fix TileLang mHC pre on 64-wide wavefronts and use the fused kernels on the AMD path #56342 touch other kernels in the same file, not the reduce.

Test results

Unit tests, on the branch, MI355X:

$ pytest tests/kernels/attention/test_rocm_triton_attn_dsv4.py \
         tests/kernels/attention/test_dsv41_rocm_fused_inv_rope.py -q
69 passed in 38.64s

Kernel equivalence, MI355X: running the real decode kernels fused, then
unfused followed by _fused_inverse_rope_gptj, on identical inputs. NoPE lanes
are bit-exact and rope lanes agree to bf16 rounding at every shape tried --
(queries, heads) of (2, 3), (1, 128), (8, 128), (32, 128):

	max abs diff
NoPE lanes	0.0 (32 queries: 4.9e-4)
rope lanes	<= 2.0e-3, against \|rope\| up to 0.29

Decode capture, DeepSeek-V4.1-Flash TP4 on MI355X, 10 engine steps at
concurrency 1, MTP spec decode. Per-step kernel time, mean over the four TP
ranks:

kernel	main	this PR
`_inverse_rope_gptj_kernel`	201.6 us	0.0 us (430 launches -> 0)
`_sparse_attn_decode_reduce_kernel`	218.7 us	227.0 us

The pass disappears on decode-only steps and the reduce absorbs it for 8.3 us,
so 193 us per step net, against the 196 us the trace analysis predicted.

Accuracy, GSM8K 5-shot, greedy, concurrency 32 score remains the same

AI assistance was used to write this change. I have reviewed every line, ran
the tests and evals above myself, and am responsible for it.

The decode reduce stores its combined output only for the standalone inverse RoPE kernel to read the whole [T, H, 512] tensor back, rotate the trailing rope lanes and write it out again -- a full round trip per layer for an epilogue on data that was already in registers. Rotate in the reduce instead. Reshaping the row into adjacent pairs puts the rope lanes in the last 32 of 256 pairs, so feeding cos=1/sin=0 to the NoPE pairs covers the whole row with one expression. _o_proj then rotates only the rows the kernel reports it did not, which on a decode-only step is none of them, and does it in place. FlashMLA's mega-attention kernel already folds this rotation (and the FP8 cast) into its epilogue on SM100; this gives the ROCm Triton path the same shape. Co-authored-by: Cursor Agent Signed-off-by: Fangzhou Ai <fangzhou.ai@amd.com>

The decode reduce rotates the rows it produces, so only prefill rows still owe an inverse RoPE. Handing that row count to _o_proj was unsound: o_proj runs inside the compiled region, and vLLM compiles with skip_all_guards_unsafe, so the value froze at its trace-time value and rows ended up rotated zero or two times. GSM8K fell to 0.087 against a 0.902 baseline, with fluent output and no error. Resolving the split in the eager attention segment keeps it correct per step and still drops the separate pass entirely on decode-only steps. Co-authored-by: Cursor <cursoragent@cursor.com> Signed-off-by: Fangzhou Ai <fangzhou.ai@amd.com> Co-authored-by: Cursor <cursoragent@cursor.com>

claude

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

AndreasKaratzas · 2026-09-18T01:42:51Z

/ci run

github-actions · 2026-09-18T01:43:10Z

✅ Triggered Buildkite CI #89754 for commit 10a6d935cf9f.

tjtanaa · 2026-09-19T04:34:05Z


 logger = init_logger(__name__)

-FP8_DTYPE = current_platform.fp8_dtype()


current_platform.fp8_dtype() is cached as it will invoke unnecessary host ops to retrieve this metadata during prefill. Can we revert this? Is there any issue with using FP8_DTYPE?

Reverted in 61abd6c. There was no issue with FP8_DTYPE — the per-call form was an unintended rebase artifact, and your point about the host query in prefill is exactly why it should stay cached. All eight call sites are back on the module constant.

tjtanaa · 2026-09-19T10:11:59Z

        BLOCK_H=1,
        NUM_SPLITS=num_splits,
        SPLITS_PAD=1 << (num_splits - 1).bit_length(),
+        FUSE_INV_ROPE=False,


Let's add a coverage for FUSE_INV_ROPE=true

Added in 61abd6c as test_sparse_attn_decode_reduce_inverse_rope_epilogue, which drives _launch_sparse_decode_reduce with real positions and a cos_sin_cache so FUSE_INV_ROPE=True runs, parametrized over adaptive_splits in [False, True]. It asserts the fused result matches the unfused reduce followed by the standalone rotation, that NoPE lanes match the unrotated output bit for bit (this pins the pair split to nope_head_dim // 2), and that RoPE lanes do not — so an epilogue that quietly did nothing fails. pytest tests/kernels/attention/test_rocm_triton_attn_dsv4.py -q → 70 passed on MI355X.

tjtanaa · 2026-09-19T10:12:55Z

@Fangzhou-Ai could you attach the commands and lm eval score of the models conc 256 numshot 30? Thanks.

Review feedback on vllm-project#57435. The test added with the fusion compared two torch reimplementations of the rotation against each other, so it proved the index algebra self-consistent and nothing about the kernel that ships. Meanwhile the reduce kernel's own harness only ever passed FUSE_INV_ROPE=False, leaving the new epilogue with no coverage at all. Drive the epilogue through `_sparse_attn_decode_reduce_kernel` instead: reduce once with the epilogue off and rotate with the standalone `_fused_inverse_rope_gptj`, reduce again with it on, and require the two to agree. The NoPE lanes must match the unrotated output bit for bit, which is what pins the pair split to `nope_head_dim // 2`, and the RoPE lanes must not, which an epilogue that quietly did nothing would fail. Both arms run against adaptive and fixed splits. The torch-only file goes away. Also restore the module-level `FP8_DTYPE`: replacing it with per-call `current_platform.fp8_dtype()` put a host-side platform query back in the prefill path, and it was never part of this change. Tested on MI355X (gfx950), ROCm 7.2.3: pytest tests/kernels/attention/test_rocm_triton_attn_dsv4.py -q 70 passed Signed-off-by: Fangzhou Ai <fangzhou.ai@amd.com> Co-authored-by: Cursor <cursoragent@cursor.com>

Fangzhou-Ai · 2026-09-21T03:02:51Z

Both review points are addressed in 61abd6c, and the lm_eval numbers are below.

FP8_DTYPE — restored to the module-level constant. Replacing it with per-call current_platform.fp8_dtype() put a host-side platform query back in the prefill path and was never part of this change.

Tests — the fusion's original test compared two torch reimplementations of the rotation, so it proved the index algebra self-consistent and nothing about the shipped kernel, while the reduce kernel's own harness only ever passed FUSE_INV_ROPE=False. The epilogue now runs through _sparse_attn_decode_reduce_kernel: reduce with the epilogue off plus the standalone _fused_inverse_rope_gptj, reduce again with it on, and the two must agree. NoPE lanes must match the unrotated output bit for bit (this pins the pair split to nope_head_dim // 2) and RoPE lanes must not (an epilogue that quietly did nothing would fail). Both adaptive and fixed splits are covered; the torch-only file is deleted. pytest tests/kernels/attention/test_rocm_triton_attn_dsv4.py -q → 70 passed on MI355X (gfx950, ROCm 7.2.3).

GSM8K, conc 256, 30-shot, 1319 questions, this branch vs. the upstream main it merges (2c88fb1), one TP=4 server each on the same node:

	strict-match	flexible-extract
this PR	0.9735 ± 0.0044	0.9727 ± 0.0045
main	0.9735 ± 0.0044	0.9727 ± 0.0045

Both score 1283/1319. 447 generations are textually identical and 10 questions flip, 5 in each direction — bf16-level divergence with no accuracy change. For reference, InferenceX CI for this recipe reports 0.9689 at 5-shot/conc32.

Server and client match the InferenceX MI355X DSv4.1-Flash recipe, including dspark MTP with real block rejection:

vllm serve deepseek-ai/DeepSeek-V4.1-Flash --port 8151 --tensor-parallel-size 4 \
  --language-model-only --tokenizer-mode deepseek_v41 \
  --tool-call-parser deepseek_v41 --enable-auto-tool-choice \
  --reasoning-parser deepseek_v41 --moe-backend aiter \
  --gpu-memory-utilization 0.9 --max-model-len 1048576 \
  --max-cudagraph-capture-size 2048 --max-num-batched-tokens 16384 \
  --speculative-config '{"method":"dspark","num_speculative_tokens":5,"draft_sample_method":"probabilistic","rejection_sample_method":"block","enable_adaptive_verification":false}'

python -m lm_eval --model local-chat-completions --apply_chat_template \
  --tasks infx/evals/gsm8k.yaml --num_fewshot 30 --log_samples \
  --model_args model=deepseek-ai/DeepSeek-V4.1-Flash,base_url=http://127.0.0.1:8151/v1/chat/completions,api_key=EMPTY,eos_string='</s>',max_retries=5,num_concurrent=256,timeout=1800,tokenized_requests=False,max_length=16384 \
  --gen_kwargs max_tokens=12288,temperature=0,top_p=1

lm_eval runs with InferenceX's lm_eval_sitecustomize.py patch, which reads reasoning_content when a reasoning model returns empty content.

AI assistance was used for this work.

github-actions · 2026-09-21T03:06:45Z

✅ @Fangzhou-Ai, CI is now available for this PR.

/ci run starts upstream CI; /amd-ci run starts AMD CI only.
Your branch must contain every commit currently on its upstream target branch. Merge or rebase onto the latest target branch, then rerun the command. Append --allow-stale to a run command to test an outdated branch at your own risk.
/ci retry retries failed jobs in the CI build for the current PR head. If the current head has no CI build, it starts a new CI build for the current head containing only jobs that failed in the latest earlier CI build for this PR.
/amd-ci retry retries failed jobs in AMD CI for the current PR head. Use /amd-ci run when the current head has no AMD CI build.
/ci cancel cancels scheduled or running CI builds for this PR branch; /amd-ci cancel does the same for AMD CI only.

Fangzhou-Ai · 2026-09-21T03:06:47Z

/ci run

github-actions · 2026-09-21T03:07:05Z

✅ Triggered Buildkite CI #90123 for commit 6a33016ddb0b.

tjtanaa · 2026-09-22T06:10:13Z

/ci run

github-actions · 2026-09-22T06:10:32Z

✅ Triggered Buildkite CI #90378 for commit b1465b142c4c.

DeepSeek-V4 shares the ROCm sparse decode kernels with V4.1, so the reduce epilogue added for V4.1 in vllm-project#57435 applies here unchanged: pass the positions and the cos/sin cache into the decode, and the rotation happens on the fp32 accumulators instead of a separate pass over the bf16 output. Which rows still owe a rotation is batch-dependent, so forward_mqa settles it from the count the decode reports rather than from _o_proj. Putting that value anywhere the compiled region can read it is unsound: vLLM compiles with skip_all_guards_unsafe, so it would freeze at its trace-time value. Unlike V4.1, V4 also has an fp8 wo_a path whose inverse_rope_group_quant rotates while it quantizes. Fusing there would rotate twice, so the fusion is gated on that path being inactive, which is exactly when _o_proj takes the BF16 einsum branch that stops rotating. Co-authored-by: Cursor <cursoragent@cursor.com> Signed-off-by: Fangzhou Ai <fangzhou.ai@amd.com>

Resolve the sparse decode signature overlap with vllm-project#57435, which fused the inverse RoPE into the decode reduce while this branch was adding the direct top-k metadata arguments. Keep both parameter sets on rocm_sparse_attn_decode, adopt the upstream int return that reports the rotated row count, and forward inv_rope_positions and inv_rope_cos_sin_cache alongside the direct top-k gating of extra_cache_nan_free and adaptive_splits. Validation: Ruff lint and format, byte-compile, and buildkite YAML parse passed, and the branch diff against main is unchanged at seven files. GPU kernel tests were not rerun. Signed-off-by: Liuyinfeng01 <yinfeliu@amd.com>

Repin dsv41flash-fp4-mi355x-vllm-agentic-dspark from nightly-rocm100-3df4ae15 to nightly-rocm100-7f1a5398 and re-measure all 16 points. Nothing else changes: the TP=4 and TP=2 rows keep concurrency 1-128 and the recipe script is untouched. The pin is mainly for vllm-project/vllm#57435, which fuses the inverse RoPE into the sparse decode reduce on the DSv4.1 ROCm path. It merged 2026-09-22, after the old pin was cut, so no point on the current curve carries it; the new pin is 176 commits ahead and contains it. 将 dsv41flash-fp4-mi355x-vllm-agentic-dspark 的镜像由 nightly-rocm100-3df4ae15 更新为 nightly-rocm100-7f1a5398，并重新测量全部 16 个数据点。其余内容不变：TP=4 与 TP=2 两行保持并发 1-128，recipe 脚本未作改动。此次固定主要是为了 vllm-project/vllm#57435：它在 DSv4.1 的 ROCm 路径上将 inverse RoPE 融合进 sparse decode 的 reduce。该 PR 于 2026-09-22 合并，晚于旧镜像的切出时间，因此当前曲线上的任何数据点都不包含该改动；新镜像领先 176 个提交，已包含该改动。 Co-authored-by: Cursor <cursoragent@cursor.com>

perf-changelog PR3404 description (dsv41flash-fp4-mi355x-vllm-agentic-dspark), lines 8849-8851: The pin is mainly for vllm-project/vllm#57435, which fuses the inverse RoPE into the sparse decode reduce on the DSv4.1 ROCm path. It merged on 2026-09-22, after nightly-rocm100-3df4ae15 was cut on 2026-09-21, so no point on the current curve carries it. The new pin is 176 commits ahead of the old one and carries the change. 在 ROCm 10.0 nightly nightly-rocm100-7f1a5398e9610d96c473931a26c0e12bbe0d0423（发布于 2026-09-24T05:45:09Z，摘要 sha256:5df261803a5b4e8c4d6bbe58a32c9cd597140b6fdfa7358c0ef0d0211c33ab32）上重新扫描 MI355X DeepSeek-V4.1-Flash vLLM AgentX 臂，替换 nightly-rocm100-3df4ae15。本次仅更换镜像：TP=4 与 TP=2 两行保持并发 1、2、4、8、16、32、64 与 128，其余 recipe 设置全部不变，包括按 TP 与并发决定的 Engram 放置、prefill 分块与序列上限阶梯、图捕获，以及吞吐用黄金接受长度 3.51、评测用真实分块验证的五 token DSpark。此次固定主要是为了 vllm-project/vllm#57435：它在 DSv4.1 的 ROCm 路径上将 inverse RoPE 融合进 sparse decode 的 reduce。该 PR 于 2026-09-22 合并，晚于 2026-09-21 切出的 nightly-rocm100-3df4ae15，因此当前曲线上的任何数据点都不包含该改动。新镜像比旧镜像领先 176 个提交，已包含该改动。 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

…ly / 在 2026-09-24 的 ROCm 10.0 nightly 上重新扫描 MI355X (#3404) * config(dsv41flash): re-sweep MI355X on the 2026-09-24 ROCm 10.0 nightly Repin dsv41flash-fp4-mi355x-vllm-agentic-dspark from nightly-rocm100-3df4ae15 to nightly-rocm100-7f1a5398 and re-measure all 16 points. Nothing else changes: the TP=4 and TP=2 rows keep concurrency 1-128 and the recipe script is untouched. The pin is mainly for vllm-project/vllm#57435, which fuses the inverse RoPE into the sparse decode reduce on the DSv4.1 ROCm path. It merged 2026-09-22, after the old pin was cut, so no point on the current curve carries it; the new pin is 176 commits ahead and contains it. 将 dsv41flash-fp4-mi355x-vllm-agentic-dspark 的镜像由 nightly-rocm100-3df4ae15 更新为 nightly-rocm100-7f1a5398，并重新测量全部 16 个数据点。其余内容不变：TP=4 与 TP=2 两行保持并发 1-128，recipe 脚本未作改动。此次固定主要是为了 vllm-project/vllm#57435：它在 DSv4.1 的 ROCm 路径上将 inverse RoPE 融合进 sparse decode 的 reduce。该 PR 于 2026-09-22 合并，晚于旧镜像的切出时间，因此当前曲线上的任何数据点都不包含该改动；新镜像领先 176 个提交，已包含该改动。 Co-authored-by: Cursor <cursoragent@cursor.com> * config(dsv41flash): document MI355X re-sweep pin rationale (PR3404) perf-changelog PR3404 description (dsv41flash-fp4-mi355x-vllm-agentic-dspark), lines 8849-8851: The pin is mainly for vllm-project/vllm#57435, which fuses the inverse RoPE into the sparse decode reduce on the DSv4.1 ROCm path. It merged on 2026-09-22, after nightly-rocm100-3df4ae15 was cut on 2026-09-21, so no point on the current curve carries it. The new pin is 176 commits ahead of the old one and carries the change. 在 ROCm 10.0 nightly nightly-rocm100-7f1a5398e9610d96c473931a26c0e12bbe0d0423（发布于 2026-09-24T05:45:09Z，摘要 sha256:5df261803a5b4e8c4d6bbe58a32c9cd597140b6fdfa7358c0ef0d0211c33ab32）上重新扫描 MI355X DeepSeek-V4.1-Flash vLLM AgentX 臂，替换 nightly-rocm100-3df4ae15。本次仅更换镜像：TP=4 与 TP=2 两行保持并发 1、2、4、8、16、32、64 与 128，其余 recipe 设置全部不变，包括按 TP 与并发决定的 Engram 放置、prefill 分块与序列上限阶梯、图捕获，以及吞吐用黄金接受长度 3.51、评测用真实分块验证的五 token DSpark。此次固定主要是为了 vllm-project/vllm#57435：它在 DSv4.1 的 ROCm 路径上将 inverse RoPE 融合进 sparse decode 的 reduce。该 PR 于 2026-09-22 合并，晚于 2026-09-21 切出的 nightly-rocm100-3df4ae15，因此当前曲线上的任何数据点都不包含该改动。新镜像比旧镜像领先 176 个提交，已包含该改动。 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: seungrokj <144636725+seungrokj@users.noreply.github.com> Co-authored-by: seungrokj <seungrok.jung@amd.com> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

mergify Bot added deepseek Related to DeepSeek models DSv4.1 Related to DeepSeek-V4.1 models rocm Related to AMD ROCm labels Sep 17, 2026

github-project-automation Bot added this to AMD Sep 17, 2026

github-project-automation Bot moved this to Todo in AMD Sep 17, 2026

Fangzhou-Ai force-pushed the rocm-dsv41-fuse-inv-rope branch 2 times, most recently from f63d46f to bde725e Compare September 17, 2026 22:06

Fangzhou-Ai force-pushed the rocm-dsv41-fuse-inv-rope branch from bde725e to 800e461 Compare September 17, 2026 22:10

This was referenced Sep 17, 2026

[ROCm][DSv4.1][Perf] Reuse the decode topk ragged metadata across layers #57434

Merged

[ROCm][DSv4][Perf] Fuse the inverse RoPE into the sparse decode reduce #57451

Merged

Fangzhou-Ai marked this pull request as ready for review September 18, 2026 01:40

Fangzhou-Ai requested review from AndreasKaratzas, dllehr-amd, hongxiayang and tjtanaa as code owners September 18, 2026 01:40

claude Bot reviewed Sep 18, 2026

View reviewed changes

Merge branch 'main' into rocm-dsv41-fuse-inv-rope

10a6d93

tjtanaa reviewed Sep 19, 2026

View reviewed changes

tjtanaa requested changes Sep 19, 2026

View reviewed changes

Comment thread tests/kernels/attention/test_dsv41_rocm_fused_inv_rope.py Outdated

tjtanaa reviewed Sep 19, 2026

View reviewed changes

Fangzhou-Ai added the ready ONLY add when PR is ready to merge/full CI is needed label Sep 21, 2026

Merge branch 'main' into rocm-dsv41-fuse-inv-rope

6a33016

Fangzhou-Ai mentioned this pull request Sep 21, 2026

[ROCm][DSv4][DSV4.1][Perf] Fill decode wave counts and fuse the reduce at one split #57880

Open

tjtanaa approved these changes Sep 22, 2026

View reviewed changes

Merge branch 'main' into rocm-dsv41-fuse-inv-rope

b1465b1

tjtanaa enabled auto-merge (squash) September 22, 2026 06:15

tjtanaa merged commit ca831d1 into vllm-project:main Sep 22, 2026
159 of 160 checks passed

github-project-automation Bot moved this from Todo to Done in AMD Sep 22, 2026

yueliu14 mentioned this pull request Oct 10, 2026

[kb] Add AgentX/EP identities and replayable kernel overlays AMD-AGI/GEAK#507

Open

XiaobingSuper mentioned this pull request Oct 13, 2026

[Kernel] Scale elementwise kernel launches with the device, not just batch size #61404

Open


		logger = init_logger(__name__)

		FP8_DTYPE = current_platform.fp8_dtype()

Conversation

Fangzhou-Ai commented Sep 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Purpose

What changes

Not a duplicate

Test results

Uh oh!

claude Bot left a comment

Choose a reason for hiding this comment

Claude Code Review

Uh oh!

AndreasKaratzas commented Sep 18, 2026

Uh oh!

github-actions Bot commented Sep 18, 2026

Uh oh!

tjtanaa Sep 19, 2026

Choose a reason for hiding this comment

Uh oh!

Fangzhou-Ai Sep 21, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

tjtanaa Sep 19, 2026

Choose a reason for hiding this comment

Uh oh!

Fangzhou-Ai Sep 21, 2026

Choose a reason for hiding this comment

Uh oh!

tjtanaa commented Sep 19, 2026

Uh oh!

Fangzhou-Ai commented Sep 21, 2026

Uh oh!

github-actions Bot commented Sep 21, 2026

Uh oh!

Fangzhou-Ai commented Sep 21, 2026

Uh oh!

github-actions Bot commented Sep 21, 2026

Uh oh!

tjtanaa commented Sep 22, 2026

Uh oh!

github-actions Bot commented Sep 22, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Fangzhou-Ai commented Sep 17, 2026 •

edited

Loading