Skip to content

diffusion: rotary embedding kernel #14302

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
RubiaCx wants to merge 45 commits into
base: main
Choose a base branch
from
Open

diffusion: rotary embedding kernel #14302

Show file tree
Hide file tree
Changes from 31 commits
Commits
Show all changes
45 commits
Select commit Hold shift + click to select a range
9081420
Integrate rotary embedding CUDA kernel with comprehensive testing and…
Dec 2, 2025
2aff99b
clean and format
RubiaCx Dec 2, 2025
28e337f
Merge remote-tracking branch 'origin/main' into cx/rotary
RubiaCx Dec 10, 2025
1fc1259
WIP
RubiaCx Dec 11, 2025
ce00ef6
Add diffusion rotary kernel path and unify op naming
RubiaCx Dec 12, 2025
db3cf33
typo
RubiaCx Dec 12, 2025
e4bc7c8
perf: optimize rotary embedding via smem caching and vectorized loads
Ther-LF Dec 16, 2025
00cf605
replace __ldg with SGLANG_LDG
Ther-LF Dec 16, 2025
d23edf2
perf:vec load and compute;modify threads_per_block
Ther-LF Dec 16, 2025
4a31c1b
fix lint
Ther-LF Dec 16, 2025
0735ba3
perf:replace int div/mod by bit-wise op
Ther-LF Dec 16, 2025
0466c59
perf:replace int div/mod by bit-wise op
Ther-LF Dec 16, 2025
aa02f4d
perf:remove smem add vec load for sin/cos
Ther-LF Dec 16, 2025
ead2093
perf:pipeline
Ther-LF Dec 17, 2025
9bbc0a4
Bench: add qwen-image shape
RubiaCx Dec 17, 2025
75a63c3
Merge remote-tracking branch 'origin/main' into cx/rotary
RubiaCx Dec 17, 2025
7660b83
Bench
RubiaCx Dec 17, 2025
a62f591
Delete sgl-kernel/benchmark/profile_rotary_embedding.py
Ther-LF Dec 18, 2025
2610ae6
[sgl-kernel]Optimize RoPE cos/sin kernel and clean up comments
RubiaCx Dec 18, 2025
6d1e0f8
Fix formatting issues from pre-commit hooks
RubiaCx Dec 19, 2025
745a55f
Fix typo and fix RoPE rotary embedding kernel/interface compatibility.
RubiaCx Dec 20, 2025
8905e0c
Merge branch 'main' into cx/rotary
RubiaCx Dec 29, 2025
cb0724b
Align with JIT kernels guide; keep behavior consistent with the AOT i…
RubiaCx Dec 30, 2025
b76e0fa
Align with JIT kernels guide; keep behavior consistent with the AOT i…
RubiaCx Dec 30, 2025
c663739
Update rotary embedding test/bench to import AOT RoPE kernel
RubiaCx Dec 30, 2025
c9396b6
Merge branch 'main' into cx/rotary
RubiaCx Dec 30, 2025
47e52a0
fix fallback for qk_norm
RubiaCx Dec 30, 2025
fc2448c
Relax QK-norm inplace contiguity + avoid fallback copy
RubiaCx Dec 31, 2025
f68646d
Merge remote-tracking branch 'origin/main' into cx/rotary
RubiaCx Dec 31, 2025
773c443
Merge branch 'main' into cx/rotary
RubiaCx Jan 3, 2026
d54d9ce
Add FlashInfer RoPE to bench and use it as speedup baseline
RubiaCx Jan 3, 2026
36c39e9
Move the ROT specialization to the Python JIT layer
RubiaCx Jan 3, 2026
09e7135
cache RoPE split/cast
RubiaCx Jan 3, 2026
2ccb1d3
jit: fuse rope cache gathering into kernel to reduce overhead
Ther-LF Jan 4, 2026
e19f5a9
jit: fuse rope cache gathering into kernel to reduce overhead
Ther-LF Jan 4, 2026
b53470a
jit: fuse rope cache gathering into kernel to reduce overhead
Ther-LF Jan 4, 2026
e37a0fa
jit: fuse rope cache gathering into kernel to reduce overhead
Ther-LF Jan 4, 2026
8185631
jit: fuse rope cache gathering into kernel to reduce overhead
Ther-LF Jan 4, 2026
29ca17d
jit: fuse rope cache gathering into kernel to reduce overhead
Ther-LF Jan 4, 2026
2315e41
bench: jit fused overhead
Ther-LF Jan 4, 2026
ff398c2
Merge branch 'main' into cx/rotary
RubiaCx Jan 5, 2026
1a4b874
clean up; fix 1D prefetch UB; align RoPE benchmarks/tests with positi…
RubiaCx Jan 5, 2026
4298be8
Refactor RoPE benchmark; remove cache; use view; relax test tolerances.
RubiaCx Jan 5, 2026
2d2d8c3
Refactor RoPE benchmark.
RubiaCx Jan 5, 2026
4525f82
Merge branch 'main' into cx/rotary
RubiaCx Jan 7, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Loading
Loading