Skip to content

[TRTLLM-35882][feat] cute dsl gvr-topk load-balance optimization #15304

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
yuxianq merged 69 commits into from
Jun 24, 2026
Merged

[TRTLLM-35882][feat] cute dsl gvr-topk load-balance optimization #15304

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
69 commits
Select commit Hold shift + click to select a range
4a2376a
Long's cuda dsl gvr initial impl.
limin2021 May 20, 2026
bec3708
Switch DSL gvr_topk_decode to fake_tensor compile + fix test pre_idx …
limin2021 May 20, 2026
b0bbf2a
Align gvr_topk_decode with CUDA reference (Batch 1: barriers / snap_l…
limin2021 May 20, 2026
3b11d0f
Align gvr_topk_decode Phase 2/3 loops with CUDA (Batch 2: runtime whi…
limin2021 May 20, 2026
89edd73
Align gvr_topk_decode with CUDA: AddressSpace.gmem + preIdx unroll
limin2021 May 21, 2026
b459a8f
Align block_count_ge with CUDA: 4-way unrolled vec loop (fast-path + …
limin2021 May 21, 2026
a01f6c5
Add strided layout + 2-way medium cascade with dtype-aware defaults
limin2021 May 21, 2026
5b3d34c
Add phase3 unroll + constant_hint switches; cascade for phase3_collect
limin2021 May 25, 2026
45c53c5
Switch Phase 2/3 to cutlass.range(unroll=4); add 3-tier min_blocks he…
limin2021 May 25, 2026
44035a5
Add use_256bit_load and configurable num_threads_per_block
limin2021 May 26, 2026
882c767
Add enable_warp_parallel_reduce switch and clean dead unroll/layout k…
limin2021 May 27, 2026
19b1682
Align DSL GvrParams kFTarget=kK with CUDA PR #14413
limin2021 May 27, 2026
cd656ad
Consolidate kernel constants into GvrTopKKernel.__init__
limin2021 May 27, 2026
b610d13
Add compress_ratio support for DSv4 indexer (cr=4)
limin2021 May 27, 2026
6bf5c87
Auto-resolve host wrapper defaults: wp_parallel + per-(T,dtype) tier-…
limin2021 May 27, 2026
eb261b9
Auto-resolve num_threads_per_block + use_256bit_load in host wrapper
limin2021 May 27, 2026
2ef50d3
Add max_seq_len hint for graph-safe (T, V) selection
limin2021 May 27, 2026
99340a0
Use N_dec for min_blocks tier check too (consistency with T/V)
limin2021 May 27, 2026
15af2ef
Wave-fit aware tier-3 mb for fp32: mb=3 when num_rows in (2*sms, 3*sms]
limin2021 May 28, 2026
d8b3f13
Add wave-fit N cutoff for fp32 tier-3 + cleanup file comments
limin2021 May 28, 2026
d474eb9
Add cute_dsl_gvr_topk_decode custom op + unittest
limin2021 May 28, 2026
3ccd70e
Skip STG.value path in GVR Top-K when caller only consumes indices
limin2021 May 29, 2026
ddd872f
Address PR review on cute_dsl_gvr_topk_decode tests
limin2021 May 29, 2026
335dd11
Validate logits.shape[0] divisibility by next_n in GVR op
limin2021 May 29, 2026
48d44e2
Inline N_cols + key info_once on shape signature
limin2021 May 29, 2026
c055ad6
Mirror kernel N_eff formula in test reference
limin2021 May 29, 2026
e219a00
Address PR review feedback for GVR Top-K DSL port
limin2021 Jun 1, 2026
d23ba74
Address second-round PR review for GVR Top-K DSL port
limin2021 Jun 1, 2026
97c47dc
Remove unused s_thr_extra smem allocation in GVR Top-K kernel
limin2021 Jun 1, 2026
8cb1e34
[None][feat] GVR Top-K V5 + cluster: multi-CTA-per-row MVP
limin2021 Jun 3, 2026
0e7220a
[None][perf] GVR Top-K cluster: Shift D — cluster-parallel Phase 3 + …
limin2021 Jun 3, 2026
513890f
[None][perf] GVR Top-K cluster: auto-dispatch cluster_size by (N, BS)
limin2021 Jun 3, 2026
f42c9e2
[None][exp] GVR Top-K cluster: Shift E SMEM slice cache (disabled by …
limin2021 Jun 4, 2026
34ccad2
[None][feat] GVR Top-K cluster: varlen (per-row seq_len) support
limin2021 Jun 4, 2026
fc31330
Merge remote-tracking branch 'github-upstream/main' into cute-dsl-gvr…
limin2021 Jun 10, 2026
10505e3
[None][feat] GVR Top-K: unify single-CTA and multi-CTA cluster paths
limin2021 Jun 10, 2026
54291a5
[None][chore] GVR Top-K: polish comments and docstrings
limin2021 Jun 10, 2026
89ca42c
[None][fix] GVR Top-K: clamp cluster_size to device hardware limit
limin2021 Jun 10, 2026
795c35e
[None][chore] GVR Top-K: fix misleading "cs=4 can crash" comment
limin2021 Jun 10, 2026
dda0a04
[None][perf] GVR Top-K: retune cluster_size auto-dispatch (add cs=8, …
limin2021 Jun 10, 2026
6110513
[None][chore] GVR Top-K: delete legacy cluster fork files
limin2021 Jun 10, 2026
b95fef1
[None][refactor] GVR Top-K: extract kernel body into run_one_row method
limin2021 Jun 11, 2026
b119960
[None][chore] GVR Top-K: drop dead do_cluster_aggregation parameter
limin2021 Jun 11, 2026
bc6d0e8
[None][fix] GVR Top-K: drop relaxed arrive on DSMEM producer→consumer…
limin2021 Jun 11, 2026
62ba581
[None][fix] GVR Top-K: cap peer_cnt to kC in cluster leader gather
limin2021 Jun 11, 2026
f4454df
[None][test] GVR Top-K: add (fp16, K=1024) cell to keep fp16 path cov…
limin2021 Jun 11, 2026
8fb7e58
[None][docs] GVR Top-K op: document the cluster_size arg
limin2021 Jun 11, 2026
55d21cc
[None][feat] GVR Top-K LB: prepare kernel + heterogeneous-cluster mai…
limin2021 Jun 11, 2026
61b6d9d
[None][test] GVR Top-K LB: decouple prepare + add unit tests
limin2021 Jun 11, 2026
005966d
[None][refactor] GVR Top-K LB: API cleanup and test simplification
limin2021 Jun 11, 2026
362dc16
[None][refactor] GVR Top-K LB: hoist wrappers to run_gvr_topk.py for …
limin2021 Jun 12, 2026
c9547bd
[None][feat] GVR Top-K: seqlen_sorted entry path via order_row indire…
limin2021 Jun 12, 2026
31e9035
[None][feat] GVR Top-K wrapper: plumb seqlen_sorted + add gvr_topk_so…
limin2021 Jun 12, 2026
f30e724
[None][feat] GVR Top-K custom op + UT: order_row LJF dispatch path
limin2021 Jun 12, 2026
3416524
Merge remote-tracking branch 'github-upstream/main' into gvr-topk-loa…
limin2021 Jun 15, 2026
be52699
[None][test] GVR Top-K: drop conftest.py onnx stub
limin2021 Jun 15, 2026
ad410c9
[None][fix] GVR Top-K: address PR review nits (coderabbitai)
limin2021 Jun 15, 2026
e51c837
[None][docs] GVR Top-K op: add TODO note for dsa.py order_row wiring
limin2021 Jun 15, 2026
b435266
[None][docs] GVR Top-K op: rename LJF -> row reorder (LJF) in comments
limin2021 Jun 15, 2026
6ee3093
[None][fix] GVR Top-K LB: assert max_batch_size pow2 in [64, 1024]
limin2021 Jun 16, 2026
baf3aec
[None][fix] GVR Top-K: address remaining PR review should-fix items
limin2021 Jun 16, 2026
a99fb4f
[None][test] GVR Top-K LB: address review nits + extend LB UT to next…
limin2021 Jun 16, 2026
d085398
[None][fix] GVR Top-K LB: address yuxianq review (drop unused long_th…
limin2021 Jun 16, 2026
32ddd3e
[None][docs] GVR Top-K: fix gvr_topk_sort_prepare docstring (request-…
limin2021 Jun 17, 2026
83789d9
[None][refactor] GVR Top-K LB: merge LBDecodeRunner into decode Runne…
limin2021 Jun 17, 2026
bc53988
[None][test] GVR Top-K: inline argsort, drop run_gvr_topk dependency
limin2021 Jun 17, 2026
fdd4ad3
[None][docs] GVR Top-K decode forward: fix "(tensor, ts)" typo in dis…
limin2021 Jun 17, 2026
0562e62
[None][docs] GVR Top-K: update LJF wiring TODO with measured gate (2 …
limin2021 Jun 23, 2026
9a34a89
Merge branch 'main' into gvr-topk-load-balance
xxi-nv Jun 24, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Loading
Loading