[TRTLLM-13234][feat] Complete TorchSampler beam search: length_penalty, diversity_rate, early_stopping, VBWS, and CBA performance - #16620

Merged

zhaoyangwang-nvidia merged 36 commits into

NVIDIA:mainfrom

zhaoyangwang-nvidia:beam-length-penalty

Aug 12, 2026

Commits on Aug 10, 2026

[TRTLLM-13234][feat] Complete TorchSampler beam search: length_penalty, diversity_rate, early_stopping, VBWS, and CBA performance
zhaoyangwang-nvidia
committed
65d4801
[TRTLLM-13234][fix] Match the actual beam-width rejection message in test
zhaoyangwang-nvidia
committed
41a36a0
[TRTLLM-13234][fix] Address review: disagg check, early_stopping schema, VBWS tests
zhaoyangwang-nvidia
committed
621580c
[TRTLLM-13234][test] Assert the per-iteration beam width in the VBWS e2e test
zhaoyangwang-nvidia
committed
a461406
[TRTLLM-13234][fix] Clamp getBeamWidthByIter with the actual beam width array length
zhaoyangwang-nvidia
committed
08bcc63
[TRTLLM-13234][test] Decode further past the beam width array in the VBWS e2e test
zhaoyangwang-nvidia
committed
06135e8
[TRTLLM-13234][fix] Exclude every kind of dummy request from the beam width check
zhaoyangwang-nvidia
committed
181060f
[TRTLLM-13234][fix] Offset generation logits by the static beam width
zhaoyangwang-nvidia
committed
2eb914d
[TRTLLM-13234][fix] Pass row_stride in the abstract BeamSearchStep test
zhaoyangwang-nvidia
committed
196365e
[TRTLLM-13234][fix] Serve every early_stopping mode from the CBA path
zhaoyangwang-nvidia
committed
a4bfc22
[TRTLLM-13234][chore] Update comments left stale by the CBA unification
zhaoyangwang-nvidia
committed
cf9325a
[TRTLLM-13234][fix] Key beam-search strategy groups regardless of tuple arity
zhaoyangwang-nvidia
committed
2f4727e
[TRTLLM-13234][fix] Stop the beam padding sentinel reaching request histories
zhaoyangwang-nvidia
committed
d83e91a
[TRTLLM-13234][chore] Refresh comments the CBA unification invalidated
zhaoyangwang-nvidia
committed
3e162e5
[TRTLLM-13234][doc] Document the beam-search behaviour changes
zhaoyangwang-nvidia
committed
f494470
[TRTLLM-13234][test] Cover the VBWS feature combinations end to end
zhaoyangwang-nvidia
committed
416f8c0
[TRTLLM-13234][test] Cover length_penalty on the candidate-beams-array path
zhaoyangwang-nvidia
committed
b9ef58e
[TRTLLM-13234][chore] Write down the _cba_step_math compile contract
zhaoyangwang-nvidia
committed
148d751
[TRTLLM-13234][chore] Delete the pool-free beam-search path
zhaoyangwang-nvidia
committed
6d50967
[TRTLLM-13234][chore] Remove state left behind by the pool-free path
zhaoyangwang-nvidia
committed
ebdafe0
[TRTLLM-13234][feat] Gate speculative beam-history D2H per group
zhaoyangwang-nvidia
committed
512a13d
[TRTLLM-13234][fix] Address beam-search review findings
zhaoyangwang-nvidia
committed
f8d5589
[TRTLLM-13234][fix] Keep the earliest per-beam finish reason on the CBA path
zhaoyangwang-nvidia
committed
a2ea981
[TRTLLM-13234][fix] Report per-beam finish reasons on the CBA path
zhaoyangwang-nvidia
committed
33d2992
[TRTLLM-13234][chore] Address beam-search review nits
zhaoyangwang-nvidia
committed
e86904d
[TRTLLM-13234][fix] Give the top-p-decay mock request a beam width
zhaoyangwang-nvidia
committed
3ec7225
[TRTLLM-13234][fix] Expect LENGTH from the CBA done verdict in its unit test
zhaoyangwang-nvidia
committed
6e97568
[TRTLLM-13234][fix] Give the remaining request mocks a beam width
zhaoyangwang-nvidia
committed
d84c778
[TRTLLM-13234][fix] Give the logprobs tests Dynamo recompile headroom
zhaoyangwang-nvidia
committed
e2bf705
[TRTLLM-13234][fix] Give the compiled CBA step its own recompile headroom
zhaoyangwang-nvidia
committed
b11b928
[TRTLLM-13234][fix] Latch the CBA harvest separately from the finish reason
zhaoyangwang-nvidia
committed
acf489c
[TRTLLM-13234][fix] Annotate two returns mypy could not infer
zhaoyangwang-nvidia
committed
44612d6
[TRTLLM-13234][fix] Name _kernel_test's return type past functools.wraps
zhaoyangwang-nvidia
committed
8da271e
[TRTLLM-13234][fix] Detect syncs without parking a hostfunc on the stream
zhaoyangwang-nvidia
committed
b600855
[TRTLLM-13234][fix] Complete the disaggregated beam search handoff
zhaoyangwang-nvidia
committed
238c420

Commits on Aug 11, 2026

[TRTLLM-13234][fix] Hand off a context phase that ends on its only token
zhaoyangwang-nvidia
committed
5f4906a