Reorganize order of operations in inference context and text generation controller - #2929

Merged

tdene merged 20 commits into

NVIDIA:mainfrom

tdene:tde/active_mask_refactor

Apr 29, 2026

Commits on Apr 29, 2026

Reorganize code to slice tensors in context class
tdene
committed
c2f332d
Slice additional tensors in context class
tdene
committed
9087f55
Slice tensors by padded_active_request_count
tdene
committed
86ae997
Move context tensor padding into dedicated method
tdene
committed
7f28de2
Store logit output in static tensor
tdene
committed
aba74cd
Syntactic sugar for CG and awaiting
tdene
committed
24fd8bc
Introduce plumbing for more graphing
tdene
committed
b1970ba
Address reviewer comment: all metadata on GPU
tdene
committed
9896634
Address reviewer comment: fix padding bug
tdene
committed
72d50ad
Remove CUDAGraphCache in favor of CudaGraphManager
tdene
committed
c5e411d
Revert change being addressed by #4295
tdene
committed
60fe574
lint
tdene
committed
3fc7bd5
Defer `active_slice` logic to later PRs
tdene
committed
94c0525
Defer graph_capture_variants and _pre_forward_*
tdene
committed
c208c07
Add comment
tdene
committed
7cb7c62
Address reviewer comment: fix for spec decoding
tdene
committed
d4ae0e1
Fix CI
tdene
committed
f58657b
Address reviewer comments
tdene
committed
31c5162
Merge remote-tracking branch 'gh/main' into tde/active_mask_refactor
tdene
committed
19c7a9c
lint
tdene
committed
80c00a8