Extend dynamic inference asynchronous scheduling support - #5939

Merged

lmcafee-nvidia merged 28 commits into

NVIDIA:mainfrom

lmcafee-nvidia:async-sched-prefill

Jul 28, 2026

Commits on Jul 21, 2026

Add asynchronous prefill scheduling
lmcafee-nvidia
committed
8f70fe7
Label asynchronous prefill phases
lmcafee-nvidia
committed
cda97a7
Reorder controller methods by dependency
lmcafee-nvidia
committed
d4a9758

Commits on Jul 22, 2026

Remove serial async scheduling mode
lmcafee-nvidia
committed
da72be8
Remove obsolete async scheduling events
lmcafee-nvidia
committed
1965c8d
Remove redundant async token metadata compaction
lmcafee-nvidia
committed
25ff376
Separate async request resolution from token commits
lmcafee-nvidia
committed
7739953
Move async token count ownership out of resolution
lmcafee-nvidia
committed
529dca3
Select async scheduling by overlap
lmcafee-nvidia
committed
cc7bc5a

Commits on Jul 23, 2026

Support lifecycle fallback in async scheduling
lmcafee-nvidia
committed
661236d
Preserve counters across async dummy resets
lmcafee-nvidia
committed
7164358
Merge remote-tracking branch 'main/main' into async-sched-prefill
lmcafee-nvidia
committed
c9bb286
Run autoformat after merging main
lmcafee-nvidia
committed
4c6a1ea
Fix async scheduling step classification
lmcafee-nvidia
committed
48ff876
Simplify async scheduling log labels
lmcafee-nvidia
committed
ddc839a
Merge remote-tracking branch 'main/main' into async-sched-prefill
lmcafee-nvidia
committed
06f8912

Commits on Jul 24, 2026

Merge remote-tracking branch 'main/main' into async-sched-prefill
lmcafee-nvidia
committed
61ada4f

Commits on Jul 27, 2026

Merge remote-tracking branch 'main/main' into async-sched-prefill
lmcafee-nvidia
committed
b819e71
Fix async sampling buffer test
lmcafee-nvidia
committed
f00aa8d
Remove async resolution forward synchronization
lmcafee-nvidia
committed
d85e283

Commits on Jul 28, 2026

Support Mamba async scheduling
lmcafee-nvidia
committed
19535cc
Support async prefix caching and chunked prefill
lmcafee-nvidia
committed
71e2e18
Support sampling modes with async scheduling
lmcafee-nvidia
committed
b771239
Support log probabilities with async scheduling
lmcafee-nvidia
committed
cc4187c
Support stop words with async scheduling
lmcafee-nvidia
committed
ce7b988
Merge remote-tracking branch 'main/main' into async-sched-prefill
lmcafee-nvidia
committed
5b236ba
Remove async scheduling functional test recipe
lmcafee-nvidia
committed
403b1a9
Merge remote-tracking branch 'main/main' into async-sched-prefill
lmcafee-nvidia
committed
2c957eb