Allow inference to asynchronously await GPU work completion - #4295

Draft

tdene wants to merge 10 commits into

NVIDIA:mainfrom

tdene:tde/exclusive_asyncio_gpu_work

Commits on May 6, 2026

Allow GPU work exclusive ownership of asyncio loop
tdene
committed
27aca26
Utilize the exclusive GPU work sugar
tdene
committed
14cbdaf
Undo the asyncio exclusive ownership overhead
tdene
committed
22b3bd1
Get rid of new file, now that code is simpler
tdene
committed
ea9d427
Undo synchronize changes in engine; out of place
tdene
committed
2ed33f2
Bring back exclusive ownership
tdene
committed
1587f74
Move await past sampling enqueue
tdene
committed
696a7d2
Make faux-interrupt optional
tdene
committed
e26743d
Move `_dynamic_step_log_probs_bookkeeping`
tdene
committed
911cab4
lint
tdene
committed
8d3f778