feat(generation): add TensorRT-LLM rollout backend - #2420

Merged

terrykong merged 56 commits into

NVIDIA-NeMo:mainfrom

shuyixiong:shuyix/trtllm

Jul 23, 2026

Commits on Jul 23, 2026

feat(trtllm): add TRT-LLM generation backend

authored and
joyang-nv
committed
804d3ba
Custom build wheel

shuyixiong
authored and
joyang-nv
committed
bf2e4b4
build(trtllm): migrate from UV_FIND_LINKS stub wheel to PEP 517 path source

shuyixiong
authored and
joyang-nv
committed
ea5db8e
fix(trtllm): correct workspace deps and scope cutlass override to vllm

shuyixiong
authored and
joyang-nv
committed
96b930a
Pin nvidia-cutlass-dsl version in vllm

shuyixiong
authored and
joyang-nv
committed
0fdd97e
Drop custom modelopt build

shuyixiong
authored and
joyang-nv
committed
1d3ec3d
Fix eos_id issue

shuyixiong
authored and
joyang-nv
committed
aecea09
Rename trtllm workspace

shuyixiong
authored and
joyang-nv
committed
b4383ae
Clean code

shuyixiong
authored and
joyang-nv
committed
c890270
allow TRT-LLM backend for async GRPO

shikicloud
authored and
joyang-nv
committed
a9b0ecb
Deprecate sync trtllm_worker code

shuyixiong
authored and
joyang-nv
committed
054e598
refactor(trtllm): remove TRT-LLM HTTP server (to be added back in the agentRL PR)

shuyixiong
authored and
joyang-nv
committed
cb5fdf1
Clean code

shuyixiong
authored and
joyang-nv
committed
639df96
Resolve rebase conflict

shuyixiong
authored and
joyang-nv
committed
2c71521
Reorder uv sync step and move cuDNN env setup before TE build

shuyixiong
authored and
joyang-nv
committed
0f662bc
Resolve review comment

shuyixiong
authored and
joyang-nv
committed
88ffff4
Fix copyright-check and pre-commit changes

shuyixiong
authored and
joyang-nv
committed
0f71d14
fixup! Resolve review comment

shuyixiong
authored and
joyang-nv
committed
e63554b
Add dtensor nightly tests and release test

shuyixiong
authored and
joyang-nv
committed
29e874c
Add trtllm setup dependency

shuyixiong
authored and
joyang-nv
committed
e2b4f87
add ccache

shikicloud
authored and
joyang-nv
committed
595940d
fix(recipe): disable async checkpoint save for nanov3-30BA3B trtllm to avoid colocated wake OOM

shuyixiong
authored and
joyang-nv
committed
eb38a90
add trtllm ci test

shikicloud
authored and
joyang-nv
committed
d76c757
Resolve rebase conflict

shuyixiong
authored and
joyang-nv
committed
1881b66
Change description on CI test

shikicloud
authored and
joyang-nv
committed
430caca
Pre-commit changes

shuyixiong
authored and
joyang-nv
committed
17600b6
Apply the taplo-format hook's changes manually

shuyixiong
authored and
joyang-nv
committed
94d8e83
Add newly added files to pyrefly.toml

shuyixiong
authored and
joyang-nv
committed
50f2315
Add IPC-ZMQ refit unit tests

shikicloud
authored and
joyang-nv
committed
e23c46b
Resolve review comment

shuyixiong
authored and
joyang-nv
committed
3c5de7c
Resolve review comment

shuyixiong
authored and
joyang-nv
committed
2b789a2
Minor fix

shuyixiong
authored and
joyang-nv
committed
2a14252
Fix rebase conflict and Add logprobs_simple_format=True

shuyixiong
authored and
joyang-nv
committed
bcb8e50
build(trtllm): single-source the fork ref in [tool.trtllm]

shuyixiong
authored and
joyang-nv
committed
f6a58f2
Reduce compile jobs to reduce memory pressure

shuyixiong
authored and
joyang-nv
committed
a107f87
validate ccache reuse

shikicloud
authored and
joyang-nv
committed
65f2baa
Resolve rebase conflict

shuyixiong
authored and
joyang-nv
committed
d170371
Add TRT-LLM non-colocated async recipe

shikicloud
authored and
joyang-nv
committed
71bf08e
build: refresh uv lock

shikicloud
authored and
joyang-nv
committed
4c8efbd
Support save partial ccache and fix L0 bugs

shikicloud
authored and
joyang-nv
committed
4e43b6a
ci: reuse TRT-LLM ccache by base image
joyang-nv
committed
39ff598
ci: cache TRT-LLM uv builds
joyang-nv
committed
ed00a39
ci: reduce TRT-LLM build log noise
joyang-nv
committed
6699ee2
ci: focus TRT-LLM build logging
joyang-nv
committed
f05b002
ci: finalize TRT-LLM build caching
joyang-nv
committed
8984ac3
ci: skip TRT-LLM cache work when disabled
joyang-nv
committed
74711d9
ci: optimize TRT-LLM cache reuse
joyang-nv
committed
206a2ae
ci: simplify TRT-LLM build caching
joyang-nv
committed
f8ad15e
ci: document TRT-LLM lockfile correctness guard
joyang-nv
committed
340c8b5
ci: clarify late lockfile consistency check
joyang-nv
committed
e927101
ci: skip empty TRT-LLM cache deltas
joyang-nv
committed
b479c39
ci: reuse prebuilt TRT-LLM wheel in L0 tests
joyang-nv
committed
c9dd7ec
test: pass required NeMo Gym rollout arguments
joyang-nv
committed
205d9d2
Revert "test: pass required NeMo Gym rollout arguments"
joyang-nv
committed
da2030d
test: consume async NeMo Gym rollout results

RayenTian
authored and
joyang-nv
committed
cb95347
Merge branch 'main' into shuyix/trtllm
joyang-nv
authored
a119d48