feat(ppo): support async ppo - #3410

Merged

yuki-97 merged 39 commits into

NVIDIA-NeMo:mainfrom

bg51717:feat/ppo-async-clean

Aug 20, 2026

Commits on Jul 15, 2026

feat(ppo): support non-colocated generation
bg51717
committed
46a2615
test(ppo): add non-colocated e2e recipe
bg51717
committed
43524ee
chore(ppo): remove redundant colocated config comments
bg51717
committed
8d36cfb

Commits on Jul 16, 2026

Commits on Jul 17, 2026

Merge remote-tracking branch 'upstream/main' into feat/ppo-noncolocated-clean
bg51717
committed
4fe31a4
test(ppo): reduce non-colocated nightly timeout
bg51717
committed
5910124

Commits on Jul 19, 2026

Commits on Jul 20, 2026

test(ppo): fix async test configurations
bg51717
committed
bde5407

Commits on Jul 29, 2026

Commits on Jul 30, 2026

fix(ppo): harden async PPO startup and actor setup
bg51717
committed
d7e1355

Commits on Aug 9, 2026

Merge upstream/main into feat/ppo-async-clean
bg51717
committed
0b90178

Commits on Aug 10, 2026

fix(ppo): narrow collector config for gym helper
bg51717
committed
4403ab2
refactor(ppo): migrate PPOConfig to BaseModel
bg51717
committed
e93a42f
test(ppo): align critic loss checks
bg51717
committed
ba6cb5a

Commits on Aug 13, 2026

Commits on Aug 14, 2026

Merge remote-tracking branch 'upstream/main' into feat/ppo-async-clean
bg51717
committed
0c83974
fix(ppo): align async rollout lifecycle
bg51717
committed
700b032
test(ppo): address async review coverage
bg51717
committed
3099536

Commits on Aug 16, 2026

Commits on Aug 18, 2026

Merge remote-tracking branch 'upstream/main' into feat/ppo-async-clean
bg51717
committed
80edfcb
fix(async): require explicit replay restore behavior
bg51717
committed
20835de
fix(async): align runtime configuration semantics
bg51717
committed
aac7eed

Commits on Aug 19, 2026

Commits on Aug 20, 2026