[algo] fix: correctly aggregate kl metrics in PPO actor#2259

Merged

vermouth1992 merged 3 commits intoverl-project:mainfrom

Jun 30, 2025

Commits on Jun 29, 2025

[actor] fix: wrong kl metrics collection in DataParallelPPOActor
0x404
committed
[critic] refactor: improve micro-batch metrics collection in DataParallelPPOCritic
0x404
committed
[critic] fix: ensure critic learning rate scheduler is updated correctly
0x404
committed