fix: fix skip_reference_policy_logprobs_calculation and skip_prev_logprobs - #2443

Merged

yuki-97 merged 13 commits into

NVIDIA-NeMo:mainfrom

jinglinglingling:skip-prev-logprobs-force-onpolicy-rebased

May 12, 2026

Commits on May 9, 2026

fix: skip prev_logprobs computation when force_on_policy_ratio is true

authored and
jinglinglingling
committed
63b6a6b
style: apply ruff format to PR-modified files

jinglinglingling
and
cursoragent
committed
3f55765
refactor(grpo): move skip_reference_policy_logprobs_calculation assert into setup

jinglinglingling
and
cursoragent
committed
473e7e9
fix: skip_reference_policy_logprobs_calculation=true crashes training

Nemo Assist
authored and
jinglinglingling
committed
2566326
style: apply ruff format to PR #2174 worker files

jinglinglingling
and
cursoragent
committed
21c6591
fix: skip loading reference model when KL penalty is zero

authored and
jinglinglingling
committed
bd85091
feat(grpo): auto-enable skip_reference_policy_logprobs when KL=0

jinglinglingling
and
cursoragent
committed
c31332e
test(functional): grpo_kl_zero.sh covers reference_policy_kl_penalty=0

jinglinglingling
and
cursoragent
committed
9ee54dd
test(grpo): add unit tests for skip-ref-logprobs and force-on-policy-ratio

jinglinglingling
and
cursoragent
committed
b430912
revert: drop worker-level reference-model guards (covered by grpo.py)

jinglinglingling
and
cursoragent
committed
d5c1532
test(grpo): add reference_policy_kl_penalty key to sglang setup test

jinglinglingling
and
cursoragent
committed
92aa6ba
test: provide real tensors for grpo_train logprob phase mocks

jinglinglingling
and
cursoragent
committed
c447a0d

Commits on May 11, 2026

Apply suggestions from code review

yfw
and
yuki-97
authored
2c57451