feat(grpo): port grouped validation (pass_k) to validate_sync - #3686
Open
lonexreb wants to merge 1 commit into
Open
feat(grpo): port grouped validation (pass_k) to validate_sync#3686lonexreb wants to merge 1 commit into
lonexreb wants to merge 1 commit into
Conversation
NVIDIA-NeMo#3401 gave grpo.validate() grouped validation: with grpo.val_num_generations_per_prompt = k > 1 each val prompt is rolled out k times (repeat_interleave, contiguous per prompt) and a pass_k metric (pass@k over each prompt's k rollouts) is reported alongside accuracy, usable as a stop_at_validation_metric. The sync dataplane's validate_sync never received the port and silently ignored the knob. Mirror the legacy implementation: expand the batch before rollout_to_tq (sizing the val partition for the expanded batch) and compute pass_k from the per-row rewards, which rollout_to_tq returns in input order. Config validation (k >= 1, pass_k stop-metric requires k > 1) already applies -- the sync driver uses the shared grpo.setup(). Tracked in NVIDIA-NeMo#3336 (port of NVIDIA-NeMo#3401 to grpo_sync). Add a validate_sync unit test mirroring test_grouped_validation_reports_pass_k. Signed-off-by: Shubhankar Tripathy <reach2shubhankar@gmail.com>
Contributor
Author
|
/ok to test c73b5f4 |
svcnvidia-nemo-ci
added
the
waiting-on-maintainers
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do ?
Ports grouped validation (#3401) to the sync dataplane's
validate_sync, as tracked in #3336:grpo.val_num_generations_per_prompt = k > 1, each validation prompt is now rolled outktimes (repeat_interleave, contiguous per prompt — same asgrpo.validate()), and the val TQ partition is sized for the expanded batch.pass_kmetric (pass@k over each prompt'skrollouts) is reported alongsideaccuracy, sogrpo.stop_at_validation_metric: pass_know works on the sync path too (the sync driver already threadsstop_at_validation_metricthrough).pass_kgrouping relies onrollout_to_tqreturning per-rowdriver_carryin input order — the same alignment the existing message-log reconstruction (turn_roles[i]/turn_contents[i]) already depends on.grpo.val_num_generations_per_prompt, and thek >= 1/ pass_k-stop-metric asserts already apply since the sync driver runs the sharedgrpo.setup().Previously
validate_syncsilently ignored the knob.Issues
Part of #3336 (item: "Port grouped validation (#3401) to
validate_sync").Usage
Before your PR is "Ready for review"
test_sync_grouped_validation_reports_pass_kmirrors feat(grpo): validation-only sampling params and grouped pass@k validation #3401'stest_grouped_validation_reports_pass_k(asserts contiguous k-repeat, expanded partition sizing, accuracy vs pass_k)