fix: add mask seq with high logp err to nemo gym config

[cmunley1]

What does this PR do ?

Add a one line overview of what this PR aims to accomplish.

Issues

List issues that this PR closes (syntax):

Usage

• You can potentially add a usage example below

# Add a code snippet demonstrating how to use this

Before your PR is "Ready for review"

Pre checks:

• Make sure you read and followed Contributor guidelines
• Did you write any new necessary tests?
• Did you run the unit tests and functional tests locally? Visit our Testing Guide for how to run tests
• Did you add or update any necessary documentation? Visit our Document Development Guide for how to write, build and test the docs.

Additional Information

• ...

Summary by CodeRabbit

• New Features
  • Added a new configurable threshold parameter for reward scaling in the workplace assistant configuration.

[coderabbitai]

📝 Walkthrough

Walkthrough

This PR adds a new configuration field seq_logprob_error_threshold (set to null) under the grpo.reward_scaling section in a GRPO workplace assistant example configuration file for NeMo's Nemotron Nano v2.9B model.

Changes

Cohort / File(s)	Summary
GRPO Configuration examples/nemo_gym/grpo_workplace_assistant_nemotron_nano_v2_9b.yaml	Added seq_logprob_error_threshold: null field under grpo.reward_scaling configuration.

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~2 minutes

Possibly related PRs

• feat: Mask sequences with high logprob error #1838 — Implements the seq_logprob_error_threshold field with masking logic and tests alongside configuration updates.

Suggested labels

CI:L1

Suggested reviewers

• pjin-nvidia
• HeyyyyyyG
• yuki-97

🚥 Pre-merge checks | ✅ 4

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Test Results For Major Changes	✅ Passed	Minor configuration update adding a single parameter to YAML file; underlying feature already fully implemented, tested, and present in other config files.
Title check	✅ Passed	The title refers to adding a configuration field related to masking sequences with high log probability error to a NeMo gym config file, which aligns with the actual change of adding seq_logprob_error_threshold to the configuration.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing touches

• 📝 Generate docstrings

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch cmunley1/nemo_gym_config

Tip

Issue Planner is now in beta. Read the docs and try it out! Share your feedback on Discord.

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

🧹 Nitpick comments (1)

examples/nemo_gym/grpo_workplace_assistant_nemotron_nano_v2_9b.yaml (1)

38-38: Add an inline comment describing the field's purpose, type, and valid range.

Every non-obvious parameter in this file carries a comment (e.g., overlong_filtering, skip_reference_policy_logprobs_calculation, moe_router_load_balancing_type). seq_logprob_error_threshold is opaque without one — readers have no indication of what unit (float, log-prob delta?), what range triggers masking, or a cross-reference to docs/guides.

✏️ Suggested comment

-  seq_logprob_error_threshold: null
+  seq_logprob_error_threshold: null  # Mask sequences whose per-seq logprob error (vLLM vs MCore) exceeds this value; null disables masking. See docs/guides/grpo.md.

🤖 Prompt for AI Agents

Verify each finding against the current code and only fix it if needed.

In `@examples/nemo_gym/grpo_workplace_assistant_nemotron_nano_v2_9b.yaml` at line
38, Add an inline comment for the seq_logprob_error_threshold field explaining
its purpose (what it controls—e.g., threshold for masking tokens based on
sequence log-probability errors), its type (float), valid range (e.g., 0.0–inf
or a recommended small positive range like 0.0–10.0 and whether negatives are
allowed), the unit/meaning (log-prob delta or absolute log-prob), and a brief
pointer to docs or a config section for more details; update the line containing
seq_logprob_error_threshold to include this one-line comment referencing the
field name so readers know how to set it and what values are safe.

🤖 Prompt for all review comments with AI agents

Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In `@examples/nemo_gym/grpo_workplace_assistant_nemotron_nano_v2_9b.yaml`:
- Line 38: Add an inline comment for the seq_logprob_error_threshold field
explaining its purpose (what it controls—e.g., threshold for masking tokens
based on sequence log-probability errors), its type (float), valid range (e.g.,
0.0–inf or a recommended small positive range like 0.0–10.0 and whether
negatives are allowed), the unit/meaning (log-prob delta or absolute log-prob),
and a brief pointer to docs or a config section for more details; update the
line containing seq_logprob_error_threshold to include this one-line comment
referencing the field name so readers know how to set it and what values are
safe.