chore: fix grpo functional test metric

[RayenTian]

What does this PR do ?

Reduce the functional test acceptance threshold for train/gen_kl_error from 1.05 to 0.001 to catch subtle regressions in generation logprob/KL alignment early.

Related Context

Link: #1506 (comment)
gen_kl_error should actually be pretty low, usually we set it to be <1e-3.

Summary by CodeRabbit

• Tests
  • Updated validation thresholds for training metrics to enforce stricter quality standards during testing procedures.

_{✏️ Tip: You can customize this high-level summary in your review settings.}

[coderabbitai]

📝 Walkthrough

Walkthrough

A test threshold expectation is adjusted in the GRPO test suite, reducing the maximum tolerance for the train/gen_kl_error metric from 1.05 to 0.001, enforcing stricter validation criteria during test execution.

Changes

Cohort / File(s)	Summary
Test threshold adjustment tests/functional/grpo.sh, tests/check_metrics.py	Reduced max(data["train/gen_kl_error"]) threshold from 1.05 to 0.001 for stricter metric validation

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~5 minutes

• Simple numeric threshold modification with no logic or control flow changes
• Verify that the new threshold is intentional and achievable under test conditions

Suggested labels

CI:L1

Suggested reviewers

• yuki-97
• joyang-nv

Pre-merge checks and finishing touches

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title 'chore: fix grpo functional test metric' directly describes the main change: adjusting the functional test acceptance threshold for a GRPO metric.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Test Results For Major Changes	✅ Passed	Changes are minor test threshold adjustments in functional test suite, not production code modifications. No new features, breaking changes, or significant refactoring introduced.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.

✨ Finishing touches

• 📝 Generate docstrings

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment
• Commit unit tests in branch ruit/fix_grpo_functional_test

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}