Fix #447: norm_adv_by_std_in_grpo should be from rllm.algorithm.norm_...

[JiwaniZakir]

Closes #447

Summary

norm_adv_by_std_in_grpo is defined under rllm.algorithm in config files, but was mistakenly being read from rllm.stepwise_advantage in two places. This caused the flag to always fall back to the default value (True) rather than respecting the user's configured value.

Type of change

• Feature
• Fix
• Docs
• Refactor
• Example / Project
• Infra / CI

What changed

• In rllm/experimental/common/config.py (AlgorithmConfig.from_config): changed config.rllm.stepwise_advantage.get("norm_adv_by_std_in_grpo", True) to config.rllm.algorithm.get("norm_adv_by_std_in_grpo", True)
• In rllm/experimental/unified_trainer.py (UnifiedTrainer): applied the same fix, changing self.rllm_config.stepwise_advantage.get("norm_adv_by_std_in_grpo", True) to self.rllm_config.algorithm.get("norm_adv_by_std_in_grpo", True)
• Added tests/unified_trainer/test_algorithm_config.py with two tests that verify AlgorithmConfig.from_config correctly reads norm_adv_by_std_in_grpo from rllm.algorithm (with the key intentionally absent from stepwise_advantage to catch any regression)

Validation

• pre-commit run --all-files
• Targeted tests: pytest tests/unified_trainer/test_algorithm_config.py
• Manual validation performed
• Not run (reason below)

Validation details:

• New tests in test_algorithm_config.py cover both True and False values for norm_adv_by_std_in_grpo, using a minimal OmegaConf config that omits the key from rllm.stepwise_advantage to confirm the correct lookup path.

Breaking changes / migration notes

• None

Docs / examples

• Not needed

Related issues / PRs

• Fixes norm_adv_by_std_in_grpo should be from rllm.algorithm.norm_adv_by_std_in_grpo? #447
• Related to #
• Stacked on / depends on #

Screenshots / logs

N/A

---

This PR was created with AI assistance (Claude). The changes were reviewed by quality gates and a critic model before submission.

[kylemontgomery1]

Thanks for catching this!