[jinglinglingling]

Summary

• add FlashREINFORCE binary-KL gating at threshold 0.01 and prompt-mean-token-mean aggregation
• add Dr.GRPO and preserved REINFORCE group-baseline advantage variants over the complete logical-owner population
• keep context compaction/trajectory splitting compatible with full-window advantage and loss computation
• add an offline DAPO-band freezer and RFC0040 iter-881 B8/N8 plus target B17/N16 recipes with R3/router replay and LR 5e-6
• propagate active logical generation indices to Gym so AgentEnv can initialize one OSWorld task and fork exactly the selected owners
• make the OSWorld launcher select AgentEnv/OpenSandbox at runtime, mount the pinned CA securely, and preflight the chosen backend
• update the Gym submodule to the companion AgentEnv integration in feat(osworld): add AgentEnv fork oversampling Gym#3

Semantics

• B17/N16 gives 272 logical training owners per optimizer step (17 * 16 > 16 * 16)
• AgentEnv fork oversampling reuses one initialized task state across the configured N owners; it does not add extra optimizer rows
• reinforce_baseline and dr_grpo are separate recipe alternatives

Verification

• focused FlashREINFORCE/advantage/CC/dataset suite: 177 passed
• AgentEnv recipe composition and rollout metadata suite: 14 passed
• qualified-container node preflight and resolved B8/N8 recipe validation passed
• final AgentEnv submit dry-run passed
• live Cell-3 provider fork and real OSWorld task setup/fork smoke passed; all temporary sandboxes were released
• git diff --check, shell syntax, Python compile, and IDE lint checks passed

Base

Built directly on the delivered NeMo RL OSWorld v2 integration commit aeb900628a5c9ac261ffd482fc3774086f4c34b0.

[jinglinglingling]

Superseded by the cross-fork PR to the official repository: NVIDIA-NeMo#4581