Vitalyk/multiturn v2

[yobibyte]

What does this PR do?

• This PR reverts the revert of Multiturn rollout support prep #2966.
• Fixes the faulty test configuration and updates its golden values.

Why was #2966 reverted?

We failed a gitlab CI test on timings. Every second iteration of the post-change test was extremely fast.

Why did it happen?

This was the configuration of a faulty test:

gpt/gpt_grpo_tp4_pp1_dp2_8b_throughput/model_config.yaml
78:  --global-batch-size: 2
79:  --grpo-group-size: 2
80:  --grpo-prompts-per-step: 2

We sampled 4 rollouts each before and were supposed to consume 2 rollouts per training step (global bs = 2).
Before #2966, sequence packing had a bug which made it sample every iteration no matter on whether we had consumed all the samples or not.
The huge timings you saw in the test were mostly for inference, training step did not take long.

Why didn't it fail on Github CI?

Our github CI used a correct batch size which made it consume all data that was sampled and not being affected by a bug I fixed in #2966.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[yobibyte]

/ok to test ee6228e

[ko3n1g]

Fast-merging this replay