Fix SFT loss type rewards being overwritten in dpo_loss()

[Mr-Neutr0n]

Summary

• In dpo_loss(), the sft loss branch correctly sets chosen_rewards and rejected_rewards to zeros (since SFT has no preference rewards), but the unconditional reward computation at the end of the method overwrites these with standard DPO implicit rewards
• This causes inflated reward metrics when using SFT as part of a multi-loss configuration (e.g., MPO with ["sigmoid", "bco_pair", "sft"]), because the SFT component contributes DPO-style rewards instead of the intended zeros
• The fix guards the reward computation so it only runs for non-SFT loss types, preserving the correct zero rewards for SFT

Test plan

• Existing test_train_with_multiple_loss_types test continues to pass
• Run MPO training with loss_type=["sigmoid", "sft"] and verify reward metrics are no longer inflated by the SFT component
• Verify single loss type training (e.g., loss_type="sigmoid") is unaffected

[qgallouedec]

thanks, this makes sense. Note that DPO is being refactored in #3906, I'm checking if we're not doing the same mistake.

[HuggingFaceDocBuilderDev]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.