Skip to content

feat: Add linear CE loss fusion for DPO#2139

Draft
pengdurice wants to merge 5 commits intoNVIDIA-NeMo:mainfrom
pengdurice:peng-add-linear-ce-fusion-v2
Draft

feat: Add linear CE loss fusion for DPO#2139
pengdurice wants to merge 5 commits intoNVIDIA-NeMo:mainfrom
pengdurice:peng-add-linear-ce-fusion-v2

Conversation

@pengdurice
Copy link
Contributor

@pengdurice pengdurice commented Mar 22, 2026

What does this PR do ?

Suport Linear CE Loss Fusion for DPO
On top of #2036 where Linear CE loss fusion support is added for SFT. This PR adds the support to DPO loss.

Issues

NA

Tests

  1. Unit tests passed
  2. Local tests (running the sh file) passed
  3. A side by side loss curve comparison between baseline and experiment(linear ce loss fusion enabled)
image

Usage

# add the following to your dpo.yaml file.
megatron_cfg:
    enabled: true
    use_linear_ce_fusion_loss: true
    linear_ce_fusion_chunk_size: 256 # or other numbers 

Before your PR is "Ready for review"

Pre checks:

  • Make sure you read and followed Contributor guidelines
  • Did you write any new necessary tests?
  • Did you run the unit tests and functional tests locally? Visit our Testing Guide for how to run tests
  • Did you add or update any necessary documentation? Visit our Document Development Guide for how to write, build and test the docs.

Additional Information

  • ...

…the loss values being nearly identical between base and exp.

Signed-off-by: pengdurice <pengduhit@gmail.com>
Signed-off-by: pengdurice <pengduhit@gmail.com>
Signed-off-by: pengdurice <pengduhit@gmail.com>
@copy-pr-bot
Copy link

copy-pr-bot bot commented Mar 22, 2026

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

Signed-off-by: pengdurice <pengduhit@gmail.com>
@github-actions github-actions bot added the documentation Improvements or additions to documentation label Mar 23, 2026
Signed-off-by: pengdurice <pengduhit@gmail.com>
@pengdurice pengdurice changed the title Add linear CE loss fusion for DPO feat: Add linear CE loss fusion for DPO Mar 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

community-request documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants