Skip to content

perf: Performance script tuning - #2473

Merged
terrykong merged 3 commits into
NVIDIA-NeMo:mainfrom
guyueh1:perf_script_tuning
May 26, 2026
Merged

perf: Performance script tuning#2473
terrykong merged 3 commits into
NVIDIA-NeMo:mainfrom
guyueh1:perf_script_tuning

Conversation

@guyueh1

@guyueh1 guyueh1 commented May 12, 2026

Copy link
Copy Markdown
Contributor

What does this PR do ?

Performance script tuning

  1. use force_on_policy_ratio to skip prev_logprobs compute in dsv3 and qwen3-235b
  2. use fuse loss in qwen3-235b
  3. use flashinfer flags for dsv3 fp8

Issues

List issues that this PR closes (syntax):

Usage

  • You can potentially add a usage example below
# Add a code snippet demonstrating how to use this

Before your PR is "Ready for review"

Pre checks:

  • Make sure you read and followed Contributor guidelines
  • Did you write any new necessary tests?
  • Did you run the unit tests and functional tests locally? Visit our Testing Guide for how to run tests
  • Did you add or update any necessary documentation? Visit our Document Development Guide for how to write, build and test the docs.

Additional Information

  • ...

@guyueh1
guyueh1 requested review from a team as code owners May 12, 2026 15:24
@copy-pr-bot

copy-pr-bot Bot commented May 12, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@youngeunkwon0405

Copy link
Copy Markdown
Contributor

Hi @guyueh1, can I ask what the condition for enabling the fuse_loss is? I am curious why it is only enabled in the qwen3-235b model, but not for all models.

@guyueh1

guyueh1 commented May 18, 2026

Copy link
Copy Markdown
Contributor Author

Hi @guyueh1, can I ask what the condition for enabling the fuse_loss is? I am curious why it is only enabled in the qwen3-235b model, but not for all models.

It is generally applicable to all cases when sequence_packing is enabled; it was already enabled in deepseek previously, I didn't add it for dense models since our release tracker stopped tracking them; But I should probably add for 30B too

@youngeunkwon0405

Copy link
Copy Markdown
Contributor

Hi @guyueh1, can I ask what the condition for enabling the fuse_loss is? I am curious why it is only enabled in the qwen3-235b model, but not for all models.

It is generally applicable to all cases when sequence_packing is enabled; it was already enabled in deepseek previously, I didn't add it for dense models since our release tracker stopped tracking them; But I should probably add for 30B too

Thank you for the clarification.

@guyueh1 guyueh1 self-assigned this May 19, 2026
@guyueh1 guyueh1 added the CI:Lfast Runs a fast test suite and re-use nightly `main` container (but sync dependencies to PRs version) label May 19, 2026
@guyueh1

guyueh1 commented May 19, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test 879d913

@guyueh1

guyueh1 commented May 19, 2026

Copy link
Copy Markdown
Contributor Author

@youngeunkwon0405 added them for the Qwen30B and a DAPO deepseek-v3;
I suggest we merge this once the fast CI is passing; it's expensive for me to run perf regression myself, the impact should be reflected in the next weekly perf CI.

@guyueh1 guyueh1 added CI:L2 Run doctests, unit tests, functional tests, and convergence tests and removed CI:Lfast Runs a fast test suite and re-use nightly `main` container (but sync dependencies to PRs version) labels May 19, 2026
@guyueh1

guyueh1 commented May 19, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test 879d913

@guyueh1

guyueh1 commented May 19, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test 8730bfe

@guyueh1

guyueh1 commented May 19, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test ca1755a

@guyueh1

guyueh1 commented May 25, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test 5e52cf9

guyueh1 and others added 3 commits May 24, 2026 16:08
1. use force_on_policy_ratio to skip prev_logprobs compute in dsv3 and qwen3-235b
2. use fuse loss in qwen3-235b
3. use flashinfer flags for dsv3 fp8

Signed-off-by: Guyue Huang <guyueh@nvidia.com>
Signed-off-by: Guyue Huang <guyueh@nvidia.com>
Signed-off-by: Guyue Huang <140554423+guyueh1@users.noreply.github.com>
@guyueh1
guyueh1 force-pushed the perf_script_tuning branch from 5e52cf9 to 27c2c8d Compare May 25, 2026 02:08
@guyueh1

guyueh1 commented May 25, 2026

Copy link
Copy Markdown
Contributor Author

/ok to test 27c2c8d

@guyueh1

guyueh1 commented May 25, 2026

Copy link
Copy Markdown
Contributor Author

@youngeunkwon0405 could you approve?

@terrykong
terrykong merged commit ed685b6 into NVIDIA-NeMo:main May 26, 2026
37 checks passed
yfw pushed a commit that referenced this pull request May 27, 2026
Signed-off-by: Guyue Huang <guyueh@nvidia.com>
Signed-off-by: Guyue Huang <140554423+guyueh1@users.noreply.github.com>
vigneshwaran pushed a commit to vigneshwaran/RL that referenced this pull request May 28, 2026
Signed-off-by: Guyue Huang <guyueh@nvidia.com>
Signed-off-by: Guyue Huang <140554423+guyueh1@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CI:L2 Run doctests, unit tests, functional tests, and convergence tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants