perf: Performance script tuning - #2473
Conversation
|
Hi @guyueh1, can I ask what the condition for enabling the fuse_loss is? I am curious why it is only enabled in the qwen3-235b model, but not for all models. |
It is generally applicable to all cases when |
Thank you for the clarification. |
|
/ok to test 879d913 |
|
@youngeunkwon0405 added them for the Qwen30B and a DAPO deepseek-v3; |
|
/ok to test 879d913 |
|
/ok to test 8730bfe |
|
/ok to test ca1755a |
|
/ok to test 5e52cf9 |
1. use force_on_policy_ratio to skip prev_logprobs compute in dsv3 and qwen3-235b 2. use fuse loss in qwen3-235b 3. use flashinfer flags for dsv3 fp8 Signed-off-by: Guyue Huang <guyueh@nvidia.com>
Signed-off-by: Guyue Huang <guyueh@nvidia.com>
Signed-off-by: Guyue Huang <140554423+guyueh1@users.noreply.github.com>
5e52cf9 to
27c2c8d
Compare
|
/ok to test 27c2c8d |
|
@youngeunkwon0405 could you approve? |
Signed-off-by: Guyue Huang <guyueh@nvidia.com> Signed-off-by: Guyue Huang <140554423+guyueh1@users.noreply.github.com>
Signed-off-by: Guyue Huang <guyueh@nvidia.com> Signed-off-by: Guyue Huang <140554423+guyueh1@users.noreply.github.com>
What does this PR do ?
Performance script tuning
Issues
List issues that this PR closes (syntax):
Usage
# Add a code snippet demonstrating how to use thisBefore your PR is "Ready for review"
Pre checks:
Additional Information