fix: fix qwen3-235b deepseek-v3 h100 perf tests - #2703
Conversation
yuki-97
added
the
CI:Lfast
|
/ok to test d5b2fa8 |
Signed-off-by: Yuki Huang <yukih@nvidia.com>
Signed-off-by: Yuki Huang <yukih@nvidia.com>
Signed-off-by: Yuki Huang <yukih@nvidia.com>
a1d2059 to
46336be
Compare
Signed-off-by: Yuki Huang <yukih@nvidia.com>
yuki-97
changed the title
|
/ok to test cd840e7 |
|
/ok to test 810268f |
Signed-off-by: Yuki Huang <yukih@nvidia.com>
810268f to
68b68fa
Compare
|
/ok to test 68b68fa |
terrykong
left a comment
terrykong
left a comment
There was a problem hiding this comment.
Reviewed by an agent team (5 agents). No production bugs found — the core change is sound and behavior-preserving.
The prepare_for_lp_inference() restructure fixes a real bug: previously prepare_for_lp_inference() ran only when not skip_prev_logprobs, yet get_reference_policy_logprobs could still execute afterward (when force_on_policy_ratio=True and a reference forward was needed), forwarding on a model that hadn't been onloaded. Decoupling the two is correct. The to_compute_kl removal is equivalent given setup() auto-enables skip_reference_policy_logprobs_calculation when reference_policy_kl_penalty==0 and asserts the flag requires kl==0. All six perf tests named in the description inherit moe_backend: triton through config defaults:.
Comments below are all non-blocking suggestions/questions. Thanks for the fix!
Generated by Claude Code
terrykong
left a comment
terrykong
left a comment
There was a problem hiding this comment.
lgtm modulo those comments
…omponents fixture Signed-off-by: Yuki Huang <yukih@nvidia.com>
…100 perf tests Signed-off-by: Yuki Huang <yukih@nvidia.com>
Signed-off-by: Yuki Huang <yukih@nvidia.com>
|
/ok to test f0f9682 |
Signed-off-by: Yuki Huang <yukih@nvidia.com>
2 participants
Issue
related to #2579.
Summary
fix several h100 perf tests list below, gb200 related perf tests still have other problems.
grpo-qwen3-235b-16n8g,grpo-qwen3-235b-32n8g,grpo-qwen3-235b-32n8g-async-1offgrpo-deepseek-v3-32n8g,grpo-deepseek-v3-64n8g,grpo-deepseek-v3-64n8g-async-1offChanges
prepare_for_lp_inference, only skip when bothskip_prev_logprobs+skip_reference_logprobs. previously only whenskip_prev_logprobs.vllm_kwargs.moe_backend=tritonto keep same behavior with vLLM==0.17.to_compute_klsinceskip_reference_policy_logprobs_calculationwill be set toTrueinsetupwhenreference_policy_kl_penalty==0.grpo-qwen3-235b-16n8g.