Skip to content

perf: Performance script tuning #2473

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
terrykong merged 3 commits into from
May 26, 2026
Merged

perf: Performance script tuning #2473

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
e184172
Perf script tuning
guyueh1 May 12, 2026
36f145f
Use fuse_loss in more moe benchmarks
guyueh1 May 19, 2026
27c2c8d
Update dapo-deepseek-v3-64n8g.v2.yaml
guyueh1 May 25, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions examples/configs/recipes/llm/performance/dapo-deepseek-v3-64n8g.v2.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,7 @@ policy:
lr_warmup_init: 5.0e-08
sequence_packing:
enabled: true
fuse_loss: true
generation:
max_new_tokens: 1536
vllm_cfg:
Expand Down
1 change: 1 addition & 0 deletions examples/configs/recipes/llm/performance/grpo-deepseek-v3-32n8g.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ grpo:
max_val_samples: 16
loss_fn:
use_importance_sampling_correction: true
force_on_policy_ratio: true
checkpointing:
checkpoint_dir: results/grpo-deepseek-v3-32n8g
policy:
Expand Down
3 changes: 3 additions & 0 deletions examples/configs/recipes/llm/performance/grpo-qwen3-235b-16n8g.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@ grpo:
max_val_samples: 16
loss_fn:
use_importance_sampling_correction: true
force_on_policy_ratio: true
checkpointing:
checkpoint_dir: results/grpo-qwen3-235b-16n8g
policy:
Expand All @@ -17,6 +18,8 @@ policy:
logprob_batch_size: 1
max_total_sequence_length: 8192
make_sequence_length_divisible_by: 8
sequence_packing:
fuse_loss: true
dtensor_cfg:
enabled: false
megatron_cfg:
Expand Down
2 changes: 2 additions & 0 deletions examples/configs/recipes/llm/performance/grpo-qwen3-30ba3b-4n4g.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,8 @@ policy:
train_micro_batch_size: 1
logprob_batch_size: 2
max_total_sequence_length: 4096
sequence_packing:
fuse_loss: true
dtensor_cfg:
enabled: false
optimizer: null
Expand Down
2 changes: 2 additions & 0 deletions examples/configs/recipes/llm/performance/grpo-qwen3-30ba3b-4n8g-40K.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,8 @@ policy:
train_micro_batch_size: 1
logprob_batch_size: 4
max_total_sequence_length: 40960
sequence_packing:
fuse_loss: true
dtensor_cfg:
enabled: false
optimizer: null
Expand Down
2 changes: 2 additions & 0 deletions examples/configs/recipes/llm/performance/grpo-qwen3-30ba3b-4n8g.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,8 @@ policy:
train_micro_batch_size: 1
logprob_batch_size: 4
max_total_sequence_length: 4096
sequence_packing:
fuse_loss: true
dtensor_cfg:
enabled: false
optimizer: null
Expand Down
6 changes: 6 additions & 0 deletions tests/test_suites/llm/performance/grpo-deepseek-v3-64n8g-fp8-async-1off.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,12 @@ NUM_MINUTES=240

exit_if_max_steps_reached

# envvars for better fp8 inference performance
export VLLM_USE_FLASHINFER_MOE_FP8=1
export VLLM_FLASHINFER_MOE_BACKEND=latency
export VLLM_FLASHINFER_ALLREDUCE_BACKEND=mnnvl
export VLLM_ALLREDUCE_USE_FLASHINFER=1

# Run the experiment
cd $PROJECT_ROOT
uv run examples/run_grpo.py \
Expand Down
Loading