Skip to content

[None][test] Enable overlap schedule on qa side for disagg perf #17390

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
fredricz-20070104 merged 5 commits into from
Aug 7, 2026
Merged

[None][test] Enable overlap schedule on qa side for disagg perf #17390

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
aa14ef0
enable overlap schedule on qa side for disagg perf
fredricz-20070104 Aug 7, 2026
b613e66
disable overlap schedule for some of the cases for coverage reason, s…
fredricz-20070104 Aug 7, 2026
646f409
Merge remote-tracking branch 'upstream/main' into feature/enable_over…
fredricz-20070104 Aug 7, 2026
c50439c
[None][test] Remove stale Kimi entries from llm_perf_disagg.yml
fredricz-20070104 Aug 7, 2026
d47550d
enable overlap schedule for both ci and qa side
fredricz-20070104 Aug 7, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions ...gregated/gb200_deepseek-r1-fp4_128k8k_con128_ctx1_pp8_gen1_dep16_eplb0_mtp1_ccb-NIXL.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,7 @@ worker_config:
backend: NIXL
transceiver_runtime: PYTHON
kv_transfer_timeout_ms: 600000
disable_overlap_scheduler: true
disable_overlap_scheduler: false
speculative_config: &id001
decoding_type: MTP
num_nextn_predict_layers: 1
Expand Down Expand Up @@ -104,5 +104,5 @@ worker_config:
backend: NIXL
transceiver_runtime: PYTHON
kv_transfer_timeout_ms: 600000
disable_overlap_scheduler: true
disable_overlap_scheduler: false
speculative_config: *id001
4 changes: 2 additions & 2 deletions ...ggregated/gb200_deepseek-r1-fp4_128k8k_con128_ctx1_pp8_gen1_dep16_eplb0_mtp1_ccb-UCX.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,7 @@ worker_config:
cache_transceiver_config:
max_tokens_in_buffer: 131104
backend: UCX
disable_overlap_scheduler: true
disable_overlap_scheduler: false
speculative_config: &id001
decoding_type: MTP
max_draft_len: 1
Expand All @@ -100,5 +100,5 @@ worker_config:
cache_transceiver_config:
max_tokens_in_buffer: 131104
backend: UCX
disable_overlap_scheduler: true
disable_overlap_scheduler: false
speculative_config: *id001
4 changes: 2 additions & 2 deletions ...saggregated/gb200_deepseek-r1-fp4_128k8k_con1_ctx1_pp8_gen1_tep8_eplb0_mtp3_ccb-NIXL.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,7 @@ worker_config:
max_tokens_in_buffer: 131104
backend: NIXL
transceiver_runtime: PYTHON
disable_overlap_scheduler: true
disable_overlap_scheduler: false
speculative_config: &id001
decoding_type: MTP
num_nextn_predict_layers: 3
Expand Down Expand Up @@ -101,5 +101,5 @@ worker_config:
max_tokens_in_buffer: 131104
backend: NIXL
transceiver_runtime: PYTHON
disable_overlap_scheduler: true
disable_overlap_scheduler: false
speculative_config: *id001
4 changes: 2 additions & 2 deletions ...ggregated/gb200_deepseek-r1-fp4_128k8k_con64_ctx1_pp8_gen1_dep32_eplb0_mtp3_ccb-NIXL.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,7 @@ worker_config:
max_tokens_in_buffer: 131104
backend: NIXL
transceiver_runtime: PYTHON
disable_overlap_scheduler: true
disable_overlap_scheduler: false
speculative_config: &id001
decoding_type: MTP
num_nextn_predict_layers: 3
Expand All @@ -102,5 +102,5 @@ worker_config:
max_tokens_in_buffer: 131104
backend: NIXL
transceiver_runtime: PYTHON
disable_overlap_scheduler: true
disable_overlap_scheduler: false
speculative_config: *id001
3 changes: 2 additions & 1 deletion ...isaggregated/gb200_deepseek-r1-fp4_128k8k_ctx1_pp4_gen8_pp4_bs2_eplb0_mtp0_con2-NIXL.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,7 @@ worker_config:
max_tokens_in_buffer: 131104
backend: NIXL
transceiver_runtime: PYTHON
disable_overlap_scheduler: false
stream_interval: 20
num_postprocess_workers: 4
allreduce_strategy: MNNVL
Expand All @@ -85,7 +86,7 @@ worker_config:
pipeline_parallel_size: 4
print_iter_log: true
cuda_graph_config: null
disable_overlap_scheduler: true
disable_overlap_scheduler: false
kv_cache_config:
enable_block_reuse: false
free_gpu_memory_fraction: 0.4
Expand Down
4 changes: 2 additions & 2 deletions ...gregated/gb200_deepseek-r1-fp4_1k1k_con1024_ctx1_dep4_gen1_dep32_eplb0_mtp3_ccb-NIXL.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ worker_config:
max_tokens_in_buffer: 16384
backend: NIXL
transceiver_runtime: PYTHON
disable_overlap_scheduler: true
disable_overlap_scheduler: false
speculative_config: &id001
decoding_type: MTP
max_draft_len: 3
Expand Down Expand Up @@ -92,5 +92,5 @@ worker_config:
max_tokens_in_buffer: 16384
backend: NIXL
transceiver_runtime: PYTHON
disable_overlap_scheduler: true
disable_overlap_scheduler: false
speculative_config: *id001
4 changes: 2 additions & 2 deletions ...isaggregated/gb200_deepseek-r1-fp4_1k1k_con1_ctx1_dep4_gen1_tep8_eplb0_mtp3_ccb-NIXL.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ worker_config:
max_tokens_in_buffer: 16384
backend: NIXL
transceiver_runtime: PYTHON
disable_overlap_scheduler: true
disable_overlap_scheduler: false
speculative_config: &id001
decoding_type: MTP
max_draft_len: 3
Expand Down Expand Up @@ -91,5 +91,5 @@ worker_config:
max_tokens_in_buffer: 16384
backend: NIXL
transceiver_runtime: PYTHON
disable_overlap_scheduler: true
disable_overlap_scheduler: false
speculative_config: *id001
4 changes: 2 additions & 2 deletions ...ggregated/gb200_deepseek-r1-fp4_1k1k_con3072_ctx1_dep4_gen1_dep4_eplb0_mtp1_ccb-NIXL.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ worker_config:
max_tokens_in_buffer: 16384
backend: NIXL
transceiver_runtime: PYTHON
disable_overlap_scheduler: true
disable_overlap_scheduler: false
speculative_config: &id001
decoding_type: MTP
num_nextn_predict_layers: 1
Expand Down Expand Up @@ -92,5 +92,5 @@ worker_config:
max_tokens_in_buffer: 16384
backend: NIXL
transceiver_runtime: PYTHON
disable_overlap_scheduler: true
disable_overlap_scheduler: false
speculative_config: *id001
4 changes: 2 additions & 2 deletions ...gregated/gb200_deepseek-r1-fp4_8k1k_con1024_ctx1_dep4_gen1_dep32_eplb0_mtp3_ccb-NIXL.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,7 @@ worker_config:
max_tokens_in_buffer: 16384
backend: NIXL
transceiver_runtime: PYTHON
disable_overlap_scheduler: true
disable_overlap_scheduler: false
speculative_config: &id001
decoding_type: MTP
num_nextn_predict_layers: 3
Expand Down Expand Up @@ -103,5 +103,5 @@ worker_config:
max_tokens_in_buffer: 16384
backend: NIXL
transceiver_runtime: PYTHON
disable_overlap_scheduler: true
disable_overlap_scheduler: false
speculative_config: *id001
4 changes: 2 additions & 2 deletions ...isaggregated/gb200_deepseek-r1-fp4_8k1k_con1_ctx1_dep4_gen1_tep8_eplb0_mtp3_ccb-NIXL.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -73,7 +73,7 @@ worker_config:
max_tokens_in_buffer: 16384
backend: NIXL
transceiver_runtime: PYTHON
disable_overlap_scheduler: true
disable_overlap_scheduler: false
speculative_config: &id001
decoding_type: MTP
num_nextn_predict_layers: 3
Expand Down Expand Up @@ -102,5 +102,5 @@ worker_config:
max_tokens_in_buffer: 16384
backend: NIXL
transceiver_runtime: PYTHON
disable_overlap_scheduler: true
disable_overlap_scheduler: false
speculative_config: *id001
2 changes: 1 addition & 1 deletion ...gregated/gb200_deepseek-r1-fp4_8k1k_con4096_ctx1_dep4_gen1_dep16_eplb0_mtp1_ccb-NIXL.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,7 @@ worker_config:
max_tokens_in_buffer: 16384
backend: NIXL
transceiver_runtime: PYTHON
disable_overlap_scheduler: true
disable_overlap_scheduler: false
speculative_config: &id001
decoding_type: MTP
num_nextn_predict_layers: 1
Expand Down
4 changes: 2 additions & 2 deletions ...ggregated/gb200_deepseek-r1-fp4_8k1k_con4096_ctx1_dep4_gen1_dep16_eplb0_mtp1_ccb-UCX.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,7 @@ worker_config:
cache_transceiver_config:
max_tokens_in_buffer: 16384
backend: UCX
disable_overlap_scheduler: true
disable_overlap_scheduler: false
speculative_config: &id001
decoding_type: MTP
max_draft_len: 1
Expand All @@ -101,5 +101,5 @@ worker_config:
cache_transceiver_config:
max_tokens_in_buffer: 16384
backend: UCX
disable_overlap_scheduler: true
disable_overlap_scheduler: false
speculative_config: *id001
3 changes: 2 additions & 1 deletion ...isaggregated/gb200_deepseek-r1-fp4_8k1k_ctx1_gen3_tep8_bs32_eplb0_mtp0_con1_ccb-NIXL.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,7 @@ worker_config:
max_tokens_in_buffer: 8448
backend: NIXL
transceiver_runtime: PYTHON
disable_overlap_scheduler: false
stream_interval: 20
num_postprocess_workers: 4
allreduce_strategy: MNNVL
Expand All @@ -88,7 +89,7 @@ worker_config:
pipeline_parallel_size: 1
print_iter_log: true
cuda_graph_config: null
disable_overlap_scheduler: true
disable_overlap_scheduler: false
kv_cache_config:
enable_block_reuse: false
free_gpu_memory_fraction: 0.75
Expand Down
4 changes: 2 additions & 2 deletions ...aggregated/gb200_deepseek-v32-fp4_32k4k_con1_ctx1_dep4_gen1_tep8_eplb0_mtp3_ccb-NIXL.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ worker_config:
max_tokens_in_buffer: 32832
backend: NIXL
transceiver_runtime: PYTHON
disable_overlap_scheduler: true
disable_overlap_scheduler: false
speculative_config: &id001
decoding_type: MTP
num_nextn_predict_layers: 3
Expand Down Expand Up @@ -115,5 +115,5 @@ worker_config:
max_tokens_in_buffer: 32832
backend: NIXL
transceiver_runtime: PYTHON
disable_overlap_scheduler: true
disable_overlap_scheduler: false
speculative_config: *id001
4 changes: 2 additions & 2 deletions ...ated/gb200_deepseek-v32-fp4_32k4k_con2048_ctx1_dep4_gen1_dep32_eplb288_mtp1_ccb-NIXL.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,7 @@ worker_config:
backend: NIXL
transceiver_runtime: PYTHON
kv_transfer_timeout_ms: 600000
disable_overlap_scheduler: true
disable_overlap_scheduler: false
Comment thread
fredricz-20070104 marked this conversation as resolved.
Resolved
speculative_config: &id001
decoding_type: MTP
num_nextn_predict_layers: 1
Expand Down Expand Up @@ -120,5 +120,5 @@ worker_config:
backend: NIXL
transceiver_runtime: PYTHON
kv_transfer_timeout_ms: 600000
disable_overlap_scheduler: true
disable_overlap_scheduler: false
Comment thread
fredricz-20070104 marked this conversation as resolved.
Resolved
speculative_config: *id001
4 changes: 2 additions & 2 deletions ...gated/gb200_deepseek-v32-fp4_32k4k_con2048_ctx1_dep4_gen1_dep32_eplb288_mtp1_ccb-UCX.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,7 @@ worker_config:
max_tokens_in_buffer: 32832
backend: UCX
kv_transfer_timeout_ms: 600000
disable_overlap_scheduler: true
disable_overlap_scheduler: false
speculative_config: &id001
decoding_type: MTP
max_draft_len: 1
Expand Down Expand Up @@ -118,5 +118,5 @@ worker_config:
max_tokens_in_buffer: 32832
backend: UCX
kv_transfer_timeout_ms: 600000
disable_overlap_scheduler: true
disable_overlap_scheduler: false
speculative_config: *id001
4 changes: 2 additions & 2 deletions ...regated/gb200_deepseek-v32-fp4_32k4k_con256_ctx1_dep4_gen1_dep32_eplb0_mtp3_ccb-NIXL.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,7 @@ worker_config:
max_tokens_in_buffer: 32832
backend: NIXL
transceiver_runtime: PYTHON
disable_overlap_scheduler: true
disable_overlap_scheduler: false
speculative_config: &id001
decoding_type: MTP
num_nextn_predict_layers: 3
Expand Down Expand Up @@ -115,5 +115,5 @@ worker_config:
max_tokens_in_buffer: 32832
backend: NIXL
transceiver_runtime: PYTHON
disable_overlap_scheduler: true
disable_overlap_scheduler: false
speculative_config: *id001
3 changes: 2 additions & 1 deletion ...gregated/gb200_deepseek-v32-fp4_32k4k_con256_ctx1_dep8_gen1_dep8_eplb0_mtp0_ccb-NIXL.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -71,6 +71,7 @@ worker_config:
max_tokens_in_buffer: 120000
backend: NIXL
transceiver_runtime: PYTHON
disable_overlap_scheduler: false
num_postprocess_workers: 8
stream_interval: 10
ctx:
Expand All @@ -84,7 +85,7 @@ worker_config:
max_num_tokens: 8192
max_seq_len: 121000
enable_chunked_prefill: true
disable_overlap_scheduler: true
disable_overlap_scheduler: false
cuda_graph_config:
enable_padding: true
max_batch_size: 32
Expand Down
4 changes: 2 additions & 2 deletions ...gated/gb200_deepseek-v32-fp4_8k1k_con1024_ctx1_dep4_gen1_dep32_eplb256_mtp3_ccb-NIXL.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,7 @@ worker_config:
max_tokens_in_buffer: 16384
backend: NIXL
transceiver_runtime: PYTHON
disable_overlap_scheduler: true
disable_overlap_scheduler: false
speculative_config: &id001
decoding_type: MTP
num_nextn_predict_layers: 3
Expand Down Expand Up @@ -114,5 +114,5 @@ worker_config:
max_tokens_in_buffer: 16384
backend: NIXL
transceiver_runtime: PYTHON
disable_overlap_scheduler: true
disable_overlap_scheduler: false
speculative_config: *id001
4 changes: 2 additions & 2 deletions ...saggregated/gb200_deepseek-v32-fp4_8k1k_con1_ctx1_dep4_gen1_tep8_eplb0_mtp3_ccb-NIXL.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,7 @@ worker_config:
max_tokens_in_buffer: 16384
backend: NIXL
transceiver_runtime: PYTHON
disable_overlap_scheduler: true
disable_overlap_scheduler: false
Comment thread
fredricz-20070104 marked this conversation as resolved.
Resolved
speculative_config: &id001
decoding_type: MTP
num_nextn_predict_layers: 3
Expand Down Expand Up @@ -110,5 +110,5 @@ worker_config:
max_tokens_in_buffer: 16384
backend: NIXL
transceiver_runtime: PYTHON
disable_overlap_scheduler: true
disable_overlap_scheduler: false
Comment thread
fredricz-20070104 marked this conversation as resolved.
Resolved
speculative_config: *id001
4 changes: 2 additions & 2 deletions ...gated/gb200_deepseek-v32-fp4_8k1k_con4096_ctx1_dep4_gen1_dep32_eplb256_mtp0_ccb-NIXL.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,7 @@ worker_config:
max_tokens_in_buffer: 16384
backend: NIXL
transceiver_runtime: PYTHON
disable_overlap_scheduler: true
disable_overlap_scheduler: false
num_postprocess_workers: 4
stream_interval: 20
nvfp4_gemm_config:
Expand Down Expand Up @@ -111,4 +111,4 @@ worker_config:
max_tokens_in_buffer: 16384
backend: NIXL
transceiver_runtime: PYTHON
disable_overlap_scheduler: true
disable_overlap_scheduler: false
4 changes: 2 additions & 2 deletions ...egated/gb200_deepseek-v32-fp4_8k1k_con4096_ctx1_dep4_gen1_dep32_eplb256_mtp0_ccb-UCX.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,7 @@ worker_config:
cache_transceiver_config:
max_tokens_in_buffer: 16384
backend: UCX
disable_overlap_scheduler: true
disable_overlap_scheduler: false
num_postprocess_workers: 4
stream_interval: 20
nvfp4_gemm_config:
Expand Down Expand Up @@ -109,4 +109,4 @@ worker_config:
cache_transceiver_config:
max_tokens_in_buffer: 16384
backend: UCX
disable_overlap_scheduler: true
disable_overlap_scheduler: false
4 changes: 2 additions & 2 deletions ...disaggregated/gb200_glm-5-fp4_8k1k_con1024_ctx1_dep4_gen1_dep8_eplb256_mtp1_ccb-NIXL.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ worker_config:
cache_transceiver_config:
max_tokens_in_buffer: 16384
backend: NIXL
disable_overlap_scheduler: true
disable_overlap_scheduler: false
speculative_config: &id001
decoding_type: MTP
num_nextn_predict_layers: 1
Expand All @@ -104,5 +104,5 @@ worker_config:
cache_transceiver_config:
max_tokens_in_buffer: 16384
backend: NIXL
disable_overlap_scheduler: true
disable_overlap_scheduler: false
speculative_config: *id001
4 changes: 2 additions & 2 deletions .../disaggregated/gb200_glm-5-fp4_8k1k_con1024_ctx1_dep4_gen1_dep8_eplb256_mtp1_ccb-UCX.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ worker_config:
cache_transceiver_config:
max_tokens_in_buffer: 16384
backend: UCX
disable_overlap_scheduler: true
disable_overlap_scheduler: false
speculative_config: &id001
decoding_type: MTP
num_nextn_predict_layers: 1
Expand All @@ -104,5 +104,5 @@ worker_config:
cache_transceiver_config:
max_tokens_in_buffer: 16384
backend: UCX
disable_overlap_scheduler: true
disable_overlap_scheduler: false
speculative_config: *id001
4 changes: 2 additions & 2 deletions ...perf/disaggregated/gb200_glm-5-fp4_8k1k_con1_ctx1_dep4_gen1_tep8_eplb0_mtp3_ccb-NIXL.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@ worker_config:
cache_transceiver_config:
max_tokens_in_buffer: 16384
backend: NIXL
disable_overlap_scheduler: true
disable_overlap_scheduler: false
speculative_config: &id001
decoding_type: MTP
num_nextn_predict_layers: 3
Expand Down Expand Up @@ -100,5 +100,5 @@ worker_config:
cache_transceiver_config:
max_tokens_in_buffer: 16384
backend: NIXL
disable_overlap_scheduler: true
disable_overlap_scheduler: false
speculative_config: *id001
Loading
Loading