Skip to content

[CI] Revert the C8 configuration for some test cases #17505

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
kunpengW-code wants to merge 1 commit into
base: main
Choose a base branch
from
Open

[CI] Revert the C8 configuration for some test cases #17505

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 0 additions & 2 deletions docs/source/tutorials/models/GLM5.2.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -214,7 +214,6 @@ The parameters are explained as follows:
--gpu-memory-utilization 0.90 \
--quantization ascend \
--compilation-config '{"cudagraph_mode": "FULL_DECODE_ONLY"}' \
--kv-cache-dtype int8 \
--attention_config.indexer_kv_dtype int8 \
Comment thread
kunpengW-code marked this conversation as resolved.
--additional-config '{"enable_dsa_cp": true, "enable_balance_scheduling": true,"fuse_muls_add":true,"multistream_overlap_shared_expert":true,"c8_enable_reshape_optim":false, "enable_reduce_sample": "True", "enable_flashcomm1": true, "enable_fused_mc2": 1}' \
--speculative-config '{"num_speculative_tokens": 3, "method": "deepseek_mtp","enforce_eager":true}'
Expand Down Expand Up @@ -262,7 +261,6 @@ The parameters are explained as follows:
--gpu-memory-utilization 0.90 \
--quantization ascend \
--compilation-config '{"cudagraph_mode": "FULL_DECODE_ONLY"}' \
--kv-cache-dtype int8 \
--attention_config.indexer_kv_dtype int8 \
--additional-config '{"enable_dsa_cp": true, "enable_balance_scheduling": true,"fuse_muls_add":true,"multistream_overlap_shared_expert":true,"c8_enable_reshape_optim":false, "enable_reduce_sample": "True", "enable_flashcomm1": true, "enable_fused_mc2": 1}' \
--speculative-config '{"num_speculative_tokens": 3, "method": "deepseek_mtp","enforce_eager":true}'
Expand Down
20 changes: 10 additions & 10 deletions docs/source/tutorials/models/GLM5.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -223,8 +223,8 @@ If you want to deploy multi-node environment, you need to set up environment on
--enable-auto-tool-choice \
--tool-call-parser glm47 \
--reasoning-parser glm45 \
--kv-cache-dtype int8 \
--attention_config.indexer_kv_dtype int8 \
--kv-cache-dtype fp8 \
--attention_config.indexer_kv_dtype fp8 \
--compilation-config '{"cudagraph_mode": "FULL_DECODE_ONLY"}' \
--hf-overrides '{"use_index_cache": true, "index_topk_freq": 4}' \
--speculative-config '{"num_speculative_tokens": 3, "method": "deepseek_mtp", "enforce_eager": true}' \
Expand Down Expand Up @@ -707,8 +707,8 @@ if __name__ == "__main__":
--enable-auto-tool-choice \
--tool-call-parser glm47 \
--reasoning-parser glm45 \
--kv-cache-dtype int8 \
--attention_config.indexer_kv_dtype int8 \
--kv-cache-dtype fp8 \
--attention_config.indexer_kv_dtype fp8 \
--speculative-config '{"num_speculative_tokens": 1, "method": "deepseek_mtp", "enforce_eager": true}' \
--hf-overrides '{"use_index_cache": true, "index_topk_freq": 4}' \
--kv-transfer-config \
Expand Down Expand Up @@ -772,8 +772,8 @@ if __name__ == "__main__":
--enable-auto-tool-choice \
--tool-call-parser glm47 \
--reasoning-parser glm45 \
--kv-cache-dtype int8 \
--attention_config.indexer_kv_dtype int8 \
--kv-cache-dtype fp8 \
--attention_config.indexer_kv_dtype fp8 \
--speculative-config '{"num_speculative_tokens": 1, "method": "deepseek_mtp", "enforce_eager": true}' \
--hf-overrides '{"use_index_cache": true, "index_topk_freq": 4}' \
--kv-transfer-config \
Expand Down Expand Up @@ -839,8 +839,8 @@ if __name__ == "__main__":
--enable-auto-tool-choice \
--tool-call-parser glm47 \
--reasoning-parser glm45 \
--kv-cache-dtype int8 \
--attention_config.indexer_kv_dtype int8 \
--kv-cache-dtype fp8 \
--attention_config.indexer_kv_dtype fp8 \
--compilation-config '{"cudagraph_mode":"FULL_DECODE_ONLY"}' \
--speculative-config '{"num_speculative_tokens": 3, "method": "deepseek_mtp", "enforce_eager": true}' \
--hf-overrides '{"use_index_cache": true, "index_topk_freq": 4}' \
Expand Down Expand Up @@ -907,8 +907,8 @@ if __name__ == "__main__":
--enable-auto-tool-choice \
--tool-call-parser glm47 \
--reasoning-parser glm45 \
--kv-cache-dtype int8 \
--attention_config.indexer_kv_dtype int8 \
--kv-cache-dtype fp8 \
--attention_config.indexer_kv_dtype fp8 \
--compilation-config '{"cudagraph_mode":"FULL_DECODE_ONLY"}' \
--speculative-config '{"num_speculative_tokens": 3, "method": "deepseek_mtp", "enforce_eager": true}' \
--hf-overrides '{"use_index_cache": true, "index_topk_freq": 4}' \
Expand Down
2 changes: 0 additions & 2 deletions tests/e2e/nightly/multi_node/internal_dp/config/GLM-5.1-W8A8C8-A3_128k_90_50.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,6 @@ deployment:
--enable-prefix-caching
--async-scheduling
--compilation-config '{"cudagraph_mode": "FULL_DECODE_ONLY"}'
--kv-cache-dtype int8
--attention_config.indexer_kv_dtype int8
--additional-config '{"enable_dsa_cp": true, "enable_balance_scheduling": true, "ascend_compilation_config": {"fuse_muls_add": true}, "multistream_overlap_shared_expert": true,"enable_fused_mc2":1,"enable_mlapo":true}'
--speculative-config '{"num_speculative_tokens": 3, "method": "deepseek_mtp","enforce_eager":true}'
Expand Down Expand Up @@ -75,7 +74,6 @@ deployment:
--enable-prefix-caching
--async-scheduling
--compilation-config '{"cudagraph_mode": "FULL_DECODE_ONLY"}'
--kv-cache-dtype int8
--attention_config.indexer_kv_dtype int8
--additional-config '{"enable_dsa_cp": true, "enable_balance_scheduling": true, "ascend_compilation_config": {"fuse_muls_add": true}, "multistream_overlap_shared_expert": true,"enable_fused_mc2":1,"enable_mlapo":true}'
--speculative-config '{"num_speculative_tokens": 3, "method": "deepseek_mtp","enforce_eager":true}'
Expand Down
16 changes: 0 additions & 16 deletions tests/e2e/weekly/multi_node/external_dp/config/GLM-5.1-W8A8C8-3.5k-1.5k-0-20-PD.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -88,10 +88,6 @@ templates:
- "1024"
- --max-model-len
- "18432"
- --kv-cache-dtype
- "int8"
- --attention_config.indexer_kv_dtype
- "int8"
- --additional-config
- '{"ascend_compilation_config": {"fuse_muls_add": true},"recompute_scheduler_enable" : false,"multistream_overlap_shared_expert": true, "enable_dsa_cp":true, "enable_fused_mc2":1}'
- --max-num-batched-tokens
Expand Down Expand Up @@ -156,10 +152,6 @@ templates:
- "1024"
- --max-model-len
- "18432"
- --kv-cache-dtype
- "int8"
- --attention_config.indexer_kv_dtype
- "int8"
- --additional-config
- '{"ascend_compilation_config": {"fuse_muls_add": true},"recompute_scheduler_enable" : false,"multistream_overlap_shared_expert": true, "enable_dsa_cp":true, "enable_fused_mc2":1}'
- --max-num-batched-tokens
Expand Down Expand Up @@ -228,10 +220,6 @@ templates:
- '{"cudagraph_mode":"FULL_DECODE_ONLY"}'
- --speculative-config
- '{"num_speculative_tokens": 3, "method":"deepseek_mtp","enforce_eager":true}'
- --kv-cache-dtype
- "int8"
- --attention_config.indexer_kv_dtype
- "int8"
- --additional-config
- '{"ascend_compilation_config": {"fuse_muls_add": true},"recompute_scheduler_enable":true,"multistream_overlap_shared_expert":true,"enable_fused_mc2":1,"enable_mlapo":true}'
- --trust-remote-code
Expand Down Expand Up @@ -297,10 +285,6 @@ templates:
- '{"cudagraph_mode":"FULL_DECODE_ONLY"}'
- --speculative-config
- '{"num_speculative_tokens": 3, "method":"deepseek_mtp","enforce_eager":true}'
- --kv-cache-dtype
- "int8"
- --attention_config.indexer_kv_dtype
- "int8"
- --additional-config
- '{"ascend_compilation_config": {"fuse_muls_add": true},"recompute_scheduler_enable":true,"multistream_overlap_shared_expert":true,"enable_fused_mc2":1,"enable_mlapo":true}'
- --max-num-seqs
Expand Down
16 changes: 0 additions & 16 deletions tests/e2e/weekly/multi_node/external_dp/config/GLM-5.1-W8A8C8-3.5k-1.5k-0-50-PD.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -89,10 +89,6 @@ templates:
- "1024"
- --max-model-len
- "18432"
- --kv-cache-dtype
- "int8"
- --attention_config.indexer_kv_dtype
- "int8"
- --additional-config
- '{"ascend_compilation_config": {"fuse_muls_add": true},"recompute_scheduler_enable" : false,"multistream_overlap_shared_expert": true, "enable_dsa_cp":true, "enable_fused_mc2":1}'
- --max-num-batched-tokens
Expand Down Expand Up @@ -157,10 +153,6 @@ templates:
- "1024"
- --max-model-len
- "18432"
- --kv-cache-dtype
- "int8"
- --attention_config.indexer_kv_dtype
- "int8"
- --additional-config
- '{"ascend_compilation_config": {"fuse_muls_add": true},"recompute_scheduler_enable" : false,"multistream_overlap_shared_expert": true, "enable_dsa_cp":true, "enable_fused_mc2":1}'
- --max-num-batched-tokens
Expand Down Expand Up @@ -229,10 +221,6 @@ templates:
- '{"cudagraph_mode":"FULL_DECODE_ONLY"}'
- --speculative-config
- '{"num_speculative_tokens": 3, "method":"deepseek_mtp","enforce_eager":true}'
- --kv-cache-dtype
- "int8"
- --attention_config.indexer_kv_dtype
- "int8"
- --additional-config
- '{"ascend_compilation_config": {"fuse_muls_add": true},"recompute_scheduler_enable":true,"multistream_overlap_shared_expert":true, "finegrained_tp_config": {"lmhead_tensor_parallel_size":16},"enable_fused_mc2":1,"enable_mlapo":true}'
- --trust-remote-code
Expand Down Expand Up @@ -298,10 +286,6 @@ templates:
- '{"cudagraph_mode":"FULL_DECODE_ONLY"}'
- --speculative-config
- '{"num_speculative_tokens": 3, "method":"deepseek_mtp","enforce_eager":true}'
- --kv-cache-dtype
- "int8"
- --attention_config.indexer_kv_dtype
- "int8"
- --additional-config
- '{"ascend_compilation_config": {"fuse_muls_add": true},"recompute_scheduler_enable":true,"multistream_overlap_shared_expert":true, "finegrained_tp_config": {"lmhead_tensor_parallel_size":16},"enable_fused_mc2":1,"enable_mlapo":true}'
- --trust-remote-code
Expand Down
8 changes: 0 additions & 8 deletions tests/e2e/weekly/multi_node/external_dp/config/GLM-5.1-W8A8C8-64k-1k-90-50-PD.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -88,8 +88,6 @@ templates:
- "1024"
- --max-model-len
- "133120"
- --kv-cache-dtype
- "int8"
- --attention_config.indexer_kv_dtype
- "int8"
- --additional-config
Expand Down Expand Up @@ -155,8 +153,6 @@ templates:
- "1024"
- --max-model-len
- "133120"
- --kv-cache-dtype
- "int8"
- --attention_config.indexer_kv_dtype
- "int8"
- --additional-config
Expand Down Expand Up @@ -226,8 +222,6 @@ templates:
- '{"cudagraph_mode":"FULL_DECODE_ONLY"}'
- --speculative-config
- '{"num_speculative_tokens": 3, "method":"deepseek_mtp","enforce_eager":true}'
- --kv-cache-dtype
- "int8"
- --attention_config.indexer_kv_dtype
- "int8"
- --additional-config
Expand Down Expand Up @@ -294,8 +288,6 @@ templates:
- '{"cudagraph_mode":"FULL_DECODE_ONLY"}'
- --speculative-config
- '{"num_speculative_tokens": 3, "method":"deepseek_mtp","enforce_eager":true}'
- --kv-cache-dtype
- "int8"
- --attention_config.indexer_kv_dtype
- "int8"
- --additional-config
Expand Down
8 changes: 0 additions & 8 deletions tests/e2e/weekly/multi_node/external_dp/config/GLM-5.2-W4A8C8-1M-PD-DCP.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -108,8 +108,6 @@ templates:
- --cp-kv-cache-interleave-size
- "128"
- --enforce-eager
- --kv-cache-dtype
- "int8"
- --attention_config.indexer_kv_dtype
- "int8"
- --additional-config
Expand Down Expand Up @@ -168,8 +166,6 @@ templates:
- --cp-kv-cache-interleave-size
- "128"
- --enforce-eager
- --kv-cache-dtype
- "int8"
- --attention_config.indexer_kv_dtype
- "int8"
- --additional-config
Expand Down Expand Up @@ -229,8 +225,6 @@ templates:
- "128"
- --compilation-config
- '{"cudagraph_mode": "FULL_DECODE_ONLY"}'
- --kv-cache-dtype
- "int8"
- --attention_config.indexer_kv_dtype
- "int8"
- --additional-config
Expand Down Expand Up @@ -290,8 +284,6 @@ templates:
- "128"
- --compilation-config
- '{"cudagraph_mode": "FULL_DECODE_ONLY"}'
- --kv-cache-dtype
- "int8"
- --attention_config.indexer_kv_dtype
- "int8"
- --additional-config
Expand Down
16 changes: 0 additions & 16 deletions tests/e2e/weekly/multi_node/external_dp/config/GLM-5.2-W8A8C8-3.5k-1.5k-0-20-PD.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -88,10 +88,6 @@ templates:
- "1024"
- --max-model-len
- "18432"
- --kv-cache-dtype
- "int8"
- --attention_config.indexer_kv_dtype
- "int8"
- --additional-config
- '{"ascend_compilation_config": {"fuse_muls_add": true},"recompute_scheduler_enable" : false,"multistream_overlap_shared_expert": true, "enable_dsa_cp":true, "enable_fused_mc2":1}'
- --max-num-batched-tokens
Expand Down Expand Up @@ -156,10 +152,6 @@ templates:
- "1024"
- --max-model-len
- "18432"
- --kv-cache-dtype
- "int8"
- --attention_config.indexer_kv_dtype
- "int8"
- --additional-config
- '{"ascend_compilation_config": {"fuse_muls_add": true},"recompute_scheduler_enable" : false,"multistream_overlap_shared_expert": true, "enable_dsa_cp":true, "enable_fused_mc2":1}'
- --max-num-batched-tokens
Expand Down Expand Up @@ -228,10 +220,6 @@ templates:
- '{"cudagraph_mode":"FULL_DECODE_ONLY"}'
- --speculative-config
- '{"num_speculative_tokens": 3, "method":"deepseek_mtp","enforce_eager":true}'
- --kv-cache-dtype
- "int8"
- --attention_config.indexer_kv_dtype
- "int8"
- --additional-config
- '{"ascend_compilation_config": {"fuse_muls_add": true},"recompute_scheduler_enable":true,"multistream_overlap_shared_expert":true,"enable_fused_mc2":1,"enable_mlapo":true}'
- --trust-remote-code
Expand Down Expand Up @@ -297,10 +285,6 @@ templates:
- '{"cudagraph_mode":"FULL_DECODE_ONLY"}'
- --speculative-config
- '{"num_speculative_tokens": 3, "method":"deepseek_mtp","enforce_eager":true}'
- --kv-cache-dtype
- "int8"
- --attention_config.indexer_kv_dtype
- "int8"
- --additional-config
- '{"ascend_compilation_config": {"fuse_muls_add": true},"recompute_scheduler_enable":true,"multistream_overlap_shared_expert":true,"enable_fused_mc2":1,"enable_mlapo":true}'
- --max-num-seqs
Expand Down
16 changes: 0 additions & 16 deletions tests/e2e/weekly/multi_node/external_dp/config/GLM-5.2-W8A8C8-3.5k-1.5k-0-50-PD.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -89,10 +89,6 @@ templates:
- "1024"
- --max-model-len
- "18432"
- --kv-cache-dtype
- "int8"
- --attention_config.indexer_kv_dtype
- "int8"
- --additional-config
- '{"ascend_compilation_config": {"fuse_muls_add": true},"recompute_scheduler_enable" : false,"multistream_overlap_shared_expert": true, "enable_dsa_cp":true, "enable_fused_mc2":1}'
- --max-num-batched-tokens
Expand Down Expand Up @@ -157,10 +153,6 @@ templates:
- "1024"
- --max-model-len
- "18432"
- --kv-cache-dtype
- "int8"
- --attention_config.indexer_kv_dtype
- "int8"
- --additional-config
- '{"ascend_compilation_config": {"fuse_muls_add": true},"recompute_scheduler_enable" : false,"multistream_overlap_shared_expert": true, "enable_dsa_cp":true, "enable_fused_mc2":1}'
- --max-num-batched-tokens
Expand Down Expand Up @@ -229,10 +221,6 @@ templates:
- '{"cudagraph_mode":"FULL_DECODE_ONLY"}'
- --speculative-config
- '{"num_speculative_tokens": 3, "method":"deepseek_mtp","enforce_eager":true}'
- --kv-cache-dtype
- "int8"
- --attention_config.indexer_kv_dtype
- "int8"
- --additional-config
- '{"ascend_compilation_config": {"fuse_muls_add": true},"recompute_scheduler_enable":true,"multistream_overlap_shared_expert":true, "finegrained_tp_config": {"lmhead_tensor_parallel_size":16},"enable_fused_mc2":1,"enable_mlapo":true}'
- --trust-remote-code
Expand Down Expand Up @@ -298,10 +286,6 @@ templates:
- '{"cudagraph_mode":"FULL_DECODE_ONLY"}'
- --speculative-config
- '{"num_speculative_tokens": 3, "method":"deepseek_mtp","enforce_eager":true}'
- --kv-cache-dtype
- "int8"
- --attention_config.indexer_kv_dtype
- "int8"
- --additional-config
- '{"ascend_compilation_config": {"fuse_muls_add": true},"recompute_scheduler_enable":true,"multistream_overlap_shared_expert":true, "finegrained_tp_config": {"lmhead_tensor_parallel_size":16},"enable_fused_mc2":1,"enable_mlapo":true}'
- --trust-remote-code
Expand Down
8 changes: 0 additions & 8 deletions tests/e2e/weekly/multi_node/external_dp/config/GLM-5.2-W8A8C8-64k-1k-90-50-PD.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -88,8 +88,6 @@ templates:
- "1024"
- --max-model-len
- "133120"
- --kv-cache-dtype
- "int8"
- --attention_config.indexer_kv_dtype
- "int8"
- --additional-config
Expand Down Expand Up @@ -156,8 +154,6 @@ templates:
- "1024"
- --max-model-len
- "133120"
- --kv-cache-dtype
- "int8"
- --attention_config.indexer_kv_dtype
- "int8"
- --additional-config
Expand Down Expand Up @@ -228,8 +224,6 @@ templates:
- '{"cudagraph_mode":"FULL_DECODE_ONLY"}'
- --speculative-config
- '{"num_speculative_tokens": 3, "method":"deepseek_mtp","enforce_eager":true}'
- --kv-cache-dtype
- "int8"
- --attention_config.indexer_kv_dtype
- "int8"
- --additional-config
Expand Down Expand Up @@ -297,8 +291,6 @@ templates:
- '{"cudagraph_mode":"FULL_DECODE_ONLY"}'
- --speculative-config
- '{"num_speculative_tokens": 3, "method":"deepseek_mtp","enforce_eager":true}'
- --kv-cache-dtype
- "int8"
- --attention_config.indexer_kv_dtype
- "int8"
- --additional-config
Expand Down
4 changes: 0 additions & 4 deletions tests/e2e/weekly/multi_node/internal_dp/config/GLM-5.1-W8A8C8-A3_3.5k_1.5k_0_20.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,8 +44,6 @@ deployment:
--no-enable-prefix-caching
--async-scheduling
--compilation-config '{"cudagraph_mode": "FULL_DECODE_ONLY"}'
--kv-cache-dtype int8
--attention_config.indexer_kv_dtype int8
--additional-config '{"enable_dsa_cp": true, "enable_balance_scheduling": true, "ascend_compilation_config": {"fuse_muls_add": true}, "multistream_overlap_shared_expert": false,"enable_fused_mc2":1,"enable_mlapo":true}'
--speculative-config '{"num_speculative_tokens": 3, "method": "deepseek_mtp","enforce_eager":true}'
-
Expand Down Expand Up @@ -78,8 +76,6 @@ deployment:
--no-enable-prefix-caching
--async-scheduling
--compilation-config '{"cudagraph_mode": "FULL_DECODE_ONLY"}'
--kv-cache-dtype int8
--attention_config.indexer_kv_dtype int8
--additional-config '{"enable_dsa_cp": true, "enable_balance_scheduling": true, "ascend_compilation_config": {"fuse_muls_add": true}, "multistream_overlap_shared_expert": false,"enable_fused_mc2":1,"enable_mlapo":true}'
--speculative-config '{"num_speculative_tokens": 3, "method": "deepseek_mtp","enforce_eager":true}'
benchmarks:
Expand Down
Loading
Loading