From 580109e918c7b6118f835ee165110ee5e8bcebe2 Mon Sep 17 00:00:00 2001 From: Chuang Zhu <111838961+chuangz0@users.noreply.github.com> Date: Thu, 20 Aug 2026 08:39:22 +0000 Subject: [PATCH] [https://nvbugs/6632606][fix] Pass server_start_timeout to Ray disagg proxy The Ray disaggregated serving example launches the disagg proxy without --server_start_timeout, so the proxy gives up waiting for the context and generation workers after the 180s default and exits silently, while the test waits 300s for port 8000. Under CPU contention on shared CI nodes the tp2 workers routinely need more than 180s to load the model and build engines, which makes the proxy exit before they are ready and the test fail with 'Disaggregated server failed to start within 5 minutes'. Align the proxy timeout with the 300s the test already waits, matching what test_disaggregated.py already does via --server-start-timeout. Verified on a B200 node: with 28 CPU-burner processes the unpatched script fails within 2 iterations; with this fix 4/4 iterations of both test_ray_disaggregated_serving[tp2] variants pass under the same load. Clean-environment runs pass 8/8 iterations. Unwaive both tests (nvbugs 6632606 and 6633930). Co-Authored-By: Claude Fable 5 Signed-off-by: Chuang Zhu <111838961+chuangz0@users.noreply.github.com> --- examples/ray_orchestrator/disaggregated/disagg_serving_local.sh | 2 +- tests/integration/test_lists/waives.txt | 2 -- 2 files changed, 1 insertion(+), 3 deletions(-) diff --git a/examples/ray_orchestrator/disaggregated/disagg_serving_local.sh b/examples/ray_orchestrator/disaggregated/disagg_serving_local.sh index b831497ad342..40ee6745e693 100644 --- a/examples/ray_orchestrator/disaggregated/disagg_serving_local.sh +++ b/examples/ray_orchestrator/disaggregated/disagg_serving_local.sh @@ -177,7 +177,7 @@ trtllm-serve $MODEL_DIR --host localhost --tp_size $TP_SIZE --port 8002 --kv_cac # Launching disaggregated server echo "Launching disaggregated server..." -trtllm-serve disaggregated -c disagg_config_local.yaml +trtllm-serve disaggregated -c disagg_config_local.yaml -t 300 # Cleanup if [[ "$RAY_STARTED" == "true" && "$ATTACH_MODE" != "true" ]]; then diff --git a/tests/integration/test_lists/waives.txt b/tests/integration/test_lists/waives.txt index 50a5f0f53800..7400b8dd4303 100644 --- a/tests/integration/test_lists/waives.txt +++ b/tests/integration/test_lists/waives.txt @@ -118,7 +118,6 @@ disaggregated/test_workers.py::test_workers_kv_cache_aware_router_deepseek_v3_li disaggregated/test_workers.py::test_workers_kv_cache_aware_router_eviction[TinyLlama-1.1B-Chat-v1.0] SKIP (https://nvbugs/6162322) examples/test_ad_speculative_decoding.py::test_autodeploy_eagle3_one_model_acceptance_rate[trtllm-torch-cudagraph] SKIP (https://nvbugs/6426841) examples/test_ad_speculative_decoding.py::test_nemotron_mtp_model_with_weights SKIP (https://nvbugs/6630699) -examples/test_ray.py::test_ray_disaggregated_serving[tp2] SKIP (https://nvbugs/6632606) examples/test_ray.py::test_ray_disaggregated_serving_python[tp2] SKIP (https://nvbugs/6601574) examples/visual_gen/test_visual_gen_cosmos3.py::test_cosmos3_feature_accuracy_against_golden[nvfp4] SKIP (https://nvbugs/6572800) examples/visual_gen/test_visual_gen_cosmos3.py::test_cosmos3_nano_t2v_lpips_against_golden SKIP (https://nvbugs/6655359) @@ -187,7 +186,6 @@ full:DGX_B200/accuracy/test_llm_api_pytorch.py::TestDeepSeekV3Lite::test_bfloat1 full:DGX_B200/accuracy/test_llm_api_pytorch.py::TestDeepSeekV4Pro::test_gsm8k_full_accuracy SKIP (https://nvbugs/6571418) full:DGX_B200/accuracy/test_llm_api_pytorch_multimodal.py::TestNanoV3Omni::test_auto_dtype[fp8_mmmu_encoder_cuda_graph] SKIP (https://nvbugs/6631019) full:DGX_B200/disaggregated/test_disaggregated.py::test_disaggregated_gpt_oss_120b_harmony[gpt_oss/gpt-oss-120b] SKIP (https://nvbugs/6594241) -full:DGX_B200/examples/test_ray.py::test_ray_disaggregated_serving_python[tp2] SKIP (https://nvbugs/6633930) full:DGX_H100/accuracy/test_llm_api_pytorch.py::TestDeepSeekV3Lite::test_bfloat16[mtp_nextn=2-attention_dp=False-cuda_graph=False-overlap_scheduler=False-torch_compile=False-enable_chunked_prefill=False-v2_kv_cache=False] SKIP (https://nvbugs/6633927) full:DGX_H100/accuracy/test_llm_api_pytorch.py::TestDeepSeekV3Lite::test_bfloat16[mtp_nextn=2-attention_dp=False-cuda_graph=False-overlap_scheduler=False-torch_compile=True-enable_chunked_prefill=False-v2_kv_cache=True] SKIP (https://nvbugs/6633927) full:DGX_H100/accuracy/test_llm_api_pytorch.py::TestDeepSeekV3Lite::test_bfloat16[mtp_nextn=2-attention_dp=False-cuda_graph=True-overlap_scheduler=False-torch_compile=False-enable_chunked_prefill=False-v2_kv_cache=True] SKIP (https://nvbugs/6633927)