fix: Add PIECEWISE cudagraph mode config for prefill server to avoid startup errors

[xbfs]

The default cudagraph configuration (FULL_AND_PIECEWISE) causes prefill instance startup errors. This change explicitly sets the cudagraph_mode to PIECEWISE for prefill servers in the disaggregated serving script.

[mergify]

Documentation preview: https://vllm--29079.org.readthedocs.build/en/29079/

[gemini-code-assist]

Code Review

This pull request addresses a startup error in the disaggregated serving example script. The error occurs in prefill server instances due to the default cudagraph_mode of FULL_AND_PIECEWISE. The proposed change correctly resolves this issue by explicitly setting the cudagraph_mode to PIECEWISE for the prefill server, using the --compilation-config argument. This is a targeted and appropriate fix, as the PIECEWISE mode is better suited for the dynamic nature of prefill operations, thus avoiding the startup failures. The implementation is correct, and I find no issues with this change.

[LucasWilkinson]

Can you please document the errors you are seeing?

[xbfs]

When the prefill instance runs to 'Capturing CUDA graphs (decode, FULL)', an error occurs:
(EngineCore_DP0 pid=1940) File "/usr/local/lib/python3.12/dist-packages/vllm/attention/layer.py", line 624, in unified_attention_with_output
(EngineCore_DP0 pid=1940) maybe_save_kv_layer_to_connector(layer_name, kv_cache)
(EngineCore_DP0 pid=1940) File "/usr/local/lib/python3.12/dist-packages/vllm/attention/layer.py", line 556, in maybe_save_kv_layer_to_connector
(EngineCore_DP0 pid=1940) connector.save_kv_layer(layer_name, kv_cache_layer,
(EngineCore_DP0 pid=1940) File "/usr/local/lib/python3.12/dist-packages/vllm/distributed/kv_transfer/kv_connector/v1/p2p/p2p_nccl_connector.py", line 267, in save_kv_layer
(EngineCore_DP0 pid=1940) connector_metadata = self._get_connector_metadata()
(EngineCore_DP0 pid=1940) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=1940) File "/usr/local/lib/python3.12/dist-packages/vllm/distributed/kv_transfer/kv_connector/v1/base.py", line 132, in _get_connector_metadata
(EngineCore_DP0 pid=1940) assert self._connector_metadata is not None
(EngineCore_DP0 pid=1940) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
(EngineCore_DP0 pid=1940) AssertionError

[LucasWilkinson]

cc @NickLucche @njhill

(potentially related: #27026)

[github-actions]

This pull request has been automatically marked as stale because it has not had any activity within 90 days. It will be automatically closed if no further activity occurs within 30 days. Leave a comment if you feel this pull request should remain open. Thank you!