Repository navigation
[PD Disagg] Add bootstrap timeout to NIXL prefill sender - #34688
ting-hong-shieh wants to merge 1 commit into
Conversation
|
Hi @ByronHsu @cctry @ShangmingCai, could one of you help coordinate this with #17845 and trigger CI for the current commit? This implements the #34510 Step 1 NIXL Prefill bootstrap-timeout item using the current The red checks currently shown were created while this PR was a draft; their gate log says If continuing #17845 is preferred, I’m happy to close this PR. |
|
Quick coordination update: #34692 was opened after the comment above and implements the same #34510 Step 1 item. The production changes overlap: both initialize Could the maintainers advise which PR should continue? I’m happy to consolidate the work or close this one if another direction is preferred. |
|
Closing in favor of #34692, now merged as I also compared the test-only rewrite I had prepared with the public No branch update was made after #34692 merged. Thanks to @jambow0320 and @ShangmingCai for carrying the fix through. |
Motivation
Part of #34510 Step 1. Related to #17845.
CommonKVSenderalready implements the Prefill bootstrap timeout. Mooncake starts the deadline when its sender is created and checks it while the request is bootstrapping. NIXL does neither, so a request can remain inKVPoll.Bootstrappingindefinitely if Decode destination metadata never arrives.Changes
NixlKVSenderinitialization time._check_bootstrap_timeout()whilepoll()reportsKVPoll.Bootstrapping.The existing helper records the failure, changes the room status to
KVPoll.Failed, and returnsKVPoll.Failed. This patch does not change the common protocol, transport abstraction, or wire format.Tests
python3 -m compileall -q python/sglang/srt/disaggregation/nixl/conn.py test/registered/unit/disaggregation/test_nixl_backend_basic.pyisort --check-only python/sglang/srt/disaggregation/nixl/conn.py test/registered/unit/disaggregation/test_nixl_backend_basic.py(isort 7.0.0)black --check python/sglang/srt/disaggregation/nixl/conn.py test/registered/unit/disaggregation/test_nixl_backend_basic.py(Black 26.1.0)ruff check --select=F401,F821,UP037 python/sglang/srt/disaggregation/nixl/conn.py test/registered/unit/disaggregation/test_nixl_backend_basic.py(Ruff 0.15.1)git diff --check HEAD^ HEADThe focused registered unit-test target was not run locally because this lightweight environment does not include NumPy, PyTorch, pyzmq, or pytest.
Accuracy and performance
Not applicable. This change affects only the Prefill bootstrap control path and does not modify model execution or KV transfer.
CI States
Latest PR Test (Base): ❌ Run #31674506695
Latest PR Test (Extra): ❌ Run #31674506531