Add IntraNode NVLink configration in PD disaggregation docs#23329
Conversation
|
Warning You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again! |
|
Please use pre-commit to format codes. docs/advanced_features/pd_disaggregation.md:136: awared ==> awarded |
Done |
|
Hi Mingyi @wisclmy0611 , thank you for your time and please help to review this PR. 🚀 |
|
Hi @TTThanos Could you pls resolve the conflict? Thanks! |
Hi @wisclmy0611 conflict solved. 😊 |
|
Hi, if this feature is still not available using docker images? |
Yep, I think so. To make sure you can leverage this feature successfully, you may need to git clone Mooncake repo and compile Mooncake enabling intraNode nvlink related cmake flags. :) |
…ect#23329) Co-authored-by: 百麒 <yaozhong.lyz@alibaba-inc.com>
…ect#23329) Co-authored-by: 百麒 <yaozhong.lyz@alibaba-inc.com>
Motivation
Add IntraNode NVLink configuration in PD disaggregation docs:
Users can launch SGlang server by using extra environment variables
export SGLANG_MOONCAKE_CUSTOM_MEM_POOL=INTRA_NODE_NVLINK export MC_INTRANODE_NVLINK=trueModifications
Accuracy Tests
Speed Tests and Profiling
Checklist
Review and Merge Process
/tag-and-rerun-ci,/tag-run-ci-label,/rerun-failed-ci