diff --git a/docs/advanced_features/pd_disaggregation.md b/docs/advanced_features/pd_disaggregation.md index b40ab11b4d01..64cb41a9fe5f 100644 --- a/docs/advanced_features/pd_disaggregation.md +++ b/docs/advanced_features/pd_disaggregation.md @@ -130,10 +130,12 @@ PD Disaggregation with Mooncake supports the following environment variables for To enable NVLink transport for KV cache transfers with the mooncake backend (recommended for NVL72 deployments), set the following environment variables. Note that auxiliary data transfer will still use TCP as a temporary workaround. ```bash -export SGLANG_MOONCAKE_CUSTOM_MEM_POOL=True +export SGLANG_MOONCAKE_CUSTOM_MEM_POOL=NVLINK export MC_FORCE_MNNVL=True ``` +The `SGLANG_MOONCAKE_CUSTOM_MEM_POOL` environment variable enables the custom memory pool. Supported values are `NVLINK` (or `True`), `BAREX`, and `INTRA_NODE_NVLINK`. + #### Prefill Server Configuration | Variable | Description | Default | |:--------:|:-----------:|:--------: diff --git a/docs/references/environment_variables.md b/docs/references/environment_variables.md index ac1b52a4b44f..2e0eebc19f33 100644 --- a/docs/references/environment_variables.md +++ b/docs/references/environment_variables.md @@ -98,6 +98,7 @@ SGLang supports various environment variables that can be used to configure its | `SGLANG_CLIP_MAX_NEW_TOKENS_ESTIMATION` | Clip max new tokens estimation for memory planning | `4096` | | `SGLANG_DETOKENIZER_MAX_STATES` | Maximum states for detokenizer | Default value based on system | | `SGLANG_ENABLE_TP_MEMORY_INBALANCE_CHECK` | Enable checks for memory imbalance across Tensor Parallel ranks | `true` | +| `SGLANG_MOONCAKE_CUSTOM_MEM_POOL` | Configure the custom memory pool type for Mooncake. Supports `NVLINK`, `BAREX`, `INTRA_NODE_NVLINK`. If set to `true`, it defaults to `NVLINK`. | `None` | ## Model-Specific Options