docs(vllm): cherry-pick worker-role flag corrections and --kv-transfer-config reference - #12666
Conversation
Cherry-pick of #12568 (27e6820) onto release/1.4.0, adapted for the pre-restructure docs layout: the fix lives at docs/fern/backends/vllm/vllm-config-reference.mdx here, not at docs/fern/pages/reference/backends/vllm-configuration.mdx. Documents --kv-transfer-config as a native vLLM argument, records that --disaggregation-mode prefill fails without it, and removes the --is-prefill-worker / --is-decode-worker entries and their validation rules, which the backend no longer accepts. Fixes 6550447 Signed-off-by: Dan Gil <dagil@nvidia.com>
github-actions
Bot
added
docs
documentation
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
| --kv-transfer-config '{"kv_connector":"NixlConnector","kv_role":"kv_both"}' | ||
| ``` | ||
|
|
||
| Only the prefill worker is required to set it, but both halves of a NIXL pair must agree on a connector for transfers to succeed. Pass the same `--kv-transfer-config` value to the decode worker, as the [disaggregated vLLM launch script](https://github.com/ai-dynamo/dynamo/blob/release/1.4.0/examples/backends/vllm/launch/disagg.sh) does. |
There was a problem hiding this comment.
🟡 Documentation link points at a release branch instead of the required main branch
The new launch-script link is written against the release/1.4.0 branch (https://github.com/ai-dynamo/dynamo/blob/release/1.4.0/examples/backends/vllm/launch/disagg.sh at docs/fern/backends/vllm/vllm-config-reference.mdx:92) instead of the branch the docs rules mandate, so the published page diverges from every other outbound source link.
Impact: Readers of the published docs follow a link pinned to an old snapshot rather than the current source, which will go stale as the example evolves.
Docs style rule on links outside docs/
docs/fern/documentation-style-guide.md:277-280 requires links to targets outside docs/ to be absolute GitHub URLs of the form https://github.com/ai-dynamo/dynamo/blob/main/<path>. Every other GitHub link on this page and in the sibling vLLM pages uses blob/main/ (e.g. docs/fern/backends/vllm/vllm-config-reference.mdx:8, docs/fern/backends/vllm/vllm-examples.mdx:48).
| Only the prefill worker is required to set it, but both halves of a NIXL pair must agree on a connector for transfers to succeed. Pass the same `--kv-transfer-config` value to the decode worker, as the [disaggregated vLLM launch script](https://github.com/ai-dynamo/dynamo/blob/release/1.4.0/examples/backends/vllm/launch/disagg.sh) does. | |
| Only the prefill worker is required to set it, but both halves of a NIXL pair must agree on a connector for transfers to succeed. Pass the same `--kv-transfer-config` value to the decode worker, as the [disaggregated vLLM launch script](https://github.com/ai-dynamo/dynamo/blob/main/examples/backends/vllm/launch/disagg.sh) does. |
Was this helpful? React with 👍 or 👎 to provide feedback.
![dynamo-review-agent[bot]](https://avatars.githubusercontent.com/u/201626793?s=60&v=4){kind=small}
2 participants
Cherry-pick of #12568 (merged to
mainas 27e6820 on Aug 3) ontorelease/1.4.0, for NVBug 6550447 / DYN-3726 (P0).Why this is not a clean cherry-pick
mainhas since restructured the Fern docs tree. The file lives atdocs/fern/pages/reference/backends/vllm-configuration.mdxonmainand atdocs/fern/backends/vllm/vllm-config-reference.mdxhere, sogit cherry-pickdoes not apply. The content change is the same; the relative link to the runtime config reference uses this branch's path, and the launch-script link points atrelease/1.4.0instead ofmain.What changed
--kv-transfer-configas a native vLLM engine argument, and warns that--disaggregation-mode prefillraises aValueErrorat argument parsing without it.--connector/DYN_CONNECTORare no longer accepted, with the message each rejected value produces.--disaggregation-modefield.--is-prefill-workerand--is-decode-workerentries and their two validation rules. The backend no longer accepts either flag, so the page documented them as working.Following the page as written on this branch fails: the removed role flags are shown as usable and the mandatory
--kv-transfer-configis absent.Fixes 6550447