Skip to content

docs(vllm): cherry-pick worker-role flag corrections and --kv-transfer-config reference #12666

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
pvijayakrish merged 1 commit into from
Aug 4, 2026
Merged

docs(vllm): cherry-pick worker-role flag corrections and --kv-transfer-config reference #12666

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 28 additions & 15 deletions docs/fern/backends/vllm/vllm-config-reference.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ subtitle: Field reference for the Dynamo-specific CLI flags and environment vari
`DynamoVllmConfig` holds the Dynamo-specific configuration for the vLLM backend (`python -m dynamo.vllm`). Every field, type, default, and choice on this page comes from the [`DynamoVllmArgGroup` and `DynamoVllmConfig`](https://github.com/ai-dynamo/dynamo/blob/main/components/src/dynamo/vllm/backend_args.py) definitions. For features and operational details, see the [vLLM Reference Guide](vllm-reference-guide.md).

<Note>
These are **only** the Dynamo wrapper flags. The vLLM backend also accepts every native vLLM `EngineArgs` argument (`--model`, `--tensor-parallel-size`, `--max-model-len`, and so on) in the same command, plus the cross-cutting [Dynamo Runtime](../../reference/runtime-config-reference.mdx) flags (`--namespace`, `--endpoint`, and others). Except for the KV event interoperability note below, this page covers neither — only the vLLM-specific `DYN_VLLM_*` surface.
These are **only** the Dynamo wrapper flags. The vLLM backend also accepts every native vLLM `EngineArgs` argument (`--model`, `--tensor-parallel-size`, `--max-model-len`, and so on) in the same command, plus the cross-cutting [Dynamo Runtime](../../reference/runtime-config-reference.mdx) flags (`--namespace`, `--endpoint`, and others). Except for the native KV event and KV transfer sections below, this page covers neither — only the vLLM-specific `DYN_VLLM_*` surface.
</Note>

## How the config is loaded
Expand Down Expand Up @@ -72,13 +72,40 @@ The `endpoint` value is the base ZeroMQ port. vLLM assigns each data-parallel ra

If workers do not publish KV events, configure the frontend with `--no-router-kv-events` for prediction-based KV routing or `--load-aware` for load-only routing.

## Native KV transfer configuration

`--kv-transfer-config` is a native vLLM engine argument rather than a `DynamoVllmConfig` field, so it has no `DYN_VLLM_*` environment variable. It selects the KV connector that moves cache blocks between prefill and decode workers.

<Warning>
A worker started with `--disaggregation-mode prefill` must be passed `--kv-transfer-config` explicitly. Without it, the worker raises a `ValueError` during argument parsing and never starts. All non-prefill modes — `agg`, `pd`, `decode`, and `encode` — do not enforce this check.
</Warning>

The value is a JSON object. For NIXL-based prefill/decode disaggregation:

```bash
python -m dynamo.vllm \
--model Qwen/Qwen3-0.6B \
--disaggregation-mode prefill \
--kv-transfer-config '{"kv_connector":"NixlConnector","kv_role":"kv_both"}'
```

Only the prefill worker is required to set it, but both halves of a NIXL pair must agree on a connector for transfers to succeed. Pass the same `--kv-transfer-config` value to the decode worker, as the [disaggregated vLLM launch script](https://github.com/ai-dynamo/dynamo/blob/release/1.4.0/examples/backends/vllm/launch/disagg.sh) does.

@devin-ai-integration devin-ai-integration Bot Aug 4, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Documentation link points at a release branch instead of the required main branch

The new launch-script link is written against the release/1.4.0 branch (https://github.com/ai-dynamo/dynamo/blob/release/1.4.0/examples/backends/vllm/launch/disagg.sh at docs/fern/backends/vllm/vllm-config-reference.mdx:92) instead of the branch the docs rules mandate, so the published page diverges from every other outbound source link.
Impact: Readers of the published docs follow a link pinned to an old snapshot rather than the current source, which will go stale as the example evolves.

Docs style rule on links outside docs/

docs/fern/documentation-style-guide.md:277-280 requires links to targets outside docs/ to be absolute GitHub URLs of the form https://github.com/ai-dynamo/dynamo/blob/main/<path>. Every other GitHub link on this page and in the sibling vLLM pages uses blob/main/ (e.g. docs/fern/backends/vllm/vllm-config-reference.mdx:8, docs/fern/backends/vllm/vllm-examples.mdx:48).

Suggested change
Only the prefill worker is required to set it, but both halves of a NIXL pair must agree on a connector for transfers to succeed. Pass the same `--kv-transfer-config` value to the decode worker, as the [disaggregated vLLM launch script](https://github.com/ai-dynamo/dynamo/blob/release/1.4.0/examples/backends/vllm/launch/disagg.sh) does.
Only the prefill worker is required to set it, but both halves of a NIXL pair must agree on a connector for transfers to succeed. Pass the same `--kv-transfer-config` value to the decode worker, as the [disaggregated vLLM launch script](https://github.com/ai-dynamo/dynamo/blob/main/examples/backends/vllm/launch/disagg.sh) does.
Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.


The earlier `--connector` flag is no longer accepted by the vLLM backend. Setting it — on the command line or through the `DYN_CONNECTOR` environment variable — raises a `ValueError` during argument parsing. The message depends on the value:

- An active connector, such as `--connector nixl` or `DYN_CONNECTOR=nixl`, reports the equivalent `--kv-transfer-config` JSON to use instead.
- `--connector none` or `--connector null` reports that the flag is no longer needed, because no connector is already the default. There is no equivalent value to migrate to, so none is shown.
- `DYN_CONNECTOR` set to an empty or whitespace-only value reports that the variable is no longer supported, without an equivalent value.

## Worker role and disaggregation

These flags control which role this worker plays in a disaggregated deployment. The default when no `--disaggregation-mode` is set is aggregated (`agg`).

<ParamField path="--disaggregation-mode" type="string" default="null">
Worker disaggregation mode. `agg` (default when unset) runs a combined aggregated prefill+decode worker. `pd` is a legacy alias for `agg`. `prefill` and `decode` split the pipeline for prefill/decode disaggregation. `encode` starts a multimodal encode-only worker.

`prefill` additionally requires the native `--kv-transfer-config` argument - see [Native KV transfer configuration](#native-kv-transfer-configuration).

<span className="enum-values"><span className="enum-label">Allowed values:</span> <Badge intent="note" minimal>pd</Badge> <Badge intent="note" minimal>agg</Badge> <Badge intent="note" minimal>prefill</Badge> <Badge intent="note" minimal>decode</Badge> <Badge intent="note" minimal>encode</Badge></span>

Environment variable: `DYN_VLLM_DISAGGREGATION_MODE`
Expand Down Expand Up @@ -204,18 +231,6 @@ These flags control the self-benchmark sweep that runs on startup before the wor

These flags are retained for backward compatibility and will be removed in a future release. Each is mapped to its replacement at startup with a deprecation warning.

<ParamField path="--is-prefill-worker" type="boolean" default="false" deprecated={true}>
**Deprecated** — use `--disaggregation-mode=prefill`. Enable prefill functionality for this worker.

Environment variable: `DYN_VLLM_IS_PREFILL_WORKER`
</ParamField>

<ParamField path="--is-decode-worker" type="boolean" default="false" deprecated={true}>
**Deprecated** — use `--disaggregation-mode=decode`. Mark this as a decode worker that does not publish KV events.

Environment variable: `DYN_VLLM_IS_DECODE_WORKER`
</ParamField>

<ParamField path="--model-express-url" type="string" default="null" deprecated={true}>
**Deprecated** — accepted for compatibility with older ModelExpress manifests only. The vLLM ModelExpress plugin reads its own configuration.

Expand All @@ -224,8 +239,6 @@ These flags are retained for backward compatibility and will be removed in a fut

## Validation rules

- `--disaggregation-mode` cannot be combined with `--is-prefill-worker` or `--is-decode-worker`.
- `--is-prefill-worker` and `--is-decode-worker` cannot both be set at the same time.
- `--embedding-worker` is only valid with `--disaggregation-mode=agg` (or the default aggregated mode) and cannot be combined with `--enable-multimodal` or `--benchmark-mode`.

## Related pages
Expand Down
Loading