Skip to content

[Bugfix][KV Offload][P2P] Support IPv6 literals in the ZMQ control channel - #64006

Open
HaiQiao-Wang wants to merge 3 commits into
vllm-project:mainfrom
HaiQiao-Wang:fix/p2p-ipv6-control
Open

HaiQiao-Wang wants to merge 3 commits into
vllm-project:mainfrom
HaiQiao-Wang:fix/p2p-ipv6-control

Conversation

@HaiQiao-Wang

@HaiQiao-Wang HaiQiao-Wang commented Oct 27, 2026 •

Copy link
Copy Markdown
Contributor

Overview

The P2P KV-offload tier's ZMQ control channel could not bind or dial IPv6 addresses.
This fixes the endpoint construction, enables ZMQ_IPV6 on the control sockets, and
normalizes bracketed IPv6 literals, so VLLM_P2P_SIDE_CHANNEL_HOST=::1 (or [::1])
works.

Claims

  • ZmqTransport now builds ZMQ endpoints via make_zmq_path(), which brackets IPv6
    literals (tcp://[::1]:5710), replacing the ambiguous tcp://::1:5710 the old
    f-string produced.
  • The ROUTER and DEALER sockets set ZMQ_IPV6=1 when the configured host is an IPv6
    literal. Without it libzmq rejects IPv6 endpoints regardless of the URL form.
  • Bracketed literals ([::1]) are accepted and normalized to the bare form, both for
    the configured host and for a host parsed back out of a peer id. Previously a
    bracketed host was not recognized as IPv6 at all, so ZMQ_IPV6 stayed off and the
    bind failed with an unrelated error.
  • test_bidirectional_messaging is parametrized over IPv4/IPv6 on both sides, covering
    the mixed-family pair (IPv4 <-> IPv6), and now asserts the DEALER identity and the
    first message the ROUTER sees.
  • Docs note that IPv6 literals are accepted in either form, with the port separate.

Validation

Before/after on the same test file, each fix reverted in place while keeping the tests:

Code under test Result
Without any fix 3 failed, 20 passed — all three cases involving ::1 fail to bind/dial
With make_zmq_path + ZMQ_IPV6 only 24 passed
Current (adds host normalization) 24 passed

The normalization-specific test was also confirmed to fail with the normalization
removed:

FAILED test_zmq_transport.py::TestZmqTransportConnectivity::test_bracketed_ipv6_host_is_accepted
1 failed, 23 deselected

The ::1 failures looked like this before the fix:

FAILED test_zmq_transport.py::TestZmqTransportConnectivity::test_bidirectional_messaging[127.0.0.1-::1]
FAILED test_zmq_transport.py::TestZmqTransportConnectivity::test_bidirectional_messaging[::1-127.0.0.1]
FAILED test_zmq_transport.py::TestZmqTransportConnectivity::test_bidirectional_messaging[::1-::1]

Reproduce:

.venv/bin/python -m pytest tests/v1/kv_offload/tiering/p2p/test_zmq_transport.py -q

The test file is already wired into Buildkite CI (check-test-tethering passes). No GPU
is required. This is a control-plane change: it does not touch the data path, model
execution, or output, so no model evaluation applies.

Details

Root cause: _tcp_addr() interpolated f"tcp://{host}:{port}". An IPv6 literal makes
that string ambiguous (tcp://::1:5710), and libzmq additionally requires ZMQ_IPV6 to
be enabled to use IPv6 transports at all. Both had to be fixed; fixing only the URL form
still fails to bind. A third case is a bracketed literal: is_valid_ipv6_address() only
recognizes the bare form, so [::1] silently took the IPv4 path. The approach mirrors
the existing mooncake control channel
(vllm/distributed/ec_transfer/ec_connector/mooncake/control.py), which uses the same
make_zmq_path + ZMQ_IPV6 pairing.

Normalization happens at both entry points — the configured host in __init__ and the
host parsed out of a peer_id in _open_connection — so a bracketed value cannot reach
is_valid_ipv6_address() from either direction.

The peer identity format is unchanged: P2POffloadingTier builds _local_id and
_peer_id_from_params as bare host:port, and _open_connection parses it back with
rsplit(":", 1). For IPv6 that yields the correct host and port, so no identity
migration is needed and IPv4 behavior is untouched.

Tradeoffs and limitations:

  • Hostnames that resolve to IPv6 (rather than IPv6 literals) still take the IPv4 socket
    path, matching existing vLLM behavior.
  • Only the control channel is covered here; the data plane address handling is
    unchanged.

AI assistance

AI assistance (Kscc) was used to write this change. The human submitter reviewed every
changed line and ran the tests above.


Pull Request Checklist
  • I used vLLM's /pr-checklist skill. (Mandatory for agents, optional for humans).

  • AI assistance was used during the creation of this PR.

  • Design Fit: Minimizes impact on core components, reuses existing functionality, and justifies added complexity.

  • Testing and Validation: Validates the change and ensures any added tests are meaningful and reliable, with CI coverage or documented CI resource constraints and validation performed outside CI.

  • Code Quality and Style: Keeps code and comments clear and concise, and updates relevant documentation and examples.

  • Pull Request Contents: Includes a brief summary and relevant links, supports claims with evidence, explains root causes and implementation tradeoffs, and follows the contributing guide.

…annel

The P2P control channel built ZMQ endpoints with a bare f-string,
`tcp://{host}:{port}`, which produces an ambiguous address for IPv6
literals (`tcp://::1:5710`), and never enabled `ZMQ_IPV6` on the
ROUTER/DEALER sockets, so binding or dialing an IPv6 address failed.

- Build endpoints with `make_zmq_path`, which brackets IPv6 literals
  (`tcp://[::1]:5710`).
- Set `ZMQ_IPV6` on the ROUTER and DEALER sockets when the configured
  host is an IPv6 literal, matching the existing mooncake control
  channel.
- Parametrize the bidirectional messaging test over IPv4/IPv6 on both
  sides, and assert the DEALER identity and first message the ROUTER
  sees, so a mixed-family pair is covered.
- Document that IPv6 literals are supported and passed without brackets.

Co-authored-by: Kscc <noreply@owtffssent.com>
Signed-off-by: w_wanghaiqiao <w_wanghaiqiao@kingsoft.com>

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

@mergify

mergify Bot commented Oct 27, 2026

Copy link
Copy Markdown
Contributor

Documentation preview: https://vllm--64006.org.readthedocs.build/en/64006/

@mergify mergify Bot added documentation Improvements or additions to documentation bug Something isn't working labels Oct 27, 2026
…Q control channel

`is_valid_ipv6_address` only recognizes unbracketed literals, so a bracketed
host such as `[::1]` — which can reach the tier via `VLLM_P2P_SIDE_CHANNEL_HOST`
or a peer's `remote_host` — was not detected as IPv6, `ZMQ_IPV6` stayed off,
and the socket failed to bind with an unrelated error.

Strip the brackets once at both entry points, so the configured host and the
host parsed back out of a peer_id are treated identically. The peer identity
format is unchanged; only the socket address is normalized.

Co-authored-by: Kscc <noreply@owtffssent.com>
Signed-off-by: w_wanghaiqiao <w_wanghaiqiao@kingsoft.com>
@HaiQiao-Wang

Copy link
Copy Markdown
Contributor Author

Requesting the ready label to trigger CI on this change.

This is a 39-line control-plane fix across 3 files for the P2P KV-offload tier's ZMQ
control channel: IPv6 literals were unusable because the endpoint address was built
with a bare f-string and ZMQ_IPV6 was never enabled on the ROUTER/DEALER sockets.
Bracketed literals ([::1]) are normalized as well.

The unit test file is already wired into CI and passes locally (24 passed). Reverting
the fix reproduces 3 failures for the ::1 cases, so the tests exercise the change
rather than passing through an unchanged path. No GPU or model evaluation is involved.

Happy to adjust the scope (e.g. split the normalization into its own PR) if that is
preferred.

@mergify mergify Bot added the nixl NIXL KV connector / multi-tiered KV offloading (KVCR) label Oct 29, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working documentation Improvements or additions to documentation nixl NIXL KV connector / multi-tiered KV offloading (KVCR)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant