Repository navigation
[Bugfix][KV Offload][P2P] Support IPv6 literals in the ZMQ control channel - #64006
HaiQiao-Wang wants to merge 3 commits into
Conversation
…annel
The P2P control channel built ZMQ endpoints with a bare f-string,
`tcp://{host}:{port}`, which produces an ambiguous address for IPv6
literals (`tcp://::1:5710`), and never enabled `ZMQ_IPV6` on the
ROUTER/DEALER sockets, so binding or dialing an IPv6 address failed.
- Build endpoints with `make_zmq_path`, which brackets IPv6 literals
(`tcp://[::1]:5710`).
- Set `ZMQ_IPV6` on the ROUTER and DEALER sockets when the configured
host is an IPv6 literal, matching the existing mooncake control
channel.
- Parametrize the bidirectional messaging test over IPv4/IPv6 on both
sides, and assert the DEALER identity and first message the ROUTER
sees, so a mixed-family pair is covered.
- Document that IPv6 literals are supported and passed without brackets.
Co-authored-by: Kscc <noreply@owtffssent.com>
Signed-off-by: w_wanghaiqiao <w_wanghaiqiao@kingsoft.com>
|
Documentation preview: https://vllm--64006.org.readthedocs.build/en/64006/ |
…Q control channel `is_valid_ipv6_address` only recognizes unbracketed literals, so a bracketed host such as `[::1]` — which can reach the tier via `VLLM_P2P_SIDE_CHANNEL_HOST` or a peer's `remote_host` — was not detected as IPv6, `ZMQ_IPV6` stayed off, and the socket failed to bind with an unrelated error. Strip the brackets once at both entry points, so the configured host and the host parsed back out of a peer_id are treated identically. The peer identity format is unchanged; only the socket address is normalized. Co-authored-by: Kscc <noreply@owtffssent.com> Signed-off-by: w_wanghaiqiao <w_wanghaiqiao@kingsoft.com>
|
Requesting the This is a 39-line control-plane fix across 3 files for the P2P KV-offload tier's ZMQ The unit test file is already wired into CI and passes locally (24 passed). Reverting Happy to adjust the scope (e.g. split the normalization into its own PR) if that is |
Overview
The P2P KV-offload tier's ZMQ control channel could not bind or dial IPv6 addresses.
This fixes the endpoint construction, enables
ZMQ_IPV6on the control sockets, andnormalizes bracketed IPv6 literals, so
VLLM_P2P_SIDE_CHANNEL_HOST=::1(or[::1])works.
Claims
ZmqTransportnow builds ZMQ endpoints viamake_zmq_path(), which brackets IPv6literals (
tcp://[::1]:5710), replacing the ambiguoustcp://::1:5710the oldf-string produced.
ZMQ_IPV6=1when the configured host is an IPv6literal. Without it libzmq rejects IPv6 endpoints regardless of the URL form.
[::1]) are accepted and normalized to the bare form, both forthe configured host and for a host parsed back out of a peer id. Previously a
bracketed host was not recognized as IPv6 at all, so
ZMQ_IPV6stayed off and thebind failed with an unrelated error.
test_bidirectional_messagingis parametrized over IPv4/IPv6 on both sides, coveringthe mixed-family pair (IPv4 <-> IPv6), and now asserts the DEALER identity and the
first message the ROUTER sees.
Validation
Before/after on the same test file, each fix reverted in place while keeping the tests:
3 failed, 20 passed— all three cases involving::1fail to bind/dialmake_zmq_path+ZMQ_IPV6only24 passed24 passedThe normalization-specific test was also confirmed to fail with the normalization
removed:
The
::1failures looked like this before the fix:Reproduce:
The test file is already wired into Buildkite CI (
check-test-tetheringpasses). No GPUis required. This is a control-plane change: it does not touch the data path, model
execution, or output, so no model evaluation applies.
Details
Root cause:
_tcp_addr()interpolatedf"tcp://{host}:{port}". An IPv6 literal makesthat string ambiguous (
tcp://::1:5710), and libzmq additionally requiresZMQ_IPV6tobe enabled to use IPv6 transports at all. Both had to be fixed; fixing only the URL form
still fails to bind. A third case is a bracketed literal:
is_valid_ipv6_address()onlyrecognizes the bare form, so
[::1]silently took the IPv4 path. The approach mirrorsthe existing mooncake control channel
(
vllm/distributed/ec_transfer/ec_connector/mooncake/control.py), which uses the samemake_zmq_path+ZMQ_IPV6pairing.Normalization happens at both entry points — the configured
hostin__init__and thehost parsed out of a
peer_idin_open_connection— so a bracketed value cannot reachis_valid_ipv6_address()from either direction.The peer identity format is unchanged:
P2POffloadingTierbuilds_local_idand_peer_id_from_paramsas barehost:port, and_open_connectionparses it back withrsplit(":", 1). For IPv6 that yields the correct host and port, so no identitymigration is needed and IPv4 behavior is untouched.
Tradeoffs and limitations:
path, matching existing vLLM behavior.
unchanged.
AI assistance
AI assistance (Kscc) was used to write this change. The human submitter reviewed every
changed line and ran the tests above.
Pull Request Checklist
I used vLLM's
/pr-checklistskill. (Mandatory for agents, optional for humans).AI assistance was used during the creation of this PR.
Design Fit: Minimizes impact on core components, reuses existing functionality, and justifies added complexity.
Testing and Validation: Validates the change and ensures any added tests are meaningful and reliable, with CI coverage or documented CI resource constraints and validation performed outside CI.
Code Quality and Style: Keeps code and comments clear and concise, and updates relevant documentation and examples.
Pull Request Contents: Includes a brief summary and relevant links, supports claims with evidence, explains root causes and implementation tradeoffs, and follows the contributing guide.