From 1d4751623eebd0f6fc7a4b02221a148f6796d5d8 Mon Sep 17 00:00:00 2001 From: Russell Bryant Date: Fri, 25 Apr 2025 12:03:20 -0400 Subject: [PATCH] Don't bind tcp zmq socket to all interfaces This change addresses https://github.com/vllm-project/vllm/security/advisories/GHSA-9f8f-2vmf-885j Stop binding to all interfaces for the XPUB socket in shm_broadcast.py. We already determine the address that other hosts will use to connect to this socket. If `VLLM_HOST_IP` is set, that will be the value. By binding to all sockets, we expose private data to more networks than intended. Signed-off-by: Russell Bryant --- docs/source/serving/distributed_serving.md | 4 ++++ vllm/distributed/device_communicators/shm_broadcast.py | 2 +- 2 files changed, 5 insertions(+), 1 deletion(-) diff --git a/docs/source/serving/distributed_serving.md b/docs/source/serving/distributed_serving.md index 591acc2c9b75..c285ef3e8e1c 100644 --- a/docs/source/serving/distributed_serving.md +++ b/docs/source/serving/distributed_serving.md @@ -77,6 +77,10 @@ bash run_cluster.sh \ Then you get a ray cluster of **containers**. Note that you need to keep the shells running these commands alive to hold the cluster. Any shell disconnect will terminate the cluster. In addition, please note that the argument `ip_of_head_node` should be the IP address of the head node, which is accessible by all the worker nodes. The IP addresses of each worker node should be specified in the `VLLM_HOST_IP` environment variable, and should be different for each worker node. Please check the network configuration of your cluster to make sure the nodes can communicate with each other through the specified IP addresses. +:::{warning} +It is considered best practice to set `VLLM_HOST_IP` to an address on a private network segment for the vLLM cluster. The traffic sent here is not encrypted. The endpoints are also exchanging data in a format that could be exploited to execute arbitrary code should a malicious party gain access to the network. Please ensure that this network is not reachable by any untrusted parties. +::: + :::{warning} Since this is a ray cluster of **containers**, all the following commands should be executed in the **containers**, otherwise you are executing the commands on the host machine, which is not connected to the ray cluster. To enter the container, you can use `docker exec -it node /bin/bash`. ::: diff --git a/vllm/distributed/device_communicators/shm_broadcast.py b/vllm/distributed/device_communicators/shm_broadcast.py index 49a65bd0d118..723719c79e9c 100644 --- a/vllm/distributed/device_communicators/shm_broadcast.py +++ b/vllm/distributed/device_communicators/shm_broadcast.py @@ -241,7 +241,7 @@ def __init__( self.remote_socket.setsockopt(IPV6, 1) remote_addr_ipv6 = True connect_ip = f"[{connect_ip}]" - socket_addr = f"tcp://*:{remote_subscribe_port}" + socket_addr = f"tcp://{connect_ip}:{remote_subscribe_port}" self.remote_socket.bind(socket_addr) remote_subscribe_addr = f"tcp://{connect_ip}:{remote_subscribe_port}" else: