vmnet: Support socket_vmnet; deprecate vde_vmnet

socket_vmnet is similar to vde_vmnet but does not depend on VDE. https://github.com/lima-vm/socket_vmnet See docs/network.md for how to create networks.yaml with socketVMNet. When both socketVMNet and vdeVMNet (deprecated) are present in the YAML, socketVMNet is chosen. Signed-off-by: Akihiro Suda <[email protected]>
lima-vm · Sep 10, 2022 · b5e0d5a · b5e0d5a
1 parent ba2d6e3
commit b5e0d5a
Show file tree

Hide file tree

Showing 21 changed files with 567 additions and 154 deletions.
diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml
@@ -105,22 +105,6 @@ jobs:
         time brew update
         time brew install qemu bash coreutils curl jq
         time brew upgrade
-    - name: Install vde_switch and vde_vmnet
-      env:
-        VDE_VMNET_VERSION: v0.6.0
-      run: |
-        (
-          brew install autoconf automake
-          cd ~
-          git clone https://github.com/lima-vm/vde_vmnet
-          cd vde_vmnet
-          git checkout $VDE_VMNET_VERSION
-          sudo git config --global --add safe.directory /Users/runner/vde_vmnet
-          sudo make PREFIX=/opt/vde install
-        )
-        (
-          limactl sudoers | sudo tee /etc/sudoers.d/lima
-        )
     - name: Cache ~/Library/Caches/lima/download
       uses: actions/cache@v3
       with:
@@ -147,15 +131,78 @@ jobs:
         retry_on: error
         max_attempts: 3
         command: ./hack/test-example.sh examples/experimental/9p.yaml
-    - name: "Test vmnet.yaml"
+    # GHA macOS is slow and flaky, so we only test a few YAMLS here.
+    # Other yamls are tested on Linux instances of Cirrus.
+
+  vmnet:
+    name: "VMNet test"
+    runs-on: macos-11
+    timeout-minutes: 120
+    steps:
+    - uses: actions/setup-go@v3
+      with:
+        go-version: 1.19.x
+    - uses: actions/checkout@v3
+      with:
+        fetch-depth: 1
+    - name: Make
+      run: make
+    - name: Install
+      run: make install
+    - name: Install test dependencies
+      run: brew install qemu bash coreutils iperf3
+    - name: Cache ~/Library/Caches/lima/download
+      uses: actions/cache@v3
+      with:
+        path: ~/Library/Caches/lima/download
+        key: ${{ runner.os }}-vmnet
+    - name: Install vde_switch and vde_vmnet (Deprecated)
+      env:
+        VDE_VMNET_VERSION: v0.6.0
+      run: |
+        (
+          brew install autoconf automake
+          cd ~
+          git clone https://github.com/lima-vm/vde_vmnet
+          cd vde_vmnet
+          git checkout $VDE_VMNET_VERSION
+          sudo git config --global --add safe.directory /Users/runner/vde_vmnet
+          sudo make PREFIX=/opt/vde install
+        )
+        limactl sudoers | sudo tee /etc/sudoers.d/lima
+    - name: Unit test (pkg/networks) with vde_vmnet (Deprecated)
+      # Set -count=1 to disable cache
+      run: go test -v -count=1 ./pkg/networks/...
+    - name: Test vde_vmnet (Deprecated)
+      uses: nick-invision/retry@v2
+      with:
+        timeout_minutes: 30
+        retry_on: error
+        max_attempts: 3
+        command: ./hack/test-example.sh examples/vmnet.yaml
+    - name: Install socket_vmnet
+      env:
+        SOCKET_VMNET_VERSION: v1.0.0-alpha.0
+      run: |
+        (
+          cd ~
+          git clone https://github.com/lima-vm/socket_vmnet
+          cd socket_vmnet
+          git checkout $SOCKET_VMNET_VERSION
+          sudo git config --global --add safe.directory /Users/runner/socket_vmnet
+          sudo make PREFIX=/opt/socket_vmnet install
+        )
+        limactl sudoers | sudo tee /etc/sudoers.d/lima
+    - name: Unit test (pkg/networks) with socket_vmnet
+      # Set -count=1 to disable cache
+      run: go test -v -count=1 ./pkg/networks/...
+    - name: Test socket_vmnet
       uses: nick-invision/retry@v2
       with:
         timeout_minutes: 30
         retry_on: error
         max_attempts: 3
         command: ./hack/test-example.sh examples/vmnet.yaml
-    # GHA macOS is slow and flaky, so we only test a few YAMLS here.
-    # Other yamls are tested on Linux instances of Cirrus.
 
   upgrade:
     name: "Upgrade test"

diff --git a/README.md b/README.md
@@ -435,7 +435,8 @@ The `copy` command only works for instances that have been created by lima 0.5.0
 
 The default guest IP 192.168.5.15 is not accessible from the host and other guests.
 
-To add another IP address that is accessible from the host and other virtual machines, enable [`vde_vmnet`](https://github.com/lima-vm/vde_vmnet).
+To add another IP address that is accessible from the host and other virtual machines, enable [`socket_vmnet`](https://github.com/lima-vm/socket_vmnet) (since Lima v0.12)
+or [`vde_vmnet`](https://github.com/lima-vm/vde_vmnet) (Deprecated).
 
 See [`./docs/network.md`](./docs/network.md).
 

diff --git a/docs/network.md b/docs/network.md
@@ -10,7 +10,7 @@ The guest IP address is set to `192.168.5.15`.
 
 This IP address is not accessible from the host by design.
 
-Use [vde_vmnet](https://github.com/lima-vm/vde_vmnet) to allow accessing the guest IP from the host and other guests.
+Use VMNet (see below) to allow accessing the guest IP from the host and other guests.
 
 ### Host IP (192.168.5.2)
 
@@ -42,50 +42,32 @@ During initial cloud-init bootstrap, `iptables` may not yet be installed. In tha
 
 If `useHostResolver` is false, then DNS servers can be configured manually in `lima.yaml` via the `dns` setting. If that list is empty, then Lima will either use the slirp DNS (on Linux), or the nameservers from the first host interface in service order that has an assigned IPv4 address (on macOS).
 
-## `vde_vmnet` (192.168.105.0/24)
+## Managed VMNet networks (192.168.105.0/24)
 
-[`vde_vmnet`](https://github.com/lima-vm/vde_vmnet) is required for adding another guest IP that is accessible from
-the host and other guests.
-
-To enable `vde_vmnet` (in addition the user-mode network), add the following lines to the YAML after installing `vde_vmnet`.
-
-```yaml
-networks:
-  # vnl (virtual network locator) points to the vde_switch socket directory,
-  # optionally with vde:// prefix
-  # - vnl: "vde:///var/run/vde.ctl"
-  #   # VDE Switch port number (not TCP/UDP port number). Set to 65535 for PTP mode.
-  #   # Default: 0
-  #   switchPort: 0
-  #   # MAC address of the instance; lima will pick one based on the instance name,
-  #   # so DHCP assigned ip addresses should remain constant over instance restarts.
-  #   macAddress: ""
-  #   # Interface name, defaults to "lima0", "lima1", etc.
-  #   interface: ""
-```
-
-The IP address range is typically `192.168.105.0/24`, but depends on the configuration of `vde_vmnet`.
-See [the documentation of `vde_vmnet`](https://github.com/lima-vm/vde_vmnet) for further information.
-
-## Managed VMNet networks (via vde_vmnet)
+Either [`socket_vmnet`](https://github.com/lima-vm/socket_vmnet) (since Lima v0.12) or [`vde_vmnet`](https://github.com/lima-vm/vde_vmnet) (Deprecated)
+is required for adding another guest IP that is accessible from the host and other guests.
 
 Starting with version v0.7.0 lima can manage the networking daemons automatically. Networks are defined in
 `$LIMA_HOME/_config/networks.yaml`. If this file doesn't already exist, it will be created with these default
 settings:
 
 ```yaml
-# Paths to vde executables. Because vde_vmnet is invoked via sudo it should be
+# Path to socket_vmnet executable. Because socket_vmnet is invoked via sudo it should be
 # installed where only root can modify/replace it. This means also none of the
 # parent directories should be writable by the user.
 #
 # The varRun directory also must not be writable by the user because it will
-# include the vde_vmnet pid files. Those will be terminated via sudo, so replacing
-# the pid files would allow killing of arbitrary privileged processes. varRun
+# include the socket_vmnet pid file. Those will be terminated via sudo, so replacing
+# the pid file would allow killing of arbitrary privileged processes. varRun
 # however MUST be writable by the daemon user.
 #
 # None of the paths segments may be symlinks, why it has to be /private/var
 # instead of /var etc.
 paths:
+# socketVMNet requires Lima >= 0.12 .
+# socketVMNet has precedence over vdeVMNet.
+  socketVMNet: /opt/socket_vmnet/bin/socket_vmnet
+# vdeSwitch and vdeVMNet are DEPRECATED.
   vdeSwitch: /opt/vde/bin/vde_switch
   vdeVMNet: /opt/vde/bin/vde_vmnet
   varRun: /private/var/run/lima
@@ -115,8 +97,9 @@ Instances can then reference these networks from their `lima.yaml` file:
 ```yaml
 networks:
   # Lima can manage daemons for networks defined in $LIMA_HOME/_config/networks.yaml
-  # automatically. Both vde_switch and vde_vmnet binaries must be installed into
+  # automatically. The socket_vmnet must be installed into
   # secure locations only alterable by the "root" user.
+  # The same applies to vde_switch and vde_vmnet for the deprecated VDE mode.
   # - lima: shared
   #   # MAC address of the instance; lima will pick one based on the instance name,
   #   # so DHCP assigned ip addresses should remain constant over instance restarts.
@@ -129,10 +112,38 @@ The network daemons are started automatically when the first instance referencin
 and will stop automatically once the last instance has stopped. Daemon logs will be stored in the
 `$LIMA_HOME/_networks` directory.
 
-Since the commands to start and stop the `vde_vmnet` daemon requires root, the user either must
+Since the commands to start and stop the `socket_vmnet` daemon (or the `vde_vmnet` daemon) requires root, the user either must
 have password-less `sudo` enabled, or add the required commands to a `sudoers` file. This can
 be done via:
 
 ```shell
 limactl sudoers | sudo tee /etc/sudoers.d/lima
 ```
+
+## Unmanaged VMNet networks
+For Lima >= 0.12:
+```yaml
+networks:
+  # Lima can also connect to "unmanaged" networks addressed by "socket". This
+  # means that the daemons will not be controlled by Lima, but must be started
+  # before the instance.  The interface type (host, shared, or bridged) is
+  # configured in socket_vmnet and not in lima.
+  # - socket: "/var/run/socket_vmnet"
+```
+
+For older Lima releases:
+```yaml
+networks:
+  # vnl (virtual network locator) points to the vde_switch socket directory,
+  # optionally with vde:// prefix
+  # ⚠️  vnl is deprecated, use socket.
+  # - vnl: "vde:///var/run/vde.ctl"
+  #   # VDE Switch port number (not TCP/UDP port number). Set to 65535 for PTP mode.
+  #   # Builtin default: 0
+  #   switchPort: 0
+  #   # MAC address of the instance; lima will pick one based on the instance name,
+  #   # so DHCP assigned ip addresses should remain constant over instance restarts.
+  #   macAddress: ""
+  #   # Interface name, defaults to "lima0", "lima1", etc.
+  #   interface: ""
+```
diff --git a/examples/default.yaml b/examples/default.yaml
@@ -217,25 +217,29 @@ video:
   display: null
 
 # The instance can get routable IP addresses from the vmnet framework using
-# https://github.com/lima-vm/vde_vmnet.
+# https://github.com/lima-vm/socket_vmnet.
 # 🟢 Builtin default: null
 networks:
 # Lima can manage daemons for networks defined in $LIMA_HOME/_config/networks.yaml
-# automatically. Both vde_switch and vde_vmnet binaries must be installed into
+# automatically. The socket_vmnet binary must be installed into
 # secure locations only alterable by the "root" user.
+# The same applies to vde_switch and vde_vmnet for the deprecated VDE mode.
 # - lima: shared
 #   # MAC address of the instance; lima will pick one based on the instance name,
 #   # so DHCP assigned ip addresses should remain constant over instance restarts.
 #   macAddress: ""
 #   # Interface name, defaults to "lima0", "lima1", etc.
 #   interface: ""
 #
-# Lima can also connect to "unmanaged" vde networks addressed by "vnl". This
+# Lima can also connect to "unmanaged" networks addressed by "socket". This
 # means that the daemons will not be controlled by Lima, but must be started
 # before the instance.  The interface type (host, shared, or bridged) is
-# configured in vde_vmnet and not in lima.
+# configured in socket_vmnet and not in lima.
+# - socket: "/var/run/socket_vmnet"
+
 # vnl (virtual network locator) points to the vde_switch socket directory,
 # optionally with vde:// prefix
+# ⚠️  vnl is deprecated, use socket.
 # - vnl: "vde:///var/run/vde.ctl"
 #   # VDE Switch port number (not TCP/UDP port number). Set to 65535 for PTP mode.
 #   # Builtin default: 0

diff --git a/examples/vmnet.yaml b/examples/vmnet.yaml
@@ -21,7 +21,14 @@ mounts:
   writable: true
 networks:
 # The instance can get routable IP addresses from the vmnet framework using
-# https://github.com/lima-vm/vde_vmnet. Available networks are defined in
+# https://github.com/lima-vm/socket_vmnet (since Lima v0.12) or
+# https://github.com/lima-vm/vde_vmnet (deprecated) .
+#
+# Available networks are defined in
 # $LIMA_HOME/_config/networks.yaml. Supported network types are "host",
 # "shared", or "bridged".
+#
+# Interface "lima0": shared mode  (IP is assigned by macOS's bootpd)
 - lima: shared
+# Interface "lima1": bridged mode (IP is assigned by a DHCP server on the physical network)
+# - lima: bridged
diff --git a/hack/test-example.sh b/hack/test-example.sh
@@ -23,6 +23,7 @@ declare -A CHECKS=(
 	["containerd-user"]="1"
 	["restart"]="1"
 	["port-forwards"]="1"
+	["vmnet"]=""
 )
 
 case "$NAME" in
@@ -41,6 +42,9 @@ case "$NAME" in
 	# ● run-r2b459797f5b04262bfa79984077a65c7.service                                       loaded failed failed    /usr/bin/systemctl start man-db-cache-update
 	CHECKS["systemd-strict"]=
 	;;
+"vmnet")
+	CHECKS["vmnet"]=1
+	;;
 esac
 
 if limactl ls -q | grep -q "$NAME"; then
@@ -217,6 +221,22 @@ if [[ -n ${CHECKS["port-forwards"]} ]]; then
 	set +x
 fi
 
+if [[ -n ${CHECKS["vmnet"]} ]]; then
+	INFO "Testing vmnet functionality"
+	guestip="$(limactl shell "$NAME" ip -4 -j addr show dev lima0 | jq -r '.[0].addr_info[0].local')"
+	INFO "Pinging the guest IP ${guestip}"
+	set -x
+	ping -c 3 "$guestip"
+	set +x
+	INFO "Benchmarking with iperf3"
+	set -x
+	limactl shell "$NAME" sudo apt-get install -y iperf3
+	limactl shell "$NAME" iperf3 -s -1 -D
+	iperf3 -c "$guestip"
+	set +x
+	# NOTE: we only test the shared interface here, as the bridged interface cannot be used on GHA (and systemd-networkd-wait-online.service will fail)
+fi
+
 if [[ -n ${CHECKS["restart"]} ]]; then
 	INFO "Create file in the guest home directory and verify that it still exists after a restart"
 	# shellcheck disable=SC2016