hyperkit: /var/db/dhcpd_leases: no such file or directory -> could not find an IP address

[echeadle]

I have reinstalled minikube using brew and it works if the driver is virtual box. I have docker desktop working and I can select between docker desktop kubernetes and minikube if minikube is running on virtualbox.

I added a dhcpd_leases file in the /var/db directory and the error changed to:
error: could not find an IP address for de:57:7:ad:86:12

The exact command to reproduce the issue:
minikube start --memory 2000MB --disksize 20000MB

The full output of the command that failed:

minikube v1.6.1 on Darwin 10.14.6
✨ Automatically selected the 'hyperkit' driver (alternates: [virtualbox])
🔥 Creating hyperkit VM (CPUs=2, Memory=2000MB, Disk=20000MB) ...
🔄 Retriable failure: create: Error creating machine: Error in driver during machine creation: IP address never found in dhcp leases file Temporary error: open /var/db/dhcpd_leases: no such file or directory
🔥 Deleting "minikube" in hyperkit ...
🔥 Creating hyperkit VM (CPUs=2, Memory=2000MB, Disk=20000MB) ...
🔄 Retriable failure: create: Error creating machine: Error in driver during machine creation: IP address never found in dhcp leases file Temporary error: open /var/db/dhcpd_leases: no such file or directory
🔥 Deleting "minikube" in hyperkit ...
🔥 Creating hyperkit VM (CPUs=2, Memory=2000MB, Disk=20000MB) ...
🔄 Retriable failure: create: Error creating machine: Error in driver during machine creation: IP address never found in dhcp leases file Temporary error: open /var/db/dhcpd_leases: no such file or directory
🔥 Deleting "minikube" in hyperkit ...

💣 Unable to start VM: create: Error creating machine: Error in driver during machine creation: IP address never found in dhcp leases file Temporary error: open /var/db/dhcpd_leases: no such file or directory

The output of the minikube logs command:

💣 api load: filestore "minikube": Docker machine "minikube" does not exist. Use "docker-machine ls" to list machines. Use "docker-machine create" to add a new one.

😿 Sorry that minikube crashed. If this was unexpected, we would love to hear from you:
👉 https://github.com/kubernetes/minikube/issues/new/choose

The operating system version:
macOS Mojave 10.14.6

[tstromberg]

I suspect that the InternetSharing daemon or bootpd may be busted or disabled in some way on this host, which the docker-machine-hyperkit driver uses.

Before running minikube start, can you run:

• sudo fs_usage -w -f pathname InternetSharing

and share the result? You should see some entries like:

                                        0.000049   InternetSharing.81461
17:09:27.359019  close             F=11                                                                                                                                                                                       0.000001   InternetSharing.81461
17:09:48.803558  open              F=11       (R_____)  private/var/db/dhcpd_leases                                                                                                                                           0.000599   InternetSharing.82050
17:09:48.805938  close             F=11                                                                                                                                                                                       0.000034   InternetSharing.82050
17:09:48.807850  open              F=11       (R_____)  private/var/db/dhcpd_leases                                                                                                                                           0.000571   InternetSharing.82050
17:09:48.810291  close             F=11                                                                                                                                                                                       0.000010   InternetSharing.82050

Also, open up the Console application, and search for "dhcp". You should see something like:

It's interesting that Docker works but minikube doesn't, as both use hyperkit. I wonder if there is anything unusual in the DfD network configuration which may be affecting things.

My apologies for the issue, and thanks for helping us solve this!

[tstromberg]

Related, but likely has a different root cause: #4206

[echeadle]

I don't think I am going to be very helpful in solving this issue. I did attach a file with the output of fs_usage just in case it might help.
fs_usage-output.txt

I work for a healthcare company and I suspect our security folk have done something with the proxy that is causing this error. I believe minikube was working a few weeks ago, but I recently upgrade to mojave. I have not heard of any changes in our security polices, but I would not be surprised that with the update of the OS, they put in some changes to the way the mac proxy works. If this is true, I probably can't back out the changes or make changes that could test any fixes. It does work with virtualbox and my docker desktop version of kubernetes is working.

I would be happy to try things, but I may not be able make changes to my machine that might solve this issue.

[phantooom]

the same issue
mac os catalina
virtualbox 6.1
minikube 1.5.2

➜  deploymgmt git:(master) ✗ sudo fs_usage -w -f pathname InternetSharing
Password:
13:07:38.170525  lstat64                                private/var/root/Library                                                                                                                                              0.000031   InternetSharing.887386
13:07:38.170541  open                   [  2] (R___________)  private/var/root/Library/LaunchDaemons>>>>>>>>>>>>>>>>>>                                                                                                        0.000013   InternetSharing.887386
13:07:38.170550  lstat64                                /Library                                                                                                                                                              0.000008   InternetSharing.887386
13:07:38.170559  open                   [  2] (R___________)  /Library/LaunchDaemons/com.apple.ftp-proxy.plist                                                                                                                0.000007   InternetSharing.887386
13:07:38.170568  lstat64                                /System/Library                                                                                                                                                       0.000008   InternetSharing.887386
13:07:38.170618  open              F=4        (R___________)  /System/Library/LaunchDaemons/com.apple.ftp-proxy.plist                                                                                                         0.000049   InternetSharing.887386
13:07:38.210261  close             F=4                                                                                                                                                                                        0.000011   InternetSharing.887386
13:07:38.210385  open              F=4        (R___________)  /System/Library/LaunchDaemons/com.apple.ftp-proxy.plist                                                                                                         0.000100   InternetSharing.887386
13:07:38.210453  close             F=4                                                                                                                                                                                        0.000004   InternetSharing.887386

[nickwilmes]

Seeing same issue with a fresh install of docker for mac and minikube. It seems to be consistent (I tried uninstalling and reinstalling at least 5 times and got the same error each time). Using Mohave on my mac.

minikube version: v1.6.2 commit: 54f28ac
hyperkit: v0.20190802-4-g97f091
docker for mac: 2.2.0.0 (42247|stable)

[tstromberg]

If anyone is running into this issue, please update this issue.

Also, it would be worth hearing if --driver=docker works instead.

[alimeerutech]

@tstromberg I had the same issue and the docker driver worked for me but hyperkit driver did not

$ sw_vers -productVersion
10.14.6

$ minikube version
minikube version: v1.8.2
commit: eb13446e786c9ef70cb0a9f85a633194e62396a1

$ /usr/local/bin/docker-machine-driver-hyperkit version
version: v1.8.2
commit: eb13446e786c9ef70cb0a9f85a633194e62396a1

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[jaysonwor]

I was running into the same issues. Turned out had nothing to do with either bootp or internet sharing, and was due to the fact the mac firewall was enabled. Hopefully will help others whom couldn't get hyperkit working on catalina (10.15.5 (19F101))

[medyagh]

@jaysonwor do you mind sharing how to disable mac frewall in terminal and what version do you use ?
and also is there a way we can detect in terminal if firewall is on ? so we could detect and advice the user?

[jaysonwor]

Sure. Click "Apple" icon -> System Preferences -> Security & Privacy -> Firewall "Turn off". Probably not the best solution but at least it worked (minikube 1.12 with hyperkit up and running)

[jaysonwor]

A bit more info about my setup just in case. I tried rm /var/db/dhcp_leases and it creates the file okay whereas before it was throwing error.

Removed all traces of the "minikube" cluster.
😄 minikube v1.12.0 on Darwin 10.15.5
✨ Using the hyperkit driver based on user configuration
👍 Starting control plane node minikube in cluster minikube
🔥 Creating hyperkit VM (CPUs=2, Memory=4000MB, Disk=20000MB) ...
🐳 Preparing Kubernetes v1.18.3 on Docker 19.03.12 ...

[adamf]

I'm also seeing this issue when I try to use minikube + hyperkit in a github action on github's macos-latest action runner image. Minikube 1.12.0:

minikube start --apiserver-port=6443 --addons=metrics-server --addons=ingress --addons=dashboard --addons=logviewer
* [minikube-dev-runner-Mac-1595621774903] minikube v1.12.0 on Darwin 10.15.6
* Using the hyperkit driver based on user configuration
* Downloading driver docker-machine-driver-hyperkit:
* minikube 1.12.1 is available! Download it: https://github.com/kubernetes/minikube/releases/tag/v1.12.1
* To disable this notice, run: 'minikube config set WantUpdateNotification false'

* The 'hyperkit' driver requires elevated permissions. The following commands will be executed:

    $ sudo chown root:wheel /Users/runner/.minikube/bin/docker-machine-driver-hyperkit 
    $ sudo chmod u+s /Users/runner/.minikube/bin/docker-machine-driver-hyperkit 


* Downloading VM boot image ...
[...]
* Starting control plane node minikube-dev-runner-Mac-1595621774903 in cluster minikube-dev-runner-Mac-1595621774903
* Downloading Kubernetes v1.18.3 preload ...
* Creating hyperkit VM (CPUs=4, Memory=6000MB, Disk=40960MB) ...
* Deleting "minikube-dev-runner-Mac-1595621774903" in hyperkit ...
* Creating hyperkit VM (CPUs=4, Memory=6000MB, Disk=40960MB) ...
* Failed to start hyperkit VM. "minikube start -p minikube-dev-runner-Mac-1595621774903" may fix it: creating host: create: Error creating machine: Error in driver during machine creation: IP address never found in dhcp leases file Temporary error: open /var/db/dhcpd_leases: no such file or directory
* 
X error provisioning host: Failed to start host: creating host: create: Error creating machine: Error in driver during machine creation: IP address never found in dhcp leases file Temporary error: open /var/db/dhcpd_leases: no such file or directory
* 
* minikube is exiting due to an error. If the above message is not useful, open an issue:
  - https://github.com/kubernetes/minikube/issues/new/choose

Interestingly sometimes re-running the job works.

[fejta-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten

[NeckBeardPrince]

Same issue

Seems to work fine if I have Docker running as the drive switches to Docker.

❯ minikube start
😄 minikube v1.15.1 on Darwin 11.1
✨ Automatically selected the hyperkit driver
🔑 The 'hyperkit' driver requires elevated permissions. The following commands will be executed:

$ sudo chown root:wheel /Users/<removed>/.minikube/bin/docker-machine-driver-hyperkit
$ sudo chmod u+s /Users/<removed>/.minikube/bin/docker-machine-driver-hyperkit

👍 Starting control plane node minikube in cluster minikube
🔥 Creating hyperkit VM (CPUs=2, Memory=4000MB, Disk=20000MB) ...
🔥 Deleting "minikube" in hyperkit ...
🤦 StartHost failed, but will try again: creating host: create: Error creating machine: Error in driver during machine creation: IP address never found in dhcp leases file Temporary error: open /var/db/dhcpd_leases: no such file or directory

[jantebeest]

Won't hyperkit be much more efficient on Mac (as opposed to Docker/VirtualBox)? I personally especially dislike VirtualBox, Docker for Mac I can live with for the time being

[echeadle]

I no longer have a mac and retired. I forgot about this issue. Sorry, I am closing it now.

[edrandall]

Refer to my comment #4206 (comment) describing pf changes which unblocked me.

[Srinath04]

❌ Exiting due to DRV_CP_ENDPOINT: Unable to get control-plane node minikube endpoint: failed to lookup ip for ""

i was struggling with the same issue for long time,
Using Mac version 15.1, vm qemu, network socket_vmnet

Just completely restarting my macbook and starting the minikube resolved the issue