Hung at "Starting VM", ssh fails with "attempted methods [none publickey], no supported methods remain"

[gitjps]

Is this a BUG REPORT or FEATURE REQUEST? (choose one):

Please provide the following details:

Environment:

Minikube version (use minikube version): v0.25.0

• OS (e.g. from /etc/os-release): macOS 10.13.4
• VM Driver (e.g. cat ~/.minikube/machines/minikube/config.json | grep DriverName): virtualbox
• ISO version (e.g. cat ~/.minikube/machines/minikube/config.json | grep -i ISO or minikube ssh cat /etc/VERSION):
• Install tools: minikube-v0.25.1.iso"
• Others:
  The above can be generated in one go with the following commands (can be copied and pasted directly into your terminal):

What happened:

minikube start hangs after

MacBook-Pro-3:~ jps$ minikube start
Starting local Kubernetes v1.9.0 cluster...
Starting VM...

Tried to find the status:

MacBook-Pro-3:~ jps$ minikube status
E0429 18:33:50.887866   19108 status.go:80] Error getting cluster bootstrapper: getting localkube bootstrapper: getting ssh client: Error dialing tcp via ssh client: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain

What you expected to happen:

running minikube cluster, valid status

How to reproduce it (as minimally and precisely as possible):

minikube start, status

Output of minikube logs (if applicable):

Anything else do we need to know:

[Patricol]

Same issue on a fresh Ubuntu 18.04 VM.

Running on Google Compute Engine with vmx enabled.

Starting from a completely fresh VM, issue appears after running these commands:

sudo apt-get update
sudo apt-get upgrade
wget -q https://www.virtualbox.org/download/oracle_vbox_2016.asc -O- | sudo apt-key add -
wget -q https://www.virtualbox.org/download/oracle_vbox.asc -O- | sudo apt-key add -
sudo add-apt-repository "deb http://download.virtualbox.org/virtualbox/debian lsb_release -cs contrib"
sudo apt-get install -y virtualbox-5.2
sudo snap install kubectl --classic
curl -Lo minikube https://storage.googleapis.com/minikube/releases/v0.26.1/minikube-linux-amd64
chmod +x minikube
sudo mv minikube /usr/local/bin/

Minikube then behaves as follows:

user@che:~$ minikube status
minikube:
cluster:
kubectl:
user@che:~$ minikube start --cpus 2 --memory 4096 --extra-config=apiserver.authorization-mode=RBAC
Starting local Kubernetes v1.10.0 cluster...
Starting VM...
Downloading Minikube ISO
150.53 MB / 150.53 MB [============================================] 100.00% 0s

[Hangs Here]

^C
user@che:~$ minikube status
E0504 20:42:58.128665 17174 status.go:80] Error getting cluster bootstrapper: getting kubeadm bootstrapper: getting ssh client: Error dialing tcp via ssh client: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain

================================================================================

An error has occurred. Would you like to opt in to sending anonymized crash
information to minikube to help prevent future errors?
To opt out of these messages, run the command:
minikube config set WantReportErrorPrompt false

================================================================================
Please enter your response [Y/n]:
user@che:~$ minikube start -v=10
Aliases:
map[string]string{}
Override:
map[string]interface {}{"v":"10"}
PFlags:
map[string]viper.FlagValue{"disable-driver-mounts":viper.pflagValue{flag:(*pflag.Flag)(0xc420491ae0)}, "nfs-share":viper.pflagValue{flag:(*pflag.Flag)(0xc42045e3c0)}, "insecure-registry":viper.pflagValue{flag:(*pflag.Flag)(0xc42045eb40)}, "nfs-shares-root":viper.pflagValue{flag:(*pflag.Flag)(0xc42045e500)}, "dns-domain":viper.pflagValue{flag:(*pflag.Flag)(0xc42045eaa0)}, "extra-config":viper.pflagValue{flag:(*pflag.Flag)(0xc42045f040)}, "hyperv-virtual-switch":viper.pflagValue{flag:(*pflag.Flag)(0xc42045e000)}, "memory":viper.pflagValue{flag:(*pflag.Flag)(0xc420491cc0)}, "network-plugin":viper.pflagValue{flag:(*pflag.Flag)(0xc42045ee60)}, "xhyve-disk-driver":viper.pflagValue{flag:(*pflag.Flag)(0xc42045e280)}, "profile":viper.pflagValue{flag:(*pflag.Flag)(0xc420490e60)}, "apiserver-name":viper.pflagValue{flag:(*pflag.Flag)(0xc42045e820)}, "docker-env":viper.pflagValue{flag:(*pflag.Flag)(0xc42045e640)}, "feature-gates":viper.pflagValue{flag:(*pflag.Flag)(0xc42045ef00)}, "mount":viper.pflagValue{flag:(*pflag.Flag)(0xc4204919a0)}, "registry-mirror":viper.pflagValue{flag:(*pflag.Flag)(0xc42045ebe0)}, "vm-driver":viper.pflagValue{flag:(*pflag.Flag)(0xc420491c20)}, "apiserver-names":viper.pflagValue{flag:(*pflag.Flag)(0xc42045e960)}, "container-runtime":viper.pflagValue{flag:(*pflag.Flag)(0xc42045ed20)}, "bootstrapper":viper.pflagValue{flag:(*pflag.Flag)(0xc420490f00)}, "iso-url":viper.pflagValue{flag:(*pflag.Flag)(0xc420491b80)}, "kubernetes-version":viper.pflagValue{flag:(*pflag.Flag)(0xc42045ec80)}, "kvm-network":viper.pflagValue{flag:(*pflag.Flag)(0xc42045e140)}, "cpus":viper.pflagValue{flag:(*pflag.Flag)(0xc420491d60)}, "mount-string":viper.pflagValue{flag:(*pflag.Flag)(0xc420491a40)}, "apiserver-ips":viper.pflagValue{flag:(*pflag.Flag)(0xc42045ea00)}, "cache-images":viper.pflagValue{flag:(*pflag.Flag)(0xc42045efa0)}, "disk-size":viper.pflagValue{flag:(*pflag.Flag)(0xc420491e00)}, "docker-opt":viper.pflagValue{flag:(*pflag.Flag)(0xc42045e780)}, "host-only-cidr":viper.pflagValue{flag:(*pflag.Flag)(0xc420491f40)}, "keep-context":viper.pflagValue{flag:(*pflag.Flag)(0xc420491900)}}
Env:
map[string]string{}
Key/Value Store:
map[string]interface {}{}
Config:
map[string]interface {}{}
Defaults:
map[string]interface {}{"showdriverdeprecationnotification":true, "log_dir":"", "wantupdatenotification":true, "reminderwaitperiodinhours":24, "wantkubectldownloadmsg":true, "showbootstrapperdeprecationnotification":true, "v":"0", "alsologtostderr":"false", "wantreporterror":false, "wantreporterrorprompt":true, "wantnonedriverwarning":true}
Starting local Kubernetes v1.10.0 cluster...
Starting VM...
COMMAND: /usr/bin/VBoxManage showvminfo minikube --machinereadable
STDOUT:
{
name="minikube"
groups="/"
ostype="Linux 2.6 / 3.x / 4.x (64-bit)"
UUID="0af78537-f618-4f5a-a110-6dd8c4f92e1c"
CfgFile="/home/user/.minikube/machines/minikube/minikube/minikube.vbox"
SnapFldr="/home/user/.minikube/machines/minikube/minikube/Snapshots"
LogFldr="/home/user/.minikube/machines/minikube/minikube/Logs"
hardwareuuid="0af78537-f618-4f5a-a110-6dd8c4f92e1c"
memory=4096
pagefusion="off"
vram=8
cpuexecutioncap=100
hpet="on"
chipset="piix3"
firmware="BIOS"
cpus=2
pae="on"
longmode="on"
triplefaultreset="off"
apic="on"
x2apic="off"
cpuid-portability-level=0
bootmenu="disabled"
boot1="dvd"
boot2="dvd"
boot3="disk"
boot4="none"
acpi="on"
ioapic="on"
biosapic="apic"
biossystemtimeoffset=0
rtcuseutc="on"
hwvirtex="on"
nestedpaging="on"
largepages="on"
vtxvpid="on"
vtxux="on"
paravirtprovider="default"
effparavirtprovider="kvm"
VMState="running"
VMStateChangeTime="2018-05-04T20:41:45.853000000"
monitorcount=1
accelerate3d="off"
accelerate2dvideo="off"
teleporterenabled="off"
teleporterport=0
teleporteraddress=""
teleporterpassword=""
tracing-enabled="off"
tracing-allow-vm-access="off"
tracing-config=""
autostart-enabled="off"
autostart-delay=0
defaultfrontend=""
storagecontrollername0="SATA"
storagecontrollertype0="IntelAhci"
storagecontrollerinstance0="0"
storagecontrollermaxportcount0="30"
storagecontrollerportcount0="30"
storagecontrollerbootable0="on"
"SATA-0-0"="/home/user/.minikube/machines/minikube/boot2docker.iso"
"SATA-ImageUUID-0-0"="3235a970-2e54-481d-853e-2ff2a6627d13"
"SATA-tempeject"="off"
"SATA-IsEjected"="off"
"SATA-1-0"="/home/user/.minikube/machines/minikube/disk.vmdk"
"SATA-ImageUUID-1-0"="d0078cec-ab96-4660-af55-edf638120764"
"SATA-2-0"="none"
"SATA-3-0"="none"
"SATA-4-0"="none"
"SATA-5-0"="none"
"SATA-6-0"="none"
"SATA-7-0"="none"
"SATA-8-0"="none"
"SATA-9-0"="none"
"SATA-10-0"="none"
"SATA-11-0"="none"
"SATA-12-0"="none"
"SATA-13-0"="none"
"SATA-14-0"="none"
"SATA-15-0"="none"
"SATA-16-0"="none"
"SATA-17-0"="none"
"SATA-18-0"="none"
"SATA-19-0"="none"
"SATA-20-0"="none"
"SATA-21-0"="none"
"SATA-22-0"="none"
"SATA-23-0"="none"
"SATA-24-0"="none"
"SATA-25-0"="none"
"SATA-26-0"="none"
"SATA-27-0"="none"
"SATA-28-0"="none"
"SATA-29-0"="none"
natnet1="nat"
macaddress1="080027539964"
cableconnected1="on"
nic1="nat"
nictype1="virtio"
nicspeed1="0"
mtu="0"
sockSnd="64"
sockRcv="64"
tcpWndSnd="64"
tcpWndRcv="64"
Forwarding(0)="ssh,tcp,127.0.0.1,36781,,22"
hostonlyadapter2="vboxnet0"
macaddress2="0800272752C5"
cableconnected2="on"
nic2="hostonly"
nictype2="virtio"
nicspeed2="0"
nic3="none"
nic4="none"
nic5="none"
nic6="none"
nic7="none"
nic8="none"
hidpointing="ps2mouse"
hidkeyboard="ps2kbd"
uart1="off"
uart2="off"
uart3="off"
uart4="off"
lpt1="off"
lpt2="off"
audio="alsa"
audio_in="false"
audio_out="false"
clipboard="disabled"
draganddrop="disabled"
SessionName="headless"
VideoMode="720,400,0"@0,0 1
vrde="off"
usb="off"
ehci="off"
xhci="off"
SharedFolderNameMachineMapping1="hosthome"
SharedFolderPathMachineMapping1="/home"
VRDEActiveConnection="off"
VRDEClients=0
videocap="off"
videocap_audio="off"
videocapscreens=0
videocapfile="/home/user/.minikube/machines/minikube/minikube/minikube.webm"
videocapres=1024x768
videocaprate=512
videocapfps=25
videocapopts=
GuestMemoryBalloon=0
GuestOSType="Linux26_64"
GuestAdditionsRunLevel=0
}
STDERR:
{
}
Waiting for SSH to be available...
Getting to WaitForSSH function...
Using SSH client type: native
&{{{ 0 [] [] []} docker [0x8375b0] 0x837560 [] 0s} 127.0.0.1 22 }
About to run SSH command:
exit 0
Error dialing TCP: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain
Error dialing TCP: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain
Error dialing TCP: ssh: handshake failed: ssh: unable to authenticate, attempted methods [publickey none], no supported methods remain
Error dialing TCP: ssh: handshake failed: ssh: unable to authenticate, attempted methods [publickey none], no supported methods remain
Error dialing TCP: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain
Error dialing TCP: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain
....

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[fejta-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten

[tstromberg]

Closing as dupe of #2126 - this seems to be an ssh authentication issue rather than a networking issue.

@davidovich mentioned to me that he suspects a data race in https://github.com/docker/machine/blob/6c90ba4d3da9a37ebc15c4c65d61479db075fca5/libmachine/ssh/keys.go#L57 - which apparently does not close files properly.