Script fails after virtual machine is restarted

[nithin-bose]

Script fails with the following error:
Restart Docker Machine ... Too many retries waiting for SSH to be available. Last error: Maximum number of retries (60) exceeded

I am not able to connect to the machine even with docker-machine ssh

[tonivdv]

Can you try to restart it again? And print the logs /var/log/bootlocal.log and /var/log/boot2docker.log.

If you can't re-access the virtual machine (hopefully not an important one), can you try again with another machine?

Also are you using last versions of everything?

[nithin-bose]

The status of the machine is "running". SSH does not work. It was not an important machine... I recreated the machine and re-ran everything. But it fails at nfs verification.

Checked that the host nfs server is running. The following is on the bootlocal.log:
mount: mounting 192.168.99.1:/Users on /Users failed: Bad file descriptor

I am running the docker-toolbox 1.9.0d. Boot2Docker version 1.9.0, build master : 16e4a2a - Tue Nov 3 19:49:22 UTC 2015. Docker version 1.9.0, build 76d6bc9

[nithin-bose]

Updated to docker-toolbox 1.9.1b. Boot2Docker version 1.9.1, build master : cef800b - Fri Nov 20 19:33:59 UTC 2015. Docker version 1.9.1, build a34a1d5.

cat /var/log/bootlocal.log 
Starting nfs client utilities.
root@test:~# /usr/local/etc/init.d/nfs-client start
/usr/local/sbin/rpcbind is already running
1051
/usr/local/sbin/rpc.statd is already running
1055
Starting nfs client utilities.
root@test:~# /usr/local/etc/init.d/nfs-client status
NFS-client is not running.

[tonivdv]

@nithin-bose Can you reboot your mac? Seems related to: #22 (comment)

[nithin-bose]

restarted.... but same as above
Using mac 10.11.1, if that matters

[nithin-bose]

Further on this, I am able to ping 192.168.99.1 (host) from 192.168.99.100 (vm) immediately after creating woth docker-machine. However after running docker-machine-nfs, ping from the host fails. However, pinging something outside, like ggogle.com works.... could it be that the network settings are affected somehow??

[tonivdv]

Certain that no firewall is blocking this? Cause another user had that case too.

[nithin-bose]

No firewalls, unless boot2docker or virtual box has one. Mac firewall is off. I have no idea why ping should fail after the script is run.

[nithin-bose]

@brandontamm I will try this out and let you know

[nithin-bose]

Tried this.... same issue... after running docker-machine-nfs, even though I can docker ssh into the machine, ping from boot2docker to host machine fails.

Boot2Docker version 1.9.1, build master : cef800b - Fri Nov 20 19:33:59 UTC 2015
Docker version 1.9.1, build a34a1d5

root@test:~# ping 192.168.99.1
PING 192.168.99.1 (192.168.99.1): 56 data bytes

--- 192.168.99.1 ping statistics ---
60 packets transmitted, 0 packets received, 100% packet loss

root@test:~# /usr/local/etc/init.d/nfs-client status
NFS-client is not running.

root@test:~# /usr/local/etc/init.d/nfs-client start
/usr/local/sbin/rpcbind is already running
1021
/usr/local/sbin/rpc.statd is already running
1024
Starting nfs client utilities.
root@test:~# /usr/local/etc/init.d/nfs-client status
NFS-client is not running.

I am not sure what is causing the issue. I really want this to work. Or at least something that passes fs events to the docker machine. :(

[tonivdv]

Hey @nithin-bose ,

When you ssh in your machine, can you ping your host ? (Not the 192.168.99.1)

[nithin-bose]

Did not get you. 192.168.99.1 is my host and 192.168.99.100 is my boot2docker vbox guest machine. The same is used for nfs as well. The commands above are from the boot2docker guest. Ping works fine if I docker-machine ssh and do it before running docker-machine-nfs.

[tonivdv]

On my machine I also have the 192.168.99.1 which is an ip from virtualbox (vboxnet4), but my actual ip from my host (mac os) is 10.0.0.X ... the ip received from my router. Can you ping that one from your guest?

Not sure this test is worth anything. I'm no expert in all underlying network stuff. Just trying to figure out why your nfs is not connecting between host<->guest.

Cheers

T

[nithin-bose]

I dont see any 10.* ip in my ifconfig. I only have a vboxnet0 adapter which shows the ip to be 192.168.99.100 :( event i dont understand why its not working.... your script is simple enough.... nfs-client seems to hang.... the same is the case with my friend too.... which versions are you on?

[tonivdv]

Hey @nithin-bose ,

My ip is 10.* because my router is in that range. But it could be very well another ip. Most routers give an ip in a range of 192.168.0.X . Anyway not sure this will tell anything.

For all the things a I use, all are at latest version.

[nithin-bose]

Oh sorry.... vbox used to give ip in the 10.* range so i thought you were refering to that. I will check this out and let you know. Thank you for your patience 👍

[paolomainardi]

Same here, with a fresh installation starting from the last docker toolbox, the nfs share doesn't work.

Darwin paolospark 14.4.0 Darwin Kernel Version 14.4.0: Thu May 28 11:35:04 PDT 2015; root:xnu-2782.30.5~1/RELEASE_X86_64 x86_64

docker-machine create dev -d virtualbox --virtualbox-disk-size 50000 --virtualbox-cpu-count 1 --virtualbox-memory 4096
docker-machine-nfs dev --nfs-config="-alldirs -maproot=0"

[cut]
[INFO] Configure Docker Machine ...         OK
[INFO] Restart Docker Machine ...       OK
[INFO] Verify NFS mount ...             FAIL

Cannot detect the NFS mount :(

~ ❯ docker-machine ssh dev docker --version                                                                                                                     
Docker version 1.9.1, build a34a1d5

~ ❯ docker-machine ssh dev ping 192.168.99.1
PING 192.168.99.1 (192.168.99.1): 56 data bytes
64 bytes from 192.168.99.1: seq=0 ttl=64 time=0.267 ms
64 bytes from 192.168.99.1: seq=1 ttl=64 time=0.356 ms
64 bytes from 192.168.99.1: seq=2 ttl=64 time=0.499 ms

~ ❯ docker-machine ssh dev mount                                                                                                                                 ⏎
tmpfs on / type tmpfs (rw,relatime,size=3641408k)
proc on /proc type proc (rw,relatime)
sysfs on /sys type sysfs (rw,relatime)
devpts on /dev/pts type devpts (rw,relatime,mode=600,ptmxmode=000)
tmpfs on /dev/shm type tmpfs (rw,relatime)
fusectl on /sys/fs/fuse/connections type fusectl (rw,relatime)
/dev/sda1 on /mnt/sda1 type ext4 (rw,relatime,data=ordered)
cgroup on /sys/fs/cgroup type tmpfs (rw,relatime,mode=755)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,relatime,cpuset)
cgroup on /sys/fs/cgroup/cpu type cgroup (rw,relatime,cpu)
cgroup on /sys/fs/cgroup/cpuacct type cgroup (rw,relatime,cpuacct)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,relatime,blkio)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,relatime,memory)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,relatime,devices)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,relatime,freezer)
cgroup on /sys/fs/cgroup/net_cls type cgroup (rw,relatime,net_cls)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,relatime,perf_event)
cgroup on /sys/fs/cgroup/net_prio type cgroup (rw,relatime,net_prio)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,relatime,hugetlb)
/dev/sda1 on /mnt/sda1/var/lib/docker/aufs type ext4 (rw,relatime,data=ordered)

~ ❯ docker-machine ssh dev "sudo /var/lib/boot2docker/bootlocal.sh"
umount: can't umount /Users: Invalid argument
/usr/local/sbin/rpcbind is already running
1021
/usr/local/sbin/rpc.statd is already running
1025
Starting nfs client utilities.
mount: RPC: Remote system error - Connection refused
mount: mounting 192.168.99.1:/

~ ❯ sudo nfsd status                                                                                                                                            
nfsd service is enabled
nfsd is running (pid 1590, 8 threads)

~ ❯ cat /etc/exports

/Users 192.168.99.100 -alldirs -maproot=0

[paolomainardi]

Problem fixed, it was missing the 127.0.0.1 localhost entry on my /etc/hosts.

[nithin-bose]

@paolomainardi which hosts file was missing this entry the host or guest?

[tonivdv]

@nithin-bose It should be on the host. Guest doesn't have to be touched on that level.