Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unexpected JSON Output #77

Open
darkn3rd opened this issue Oct 11, 2020 · 8 comments
Open

Unexpected JSON Output #77

darkn3rd opened this issue Oct 11, 2020 · 8 comments

Comments

@darkn3rd
Copy link

darkn3rd commented Oct 11, 2020

I tried to use the latest released version with docker-machine, and I am getting this error. I am not deeply familiar with KVM or libvirt, so I am not sure how this works. I have libvirt working fine with vagrant and mini-kube (using their embedded kvm2 docker-machine driver) without any problems. I am not sure how I could debug further.

$ docker-machine -D create -d kvm myengine1
Docker Machine Version:  0.16.2, build bd45ab13
Found binary path at /usr/local/bin/docker-machine-driver-kvm
Launching plugin server for driver kvm
Plugin server listening at address 127.0.0.1:41571
() Calling .GetVersion
Using API Version  1
() Calling .SetConfigRaw
() Calling .GetMachineName
(flag-lookup) Calling .GetMachineName
(flag-lookup) Calling .DriverName
(flag-lookup) Calling .GetCreateFlags
Found binary path at /usr/local/bin/docker-machine-driver-kvm
Launching plugin server for driver kvm
Plugin server listening at address 127.0.0.1:45025
() Calling .GetVersion
Using API Version  1
() Calling .SetConfigRaw
() Calling .GetMachineName
(myengine1) Calling .GetMachineName
(myengine1) Calling .DriverName
(myengine1) Calling .GetCreateFlags
(myengine1) Calling .SetConfigFromFlags
(myengine1) DBG | SetConfigFromFlags called
Reading certificate data from /home/joaquin/.docker/machine/certs/ca.pem
Decoding PEM data...
Parsing certificate...
Reading certificate data from /home/joaquin/.docker/machine/certs/cert.pem
Decoding PEM data...
Parsing certificate...
Running pre-create checks...
(myengine1) Calling .PreCreateCheck
(myengine1) DBG | About to check libvirt version
(myengine1) DBG | Validating private network
(myengine1) DBG | Validating network default
(myengine1) Calling .GetConfigRaw
Creating machine...
(myengine1) Calling .Create
(myengine1) DBG | local Boot2Docker ISO version:  v19.03.12
(myengine1) Copying /home/joaquin/.docker/machine/cache/boot2docker.iso to /home/joaquin/.docker/machine/machines/myengine1/boot2docker.iso...
(myengine1) Creating SSH key...
(myengine1) DBG | Verifying executable bit set on /home/joaquin/.docker/machine/machines/myengine1
(myengine1) DBG | Setting executable bit set on /home/joaquin/.docker/machine/machines/myengine1
(myengine1) DBG | Verifying executable bit set on /home/joaquin/.docker/machine/machines
(myengine1) DBG | Verifying executable bit set on /home/joaquin/.docker/machine
(myengine1) DBG | Verifying executable bit set on /home/joaquin/.docker
(myengine1) DBG | Verifying executable bit set on /home/joaquin
(myengine1) DBG | Verifying executable bit set on /home
(myengine1) DBG | Creating VM data disk...
(myengine1) DBG | Creating 20000 MB hard disk image...
(myengine1) DBG | Defining VM...
(myengine1) DBG | Starting VM myengine1
(myengine1) DBG | GetIP called for myengine1
(myengine1) DBG | Failed to retrieve dnsmasq leases from /var/lib/libvirt/dnsmasq/docker-machines.leases
(myengine1) Failed to decode dnsmasq lease status: unexpected end of JSON input
(myengine1) DBG | Unable to locate IP address for MAC 52:54:00:d2:f1:a0
(myengine1) DBG | Waiting for the VM to come up... 0
(myengine1) DBG | GetIP called for myengine1
(myengine1) DBG | Failed to retrieve dnsmasq leases from /var/lib/libvirt/dnsmasq/docker-machines.leases
(myengine1) DBG | Unable to locate IP address for MAC 52:54:00:d2:f1:a0
(myengine1) DBG | Waiting for the VM to come up... 1
(myengine1) Failed to decode dnsmasq lease status: unexpected end of JSON input
(myengine1) DBG | GetIP called for myengine1
(myengine1) DBG | Failed to retrieve dnsmasq leases from /var/lib/libvirt/dnsmasq/docker-machines.leases
(myengine1) Failed to decode dnsmasq lease status: unexpected end of JSON input
(myengine1) DBG | Unable to locate IP address for MAC 52:54:00:d2:f1:a0
(myengine1) DBG | Waiting for the VM to come up... 2
(myengine1) DBG | GetIP called for myengine1
(myengine1) DBG | Failed to retrieve dnsmasq leases from /var/lib/libvirt/dnsmasq/docker-machines.leases
(myengine1) Failed to decode dnsmasq lease status: unexpected end of JSON input
(myengine1) DBG | Unable to locate IP address for MAC 52:54:00:d2:f1:a0
(myengine1) DBG | Waiting for the VM to come up... 3
(myengine1) DBG | GetIP called for myengine1
(myengine1) DBG | Failed to retrieve dnsmasq leases from /var/lib/libvirt/dnsmasq/docker-machines.leases
(myengine1) Failed to decode dnsmasq lease status: unexpected end of JSON input
(myengine1) DBG | Unable to locate IP address for MAC 52:54:00:d2:f1:a0
(myengine1) DBG | Waiting for the VM to come up... 4
(myengine1) DBG | GetIP called for myengine1
(myengine1) DBG | Failed to retrieve dnsmasq leases from /var/lib/libvirt/dnsmasq/docker-machines.leases
(myengine1) Failed to decode dnsmasq lease status: unexpected end of JSON input
(myengine1) DBG | Unable to locate IP address for MAC 52:54:00:d2:f1:a0
(myengine1) DBG | Waiting for the VM to come up... 5
(myengine1) DBG | GetIP called for myengine1
(myengine1) DBG | Failed to retrieve dnsmasq leases from /var/lib/libvirt/dnsmasq/docker-machines.leases
(myengine1) Failed to decode dnsmasq lease status: unexpected end of JSON input
(myengine1) DBG | Unable to locate IP address for MAC 52:54:00:d2:f1:a0
(myengine1) DBG | Waiting for the VM to come up... 6
(myengine1) DBG | GetIP called for myengine1
(myengine1) DBG | Failed to retrieve dnsmasq leases from /var/lib/libvirt/dnsmasq/docker-machines.leases
(myengine1) Failed to decode dnsmasq lease status: unexpected end of JSON input
(myengine1) DBG | Unable to locate IP address for MAC 52:54:00:d2:f1:a0
(myengine1) DBG | Waiting for the VM to come up... 7
(myengine1) DBG | GetIP called for myengine1
(myengine1) DBG | Failed to retrieve dnsmasq leases from /var/lib/libvirt/dnsmasq/docker-machines.leases
(myengine1) Failed to decode dnsmasq lease status: unexpected end of JSON input
(myengine1) DBG | Unable to locate IP address for MAC 52:54:00:d2:f1:a0
(myengine1) DBG | Waiting for the VM to come up... 8
(myengine1) DBG | GetIP called for myengine1
(myengine1) DBG | Failed to retrieve dnsmasq leases from /var/lib/libvirt/dnsmasq/docker-machines.leases
(myengine1) Failed to decode dnsmasq lease status: unexpected end of JSON input
(myengine1) DBG | Unable to locate IP address for MAC 52:54:00:d2:f1:a0
(myengine1) DBG | Waiting for the VM to come up... 9

There's no such file /var/lib/libvirt/dnsmasq/docker-machines.leases.

$ ls -la /var/lib/libvirt/dnsmasq
total 24
drwxr-xr-x 2 root root 4096 Oct 11 03:46 .
drwxr-xr-x 7 root root 4096 Oct 11 02:00 ..
-rw-r--r-- 1 root root    0 Oct 11 02:01 default.addnhosts
-rw------- 1 root root  619 Oct 11 02:01 default.conf
-rw-r--r-- 1 root root    0 Oct 11 02:01 default.hostsfile
-rw-r--r-- 1 root root    0 Oct 11 03:25 docker-machines.addnhosts
-rw------- 1 root root  688 Oct 11 03:25 docker-machines.conf
-rw-r--r-- 1 root root    0 Oct 11 03:25 docker-machines.hostsfile
-rw-r--r-- 1 root root   85 Oct 11 03:46 virbr0.macs
-rw-r--r-- 1 root root    0 Oct 11 01:02 virbr0.status
-rw-r--r-- 1 root root   85 Oct 11 03:46 virbr1.macs
-rw-r--r-- 1 root root    0 Oct 11 03:25 virbr1.status
@afbjorklund
Copy link

This is related to libvirt/libvirt@0f87054 since libvirt doesn't use a dnsmasq file for this anymore (--leasefile-ro).

As you say, the error format is misleading. It should probably have been "file not found", not that it would have helped...

@afbjorklund
Copy link

afbjorklund commented Oct 12, 2020

Also seen in #60 and #72

minikube uses virtio now

@afbjorklund
Copy link

Actually it was not only using the correct code (status instead of leases), it also needs to use a different network card.
The current "rtl8139" is not supported with the kernel config, it doesn't have to use virtio but can at least use "e1000".

model type rtl8139 (fails)
rtl8139

model type e1000 (works)
e1000

It is possible to backport code from minikube to this driver, but since it is not supported anymore there will be no releases.
With enough interest it would be possible to make an version at https://github.com/machine-drivers/docker-machine-kvm

diff --git a/kvm.go b/kvm.go
index 1e6c891..c73de61 100644
--- a/kvm.go
+++ b/kvm.go
@@ -665,11 +667,22 @@ func (d *Driver) GetIP() (string, error) {
        if err != nil {
                return "", err
        }
-       /*
-        * TODO - Figure out what version of libvirt changed behavior and
-        *        be smarter about selecting which algorithm to use
-        */
-       ip, err := d.getIPByMACFromLeaseFile(mac)
+       conn, err := d.getConn()
+       if err != nil {
+               return "", err
+       }
+       defer conn.Close()
+
+       var ip string
+       libVersion, err := conn.GetLibVersion()
+       if err != nil {
+               return "", err
+       }
+
+       // Earlier versions of libvirt use a lease file instead of a status file
+       if libVersion < 1002006 {
+               ip, err = d.getIPByMACFromLeaseFile(mac)
+       }
        if ip == "" {
                ip, err = d.getIPByMacFromSettings(mac)
        }

@darkn3rd
Copy link
Author

As newbie to all of this, some questions:

  • These are virtual NICs from kvm/libvirt? If so, what determines use of one NIC or another?
  • For the lease file mechanism, is there any docs from libvirt on that? or just in the code? I want to learn more how it works.
  • No releases: meaning there will be no more tag versions? Or meaning this is in archival mode, no more commits for master? It would be nice to add to master or docs...

@afbjorklund
Copy link

Those were the virtual nics, the"rtl8139" is the default and I changed it to "e1000" in a local build:

--- a/kvm.go
+++ b/kvm.go
@@ -60,9 +60,11 @@ const (
     </graphics>
     <interface type='network'>
       <source network='{{.Network}}'/>
+      <model type='e1000'/>
     </interface>
     <interface type='network'>
       <source network='{{.PrivateNetwork}}'/>
+      <model type='e1000'/>
     </interface>
   </devices>
 </domain>`

The lease file used to carry the IPs allocated, but it is now in the status file (in dnsmasq) instead.

No releases: meaning there will be no more tag versions? Or meaning this is in archival mode, no more commits for master? It would be nice to add to master or docs...

See #67

Only the VirtualBox driver was officially supported by Docker Inc., and now Machine is deprecated too.

So far there has been no volunteers to take over the development and release, in the "new" organization.

As the name implies, it was originally only for drivers:

https://github.com/machine-drivers/discussion/issues

@afbjorklund
Copy link

@darkn3rd : You can find a prebuilt binary here: https://github.com/afbjorklund/docker-machine-kvm

@afbjorklund
Copy link

The missing kernel configuration for "rtl8139cp" was reported here: boot2docker/boot2docker#1407

@jernejr
Copy link

jernejr commented Jan 18, 2021

@darkn3rd : You can find a prebuilt binary here: https://github.com/afbjorklund/docker-machine-kvm

That solved this exact issue for me

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants