Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

docker: Error response from daemon #10496

Closed
horseinthesky opened this issue Feb 17, 2021 · 3 comments
Closed

docker: Error response from daemon #10496

horseinthesky opened this issue Feb 17, 2021 · 3 comments
Labels
co/docker-driver Issues related to kubernetes in container kind/support Categorizes issue or PR as a support question. os/linux

Comments

@horseinthesky
Copy link

Steps to reproduce the issue:

Unable to create cluster.

Arch
minikube 1.17.1 from pacman

Clean start attempt

  1. minikube delete
  2. docker service prune
  3. minikube start

Full output of failed command:

minikube start --alsologtostderr               

I0217 14:59:48.817206  173041 out.go:229] Setting OutFile to fd 1 ...
I0217 14:59:48.817495  173041 out.go:281] isatty.IsTerminal(1) = true
I0217 14:59:48.817514  173041 out.go:242] Setting ErrFile to fd 2...
I0217 14:59:48.817524  173041 out.go:281] isatty.IsTerminal(2) = true
I0217 14:59:48.817665  173041 root.go:291] Updating PATH: /home/horseinthesky/.minikube/bin
I0217 14:59:48.818039  173041 out.go:236] Setting JSON to false
I0217 14:59:48.819129  173041 start.go:106] hostinfo: {"hostname":"KappaA","uptime":2075384,"bootTime":1611487805,"procs":188,"os":"linux","platform":"arch","platformFamily":"arch","platformVersion":"","kernelVersion":"5.10.9-arch1-1","kernelArch":"x86_64","virtualizationSystem":"kvm","virtualizationRole":"host","hostId":"2da4b796-a999-4db9-86bd-766ff5de7268"}
I0217 14:59:48.819217  173041 start.go:116] virtualization: kvm host
I0217 14:59:48.819506  173041 out.go:119] 😄  minikube v1.17.1 on Arch 
😄  minikube v1.17.1 on Arch 
I0217 14:59:48.819656  173041 driver.go:315] Setting default libvirt URI to qemu:///system
I0217 14:59:48.819688  173041 global.go:102] Querying for installed drivers using PATH=/home/horseinthesky/.minikube/bin:/home/horseinthesky/.pyenv/bin:/usr/local/sbin:/usr/local/bin:/usr/bin:/usr/bin/site_perl:/usr/bin/vendor_perl:/usr/bin/core_perl:/home/horseinthesky/.fzf/bin:/home/horseinthesky/.local/bin
I0217 14:59:48.819751  173041 notify.go:126] Checking for updates...
I0217 14:59:48.832486  173041 global.go:110] none priority: 4, state: {Installed:true Healthy:true Running:false NeedsImprovement:false Error:<nil> Reason: Fix: Doc:}
I0217 14:59:48.976453  173041 podman.go:118] podman version: 3.0.0
I0217 14:59:48.976512  173041 global.go:110] podman priority: 3, state: {Installed:true Healthy:true Running:false NeedsImprovement:false Error:<nil> Reason: Fix: Doc:}
I0217 14:59:48.976555  173041 global.go:110] ssh priority: 4, state: {Installed:true Healthy:true Running:false NeedsImprovement:false Error:<nil> Reason: Fix: Doc:}
I0217 14:59:48.976699  173041 global.go:110] virtualbox priority: 6, state: {Installed:false Healthy:false Running:false NeedsImprovement:false Error:unable to find VBoxManage in $PATH Reason: Fix:Install VirtualBox Doc:https://minikube.sigs.k8s.io/docs/reference/drivers/virtualbox/}
I0217 14:59:48.976835  173041 global.go:110] vmware priority: 7, state: {Installed:false Healthy:false Running:false NeedsImprovement:false Error:exec: "docker-machine-driver-vmware": executable file not found in $PATH Reason: Fix:Install docker-machine-driver-vmware Doc:https://minikube.sigs.k8s.io/docs/reference/drivers/vmware/}
I0217 14:59:49.034378  173041 docker.go:115] docker version: linux-20.10.2
I0217 14:59:49.034571  173041 cli_runner.go:111] Run: docker system info --format "{{json .}}"
I0217 14:59:49.149870  173041 info.go:253] docker info: {ID:VA7H:JSAZ:6GVA:ROAA:X5OG:4J5F:5VRP:KOWP:D7F4:V5VC:4FMM:BV6D Containers:0 ContainersRunning:0 ContainersPaused:0 ContainersStopped:0 Images:1 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Native Overlay Diff false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:true KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:25 OomKillDisable:true NGoroutines:40 SystemTime:2021-02-17 14:59:49.070474755 +0300 MSK LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.10.9-arch1-1 OperatingSystem:Arch Linux OSType:linux Architecture:x86_64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:4 MemTotal:16784654336 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:KappaA Labels:[] ExperimentalBuild:false ServerVersion:20.10.2 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:269548fa27e0089a8b8278fc4fc781d7f65a939b.m Expected:269548fa27e0089a8b8278fc4fc781d7f65a939b.m} RuncCommit:{ID:12644e614e25b05da6fd08a38ffa0cfe1903fdec Expected:12644e614e25b05da6fd08a38ffa0cfe1903fdec} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=seccomp,profile=default] ProductLicense: Warnings:[WARNING: No blkio weight support WARNING: No blkio weight_device support] ServerErrors:[] ClientInfo:{Debug:false Plugins:[map[Experimental:true Name:app Path:/usr/lib/docker/cli-plugins/docker-app SchemaVersion:0.1.0 ShortDescription:Docker App Vendor:Docker Inc. Version:v0.9.1-beta3] map[Experimental:true Name:buildx Path:/usr/lib/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Build with BuildKit Vendor:Docker Inc. Version:v0.5.1-tp-docker]] Warnings:<nil>}}
I0217 14:59:49.150027  173041 docker.go:145] overlay module found
I0217 14:59:49.150046  173041 global.go:110] docker priority: 9, state: {Installed:true Healthy:true Running:false NeedsImprovement:false Error:<nil> Reason: Fix: Doc:}
I0217 14:59:49.150109  173041 global.go:110] kvm2 priority: 8, state: {Installed:false Healthy:false Running:false NeedsImprovement:false Error:exec: "virsh": executable file not found in $PATH Reason: Fix:Install libvirt Doc:https://minikube.sigs.k8s.io/docs/reference/drivers/kvm2/}
I0217 14:59:49.150138  173041 driver.go:257] not recommending "none" due to priority: 4
I0217 14:59:49.150153  173041 driver.go:257] not recommending "ssh" due to priority: 4
I0217 14:59:49.150171  173041 driver.go:257] not recommending "podman" due to priority: 3
I0217 14:59:49.150192  173041 driver.go:286] Picked: docker
I0217 14:59:49.150209  173041 driver.go:287] Alternatives: [none ssh podman (experimental)]
I0217 14:59:49.150226  173041 driver.go:288] Rejects: [virtualbox vmware kvm2]
I0217 14:59:49.150410  173041 out.go:119] ✨  Automatically selected the docker driver. Other choices: none, ssh, podman (experimental)
✨  Automatically selected the docker driver. Other choices: none, ssh, podman (experimental)
I0217 14:59:49.150444  173041 start.go:279] selected driver: docker
I0217 14:59:49.150454  173041 start.go:702] validating driver "docker" against <nil>
I0217 14:59:49.150475  173041 start.go:713] status for docker: {Installed:true Healthy:true Running:false NeedsImprovement:false Error:<nil> Reason: Fix: Doc:}
I0217 14:59:49.150552  173041 cli_runner.go:111] Run: docker system info --format "{{json .}}"
I0217 14:59:49.266288  173041 info.go:253] docker info: {ID:VA7H:JSAZ:6GVA:ROAA:X5OG:4J5F:5VRP:KOWP:D7F4:V5VC:4FMM:BV6D Containers:0 ContainersRunning:0 ContainersPaused:0 ContainersStopped:0 Images:1 Driver:overlay2 DriverStatus:[[Backing Filesystem extfs] [Supports d_type true] [Native Overlay Diff false]] SystemStatus:<nil> Plugins:{Volume:[local] Network:[bridge host ipvlan macvlan null overlay] Authorization:<nil> Log:[awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog]} MemoryLimit:true SwapLimit:true KernelMemory:true KernelMemoryTCP:true CPUCfsPeriod:true CPUCfsQuota:true CPUShares:true CPUSet:true PidsLimit:true IPv4Forwarding:true BridgeNfIptables:true BridgeNfIP6Tables:true Debug:false NFd:25 OomKillDisable:true NGoroutines:40 SystemTime:2021-02-17 14:59:49.186238155 +0300 MSK LoggingDriver:json-file CgroupDriver:cgroupfs NEventsListener:0 KernelVersion:5.10.9-arch1-1 OperatingSystem:Arch Linux OSType:linux Architecture:x86_64 IndexServerAddress:https://index.docker.io/v1/ RegistryConfig:{AllowNondistributableArtifactsCIDRs:[] AllowNondistributableArtifactsHostnames:[] InsecureRegistryCIDRs:[127.0.0.0/8] IndexConfigs:{DockerIo:{Name:docker.io Mirrors:[] Secure:true Official:true}} Mirrors:[]} NCPU:4 MemTotal:16784654336 GenericResources:<nil> DockerRootDir:/var/lib/docker HTTPProxy: HTTPSProxy: NoProxy: Name:KappaA Labels:[] ExperimentalBuild:false ServerVersion:20.10.2 ClusterStore: ClusterAdvertise: Runtimes:{Runc:{Path:runc}} DefaultRuntime:runc Swarm:{NodeID: NodeAddr: LocalNodeState:inactive ControlAvailable:false Error: RemoteManagers:<nil>} LiveRestoreEnabled:false Isolation: InitBinary:docker-init ContainerdCommit:{ID:269548fa27e0089a8b8278fc4fc781d7f65a939b.m Expected:269548fa27e0089a8b8278fc4fc781d7f65a939b.m} RuncCommit:{ID:12644e614e25b05da6fd08a38ffa0cfe1903fdec Expected:12644e614e25b05da6fd08a38ffa0cfe1903fdec} InitCommit:{ID:de40ad0 Expected:de40ad0} SecurityOptions:[name=seccomp,profile=default] ProductLicense: Warnings:[WARNING: No blkio weight support WARNING: No blkio weight_device support] ServerErrors:[] ClientInfo:{Debug:false Plugins:[map[Experimental:true Name:app Path:/usr/lib/docker/cli-plugins/docker-app SchemaVersion:0.1.0 ShortDescription:Docker App Vendor:Docker Inc. Version:v0.9.1-beta3] map[Experimental:true Name:buildx Path:/usr/lib/docker/cli-plugins/docker-buildx SchemaVersion:0.1.0 ShortDescription:Build with BuildKit Vendor:Docker Inc. Version:v0.5.1-tp-docker]] Warnings:<nil>}}
I0217 14:59:49.266504  173041 start_flags.go:249] no existing cluster config was found, will generate one from the flags 
I0217 14:59:49.267135  173041 start_flags.go:267] Using suggested 4000MB memory alloc based on sys=16007MB, container=16007MB
I0217 14:59:49.267376  173041 start_flags.go:671] Wait components to verify : map[apiserver:true system_pods:true]
I0217 14:59:49.267422  173041 cni.go:74] Creating CNI manager for ""
I0217 14:59:49.267450  173041 cni.go:139] CNI unnecessary in this configuration, recommending no CNI
I0217 14:59:49.267469  173041 start_flags.go:390] config:
{Name:minikube KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase:v0.0.17@sha256:1cd2e039ec9d418e6380b2fa0280503a72e5b282adea674ee67882f59f4f546e Memory:4000 CPUs:2 DiskSize:20000 VMDriver: Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.99.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.2 ClusterName:minikube Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[] Addons:map[] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] Network: MultiNodeRequested:false}
I0217 14:59:49.267771  173041 out.go:119] 👍  Starting control plane node minikube in cluster minikube
👍  Starting control plane node minikube in cluster minikube
I0217 14:59:49.308416  173041 image.go:92] Found gcr.io/k8s-minikube/kicbase:v0.0.17@sha256:1cd2e039ec9d418e6380b2fa0280503a72e5b282adea674ee67882f59f4f546e in local docker daemon, skipping pull
I0217 14:59:49.308468  173041 cache.go:116] gcr.io/k8s-minikube/kicbase:v0.0.17@sha256:1cd2e039ec9d418e6380b2fa0280503a72e5b282adea674ee67882f59f4f546e exists in daemon, skipping pull
I0217 14:59:49.308495  173041 preload.go:97] Checking if preload exists for k8s version v1.20.2 and runtime docker
I0217 14:59:49.308579  173041 preload.go:105] Found local preload: /home/horseinthesky/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v8-v1.20.2-docker-overlay2-amd64.tar.lz4
I0217 14:59:49.308606  173041 cache.go:54] Caching tarball of preloaded images
I0217 14:59:49.308648  173041 preload.go:131] Found /home/horseinthesky/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v8-v1.20.2-docker-overlay2-amd64.tar.lz4 in cache, skipping download
I0217 14:59:49.308668  173041 cache.go:57] Finished verifying existence of preloaded tar for  v1.20.2 on docker
I0217 14:59:49.309275  173041 profile.go:148] Saving config to /home/horseinthesky/.minikube/profiles/minikube/config.json ...
I0217 14:59:49.309333  173041 lock.go:36] WriteFile acquiring /home/horseinthesky/.minikube/profiles/minikube/config.json: {Name:mk5cb7d5c4de4d8ca19444cc45be9caa15c4f0dc Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0217 14:59:49.309911  173041 cache.go:185] Successfully downloaded all kic artifacts
I0217 14:59:49.309960  173041 start.go:313] acquiring machines lock for minikube: {Name:mk2ca001533c4eb542be71a7f8bf1a970f80797f Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0217 14:59:49.310061  173041 start.go:317] acquired machines lock for "minikube" in 69.102µs
I0217 14:59:49.310101  173041 start.go:89] Provisioning new machine with config: &{Name:minikube KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase:v0.0.17@sha256:1cd2e039ec9d418e6380b2fa0280503a72e5b282adea674ee67882f59f4f546e Memory:4000 CPUs:2 DiskSize:20000 VMDriver: Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.99.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.2 ClusterName:minikube Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.20.2 ControlPlane:true Worker:true}] Addons:map[] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] Network: MultiNodeRequested:false} &{Name: IP: Port:8443 KubernetesVersion:v1.20.2 ControlPlane:true Worker:true}
I0217 14:59:49.310198  173041 start.go:126] createHost starting for "" (driver="docker")
I0217 14:59:49.310472  173041 out.go:140] 🔥  Creating docker container (CPUs=2, Memory=4000MB) ...
🔥  Creating docker container (CPUs=2, Memory=4000MB) ...| I0217 14:59:49.310831  173041 start.go:160] libmachine.API.Create for "minikube" (driver="docker")
I0217 14:59:49.310900  173041 client.go:168] LocalClient.Create starting
I0217 14:59:49.311123  173041 main.go:119] libmachine: Reading certificate data from /home/horseinthesky/.minikube/certs/ca.pem
I0217 14:59:49.311190  173041 main.go:119] libmachine: Decoding PEM data...
I0217 14:59:49.311377  173041 main.go:119] libmachine: Parsing certificate...
I0217 14:59:49.311615  173041 main.go:119] libmachine: Reading certificate data from /home/horseinthesky/.minikube/certs/cert.pem
I0217 14:59:49.311698  173041 main.go:119] libmachine: Decoding PEM data...
I0217 14:59:49.311739  173041 main.go:119] libmachine: Parsing certificate...
I0217 14:59:49.312375  173041 cli_runner.go:111] Run: docker network inspect minikube --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}},{{$first := true}} "ContainerIPs": [{{range $k,$v := .Containers }}{{if $first}}{{$first = false}}{{else}}, {{end}}"{{$v.IPv4Address}}"{{end}}]}"
W0217 14:59:49.353054  173041 cli_runner.go:149] docker network inspect minikube --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}},{{$first := true}} "ContainerIPs": [{{range $k,$v := .Containers }}{{if $first}}{{$first = false}}{{else}}, {{end}}"{{$v.IPv4Address}}"{{end}}]}" returned with exit code 1
I0217 14:59:49.353278  173041 network_create.go:249] running [docker network inspect minikube] to gather additional debugging logs...
I0217 14:59:49.353317  173041 cli_runner.go:111] Run: docker network inspect minikube
W0217 14:59:49.394965  173041 cli_runner.go:149] docker network inspect minikube returned with exit code 1
I0217 14:59:49.395031  173041 network_create.go:252] error running [docker network inspect minikube]: docker network inspect minikube: exit status 1
stdout:
[]

stderr:
Error: No such network: minikube
I0217 14:59:49.395066  173041 network_create.go:254] output of [docker network inspect minikube]: -- stdout --
[]

-- /stdout --
** stderr ** 
Error: No such network: minikube

** /stderr **
I0217 14:59:49.395140  173041 cli_runner.go:111] Run: docker network inspect bridge --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}},{{$first := true}} "ContainerIPs": [{{range $k,$v := .Containers }}{{if $first}}{{$first = false}}{{else}}, {{end}}"{{$v.IPv4Address}}"{{end}}]}"
/ I0217 14:59:49.436731  173041 network_create.go:104] attempt to create network 192.168.49.0/24 with subnet: minikube and gateway 192.168.49.1 and MTU of 1500 ...
I0217 14:59:49.436846  173041 cli_runner.go:111] Run: docker network create --driver=bridge --subnet=192.168.49.0/24 --gateway=192.168.49.1 -o --ip-masq -o --icc -o com.docker.network.driver.mtu=1500 --label=created_by.minikube.sigs.k8s.io=true minikube
/ I0217 14:59:49.855397  173041 kic.go:100] calculated static IP "192.168.49.2" for the "minikube" container
I0217 14:59:49.855491  173041 cli_runner.go:111] Run: docker ps -a --format {{.Names}}
I0217 14:59:49.894263  173041 cli_runner.go:111] Run: docker volume create minikube --label name.minikube.sigs.k8s.io=minikube --label created_by.minikube.sigs.k8s.io=true
- I0217 14:59:49.933543  173041 oci.go:102] Successfully created a docker volume minikube
I0217 14:59:49.933646  173041 cli_runner.go:111] Run: docker run --rm --entrypoint /usr/bin/test -v minikube:/var gcr.io/k8s-minikube/kicbase:v0.0.17@sha256:1cd2e039ec9d418e6380b2fa0280503a72e5b282adea674ee67882f59f4f546e -d /var/lib
| I0217 14:59:50.524982  173041 oci.go:106] Successfully prepared a docker volume minikube
W0217 14:59:50.525075  173041 oci.go:159] Your kernel does not support swap limit capabilities or the cgroup is not mounted.
I0217 14:59:50.525080  173041 preload.go:97] Checking if preload exists for k8s version v1.20.2 and runtime docker
I0217 14:59:50.525189  173041 preload.go:105] Found local preload: /home/horseinthesky/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v8-v1.20.2-docker-overlay2-amd64.tar.lz4
I0217 14:59:50.525246  173041 kic.go:163] Starting extracting preloaded images to volume ...
I0217 14:59:50.525373  173041 cli_runner.go:111] Run: docker run --rm --entrypoint /usr/bin/tar -v /home/horseinthesky/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v8-v1.20.2-docker-overlay2-amd64.tar.lz4:/preloaded.tar:ro -v minikube:/extractDir gcr.io/k8s-minikube/kicbase:v0.0.17@sha256:1cd2e039ec9d418e6380b2fa0280503a72e5b282adea674ee67882f59f4f546e -I lz4 -xf /preloaded.tar -C /extractDir
I0217 14:59:50.525506  173041 cli_runner.go:111] Run: docker info --format "'{{json .SecurityOptions}}'"
/ I0217 14:59:50.663622  173041 cli_runner.go:111] Run: docker run -d -t --privileged --security-opt seccomp=unconfined --tmpfs /tmp --tmpfs /run -v /lib/modules:/lib/modules:ro --hostname minikube --name minikube --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=minikube --label role.minikube.sigs.k8s.io= --label mode.minikube.sigs.k8s.io=minikube --network minikube --ip 192.168.49.2 --volume minikube:/var --security-opt apparmor=unconfined --memory=4000mb --memory-swap=4000mb --cpus=2 -e container=docker --expose 8443 --publish=127.0.0.1::8443 --publish=127.0.0.1::22 --publish=127.0.0.1::2376 --publish=127.0.0.1::5000 gcr.io/k8s-minikube/kicbase:v0.0.17@sha256:1cd2e039ec9d418e6380b2fa0280503a72e5b282adea674ee67882f59f4f546e
\ W0217 14:59:50.870423  173041 cli_runner.go:149] docker run -d -t --privileged --security-opt seccomp=unconfined --tmpfs /tmp --tmpfs /run -v /lib/modules:/lib/modules:ro --hostname minikube --name minikube --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=minikube --label role.minikube.sigs.k8s.io= --label mode.minikube.sigs.k8s.io=minikube --network minikube --ip 192.168.49.2 --volume minikube:/var --security-opt apparmor=unconfined --memory=4000mb --memory-swap=4000mb --cpus=2 -e container=docker --expose 8443 --publish=127.0.0.1::8443 --publish=127.0.0.1::22 --publish=127.0.0.1::2376 --publish=127.0.0.1::5000 gcr.io/k8s-minikube/kicbase:v0.0.17@sha256:1cd2e039ec9d418e6380b2fa0280503a72e5b282adea674ee67882f59f4f546e returned with exit code 125
I0217 14:59:50.870519  173041 client.go:171] LocalClient.Create took 1.559592915s
\ I0217 14:59:52.870802  173041 ssh_runner.go:149] Run: sh -c "df -h /var | awk 'NR==2{print $5}'"
I0217 14:59:52.870897  173041 cli_runner.go:111] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube
W0217 14:59:52.913825  173041 cli_runner.go:149] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube returned with exit code 1
I0217 14:59:52.913984  173041 retry.go:31] will retry after 276.165072ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "minikube": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube: exit status 1
stdout:


stderr:
Template parsing error: template: :1:4: executing "" at <index (index .NetworkSettings.Ports "22/tcp") 0>: error calling index: index of untyped nil
- I0217 14:59:53.190623  173041 cli_runner.go:111] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube
\ W0217 14:59:53.237053  173041 cli_runner.go:149] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube returned with exit code 1
I0217 14:59:53.237170  173041 retry.go:31] will retry after 540.190908ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "minikube": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube: exit status 1
stdout:


stderr:
Template parsing error: template: :1:4: executing "" at <index (index .NetworkSettings.Ports "22/tcp") 0>: error calling index: index of untyped nil
| I0217 14:59:53.777659  173041 cli_runner.go:111] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube
W0217 14:59:53.819396  173041 cli_runner.go:149] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube returned with exit code 1
I0217 14:59:53.819551  173041 retry.go:31] will retry after 655.06503ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "minikube": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube: exit status 1
stdout:


stderr:
Template parsing error: template: :1:4: executing "" at <index (index .NetworkSettings.Ports "22/tcp") 0>: error calling index: index of untyped nil
\ I0217 14:59:54.474981  173041 cli_runner.go:111] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube
W0217 14:59:54.520187  173041 cli_runner.go:149] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube returned with exit code 1
I0217 14:59:54.520310  173041 retry.go:31] will retry after 791.196345ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "minikube": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube: exit status 1
stdout:


stderr:
Template parsing error: template: :1:4: executing "" at <index (index .NetworkSettings.Ports "22/tcp") 0>: error calling index: index of untyped nil
\ I0217 14:59:55.311841  173041 cli_runner.go:111] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube
| W0217 14:59:55.360191  173041 cli_runner.go:149] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube returned with exit code 1
W0217 14:59:55.360324  173041 start.go:257] error running df -h /var: NewSession: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "minikube": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube: exit status 1
stdout:


stderr:
Template parsing error: template: :1:4: executing "" at <index (index .NetworkSettings.Ports "22/tcp") 0>: error calling index: index of untyped nil

W0217 14:59:55.360357  173041 start.go:239] error getting percentage of /var that is free: NewSession: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "minikube": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube: exit status 1
stdout:


stderr:
Template parsing error: template: :1:4: executing "" at <index (index .NetworkSettings.Ports "22/tcp") 0>: error calling index: index of untyped nil
I0217 14:59:55.360386  173041 start.go:129] duration metric: createHost completed in 6.050168211s
I0217 14:59:55.360399  173041 start.go:80] releasing machines lock for "minikube", held for 6.050310855s
W0217 14:59:55.360453  173041 start.go:377] error starting host: creating host: create: creating: create kic node: create container: docker run -d -t --privileged --security-opt seccomp=unconfined --tmpfs /tmp --tmpfs /run -v /lib/modules:/lib/modules:ro --hostname minikube --name minikube --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=minikube --label role.minikube.sigs.k8s.io= --label mode.minikube.sigs.k8s.io=minikube --network minikube --ip 192.168.49.2 --volume minikube:/var --security-opt apparmor=unconfined --memory=4000mb --memory-swap=4000mb --cpus=2 -e container=docker --expose 8443 --publish=127.0.0.1::8443 --publish=127.0.0.1::22 --publish=127.0.0.1::2376 --publish=127.0.0.1::5000 gcr.io/k8s-minikube/kicbase:v0.0.17@sha256:1cd2e039ec9d418e6380b2fa0280503a72e5b282adea674ee67882f59f4f546e: exit status 125
stdout:
58167a27c16956f55245cc6a188f5647ebdb4599e312df53421a3244521865e6

stderr:
docker: Error response from daemon: driver failed programming external connectivity on endpoint minikube (e876589fb9130cddf8a30a32d4bbca36446fa913b0ffbe6ccad3cafc47b475ce):  (iptables failed: iptables --wait -t nat -A DOCKER -p tcp -d 127.0.0.1 --dport 49302 -j DNAT --to-destination 192.168.49.2:8443 ! -i br-bc41f06feb34: iptables v1.8.7 (legacy): unknown option "--dport"
Try `iptables -h' or 'iptables --help' for more information.
 (exit status 2)).
I0217 14:59:55.361027  173041 cli_runner.go:111] Run: docker container inspect minikube --format={{.State.Status}}
W0217 14:59:55.401345  173041 start.go:382] delete host: Docker machine "minikube" does not exist. Use "docker-machine ls" to list machines. Use "docker-machine create" to add a new one.
W0217 14:59:55.401611  173041 out.go:181] 🤦  StartHost failed, but will try again: creating host: create: creating: create kic node: create container: docker run -d -t --privileged --security-opt seccomp=unconfined --tmpfs /tmp --tmpfs /run -v /lib/modules:/lib/modules:ro --hostname minikube --name minikube --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=minikube --label role.minikube.sigs.k8s.io= --label mode.minikube.sigs.k8s.io=minikube --network minikube --ip 192.168.49.2 --volume minikube:/var --security-opt apparmor=unconfined --memory=4000mb --memory-swap=4000mb --cpus=2 -e container=docker --expose 8443 --publish=127.0.0.1::8443 --publish=127.0.0.1::22 --publish=127.0.0.1::2376 --publish=127.0.0.1::5000 gcr.io/k8s-minikube/kicbase:v0.0.17@sha256:1cd2e039ec9d418e6380b2fa0280503a72e5b282adea674ee67882f59f4f546e: exit status 125
stdout:
58167a27c16956f55245cc6a188f5647ebdb4599e312df53421a3244521865e6

stderr:
docker: Error response from daemon: driver failed programming external connectivity on endpoint minikube (e876589fb9130cddf8a30a32d4bbca36446fa913b0ffbe6ccad3cafc47b475ce):  (iptables failed: iptables --wait -t nat -A DOCKER -p tcp -d 127.0.0.1 --dport 49302 -j DNAT --to-destination 192.168.49.2:8443 ! -i br-bc41f06feb34: iptables v1.8.7 (legacy): unknown option "--dport"
Try `iptables -h' or 'iptables --help' for more information.
 (exit status 2)).


🤦  StartHost failed, but will try again: creating host: create: creating: create kic node: create container: docker run -d -t --privileged --security-opt seccomp=unconfined --tmpfs /tmp --tmpfs /run -v /lib/modules:/lib/modules:ro --hostname minikube --name minikube --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=minikube --label role.minikube.sigs.k8s.io= --label mode.minikube.sigs.k8s.io=minikube --network minikube --ip 192.168.49.2 --volume minikube:/var --security-opt apparmor=unconfined --memory=4000mb --memory-swap=4000mb --cpus=2 -e container=docker --expose 8443 --publish=127.0.0.1::8443 --publish=127.0.0.1::22 --publish=127.0.0.1::2376 --publish=127.0.0.1::5000 gcr.io/k8s-minikube/kicbase:v0.0.17@sha256:1cd2e039ec9d418e6380b2fa0280503a72e5b282adea674ee67882f59f4f546e: exit status 125
stdout:
58167a27c16956f55245cc6a188f5647ebdb4599e312df53421a3244521865e6

stderr:
docker: Error response from daemon: driver failed programming external connectivity on endpoint minikube (e876589fb9130cddf8a30a32d4bbca36446fa913b0ffbe6ccad3cafc47b475ce):  (iptables failed: iptables --wait -t nat -A DOCKER -p tcp -d 127.0.0.1 --dport 49302 -j DNAT --to-destination 192.168.49.2:8443 ! -i br-bc41f06feb34: iptables v1.8.7 (legacy): unknown option "--dport"
Try `iptables -h' or 'iptables --help' for more information.
 (exit status 2)).

I0217 14:59:55.401699  173041 start.go:392] Will try again in 5 seconds ...
I0217 14:59:57.043489  173041 cli_runner.go:155] Completed: docker run --rm --entrypoint /usr/bin/tar -v /home/horseinthesky/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v8-v1.20.2-docker-overlay2-amd64.tar.lz4:/preloaded.tar:ro -v minikube:/extractDir gcr.io/k8s-minikube/kicbase:v0.0.17@sha256:1cd2e039ec9d418e6380b2fa0280503a72e5b282adea674ee67882f59f4f546e -I lz4 -xf /preloaded.tar -C /extractDir: (6.518020395s)
I0217 14:59:57.043540  173041 kic.go:172] duration metric: took 6.518291 seconds to extract preloaded images to volume
I0217 15:00:00.401900  173041 start.go:313] acquiring machines lock for minikube: {Name:mk2ca001533c4eb542be71a7f8bf1a970f80797f Clock:{} Delay:500ms Timeout:10m0s Cancel:<nil>}
I0217 15:00:00.402075  173041 start.go:317] acquired machines lock for "minikube" in 101.691µs
I0217 15:00:00.402107  173041 start.go:93] Skipping create...Using existing machine configuration
I0217 15:00:00.402117  173041 fix.go:54] fixHost starting: 
I0217 15:00:00.402473  173041 cli_runner.go:111] Run: docker container inspect minikube --format={{.State.Status}}
I0217 15:00:00.445225  173041 fix.go:107] recreateIfNeeded on minikube: state= err=<nil>
I0217 15:00:00.445267  173041 fix.go:112] machineExists: false. err=machine does not exist
I0217 15:00:00.445445  173041 out.go:119] 🤷  docker "minikube" container is missing, will recreate.
🤷  docker "minikube" container is missing, will recreate.
I0217 15:00:00.445476  173041 delete.go:124] DEMOLISHING minikube ...
I0217 15:00:00.445565  173041 cli_runner.go:111] Run: docker container inspect minikube --format={{.State.Status}}
I0217 15:00:00.501725  173041 stop.go:79] host is in state 
I0217 15:00:00.501801  173041 main.go:119] libmachine: Stopping "minikube"...
I0217 15:00:00.501872  173041 cli_runner.go:111] Run: docker container inspect minikube --format={{.State.Status}}
I0217 15:00:00.553685  173041 kic_runner.go:94] Run: systemctl --version
I0217 15:00:00.553718  173041 kic_runner.go:115] Args: [docker exec --privileged minikube systemctl --version]
I0217 15:00:00.594211  173041 kic_runner.go:94] Run: sudo service kubelet stop
I0217 15:00:00.594243  173041 kic_runner.go:115] Args: [docker exec --privileged minikube sudo service kubelet stop]
I0217 15:00:00.634142  173041 openrc.go:141] stop output: 
** stderr ** 
Error response from daemon: Container 58167a27c16956f55245cc6a188f5647ebdb4599e312df53421a3244521865e6 is not running

** /stderr **
W0217 15:00:00.634180  173041 kic.go:421] couldn't stop kubelet. will continue with stop anyways: sudo service kubelet stop: exit status 1
stdout:

stderr:
Error response from daemon: Container 58167a27c16956f55245cc6a188f5647ebdb4599e312df53421a3244521865e6 is not running
I0217 15:00:00.634254  173041 kic_runner.go:94] Run: sudo service kubelet stop
I0217 15:00:00.634270  173041 kic_runner.go:115] Args: [docker exec --privileged minikube sudo service kubelet stop]
I0217 15:00:00.674588  173041 openrc.go:141] stop output: 
** stderr ** 
Error response from daemon: Container 58167a27c16956f55245cc6a188f5647ebdb4599e312df53421a3244521865e6 is not running

** /stderr **
W0217 15:00:00.674627  173041 kic.go:423] couldn't force stop kubelet. will continue with stop anyways: sudo service kubelet stop: exit status 1
stdout:

stderr:
Error response from daemon: Container 58167a27c16956f55245cc6a188f5647ebdb4599e312df53421a3244521865e6 is not running
I0217 15:00:00.674718  173041 kic_runner.go:94] Run: docker ps -a --filter=name=k8s_.*_(kube-system|kubernetes-dashboard|storage-gluster|istio-operator)_ --format={{.ID}}
I0217 15:00:00.674747  173041 kic_runner.go:115] Args: [docker exec --privileged minikube docker ps -a --filter=name=k8s_.*_(kube-system|kubernetes-dashboard|storage-gluster|istio-operator)_ --format={{.ID}}]
I0217 15:00:00.715175  173041 kic.go:434] unable list containers : docker: docker ps -a --filter=name=k8s_.*_(kube-system|kubernetes-dashboard|storage-gluster|istio-operator)_ --format={{.ID}}: exit status 1
stdout:

stderr:
Error response from daemon: Container 58167a27c16956f55245cc6a188f5647ebdb4599e312df53421a3244521865e6 is not running
I0217 15:00:00.715227  173041 kic.go:444] successfully stopped kubernetes!
I0217 15:00:00.715287  173041 kic_runner.go:94] Run: pgrep kube-apiserver
I0217 15:00:00.715315  173041 kic_runner.go:115] Args: [docker exec --privileged minikube pgrep kube-apiserver]
I0217 15:00:00.793725  173041 cli_runner.go:111] Run: docker container inspect minikube --format={{.State.Status}}
I0217 15:00:03.834406  173041 cli_runner.go:111] Run: docker container inspect minikube --format={{.State.Status}}
I0217 15:00:06.874407  173041 cli_runner.go:111] Run: docker container inspect minikube --format={{.State.Status}}
I0217 15:00:09.914815  173041 cli_runner.go:111] Run: docker container inspect minikube --format={{.State.Status}}
I0217 15:00:12.955527  173041 cli_runner.go:111] Run: docker container inspect minikube --format={{.State.Status}}
I0217 15:00:15.995262  173041 cli_runner.go:111] Run: docker container inspect minikube --format={{.State.Status}}
I0217 15:00:19.035189  173041 cli_runner.go:111] Run: docker container inspect minikube --format={{.State.Status}}
I0217 15:00:22.075558  173041 cli_runner.go:111] Run: docker container inspect minikube --format={{.State.Status}}
I0217 15:00:25.116445  173041 cli_runner.go:111] Run: docker container inspect minikube --format={{.State.Status}}
I0217 15:00:28.156455  173041 cli_runner.go:111] Run: docker container inspect minikube --format={{.State.Status}}
I0217 15:00:31.196240  173041 cli_runner.go:111] Run: docker container inspect minikube --format={{.State.Status}}
I0217 15:00:34.236363  173041 cli_runner.go:111] Run: docker container inspect minikube --format={{.State.Status}}
I0217 15:00:37.274868  173041 cli_runner.go:111] Run: docker container inspect minikube --format={{.State.Status}}
I0217 15:00:40.315407  173041 cli_runner.go:111] Run: docker container inspect minikube --format={{.State.Status}}
I0217 15:00:43.371797  173041 cli_runner.go:111] Run: docker container inspect minikube --format={{.State.Status}}
I0217 15:00:46.412097  173041 cli_runner.go:111] Run: docker container inspect minikube --format={{.State.Status}}
I0217 15:00:49.452928  173041 cli_runner.go:111] Run: docker container inspect minikube --format={{.State.Status}}
I0217 15:00:52.493073  173041 cli_runner.go:111] Run: docker container inspect minikube --format={{.State.Status}}
I0217 15:00:55.533445  173041 cli_runner.go:111] Run: docker container inspect minikube --format={{.State.Status}}
I0217 15:00:58.573242  173041 cli_runner.go:111] Run: docker container inspect minikube --format={{.State.Status}}
I0217 15:01:01.613635  173041 cli_runner.go:111] Run: docker container inspect minikube --format={{.State.Status}}
I0217 15:01:04.654411  173041 cli_runner.go:111] Run: docker container inspect minikube --format={{.State.Status}}
I0217 15:01:07.695231  173041 cli_runner.go:111] Run: docker container inspect minikube --format={{.State.Status}}
I0217 15:01:10.746503  173041 cli_runner.go:111] Run: docker container inspect minikube --format={{.State.Status}}
I0217 15:01:13.786361  173041 cli_runner.go:111] Run: docker container inspect minikube --format={{.State.Status}}
I0217 15:01:16.828397  173041 cli_runner.go:111] Run: docker container inspect minikube --format={{.State.Status}}
I0217 15:01:19.870180  173041 cli_runner.go:111] Run: docker container inspect minikube --format={{.State.Status}}
I0217 15:01:22.912644  173041 cli_runner.go:111] Run: docker container inspect minikube --format={{.State.Status}}
I0217 15:01:25.952490  173041 cli_runner.go:111] Run: docker container inspect minikube --format={{.State.Status}}
I0217 15:01:28.991688  173041 cli_runner.go:111] Run: docker container inspect minikube --format={{.State.Status}}
I0217 15:01:32.031645  173041 cli_runner.go:111] Run: docker container inspect minikube --format={{.State.Status}}
I0217 15:01:35.071038  173041 cli_runner.go:111] Run: docker container inspect minikube --format={{.State.Status}}
I0217 15:01:38.111571  173041 cli_runner.go:111] Run: docker container inspect minikube --format={{.State.Status}}
I0217 15:01:41.152262  173041 cli_runner.go:111] Run: docker container inspect minikube --format={{.State.Status}}
I0217 15:01:44.194381  173041 cli_runner.go:111] Run: docker container inspect minikube --format={{.State.Status}}
I0217 15:01:47.248741  173041 cli_runner.go:111] Run: docker container inspect minikube --format={{.State.Status}}
I0217 15:01:50.289938  173041 cli_runner.go:111] Run: docker container inspect minikube --format={{.State.Status}}
I0217 15:01:53.331604  173041 cli_runner.go:111] Run: docker container inspect minikube --format={{.State.Status}}
I0217 15:01:56.378117  173041 cli_runner.go:111] Run: docker container inspect minikube --format={{.State.Status}}
I0217 15:01:59.421348  173041 cli_runner.go:111] Run: docker container inspect minikube --format={{.State.Status}}
I0217 15:02:02.461417  173041 cli_runner.go:111] Run: docker container inspect minikube --format={{.State.Status}}
I0217 15:02:05.501617  173041 cli_runner.go:111] Run: docker container inspect minikube --format={{.State.Status}}
I0217 15:02:08.543016  173041 cli_runner.go:111] Run: docker container inspect minikube --format={{.State.Status}}
I0217 15:02:11.582703  173041 cli_runner.go:111] Run: docker container inspect minikube --format={{.State.Status}}
I0217 15:02:14.622854  173041 cli_runner.go:111] Run: docker container inspect minikube --format={{.State.Status}}
I0217 15:02:17.664248  173041 cli_runner.go:111] Run: docker container inspect minikube --format={{.State.Status}}
I0217 15:02:20.704768  173041 cli_runner.go:111] Run: docker container inspect minikube --format={{.State.Status}}
I0217 15:02:23.746120  173041 cli_runner.go:111] Run: docker container inspect minikube --format={{.State.Status}}
I0217 15:02:26.804971  173041 cli_runner.go:111] Run: docker container inspect minikube --format={{.State.Status}}
I0217 15:02:29.847676  173041 cli_runner.go:111] Run: docker container inspect minikube --format={{.State.Status}}
I0217 15:02:32.887009  173041 cli_runner.go:111] Run: docker container inspect minikube --format={{.State.Status}}
I0217 15:02:35.926756  173041 cli_runner.go:111] Run: docker container inspect minikube --format={{.State.Status}}
I0217 15:02:38.966807  173041 cli_runner.go:111] Run: docker container inspect minikube --format={{.State.Status}}
I0217 15:02:42.006745  173041 cli_runner.go:111] Run: docker container inspect minikube --format={{.State.Status}}
I0217 15:02:45.045965  173041 cli_runner.go:111] Run: docker container inspect minikube --format={{.State.Status}}
I0217 15:02:48.086524  173041 cli_runner.go:111] Run: docker container inspect minikube --format={{.State.Status}}
I0217 15:02:51.126402  173041 cli_runner.go:111] Run: docker container inspect minikube --format={{.State.Status}}
I0217 15:02:54.165638  173041 cli_runner.go:111] Run: docker container inspect minikube --format={{.State.Status}}
I0217 15:02:57.205385  173041 cli_runner.go:111] Run: docker container inspect minikube --format={{.State.Status}}
I0217 15:03:00.245138  173041 cli_runner.go:111] Run: docker container inspect minikube --format={{.State.Status}}
I0217 15:03:03.286238  173041 stop.go:59] stop err: Maximum number of retries (60) exceeded
I0217 15:03:03.286726  173041 delete.go:129] stophost failed (probably ok): Temporary Error: stop: Maximum number of retries (60) exceeded
I0217 15:03:03.287870  173041 cli_runner.go:111] Run: docker container inspect minikube --format={{.State.Status}}
W0217 15:03:03.329465  173041 delete.go:135] deletehost failed: Docker machine "minikube" does not exist. Use "docker-machine ls" to list machines. Use "docker-machine create" to add a new one.
I0217 15:03:03.329863  173041 cli_runner.go:111] Run: docker container inspect -f {{.Id}} minikube
I0217 15:03:03.372477  173041 cli_runner.go:111] Run: docker container inspect minikube --format={{.State.Status}}
I0217 15:03:03.413333  173041 cli_runner.go:111] Run: docker exec --privileged -t minikube /bin/bash -c "sudo init 0"
W0217 15:03:03.454597  173041 cli_runner.go:149] docker exec --privileged -t minikube /bin/bash -c "sudo init 0" returned with exit code 1
I0217 15:03:03.455000  173041 oci.go:600] error shutdown minikube: docker exec --privileged -t minikube /bin/bash -c "sudo init 0": exit status 1
stdout:

stderr:
Error response from daemon: Container 58167a27c16956f55245cc6a188f5647ebdb4599e312df53421a3244521865e6 is not running
I0217 15:03:04.455629  173041 cli_runner.go:111] Run: docker container inspect minikube --format={{.State.Status}}
I0217 15:03:04.498433  173041 oci.go:614] temporary error: container minikube status is  but expect it to be exited
I0217 15:03:04.498476  173041 oci.go:620] Successfully shutdown container minikube
I0217 15:03:04.498537  173041 cli_runner.go:111] Run: docker rm -f -v minikube
I0217 15:03:04.543153  173041 cli_runner.go:111] Run: docker container inspect -f {{.Id}} minikube
W0217 15:03:04.583517  173041 cli_runner.go:149] docker container inspect -f {{.Id}} minikube returned with exit code 1
I0217 15:03:04.583619  173041 cli_runner.go:111] Run: docker network inspect minikube --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}},{{$first := true}} "ContainerIPs": [{{range $k,$v := .Containers }}{{if $first}}{{$first = false}}{{else}}, {{end}}"{{$v.IPv4Address}}"{{end}}]}"
I0217 15:03:04.623284  173041 cli_runner.go:111] Run: docker network rm minikube
W0217 15:03:05.044075  173041 delete.go:139] delete failed (probably ok) <nil>
I0217 15:03:05.044109  173041 fix.go:119] Sleeping 1 second for extra luck!
I0217 15:03:06.044335  173041 start.go:126] createHost starting for "" (driver="docker")
I0217 15:03:06.044629  173041 out.go:140] 🔥  Creating docker container (CPUs=2, Memory=4000MB) ...
🔥  Creating docker container (CPUs=2, Memory=4000MB) ...I0217 15:03:06.044754  173041 start.go:160] libmachine.API.Create for "minikube" (driver="docker")
| I0217 15:03:06.044795  173041 client.go:168] LocalClient.Create starting
I0217 15:03:06.044887  173041 main.go:119] libmachine: Reading certificate data from /home/horseinthesky/.minikube/certs/ca.pem
I0217 15:03:06.044951  173041 main.go:119] libmachine: Decoding PEM data...
I0217 15:03:06.045009  173041 main.go:119] libmachine: Parsing certificate...
I0217 15:03:06.045192  173041 main.go:119] libmachine: Reading certificate data from /home/horseinthesky/.minikube/certs/cert.pem
I0217 15:03:06.045234  173041 main.go:119] libmachine: Decoding PEM data...
I0217 15:03:06.045257  173041 main.go:119] libmachine: Parsing certificate...
I0217 15:03:06.045574  173041 cli_runner.go:111] Run: docker network inspect minikube --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}},{{$first := true}} "ContainerIPs": [{{range $k,$v := .Containers }}{{if $first}}{{$first = false}}{{else}}, {{end}}"{{$v.IPv4Address}}"{{end}}]}"
W0217 15:03:06.085668  173041 cli_runner.go:149] docker network inspect minikube --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}},{{$first := true}} "ContainerIPs": [{{range $k,$v := .Containers }}{{if $first}}{{$first = false}}{{else}}, {{end}}"{{$v.IPv4Address}}"{{end}}]}" returned with exit code 1
I0217 15:03:06.085805  173041 network_create.go:249] running [docker network inspect minikube] to gather additional debugging logs...
I0217 15:03:06.085852  173041 cli_runner.go:111] Run: docker network inspect minikube
W0217 15:03:06.124133  173041 cli_runner.go:149] docker network inspect minikube returned with exit code 1
I0217 15:03:06.124208  173041 network_create.go:252] error running [docker network inspect minikube]: docker network inspect minikube: exit status 1
stdout:
[]

stderr:
Error: No such network: minikube
I0217 15:03:06.124326  173041 network_create.go:254] output of [docker network inspect minikube]: -- stdout --
[]

-- /stdout --
** stderr ** 
Error: No such network: minikube

** /stderr **
I0217 15:03:06.124528  173041 cli_runner.go:111] Run: docker network inspect bridge --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}},{{$first := true}} "ContainerIPs": [{{range $k,$v := .Containers }}{{if $first}}{{$first = false}}{{else}}, {{end}}"{{$v.IPv4Address}}"{{end}}]}"
/ I0217 15:03:06.163695  173041 network_create.go:104] attempt to create network 192.168.49.0/24 with subnet: minikube and gateway 192.168.49.1 and MTU of 1500 ...
I0217 15:03:06.163835  173041 cli_runner.go:111] Run: docker network create --driver=bridge --subnet=192.168.49.0/24 --gateway=192.168.49.1 -o --ip-masq -o --icc -o com.docker.network.driver.mtu=1500 --label=created_by.minikube.sigs.k8s.io=true minikube
| I0217 15:03:06.535477  173041 kic.go:100] calculated static IP "192.168.49.2" for the "minikube" container
I0217 15:03:06.535567  173041 cli_runner.go:111] Run: docker ps -a --format {{.Names}}
/ I0217 15:03:06.575396  173041 cli_runner.go:111] Run: docker volume create minikube --label name.minikube.sigs.k8s.io=minikube --label created_by.minikube.sigs.k8s.io=true
I0217 15:03:06.615224  173041 oci.go:102] Successfully created a docker volume minikube
I0217 15:03:06.615312  173041 cli_runner.go:111] Run: docker run --rm --entrypoint /usr/bin/test -v minikube:/var gcr.io/k8s-minikube/kicbase:v0.0.17@sha256:1cd2e039ec9d418e6380b2fa0280503a72e5b282adea674ee67882f59f4f546e -d /var/lib
- I0217 15:03:07.115245  173041 oci.go:106] Successfully prepared a docker volume minikube
W0217 15:03:07.115333  173041 oci.go:159] Your kernel does not support swap limit capabilities or the cgroup is not mounted.
I0217 15:03:07.115345  173041 preload.go:97] Checking if preload exists for k8s version v1.20.2 and runtime docker
I0217 15:03:07.115396  173041 preload.go:105] Found local preload: /home/horseinthesky/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v8-v1.20.2-docker-overlay2-amd64.tar.lz4
I0217 15:03:07.115435  173041 cli_runner.go:111] Run: docker info --format "'{{json .SecurityOptions}}'"
I0217 15:03:07.115435  173041 kic.go:163] Starting extracting preloaded images to volume ...
I0217 15:03:07.115530  173041 cli_runner.go:111] Run: docker run --rm --entrypoint /usr/bin/tar -v /home/horseinthesky/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v8-v1.20.2-docker-overlay2-amd64.tar.lz4:/preloaded.tar:ro -v minikube:/extractDir gcr.io/k8s-minikube/kicbase:v0.0.17@sha256:1cd2e039ec9d418e6380b2fa0280503a72e5b282adea674ee67882f59f4f546e -I lz4 -xf /preloaded.tar -C /extractDir
| I0217 15:03:07.260627  173041 cli_runner.go:111] Run: docker run -d -t --privileged --security-opt seccomp=unconfined --tmpfs /tmp --tmpfs /run -v /lib/modules:/lib/modules:ro --hostname minikube --name minikube --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=minikube --label role.minikube.sigs.k8s.io= --label mode.minikube.sigs.k8s.io=minikube --network minikube --ip 192.168.49.2 --volume minikube:/var --security-opt apparmor=unconfined --memory=4000mb --memory-swap=4000mb --cpus=2 -e container=docker --expose 8443 --publish=127.0.0.1::8443 --publish=127.0.0.1::22 --publish=127.0.0.1::2376 --publish=127.0.0.1::5000 gcr.io/k8s-minikube/kicbase:v0.0.17@sha256:1cd2e039ec9d418e6380b2fa0280503a72e5b282adea674ee67882f59f4f546e
- W0217 15:03:07.499307  173041 cli_runner.go:149] docker run -d -t --privileged --security-opt seccomp=unconfined --tmpfs /tmp --tmpfs /run -v /lib/modules:/lib/modules:ro --hostname minikube --name minikube --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=minikube --label role.minikube.sigs.k8s.io= --label mode.minikube.sigs.k8s.io=minikube --network minikube --ip 192.168.49.2 --volume minikube:/var --security-opt apparmor=unconfined --memory=4000mb --memory-swap=4000mb --cpus=2 -e container=docker --expose 8443 --publish=127.0.0.1::8443 --publish=127.0.0.1::22 --publish=127.0.0.1::2376 --publish=127.0.0.1::5000 gcr.io/k8s-minikube/kicbase:v0.0.17@sha256:1cd2e039ec9d418e6380b2fa0280503a72e5b282adea674ee67882f59f4f546e returned with exit code 125
I0217 15:03:07.499418  173041 client.go:171] LocalClient.Create took 1.454598828s
- I0217 15:03:09.499717  173041 ssh_runner.go:149] Run: sh -c "df -h /var | awk 'NR==2{print $5}'"
I0217 15:03:09.499885  173041 cli_runner.go:111] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube
W0217 15:03:09.539844  173041 cli_runner.go:149] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube returned with exit code 1
I0217 15:03:09.540027  173041 retry.go:31] will retry after 231.159374ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "minikube": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube: exit status 1
stdout:


stderr:
Template parsing error: template: :1:4: executing "" at <index (index .NetworkSettings.Ports "22/tcp") 0>: error calling index: index of untyped nil
/ I0217 15:03:09.771580  173041 cli_runner.go:111] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube
W0217 15:03:09.827199  173041 cli_runner.go:149] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube returned with exit code 1
I0217 15:03:09.827315  173041 retry.go:31] will retry after 445.058653ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "minikube": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube: exit status 1
stdout:


stderr:
Template parsing error: template: :1:4: executing "" at <index (index .NetworkSettings.Ports "22/tcp") 0>: error calling index: index of untyped nil
- I0217 15:03:10.272612  173041 cli_runner.go:111] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube
W0217 15:03:10.317987  173041 cli_runner.go:149] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube returned with exit code 1
I0217 15:03:10.318109  173041 retry.go:31] will retry after 318.170823ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "minikube": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube: exit status 1
stdout:


stderr:
Template parsing error: template: :1:4: executing "" at <index (index .NetworkSettings.Ports "22/tcp") 0>: error calling index: index of untyped nil
/ I0217 15:03:10.636512  173041 cli_runner.go:111] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube
- W0217 15:03:10.681184  173041 cli_runner.go:149] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube returned with exit code 1
I0217 15:03:10.681310  173041 retry.go:31] will retry after 553.938121ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "minikube": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube: exit status 1
stdout:


stderr:
Template parsing error: template: :1:4: executing "" at <index (index .NetworkSettings.Ports "22/tcp") 0>: error calling index: index of untyped nil
\ I0217 15:03:11.235405  173041 cli_runner.go:111] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube
| W0217 15:03:11.281837  173041 cli_runner.go:149] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube returned with exit code 1
I0217 15:03:11.281953  173041 retry.go:31] will retry after 755.539547ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "minikube": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube: exit status 1
stdout:


stderr:
Template parsing error: template: :1:4: executing "" at <index (index .NetworkSettings.Ports "22/tcp") 0>: error calling index: index of untyped nil
\ I0217 15:03:12.037786  173041 cli_runner.go:111] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube
| W0217 15:03:12.078919  173041 cli_runner.go:149] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube returned with exit code 1
W0217 15:03:12.079049  173041 start.go:257] error running df -h /var: NewSession: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "minikube": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube: exit status 1
stdout:


stderr:
Template parsing error: template: :1:4: executing "" at <index (index .NetworkSettings.Ports "22/tcp") 0>: error calling index: index of untyped nil

W0217 15:03:12.079078  173041 start.go:239] error getting percentage of /var that is free: NewSession: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "minikube": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube: exit status 1
stdout:


stderr:
Template parsing error: template: :1:4: executing "" at <index (index .NetworkSettings.Ports "22/tcp") 0>: error calling index: index of untyped nil
I0217 15:03:12.079094  173041 start.go:129] duration metric: createHost completed in 6.034706s
I0217 15:03:12.079154  173041 ssh_runner.go:149] Run: sh -c "df -h /var | awk 'NR==2{print $5}'"
I0217 15:03:12.079203  173041 cli_runner.go:111] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube
W0217 15:03:12.119648  173041 cli_runner.go:149] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube returned with exit code 1
I0217 15:03:12.120168  173041 retry.go:31] will retry after 200.227965ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "minikube": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube: exit status 1
stdout:


stderr:
Template parsing error: template: :1:4: executing "" at <index (index .NetworkSettings.Ports "22/tcp") 0>: error calling index: index of untyped nil
- I0217 15:03:12.321365  173041 cli_runner.go:111] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube
\ W0217 15:03:12.361446  173041 cli_runner.go:149] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube returned with exit code 1
I0217 15:03:12.361569  173041 retry.go:31] will retry after 380.704736ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "minikube": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube: exit status 1
stdout:


stderr:
Template parsing error: template: :1:4: executing "" at <index (index .NetworkSettings.Ports "22/tcp") 0>: error calling index: index of untyped nil
- I0217 15:03:12.742542  173041 cli_runner.go:111] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube
\ W0217 15:03:12.782064  173041 cli_runner.go:149] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube returned with exit code 1
I0217 15:03:12.782183  173041 retry.go:31] will retry after 738.922478ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "minikube": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube: exit status 1
stdout:


stderr:
Template parsing error: template: :1:4: executing "" at <index (index .NetworkSettings.Ports "22/tcp") 0>: error calling index: index of untyped nil
- I0217 15:03:13.521284  173041 cli_runner.go:111] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube
\ W0217 15:03:13.562957  173041 cli_runner.go:149] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube returned with exit code 1
I0217 15:03:13.563066  173041 retry.go:31] will retry after 602.660142ms: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "minikube": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube: exit status 1
stdout:


stderr:
Template parsing error: template: :1:4: executing "" at <index (index .NetworkSettings.Ports "22/tcp") 0>: error calling index: index of untyped nil
- I0217 15:03:13.939011  173041 cli_runner.go:155] Completed: docker run --rm --entrypoint /usr/bin/tar -v /home/horseinthesky/.minikube/cache/preloaded-tarball/preloaded-images-k8s-v8-v1.20.2-docker-overlay2-amd64.tar.lz4:/preloaded.tar:ro -v minikube:/extractDir gcr.io/k8s-minikube/kicbase:v0.0.17@sha256:1cd2e039ec9d418e6380b2fa0280503a72e5b282adea674ee67882f59f4f546e -I lz4 -xf /preloaded.tar -C /extractDir: (6.823411306s)
I0217 15:03:13.939064  173041 kic.go:172] duration metric: took 6.823629 seconds to extract preloaded images to volume
/ I0217 15:03:14.166067  173041 cli_runner.go:111] Run: docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube
W0217 15:03:14.207482  173041 cli_runner.go:149] docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube returned with exit code 1
W0217 15:03:14.207609  173041 start.go:257] error running df -h /var: NewSession: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "minikube": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube: exit status 1
stdout:


stderr:
Template parsing error: template: :1:4: executing "" at <index (index .NetworkSettings.Ports "22/tcp") 0>: error calling index: index of untyped nil

W0217 15:03:14.207656  173041 start.go:239] error getting percentage of /var that is free: NewSession: new client: new client: Error creating new ssh host from driver: Error getting ssh port for driver: get ssh host-port: get port 22 for "minikube": docker container inspect -f "'{{(index (index .NetworkSettings.Ports "22/tcp") 0).HostPort}}'" minikube: exit status 1
stdout:


stderr:
Template parsing error: template: :1:4: executing "" at <index (index .NetworkSettings.Ports "22/tcp") 0>: error calling index: index of untyped nil
I0217 15:03:14.207676  173041 fix.go:56] fixHost completed within 3m13.805558908s
I0217 15:03:14.207690  173041 start.go:80] releasing machines lock for "minikube", held for 3m13.805596118s
W0217 15:03:14.207911  173041 out.go:181] 😿  Failed to start docker container. Running "minikube delete" may fix it: recreate: creating host: create: creating: create kic node: create container: docker run -d -t --privileged --security-opt seccomp=unconfined --tmpfs /tmp --tmpfs /run -v /lib/modules:/lib/modules:ro --hostname minikube --name minikube --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=minikube --label role.minikube.sigs.k8s.io= --label mode.minikube.sigs.k8s.io=minikube --network minikube --ip 192.168.49.2 --volume minikube:/var --security-opt apparmor=unconfined --memory=4000mb --memory-swap=4000mb --cpus=2 -e container=docker --expose 8443 --publish=127.0.0.1::8443 --publish=127.0.0.1::22 --publish=127.0.0.1::2376 --publish=127.0.0.1::5000 gcr.io/k8s-minikube/kicbase:v0.0.17@sha256:1cd2e039ec9d418e6380b2fa0280503a72e5b282adea674ee67882f59f4f546e: exit status 125
stdout:
caf04d301c5521fe4de010849cd004a287c2f40e9d932f1de5b00100dd4a6d2b

stderr:
docker: Error response from daemon: driver failed programming external connectivity on endpoint minikube (d96f528b68412fdbf86f029c47c9ba9e94cd8112c77cd5338a0417b1a9a044d4):  (iptables failed: iptables --wait -t nat -A DOCKER -p tcp -d 127.0.0.1 --dport 49312 -j DNAT --to-destination 192.168.49.2:8443 ! -i br-d96ed2151c1c: iptables v1.8.7 (legacy): unknown option "--dport"
Try `iptables -h' or 'iptables --help' for more information.
 (exit status 2)).


😿  Failed to start docker container. Running "minikube delete" may fix it: recreate: creating host: create: creating: create kic node: create container: docker run -d -t --privileged --security-opt seccomp=unconfined --tmpfs /tmp --tmpfs /run -v /lib/modules:/lib/modules:ro --hostname minikube --name minikube --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=minikube --label role.minikube.sigs.k8s.io= --label mode.minikube.sigs.k8s.io=minikube --network minikube --ip 192.168.49.2 --volume minikube:/var --security-opt apparmor=unconfined --memory=4000mb --memory-swap=4000mb --cpus=2 -e container=docker --expose 8443 --publish=127.0.0.1::8443 --publish=127.0.0.1::22 --publish=127.0.0.1::2376 --publish=127.0.0.1::5000 gcr.io/k8s-minikube/kicbase:v0.0.17@sha256:1cd2e039ec9d418e6380b2fa0280503a72e5b282adea674ee67882f59f4f546e: exit status 125
stdout:
caf04d301c5521fe4de010849cd004a287c2f40e9d932f1de5b00100dd4a6d2b

stderr:
docker: Error response from daemon: driver failed programming external connectivity on endpoint minikube (d96f528b68412fdbf86f029c47c9ba9e94cd8112c77cd5338a0417b1a9a044d4):  (iptables failed: iptables --wait -t nat -A DOCKER -p tcp -d 127.0.0.1 --dport 49312 -j DNAT --to-destination 192.168.49.2:8443 ! -i br-d96ed2151c1c: iptables v1.8.7 (legacy): unknown option "--dport"
Try `iptables -h' or 'iptables --help' for more information.
 (exit status 2)).

W0217 15:03:14.208102  173041 out.go:181] ❗  Startup with docker driver failed, trying with alternate driver none: Failed to start host: recreate: creating host: create: creating: create kic node: create container: docker run -d -t --privileged --security-opt seccomp=unconfined --tmpfs /tmp --tmpfs /run -v /lib/modules:/lib/modules:ro --hostname minikube --name minikube --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=minikube --label role.minikube.sigs.k8s.io= --label mode.minikube.sigs.k8s.io=minikube --network minikube --ip 192.168.49.2 --volume minikube:/var --security-opt apparmor=unconfined --memory=4000mb --memory-swap=4000mb --cpus=2 -e container=docker --expose 8443 --publish=127.0.0.1::8443 --publish=127.0.0.1::22 --publish=127.0.0.1::2376 --publish=127.0.0.1::5000 gcr.io/k8s-minikube/kicbase:v0.0.17@sha256:1cd2e039ec9d418e6380b2fa0280503a72e5b282adea674ee67882f59f4f546e: exit status 125
stdout:
caf04d301c5521fe4de010849cd004a287c2f40e9d932f1de5b00100dd4a6d2b

stderr:
docker: Error response from daemon: driver failed programming external connectivity on endpoint minikube (d96f528b68412fdbf86f029c47c9ba9e94cd8112c77cd5338a0417b1a9a044d4):  (iptables failed: iptables --wait -t nat -A DOCKER -p tcp -d 127.0.0.1 --dport 49312 -j DNAT --to-destination 192.168.49.2:8443 ! -i br-d96ed2151c1c: iptables v1.8.7 (legacy): unknown option "--dport"
Try `iptables -h' or 'iptables --help' for more information.
 (exit status 2)).

❗  Startup with docker driver failed, trying with alternate driver none: Failed to start host: recreate: creating host: create: creating: create kic node: create container: docker run -d -t --privileged --security-opt seccomp=unconfined --tmpfs /tmp --tmpfs /run -v /lib/modules:/lib/modules:ro --hostname minikube --name minikube --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=minikube --label role.minikube.sigs.k8s.io= --label mode.minikube.sigs.k8s.io=minikube --network minikube --ip 192.168.49.2 --volume minikube:/var --security-opt apparmor=unconfined --memory=4000mb --memory-swap=4000mb --cpus=2 -e container=docker --expose 8443 --publish=127.0.0.1::8443 --publish=127.0.0.1::22 --publish=127.0.0.1::2376 --publish=127.0.0.1::5000 gcr.io/k8s-minikube/kicbase:v0.0.17@sha256:1cd2e039ec9d418e6380b2fa0280503a72e5b282adea674ee67882f59f4f546e: exit status 125
stdout:
caf04d301c5521fe4de010849cd004a287c2f40e9d932f1de5b00100dd4a6d2b

stderr:
docker: Error response from daemon: driver failed programming external connectivity on endpoint minikube (d96f528b68412fdbf86f029c47c9ba9e94cd8112c77cd5338a0417b1a9a044d4):  (iptables failed: iptables --wait -t nat -A DOCKER -p tcp -d 127.0.0.1 --dport 49312 -j DNAT --to-destination 192.168.49.2:8443 ! -i br-d96ed2151c1c: iptables v1.8.7 (legacy): unknown option "--dport"
Try `iptables -h' or 'iptables --help' for more information.
 (exit status 2)).

I0217 15:03:14.208310  173041 delete.go:280] Deleting minikube
I0217 15:03:14.208348  173041 delete.go:285] minikube configuration: &{Name:minikube KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase:v0.0.17@sha256:1cd2e039ec9d418e6380b2fa0280503a72e5b282adea674ee67882f59f4f546e Memory:4000 CPUs:2 DiskSize:20000 VMDriver: Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.99.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.20.2 ClusterName:minikube Namespace:default APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI: NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP: Port:8443 KubernetesVersion:v1.20.2 ControlPlane:true Worker:true}] Addons:map[] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] Network: MultiNodeRequested:false}
W0217 15:03:14.208490  173041 register.go:127] "Deleting" was not found within the registered steps for "Initial Minikube Setup": [Initial Minikube Setup Selecting Driver Downloading Artifacts Starting Node Pulling Base Image Running on Localhost Local OS Release Creating Container Creating VM Running Remotely Preparing Kubernetes Generating certificates Booting control plane Configuring RBAC rules Configuring CNI Configuring Localhost Environment Verifying Kubernetes Enabling Addons Done]
I0217 15:03:14.208558  173041 out.go:119] 🔥  Deleting "minikube" in docker ...
🔥  Deleting "minikube" in docker ...
I0217 15:03:14.208599  173041 delete.go:243] deleting possible KIC leftovers for minikube (driver=docker) ...
I0217 15:03:14.208635  173041 cli_runner.go:111] Run: docker ps -a --filter label=name.minikube.sigs.k8s.io=minikube --format {{.Names}}
W0217 15:03:14.249175  173041 register.go:127] "Deleting" was not found within the registered steps for "Initial Minikube Setup": [Initial Minikube Setup Selecting Driver Downloading Artifacts Starting Node Pulling Base Image Running on Localhost Local OS Release Creating Container Creating VM Running Remotely Preparing Kubernetes Generating certificates Booting control plane Configuring RBAC rules Configuring CNI Configuring Localhost Environment Verifying Kubernetes Enabling Addons Done]
I0217 15:03:14.249256  173041 out.go:119] 🔥  Deleting container "minikube" ...
🔥  Deleting container "minikube" ...
I0217 15:03:14.249316  173041 cli_runner.go:111] Run: docker container inspect minikube --format={{.State.Status}}
I0217 15:03:14.289298  173041 cli_runner.go:111] Run: docker exec --privileged -t minikube /bin/bash -c "sudo init 0"
W0217 15:03:14.329566  173041 cli_runner.go:149] docker exec --privileged -t minikube /bin/bash -c "sudo init 0" returned with exit code 1
I0217 15:03:14.329616  173041 oci.go:600] error shutdown minikube: docker exec --privileged -t minikube /bin/bash -c "sudo init 0": exit status 1
stdout:

stderr:
Error response from daemon: Container caf04d301c5521fe4de010849cd004a287c2f40e9d932f1de5b00100dd4a6d2b is not running
I0217 15:03:15.329917  173041 cli_runner.go:111] Run: docker container inspect minikube --format={{.State.Status}}
I0217 15:03:15.371200  173041 oci.go:614] temporary error: container minikube status is  but expect it to be exited
I0217 15:03:15.371242  173041 oci.go:620] Successfully shutdown container minikube
I0217 15:03:15.371334  173041 cli_runner.go:111] Run: docker rm -f -v minikube
I0217 15:03:15.415176  173041 volumes.go:36] trying to delete all docker volumes with label name.minikube.sigs.k8s.io=minikube
I0217 15:03:15.415269  173041 cli_runner.go:111] Run: docker volume ls --filter label=name.minikube.sigs.k8s.io=minikube --format {{.Name}}
I0217 15:03:15.453499  173041 cli_runner.go:111] Run: docker volume rm --force minikube
I0217 15:03:15.904972  173041 cli_runner.go:111] Run: docker network ls --filter=label=created_by.minikube.sigs.k8s.io --format {{.Name}}
I0217 15:03:15.946555  173041 cli_runner.go:111] Run: docker network inspect minikube --format "{"Name": "{{.Name}}","Driver": "{{.Driver}}","Subnet": "{{range .IPAM.Config}}{{.Subnet}}{{end}}","Gateway": "{{range .IPAM.Config}}{{.Gateway}}{{end}}","MTU": {{if (index .Options "com.docker.network.driver.mtu")}}{{(index .Options "com.docker.network.driver.mtu")}}{{else}}0{{end}},{{$first := true}} "ContainerIPs": [{{range $k,$v := .Containers }}{{if $first}}{{$first = false}}{{else}}, {{end}}"{{$v.IPv4Address}}"{{end}}]}"
I0217 15:03:15.989504  173041 cli_runner.go:111] Run: docker network rm minikube
I0217 15:03:16.392115  173041 volumes.go:58] trying to prune all docker volumes with label name.minikube.sigs.k8s.io=minikube
I0217 15:03:16.392200  173041 cli_runner.go:111] Run: docker volume prune -f --filter label=name.minikube.sigs.k8s.io=minikube
I0217 15:03:16.432562  173041 cli_runner.go:111] Run: docker container inspect minikube --format={{.State.Status}}
W0217 15:03:16.471287  173041 cli_runner.go:149] docker container inspect minikube --format={{.State.Status}} returned with exit code 1
I0217 15:03:16.471373  173041 delete.go:82] Unable to get host status for minikube, assuming it has already been deleted: state: unknown state "minikube": docker container inspect minikube --format={{.State.Status}}: exit status 1
stdout:


stderr:
Error: No such container: minikube
W0217 15:03:16.471564  173041 register.go:127] "Deleting" was not found within the registered steps for "Initial Minikube Setup": [Initial Minikube Setup Selecting Driver Downloading Artifacts Starting Node Pulling Base Image Running on Localhost Local OS Release Creating Container Creating VM Running Remotely Preparing Kubernetes Generating certificates Booting control plane Configuring RBAC rules Configuring CNI Configuring Localhost Environment Verifying Kubernetes Enabling Addons Done]
I0217 15:03:16.471640  173041 out.go:119] 🔥  Removing /home/horseinthesky/.minikube/machines/minikube ...
🔥  Removing /home/horseinthesky/.minikube/machines/minikube ...
W0217 15:03:16.472226  173041 register.go:127] "Deleting" was not found within the registered steps for "Initial Minikube Setup": [Initial Minikube Setup Selecting Driver Downloading Artifacts Starting Node Pulling Base Image Running on Localhost Local OS Release Creating Container Creating VM Running Remotely Preparing Kubernetes Generating certificates Booting control plane Configuring RBAC rules Configuring CNI Configuring Localhost Environment Verifying Kubernetes Enabling Addons Done]
I0217 15:03:16.472318  173041 out.go:119] 💀  Removed all traces of the "minikube" cluster.
💀  Removed all traces of the "minikube" cluster.
I0217 15:03:16.472337  173041 start.go:279] selected driver: none
I0217 15:03:16.472346  173041 start.go:702] validating driver "none" against <nil>
I0217 15:03:16.472365  173041 start.go:713] status for none: {Installed:true Healthy:true Running:false NeedsImprovement:false Error:<nil> Reason: Fix: Doc:}
I0217 15:03:16.472392  173041 start.go:1217] auto setting extra-config to "kubelet.resolv-conf=/run/systemd/resolve/resolv.conf".
I0217 15:03:16.472670  173041 out.go:119] 

W0217 15:03:16.472758  173041 out.go:181] ❌  Exiting due to GUEST_MISSING_CONNTRACK: Sorry, Kubernetes 1.20.2 requires conntrack to be installed in root's path
❌  Exiting due to GUEST_MISSING_CONNTRACK: Sorry, Kubernetes 1.20.2 requires conntrack to be installed in root's path
I0217 15:03:16.472779  173041 out.go:119]

Full output of minikube start command used, if not already included:

minikube start 
😄  minikube v1.17.1 on Arch 
✨  Automatically selected the docker driver. Other choices: none, ssh, podman (experimental)
👍  Starting control plane node minikube in cluster minikube
🔥  Creating docker container (CPUs=2, Memory=4000MB) ...
🤦  StartHost failed, but will try again: creating host: create: creating: create kic node: create container: docker run -d -t --privileged --security-opt seccomp=unconfined --tmpfs /tmp --tmpfs /run -v /lib/modules:/lib/modules:ro --hostname minikube --name minikube --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=minikube --label role.minikube.sigs.k8s.io= --label mode.minikube.sigs.k8s.io=minikube --network minikube --ip 192.168.49.2 --volume minikube:/var --security-opt apparmor=unconfined --memory=4000mb --memory-swap=4000mb --cpus=2 -e container=docker --expose 8443 --publish=127.0.0.1::8443 --publish=127.0.0.1::22 --publish=127.0.0.1::2376 --publish=127.0.0.1::5000 gcr.io/k8s-minikube/kicbase:v0.0.17@sha256:1cd2e039ec9d418e6380b2fa0280503a72e5b282adea674ee67882f59f4f546e: exit status 125
stdout:
1d2fa1aac1c3ae98a40e1f95bae86990c321b4fabcf76471ddb3d8f41047e971

stderr:
docker: Error response from daemon: driver failed programming external connectivity on endpoint minikube (6e59970b4936b96c63368f396a619bc227dd3d22b52ee7f26c4ea437db815558):  (iptables failed: iptables --wait -t nat -A DOCKER -p tcp -d 127.0.0.1 --dport 49282 -j DNAT --to-destination 192.168.49.2:8443 ! -i br-c463e713e81d: iptables v1.8.7 (legacy): unknown option "--dport"
Try `iptables -h' or 'iptables --help' for more information.
 (exit status 2)).

🤷  docker "minikube" container is missing, will recreate.
🔥  Creating docker container (CPUs=2, Memory=4000MB) ...
😿  Failed to start docker container. Running "minikube delete" may fix it: recreate: creating host: create: creating: create kic node: create container: docker run -d -t --privileged --security-opt seccomp=unconfined --tmpfs /tmp --tmpfs /run -v /lib/modules:/lib/modules:ro --hostname minikube --name minikube --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=minikube --label role.minikube.sigs.k8s.io= --label mode.minikube.sigs.k8s.io=minikube --network minikube --ip 192.168.49.2 --volume minikube:/var --security-opt apparmor=unconfined --memory=4000mb --memory-swap=4000mb --cpus=2 -e container=docker --expose 8443 --publish=127.0.0.1::8443 --publish=127.0.0.1::22 --publish=127.0.0.1::2376 --publish=127.0.0.1::5000 gcr.io/k8s-minikube/kicbase:v0.0.17@sha256:1cd2e039ec9d418e6380b2fa0280503a72e5b282adea674ee67882f59f4f546e: exit status 125
stdout:
51464ced5bdf62461294de70babbb5cc53d9152f1f458eb893ba2279d3e31d92

stderr:
docker: Error response from daemon: driver failed programming external connectivity on endpoint minikube (3f250bce02fb563b82df947bc1dde06c7d67fbd1f2777db4dc24c652cbc27aa6):  (iptables failed: iptables --wait -t nat -A DOCKER -p tcp -d 127.0.0.1 --dport 49292 -j DNAT --to-destination 192.168.49.2:8443 ! -i br-3af8f34ac329: iptables v1.8.7 (legacy): unknown option "--dport"
Try `iptables -h' or 'iptables --help' for more information.
 (exit status 2)).

❗  Startup with docker driver failed, trying with alternate driver none: Failed to start host: recreate: creating host: create: creating: create kic node: create container: docker run -d -t --privileged --security-opt seccomp=unconfined --tmpfs /tmp --tmpfs /run -v /lib/modules:/lib/modules:ro --hostname minikube --name minikube --label created_by.minikube.sigs.k8s.io=true --label name.minikube.sigs.k8s.io=minikube --label role.minikube.sigs.k8s.io= --label mode.minikube.sigs.k8s.io=minikube --network minikube --ip 192.168.49.2 --volume minikube:/var --security-opt apparmor=unconfined --memory=4000mb --memory-swap=4000mb --cpus=2 -e container=docker --expose 8443 --publish=127.0.0.1::8443 --publish=127.0.0.1::22 --publish=127.0.0.1::2376 --publish=127.0.0.1::5000 gcr.io/k8s-minikube/kicbase:v0.0.17@sha256:1cd2e039ec9d418e6380b2fa0280503a72e5b282adea674ee67882f59f4f546e: exit status 125
stdout:
51464ced5bdf62461294de70babbb5cc53d9152f1f458eb893ba2279d3e31d92

stderr:
docker: Error response from daemon: driver failed programming external connectivity on endpoint minikube (3f250bce02fb563b82df947bc1dde06c7d67fbd1f2777db4dc24c652cbc27aa6):  (iptables failed: iptables --wait -t nat -A DOCKER -p tcp -d 127.0.0.1 --dport 49292 -j DNAT --to-destination 192.168.49.2:8443 ! -i br-3af8f34ac329: iptables v1.8.7 (legacy): unknown option "--dport"
Try `iptables -h' or 'iptables --help' for more information.
 (exit status 2)).

🔥  Deleting "minikube" in docker ...
🔥  Deleting container "minikube" ...
🔥  Removing /home/horseinthesky/.minikube/machines/minikube ...
💀  Removed all traces of the "minikube" cluster.

❌  Exiting due to GUEST_MISSING_CONNTRACK: Sorry, Kubernetes 1.20.2 requires conntrack to be installed in root's path

Optional: Full output of minikube logs command:
``
minikube logs
🤷 Profile "minikube" not found. Run "minikube profile list" to view all profiles.
👉 To start a cluster, run: "minikube start"
@horseinthesky horseinthesky mentioned this issue Feb 18, 2021
Closed
@afbjorklund
Copy link
Collaborator

iptables failed: iptables --wait -t nat -A DOCKER -p tcp -d 127.0.0.1 --dport 49282 -j DNAT --to-destination 192.168.49.2:8443 ! -i br-c463e713e81d: : iptables v1.8.7 (legacy): unknown option "--dport"

Hmm, that is supposed to be a standard tcp option, see iptables-extensions(8)

  tcp
       These extensions can be used if `--protocol tcp' is specified. It provides the following options:

       [!] --source-port,--sport port[:port]
              Source  port or port range specification. This can either be a service name or a port number. An inclusive range can also be specified, using the format first:last.  If the first port
              is omitted, "0" is assumed; if the last is omitted, "65535" is assumed.  The flag --sport is a convenient alias for this option.

       [!] --destination-port,--dport port[:port]
              Destination port or port range specification.  The flag --dport is a convenient alias for this option.

Do running other docker containers work OK ?

@afbjorklund afbjorklund added co/docker-driver Issues related to kubernetes in container os/linux kind/support Categorizes issue or PR as a support question. labels Feb 19, 2021
@horseinthesky
Copy link
Author

@afbjorklund I'm sorry for bothering. That was an Arch issue: it's updated a kernel and unmounted old kernel modules. Reboot fixed it.

@dreamsmasher
Copy link

Just faced the same issue, rebooting solved it. Way too often, I update then try to do work that requires unmounted kernel modules hahaha.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
co/docker-driver Issues related to kubernetes in container kind/support Categorizes issue or PR as a support question. os/linux
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@afbjorklund @horseinthesky @dreamsmasher