Use the appropriate OCI for deletePossibleKicLeftOver when known #8038

tstromberg · 2020-05-07T22:23:31Z

Partially resolves #7958

Important changes:

deletePossibleKicLeftOver is now only called once
deletePossibleKicLeftOver is now called inside DeleteProfiles if a profile is known, and after DeleteProfiles if it's orphaned
deletePossibleKicLeftOver now takes a driver name. If unknown, Docker is passed in.

Other changes:

deleteContainersAndVolumes only runs docker commands if docker is found
extra logging added

Old Behavior

Deleting a known Docker profile:

I0507 15:27:06.349053   24786 cli_runner.go:108] Run: docker ps -a --filter label=name.minikube.sigs.k8s.io --format {{.Names}}
I0507 15:27:06.416720   24786 cli_runner.go:108] Run: docker ps -a --filter label=name.minikube.sigs.k8s.io=minikube --format {{.Names}}
🔥  Deleting container "minikube" ...
I0507 15:27:06.461363   24786 cli_runner.go:108] Run: docker inspect minikube --format={{.State.Status}}
I0507 15:27:06.503719   24786 cli_runner.go:108] Run: docker exec --privileged -t minikube /bin/bash -c "sudo init 0"
I0507 15:27:07.800099   24786 cli_runner.go:108] Run: docker inspect minikube --format={{.State.Status}}
I0507 15:27:07.838059   24786 oci.go:519] temporary error: container minikube status is Running but expect it to be exited
I0507 15:27:07.838109   24786 oci.go:525] Successfully shutdown container minikube
I0507 15:27:07.838230   24786 cli_runner.go:108] Run: docker rm -f -v minikube
I0507 15:27:08.213716   24786 volumes.go:34] trying to delete all docker volumes with label name.minikube.sigs.k8s.io=minikube
I0507 15:27:08.213876   24786 cli_runner.go:108] Run: docker volume ls --filter label=name.minikube.sigs.k8s.io=minikube --format {{.Name}}
I0507 15:27:08.247632   24786 cli_runner.go:108] Run: docker volume rm --force minikube
I0507 15:27:08.655286   24786 volumes.go:56] trying to prune all docker volumes with label name.minikube.sigs.k8s.io=minikube
I0507 15:27:08.655443   24786 cli_runner.go:108] Run: docker volume prune -f --filter label=name.minikube.sigs.k8s.io=minikube
I0507 15:27:08.688318   24786 cli_runner.go:108] Run: podman ps -a --filter label=name.minikube.sigs.k8s.io=minikube --format {{.Names}}
I0507 15:27:08.688363   24786 volumes.go:34] trying to delete all podman volumes with label name.minikube.sigs.k8s.io=minikube
I0507 15:27:08.688439   24786 cli_runner.go:108] Run: podman volume ls --filter label=name.minikube.sigs.k8s.io=minikube --format {{.Name}}
W0507 15:27:08.688457   24786 delete.go:211] error deleting volumes (might be okay).
To see the list of volumes run: 'docker volume ls'
:[listing volumes by label "name.minikube.sigs.k8s.io=minikube": podman volume ls --filter label=name.minikube.sigs.k8s.io=minikube --format {{.Name}}: exec: "podman": executable file not found in $PATH
stdout:

stderr:
]
I0507 15:27:08.689139   24786 volumes.go:56] trying to prune all podman volumes with label name.minikube.sigs.k8s.io=minikube
I0507 15:27:08.689249   24786 cli_runner.go:108] Run: podman volume prune -f --filter label=name.minikube.sigs.k8s.io=minikube
W0507 15:27:08.689277   24786 delete.go:216] error pruning volume (might be okay):
[prune volume by label name.minikube.sigs.k8s.io=minikube: podman volume prune -f --filter label=name.minikube.sigs.k8s.io=minikube: exec: "podman": executable file not found in $PATH
stdout:

stderr:
]
🔥  Deleting "minikube" in docker ...
I0507 15:27:08.712940   24786 cli_runner.go:108] Run: docker ps -a --filter label=name.minikube.sigs.k8s.io=minikube --format {{.Names}}
I0507 15:27:08.749170   24786 volumes.go:34] trying to delete all docker volumes with label name.minikube.sigs.k8s.io=minikube
I0507 15:27:08.749329   24786 cli_runner.go:108] Run: docker volume ls --filter label=name.minikube.sigs.k8s.io=minikube --format {{.Name}}
I0507 15:27:08.781812   24786 volumes.go:56] trying to prune all docker volumes with label name.minikube.sigs.k8s.io=minikube
I0507 15:27:08.781945   24786 cli_runner.go:108] Run: docker volume prune -f --filter label=name.minikube.sigs.k8s.io=minikube
I0507 15:27:08.815468   24786 cli_runner.go:108] Run: podman ps -a --filter label=name.minikube.sigs.k8s.io=minikube --format {{.Names}}
I0507 15:27:08.815497   24786 volumes.go:34] trying to delete all podman volumes with label name.minikube.sigs.k8s.io=minikube
I0507 15:27:08.815562   24786 cli_runner.go:108] Run: podman volume ls --filter label=name.minikube.sigs.k8s.io=minikube --format {{.Name}}
W0507 15:27:08.815581   24786 delete.go:211] error deleting volumes (might be okay).
To see the list of volumes run: 'docker volume ls'
:[listing volumes by label "name.minikube.sigs.k8s.io=minikube": podman volume ls --filter label=name.minikube.sigs.k8s.io=minikube --format {{.Name}}: exec: "podman": executable file not found in $PATH
stdout:

stderr:
]
I0507 15:27:08.815593   24786 volumes.go:56] trying to prune all podman volumes with label name.minikube.sigs.k8s.io=minikube
I0507 15:27:08.815663   24786 cli_runner.go:108] Run: podman volume prune -f --filter label=name.minikube.sigs.k8s.io=minikube
W0507 15:27:08.815682   24786 delete.go:216] error pruning volume (might be okay):
[prune volume by label name.minikube.sigs.k8s.io=minikube: podman volume prune -f --filter label=name.minikube.sigs.k8s.io=minikube: exec: "podman": executable file not found in $PATH
stdout:

stderr:
]
I0507 15:27:08.816516   24786 cli_runner.go:108] Run: docker inspect minikube --format={{.State.Status}}
I0507 15:27:08.869693   24786 delete.go:75] Unable to get host status for minikube, assuming it has already been deleted: state: unknown state "minikube": docker inspect minikube --format={{.State.Status}}: exit status 1
stdout:


stderr:
Error: No such object: minikube
🔥  Removing /Users/tstromberg/.minikube/machines/minikube ...
I0507 15:27:08.891006   24786 lock.go:35] WriteFile acquiring /Users/tstromberg/.kube/config: {Name:mke11dbcd579c84d4d098461d9cff60a03fa7a38 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
💀  Removed all traces of the "minikube" cluster.

Deleting a non-existent profile:

I0507 15:27:28.695001   24859 cli_runner.go:108] Run: docker ps -a --filter label=name.minikube.sigs.k8s.io --format {{.Names}}
🙄  "minikube" profile does not exist, trying anyways.
I0507 15:27:28.727648   24859 cli_runner.go:108] Run: docker ps -a --filter label=name.minikube.sigs.k8s.io=minikube --format {{.Names}}
I0507 15:27:28.758055   24859 volumes.go:34] trying to delete all docker volumes with label name.minikube.sigs.k8s.io=minikube
I0507 15:27:28.758224   24859 cli_runner.go:108] Run: docker volume ls --filter label=name.minikube.sigs.k8s.io=minikube --format {{.Name}}
I0507 15:27:28.790737   24859 volumes.go:56] trying to prune all docker volumes with label name.minikube.sigs.k8s.io=minikube
I0507 15:27:28.790897   24859 cli_runner.go:108] Run: docker volume prune -f --filter label=name.minikube.sigs.k8s.io=minikube
I0507 15:27:28.823814   24859 cli_runner.go:108] Run: podman ps -a --filter label=name.minikube.sigs.k8s.io=minikube --format {{.Names}}
I0507 15:27:28.824224   24859 volumes.go:34] trying to delete all podman volumes with label name.minikube.sigs.k8s.io=minikube
I0507 15:27:28.824348   24859 cli_runner.go:108] Run: podman volume ls --filter label=name.minikube.sigs.k8s.io=minikube --format {{.Name}}
W0507 15:27:28.824371   24859 delete.go:211] error deleting volumes (might be okay).
To see the list of volumes run: 'docker volume ls'
:[listing volumes by label "name.minikube.sigs.k8s.io=minikube": podman volume ls --filter label=name.minikube.sigs.k8s.io=minikube --format {{.Name}}: exec: "podman": executable file not found in $PATH
stdout:

stderr:
]
I0507 15:27:28.825341   24859 volumes.go:56] trying to prune all podman volumes with label name.minikube.sigs.k8s.io=minikube
I0507 15:27:28.825490   24859 cli_runner.go:108] Run: podman volume prune -f --filter label=name.minikube.sigs.k8s.io=minikube
W0507 15:27:28.825518   24859 delete.go:216] error pruning volume (might be okay):
[prune volume by label name.minikube.sigs.k8s.io=minikube: podman volume prune -f --filter label=name.minikube.sigs.k8s.io=minikube: exec: "podman": executable file not found in $PATH
stdout:

stderr:
]
I0507 15:27:28.827318   24859 lock.go:35] WriteFile acquiring /Users/tstromberg/.kube/config: {Name:mke11dbcd579c84d4d098461d9cff60a03fa7a38 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
💀  Removed all traces of the "minikube" cluster.

New Behavior

Deleting a known Docker profile:

I0507 15:25:17.974658   24496 cli_runner.go:108] Run: docker ps -a --filter label=name.minikube.sigs.k8s.io --format {{.Names}}
I0507 15:25:18.049231   24496 delete.go:189] DeleteProfiles
I0507 15:25:18.049256   24496 delete.go:249] Deleting minikube
I0507 15:25:18.049267   24496 delete.go:252] minikube configuration: &{Name:minikube KeepContext:false EmbedCerts:false MinikubeISO: KicBaseImage:gcr.io/k8s-minikube/kicbase:v0.0.10@sha256:f58e0c4662bac8a9b5dda7984b185bad8502ade5d9fa364bf2755d636ab51438 Memory:982 CPUs:2 DiskSize:20000 Driver:docker HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.99.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio KubernetesConfig:{KubernetesVersion:v1.18.1 ClusterName:minikube APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin: FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: ExtraOptions:[{Component:kubeadm Key:pod-network-cidr Value:10.244.0.0/16}] ShouldLoadCachedImages:true EnableDefaultCNI:false NodeIP: NodePort:8443 NodeName:} Nodes:[{Name: IP:172.17.0.3 Port:8443 KubernetesVersion:v1.18.1 ControlPlane:true Worker:true}] Addons:map[default-storageclass:true storage-provisioner:true] VerifyComponents:map[apiserver:true system_pods:true]}
🔥  Deleting "minikube" in docker ...
I0507 15:25:18.062225   24496 delete.go:212] deleting possible KIC leftovers for minikube (driver=docker) ...
I0507 15:25:18.062419   24496 cli_runner.go:108] Run: docker ps -a --filter label=name.minikube.sigs.k8s.io=minikube --format {{.Names}}
🔥  Deleting container "minikube" ...
I0507 15:25:18.107310   24496 cli_runner.go:108] Run: docker inspect minikube --format={{.State.Status}}
I0507 15:25:18.153675   24496 cli_runner.go:108] Run: docker exec --privileged -t minikube /bin/bash -c "sudo init 0"
I0507 15:25:19.449497   24496 cli_runner.go:108] Run: docker inspect minikube --format={{.State.Status}}
I0507 15:25:19.505777   24496 oci.go:519] temporary error: container minikube status is Running but expect it to be exited
I0507 15:25:19.505816   24496 oci.go:525] Successfully shutdown container minikube
I0507 15:25:19.505940   24496 cli_runner.go:108] Run: docker rm -f -v minikube
I0507 15:25:19.953547   24496 volumes.go:34] trying to delete all docker volumes with label name.minikube.sigs.k8s.io=minikube
I0507 15:25:19.953700   24496 cli_runner.go:108] Run: docker volume ls --filter label=name.minikube.sigs.k8s.io=minikube --format {{.Name}}
I0507 15:25:19.989498   24496 cli_runner.go:108] Run: docker volume rm --force minikube
I0507 15:25:20.422175   24496 volumes.go:56] trying to prune all docker volumes with label name.minikube.sigs.k8s.io=minikube
I0507 15:25:20.422329   24496 cli_runner.go:108] Run: docker volume prune -f --filter label=name.minikube.sigs.k8s.io=minikube
I0507 15:25:20.457190   24496 cli_runner.go:108] Run: docker inspect minikube --format={{.State.Status}}
I0507 15:25:20.513857   24496 delete.go:75] Unable to get host status for minikube, assuming it has already been deleted: state: unknown state "minikube": docker inspect minikube --format={{.State.Status}}: exit status 1
stdout:


stderr:
Error: No such object: minikube
🔥  Removing /Users/tstromberg/.minikube/machines/minikube ...
I0507 15:25:20.526813   24496 lock.go:35] WriteFile acquiring /Users/tstromberg/.kube/config: {Name:mke11dbcd579c84d4d098461d9cff60a03fa7a38 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
💀  Removed all traces of the "minikube" cluster.

Deleting a non-existent profile:

I0507 15:25:40.120023   24583 cli_runner.go:108] Run: docker ps -a --filter label=name.minikube.sigs.k8s.io --format {{.Names}}
🙄  "minikube" profile does not exist, trying anyways.
I0507 15:25:40.156649   24583 delete.go:189] DeleteProfiles
I0507 15:25:40.156661   24583 delete.go:249] Deleting minikube
I0507 15:25:40.156673   24583 delete.go:260] minikube has no configuration, will try to make it work anyways
I0507 15:25:40.158487   24583 lock.go:35] WriteFile acquiring /Users/tstromberg/.kube/config: {Name:mke11dbcd579c84d4d098461d9cff60a03fa7a38 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
💀  Removed all traces of the "minikube" cluster.
I0507 15:25:40.174158   24583 delete.go:212] deleting possible KIC leftovers for minikube (driver=docker) ...
I0507 15:25:40.174329   24583 cli_runner.go:108] Run: docker ps -a --filter label=name.minikube.sigs.k8s.io=minikube --format {{.Names}}
I0507 15:25:40.214958   24583 volumes.go:34] trying to delete all docker volumes with label name.minikube.sigs.k8s.io=minikube
I0507 15:25:40.215113   24583 cli_runner.go:108] Run: docker volume ls --filter label=name.minikube.sigs.k8s.io=minikube --format {{.Name}}
I0507 15:25:40.247956   24583 volumes.go:56] trying to prune all docker volumes with label name.minikube.sigs.k8s.io=minikube
I0507 15:25:40.248109   24583 cli_runner.go:108] Run: docker volume prune -f --filter label=name.minikube.sigs.k8s.io=minikube

k8s-ci-robot · 2020-05-07T22:23:44Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: tstromberg

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [tstromberg]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

tstromberg · 2020-05-07T22:27:48Z

/ok-to-test

TravisBuddy · 2020-05-07T22:28:49Z

Travis tests have failed

Hey @tstromberg,
Please read the following log in order to understand the failure reason.
It'll be awesome if you fix what's wrong and commit the changes.

TravisBuddy Request Identifier: 1df0a4f0-90b2-11ea-ba10-7b1b65697aa0

minikube-pr-bot · 2020-05-07T22:35:19Z

kvm2 Driver
docker Driver

medyagh

looks good to me with a small nit.

cmd/minikube/cmd/delete.go

TravisBuddy · 2020-05-07T23:05:28Z

Travis tests have failed

Hey @tstromberg,
Please read the following log in order to understand the failure reason.
It'll be awesome if you fix what's wrong and commit the changes.

TravisBuddy Request Identifier: 3ca2ab50-90b7-11ea-ba10-7b1b65697aa0

minikube-pr-bot · 2020-05-07T23:17:06Z

kvm2 Driver
Times for minikube: [65.824592913 78.96957645399999 79.55073856599999]
Average time for minikube: 74.78163597766667

Times for Minikube (PR 8038): [79.969328478 78.01306453800001 80.87183033199999]
Average time for Minikube (PR 8038): 79.61807444933333

Averages Time Per Log

+--------------------------------+-----------+--------------------+
|              LOG               | MINIKUBE  | MINIKUBE (PR 8038) |
+--------------------------------+-----------+--------------------+
| * minikube v1.10.0-beta.2 on   |  0.080964 |           0.066326 |
| Debian 9.11                    |           |                    |
| * Using the kvm2 driver based  |  0.026313 |           0.024036 |
| on existing profile            |           |                    |
| * Starting control plane node  |  0.003946 |           0.003360 |
| minikube in cluster minikube   |           |                    |
| * Creating kvm2 VM (CPUs=2,    | 44.145243 |          45.757450 |
| Memory=3700MB, Disk=20000MB)   |           |                    |
| ...                            |           |                    |
| * Preparing Kubernetes v1.18.1 | 27.226835 |          29.667287 |
| on Docker 19.03.8 ...          |           |                    |
| * Verifying Kubernetes         |  2.291359 |           2.971211 |
| components...                  |           |                    |
| * Enabled addons:              |  0.909938 |           0.980585 |
| default-storageclass,          |           |                    |
| storage-provisioner            |           |                    |
| * Done! kubectl is now         |  0.092585 |           0.138853 |
| configured to use "minikube"   |           |                    |
|                                |  0.004453 |           0.008967 |
+--------------------------------+-----------+--------------------+

docker Driver
Times for minikube: [33.159149469999996 29.720201285999998 26.789550778999995]
Average time for minikube: 29.889633844999995

Times for Minikube (PR 8038): [44.288395295 28.360521427 27.864910923999997]
Average time for Minikube (PR 8038): 33.50460921533333

Averages Time Per Log

+----------------------------------------+-----------+--------------------+
|                  LOG                   | MINIKUBE  | MINIKUBE (PR 8038) |
+----------------------------------------+-----------+--------------------+
| * minikube v1.10.0-beta.2 on           |  0.093460 |           0.088876 |
| Debian 9.11                            |           |                    |
| * Using the docker driver              |  0.003301 |           0.003213 |
| based on existing profile              |           |                    |
| * Starting control plane node          |  0.075695 |           0.072951 |
| minikube in cluster minikube           |           |                    |
| * Creating docker container            |  9.266447 |           8.791772 |
| (CPUs=2, Memory=3700MB) ...            |           |                    |
| * Preparing Kubernetes v1.18.1         |  0.190522 |           0.155404 |
| on Docker 19.03.2 ...                  |           |                    |
|   -                                    | 19.066068 |          19.435774 |
| kubeadm.pod-network-cidr=10.244.0.0/16 |           |                    |
| * Verifying Kubernetes                 |  1.037472 |           1.377320 |
| components...                          |           |                    |
| * Enabled addons:                      |  0.082084 |           2.732658 |
| default-storageclass,                  |           |                    |
| storage-provisioner                    |           |                    |
| * Done! kubectl is now                 |  0.069616 |           0.842030 |
| configured to use "minikube"           |           |                    |
|                                        |  0.004969 |           0.004612 |
+----------------------------------------+-----------+--------------------+

Only delete leftovers for the appropriate OCI if known

64e4799

k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label May 7, 2020

k8s-ci-robot requested review from afbjorklund and medyagh May 7, 2020 22:23

k8s-ci-robot added approved Indicates a PR has been approved by an approver from all required OWNERS files. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels May 7, 2020

tstromberg changed the title ~~Only delete leftovers for the appropriate OCI if known~~ Use the appropriate OCI for deletePossibleKicLeftOver when known May 7, 2020

k8s-ci-robot added the ok-to-test Indicates a non-member PR verified by an org member that is safe to test. label May 7, 2020

medyagh approved these changes May 7, 2020

View reviewed changes

cmd/minikube/cmd/delete.go Outdated Show resolved Hide resolved

Cite issue kubernetes#8040 for TODO's

e06e180

medyagh merged commit bf541b4 into kubernetes:master May 8, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use the appropriate OCI for deletePossibleKicLeftOver when known #8038

Use the appropriate OCI for deletePossibleKicLeftOver when known #8038

tstromberg commented May 7, 2020 •

edited

Loading

k8s-ci-robot commented May 7, 2020

tstromberg commented May 7, 2020

TravisBuddy commented May 7, 2020

minikube-pr-bot commented May 7, 2020

medyagh left a comment

TravisBuddy commented May 7, 2020

minikube-pr-bot commented May 7, 2020

Use the appropriate OCI for deletePossibleKicLeftOver when known #8038

Use the appropriate OCI for deletePossibleKicLeftOver when known #8038

Conversation

tstromberg commented May 7, 2020 • edited Loading

Old Behavior

New Behavior

k8s-ci-robot commented May 7, 2020

tstromberg commented May 7, 2020

TravisBuddy commented May 7, 2020

Travis tests have failed

TravisBuddy Request Identifier: 1df0a4f0-90b2-11ea-ba10-7b1b65697aa0

minikube-pr-bot commented May 7, 2020

medyagh left a comment

Choose a reason for hiding this comment

TravisBuddy commented May 7, 2020

Travis tests have failed

TravisBuddy Request Identifier: 3ca2ab50-90b7-11ea-ba10-7b1b65697aa0

minikube-pr-bot commented May 7, 2020

tstromberg commented May 7, 2020 •

edited

Loading