Fix none driver bugs with "pause" #6452

tstromberg · 2020-02-01T00:17:35Z

Fixes two bugs with the none driver:

"status" fails when the "none" driver is paused. This is because the driver status was based on the kubelet process status, which we kill when pausing.
"delete" fails when the "none" driver is paused. This is because kubeadm tries to reach the apiserver and hangs.
apiserver status checks fail when there is more than one apiserver pid running.

Behavioral changes

The "none" driver host status now reports based on a combination of apiserver status and kubelet status. This allows the host to appear paused.

Closes #6451

k8s-ci-robot · 2020-02-01T00:18:35Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: tstromberg

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [tstromberg]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

codecov-io · 2020-02-01T00:23:00Z

Codecov Report

Merging #6452 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #6452   +/-   ##
=======================================
  Coverage   37.84%   37.84%           
=======================================
  Files         128      128           
  Lines        8748     8748           
=======================================
  Hits         3311     3311           
  Misses       5011     5011           
  Partials      426      426

tstromberg · 2020-02-01T00:24:48Z

/ok-to-test

minikube-pr-bot · 2020-02-01T00:36:17Z

Error: running mkcmp: exit status 1

priyawadhwa

lgtm!

minikube-pr-bot · 2020-02-01T00:56:06Z

All Times minikube: [ 96.154790 95.615379 94.942786]
All Times Minikube (PR 6452): [ 94.568774 94.610937 96.071188]

Average minikube: 95.570985
Average Minikube (PR 6452): 95.083633

Averages Time Per Log

+----------------------+-----------+--------------------+
|         LOG          | MINIKUBE  | MINIKUBE (PR 6452) |
+----------------------+-----------+--------------------+
| minikube v           |  0.300100 |           0.279051 |
| Creating kvm2        | 20.974831 |          20.368994 |
| Preparing Kubernetes | 49.178473 |          48.624250 |
| Pulling images       |  3.322960 |           3.564712 |
| Launching Kubernetes | 19.133110 |          19.283358 |
| Waiting for cluster  |  1.056971 |           1.400833 |
+----------------------+-----------+--------------------+

minikube-pr-bot · 2020-02-01T01:26:08Z

All Times minikube: [ 94.460397 94.253071 97.194256]
All Times Minikube (PR 6452): [ 97.616462 94.827217 99.647928]

Average minikube: 95.302575
Average Minikube (PR 6452): 97.363869

Averages Time Per Log

+----------------------+-----------+--------------------+
|         LOG          | MINIKUBE  | MINIKUBE (PR 6452) |
+----------------------+-----------+--------------------+
| minikube v           |  0.296881 |           0.305329 |
| Creating kvm2        | 20.730766 |          20.352119 |
| Preparing Kubernetes | 48.378544 |          50.413683 |
| Pulling images       |  3.550320 |           3.490965 |
| Launching Kubernetes | 19.616320 |          19.872759 |
| Waiting for cluster  |  1.215892 |           1.390992 |
+----------------------+-----------+--------------------+

minikube-pr-bot · 2020-02-01T01:47:46Z

All Times minikube: [ 95.086994 97.051017 97.635279]
All Times Minikube (PR 6452): [ 95.791791 97.160910 98.785699]

Average minikube: 96.591097
Average Minikube (PR 6452): 97.246133

Averages Time Per Log

+----------------------+-----------+--------------------+
|         LOG          | MINIKUBE  | MINIKUBE (PR 6452) |
+----------------------+-----------+--------------------+
| minikube v           |  0.280672 |           0.284765 |
| Creating kvm2        | 20.323716 |          20.079139 |
| Preparing Kubernetes | 49.659476 |          50.058005 |
| Pulling images       |  4.189286 |           4.009000 |
| Launching Kubernetes | 18.911623 |          19.815979 |
| Waiting for cluster  |  1.382051 |           1.388445 |
+----------------------+-----------+--------------------+

minikube-pr-bot · 2020-02-01T02:48:03Z

All Times minikube: [ 93.884505 93.614791 95.225650]
All Times Minikube (PR 6452): [ 94.882588 94.738834 95.258183]

Average minikube: 94.241649
Average Minikube (PR 6452): 94.959869

Averages Time Per Log

+----------------------+-----------+--------------------+
|         LOG          | MINIKUBE  | MINIKUBE (PR 6452) |
+----------------------+-----------+--------------------+
| minikube v           |  0.298791 |           0.283177 |
| Creating kvm2        | 19.979805 |          19.552157 |
| Preparing Kubernetes | 49.237329 |          48.940015 |
| Pulling images       |  3.383737 |           3.864781 |
| Launching Kubernetes | 18.751534 |          19.521080 |
| Waiting for cluster  |  1.056526 |           1.224446 |
+----------------------+-----------+--------------------+

minikube-pr-bot · 2020-02-01T03:07:46Z

All Times minikube: [ 93.845361 92.516557 95.441738]
All Times Minikube (PR 6452): [ 94.314409 95.416252 93.546439]

Average minikube: 93.934552
Average Minikube (PR 6452): 94.425700

Averages Time Per Log

+----------------------+-----------+--------------------+
|         LOG          | MINIKUBE  | MINIKUBE (PR 6452) |
+----------------------+-----------+--------------------+
| minikube v           |  0.299097 |           0.281602 |
| Creating kvm2        | 20.073203 |          19.711166 |
| Preparing Kubernetes | 47.618370 |          49.697541 |
| Pulling images       |  3.568260 |           3.294406 |
| Launching Kubernetes | 18.834523 |          18.844178 |
| Waiting for cluster  |  1.553880 |           1.077882 |
+----------------------+-----------+--------------------+

…confusing

minikube-pr-bot · 2020-02-01T03:27:24Z

All Times minikube: [ 92.558355 92.338159 89.822738]
All Times Minikube (PR 6452): [ 96.054025 96.786575 92.833253]

Average minikube: 91.573084
Average Minikube (PR 6452): 95.224618

Averages Time Per Log

+----------------------+-----------+--------------------+
|         LOG          | MINIKUBE  | MINIKUBE (PR 6452) |
+----------------------+-----------+--------------------+
| minikube v           |  0.309202 |           0.298088 |
| Creating kvm2        | 19.707230 |          20.199696 |
| Preparing Kubernetes | 46.836696 |          49.131964 |
| Pulling images       |  3.543493 |           3.446408 |
| Launching Kubernetes | 18.302376 |          19.560101 |
| Waiting for cluster  |  1.043450 |           1.059005 |
+----------------------+-----------+--------------------+

tstromberg · 2020-02-01T03:27:55Z

Wooh, that was fun. It turns out that libmachine will repeatedly call driver.Stop() until state == stopped, so the none driver better return stopped when it is supposed to.

tstromberg · 2020-02-01T06:19:06Z

Still have some quirks to sort out so that the none driver returns the same status as other drivers when paused or stopped.

medyagh · 2020-02-03T01:04:14Z

/retest this please

minikube-pr-bot · 2020-02-03T22:35:14Z

All Times minikube: [ 100.319053 99.080102 96.714365]
All Times Minikube (PR 6452): [ 97.695928 100.659934 97.827891]

Average Minikube (PR 6452): 98.727918
Average minikube: 98.704507

Averages Time Per Log

+----------------------+-----------+--------------------+
|         LOG          | MINIKUBE  | MINIKUBE (PR 6452) |
+----------------------+-----------+--------------------+
| minikube v           |  0.332553 |           0.310830 |
| Creating kvm2        | 21.012571 |          20.358166 |
| Preparing Kubernetes | 49.684249 |          49.669207 |
| Pulling images       |  3.302688 |           4.123325 |
| Launching Kubernetes | 20.585979 |          20.441037 |
| Waiting for cluster  |  1.717287 |           1.725136 |
+----------------------+-----------+--------------------+

k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Feb 1, 2020

k8s-ci-robot requested review from blueelvis and medyagh February 1, 2020 00:18

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Feb 1, 2020

tstromberg added 3 commits January 31, 2020 16:23

Allow 'profiles list' to work even if a profile has no control plane

b3149a1

Promote Infofs to errors/warnings

40f2f7c

Improve handling of paused none clusters.

dd4ad10

tstromberg force-pushed the none-paused branch from 1cabf0c to dd4ad10 Compare February 1, 2020 00:23

tstromberg requested review from priyawadhwa and sharifelgamal February 1, 2020 00:24

k8s-ci-robot added the ok-to-test Indicates a non-member PR verified by an org member that is safe to test. label Feb 1, 2020

Merge branch 'master' into none-paused

ab93b27

priyawadhwa approved these changes Feb 1, 2020

View reviewed changes

The none driver does not stop. Improve test error messages

25401ce

Fix unset got variable

921299b

tstromberg added 2 commits January 31, 2020 18:36

Add logging of non-nil errors

6760929

Add more logging

7f0e835

tstromberg added 2 commits January 31, 2020 18:54

Make none.GetState more intelligent to fix stop issues

cee9448

More status logging

1176aa4

Re-arrange status calls to extract maximum information without being …

edbe92f

…confusing

Move kubeconfig down as we cannot get an IP if host is down

af2060a

tstromberg added this to the v1.7.0 milestone Feb 1, 2020

tstromberg mentioned this pull request Feb 3, 2020

docker driver on macOS: hangs after wake from hibernation #6462

Closed

tstromberg added 4 commits February 3, 2020 13:49

Merge branch 'master' into none-paused

04df968

Consider a paused apiserver to be controlled by a running host

4124fbd

Choose the newest apiserver pid, and choose more exactly

e5e47db

Merge branch 'master' into none-paused

cbd3327

tstromberg merged commit 901cb52 into kubernetes:master Feb 4, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix none driver bugs with "pause" #6452

Fix none driver bugs with "pause" #6452

tstromberg commented Feb 1, 2020 •

edited

Loading

k8s-ci-robot commented Feb 1, 2020

codecov-io commented Feb 1, 2020 •

edited

Loading

tstromberg commented Feb 1, 2020

minikube-pr-bot commented Feb 1, 2020

priyawadhwa left a comment

minikube-pr-bot commented Feb 1, 2020

minikube-pr-bot commented Feb 1, 2020

minikube-pr-bot commented Feb 1, 2020

minikube-pr-bot commented Feb 1, 2020

minikube-pr-bot commented Feb 1, 2020

minikube-pr-bot commented Feb 1, 2020

tstromberg commented Feb 1, 2020

tstromberg commented Feb 1, 2020

medyagh commented Feb 3, 2020

minikube-pr-bot commented Feb 3, 2020

Fix none driver bugs with "pause" #6452

Fix none driver bugs with "pause" #6452

Conversation

tstromberg commented Feb 1, 2020 • edited Loading

Behavioral changes

k8s-ci-robot commented Feb 1, 2020

codecov-io commented Feb 1, 2020 • edited Loading

Codecov Report

tstromberg commented Feb 1, 2020

minikube-pr-bot commented Feb 1, 2020

priyawadhwa left a comment

Choose a reason for hiding this comment

minikube-pr-bot commented Feb 1, 2020

minikube-pr-bot commented Feb 1, 2020

minikube-pr-bot commented Feb 1, 2020

minikube-pr-bot commented Feb 1, 2020

minikube-pr-bot commented Feb 1, 2020

minikube-pr-bot commented Feb 1, 2020

tstromberg commented Feb 1, 2020

tstromberg commented Feb 1, 2020

medyagh commented Feb 3, 2020

minikube-pr-bot commented Feb 3, 2020

tstromberg commented Feb 1, 2020 •

edited

Loading

codecov-io commented Feb 1, 2020 •

edited

Loading