Add 'pause' command to freeze Kubernetes cluster

[tstromberg]

Uses the cgroups freezer to reduce minikube's CPU overhead from 30-40% of a core to 1-2% of a core.

By default, it only pauses namespaces populated by minikube, so that applications running on top remain running. This behavior can be overridden by passing -A, similar to kubectl, or by passing --namespaces.

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: tstromberg

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [tstromberg]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[afbjorklund]

This change would look nicer if we fixed #5677 first.

[blueelvis]

What if we pause the VM in its entirety and then resume it?

[tstromberg]

What if we pause the VM in its entirety and then resume it?

I decided against this approach, for a number of reasons:

• This allows us to pause Kubernetes without pausing the apps running on top of Kubernetes. The intent is for a follow-up PR to allow for pausing only Kubernetes by default, but also allowing -A and --namespace arguments.
• The diifference between pausing the VM and pausing containers is negligible (<5% of a core)
• libmachine doesn't support suspend/resume

[medyagh]

@tstromberg are you still wokring on this ?

[tstromberg]

@medyagh - Yes.

[codecov-io]

Codecov Report

❗ No coverage uploaded for pull request base (master@5f0420f). Click here to learn what that means.
The diff coverage is 15.66%.

@@           Coverage Diff            @@
##             master   #5962   +/-   ##
========================================
  Coverage          ?   37.6%           
========================================
  Files             ?     127           
  Lines             ?    8562           
  Branches          ?       0           
========================================
  Hits              ?    3220           
  Misses            ?    4923           
  Partials          ?     419

Impacted Files	Coverage Δ
pkg/minikube/bootstrapper/images/images.go	100% <ø> (ø)
pkg/minikube/logs/logs.go	2.35% <ø> (ø)
pkg/minikube/assets/addons.go	37.93% <ø> (ø)
pkg/minikube/config/profile.go	70.1% <ø> (ø)
cmd/minikube/cmd/status.go	6.89% <0%> (ø)
cmd/minikube/cmd/pause.go	8.1% <0%> (ø)
cmd/minikube/cmd/unpause.go	7.89% <0%> (ø)
pkg/minikube/cluster/pause.go	0% <0%> (ø)
pkg/addons/addons.go	26.05% <0%> (ø)
pkg/minikube/cruntime/docker.go	37.6% <0%> (ø)
... and 1 more

[tstromberg]

/ok-to-test

[minikube-pr-bot]

Error: running mkcmp: exit status 1

[medyagh]

/ok-to-test

[priyawadhwa]

Tried locally with docker and it looks good! Just a couple comments.

[minikube-pr-bot]

All Times minikube: [ 97.815426 98.333187 94.732535]
All Times Minikube (PR 5962): [ 98.485204 95.437083 95.348961]

Average minikube: 96.960382
Average Minikube (PR 5962): 96.423749

Averages Time Per Log

+----------------------+-----------+--------------------+
|         LOG          | MINIKUBE  | MINIKUBE (PR 5962) |
+----------------------+-----------+--------------------+
| minikube v           |  0.316998 |           0.291200 |
| Creating kvm2        | 19.829913 |          19.465774 |
| Preparing Kubernetes | 49.272115 |          49.516292 |
| Pulling images       |  4.068309 |           3.271847 |
| Launching Kubernetes | 19.760251 |          20.620153 |
| Waiting for cluster  |  3.062247 |           2.722973 |
+----------------------+-----------+--------------------+