Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable or disable addons per profile #6124

Merged
merged 19 commits into from
Jan 7, 2020

Conversation

priyawadhwa
Copy link

@priyawadhwa priyawadhwa commented Dec 19, 2019

This PR makes a few changes to the addons code:

  1. It moves a lot of the addons code for enabling/disabling addons from package cmd/config/util to a new package minikube/pkg/addons

  2. We now enable/disable addons per profile, and so addons information is stored in machine configs per profile. Any addon related information in the global config is now disregarded.

  3. Since addons can no longer be globally saved, on minikube delete the state of addons is wiped

This PR should fix #4227

The new config has an Addons section:

{
	"Name": "minikube",
	"KeepContext": false,
	"EmbedCerts": false,
	"MinikubeISO": "https://storage.googleapis.com/minikube/iso/minikube-v1.6.0.iso",
	"Memory": 2000,
	"CPUs": 2,
	"DiskSize": 20000,
	"VMDriver": "hyperkit",
	"ContainerRuntime": "docker",
	"HyperkitVpnKitSock": "",
	"HyperkitVSockPorts": [],
	"DockerEnv": null,
	"InsecureRegistry": null,
	"RegistryMirror": null,
	"HostOnlyCIDR": "192.168.99.1/24",
	"HypervVirtualSwitch": "",
	"KVMNetwork": "default",
	"KVMQemuURI": "qemu:///system",
	"KVMGPU": false,
	"KVMHidden": false,
	"DockerOpt": null,
	"DisableDriverMounts": false,
	"NFSShare": [],
	"NFSSharesRoot": "/nfsshares",
	"UUID": "",
	"NoVTXCheck": false,
	"DNSProxy": false,
	"HostDNSResolver": true,
	"KubernetesConfig": {
              ...
	},
	"HostOnlyNicType": "virtio",
	"NatNicType": "virtio",
	"Addons": {
		"helm-tiller": true
	}
}

@k8s-ci-robot k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Dec 19, 2019
@priyawadhwa
Copy link
Author

/ok-to-test

@k8s-ci-robot k8s-ci-robot added the ok-to-test Indicates a non-member PR verified by an org member that is safe to test. label Dec 19, 2019
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: priyawadhwa

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added approved Indicates a PR has been approved by an approver from all required OWNERS files. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Dec 19, 2019
@minikube-bot
Copy link
Collaborator

Error: running mkcmp: exit status 1

Copy link
Contributor

@tstromberg tstromberg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work! Two minor notes.

Remove old TODO and remove function to get current profile as it can
create errors when trying to get multiple profiles.
@codecov-io
Copy link

codecov-io commented Dec 19, 2019

Codecov Report

Merging #6124 into master will increase coverage by <.01%.
The diff coverage is 23.68%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #6124      +/-   ##
==========================================
+ Coverage   38.35%   38.36%   +<.01%     
==========================================
  Files         120      122       +2     
  Lines        8103     8143      +40     
==========================================
+ Hits         3108     3124      +16     
- Misses       4591     4606      +15     
- Partials      404      413       +9
Impacted Files Coverage Δ
cmd/minikube/cmd/config/config.go 10.41% <ø> (ø) ⬆️
cmd/minikube/cmd/config/validations.go 44.64% <ø> (+5.2%) ⬆️
cmd/minikube/cmd/config/util.go 53.57% <ø> (+20.93%) ⬆️
pkg/minikube/tunnel/test_doubles.go 82.85% <0%> (-5.03%) ⬇️
cmd/minikube/cmd/config/disable.go 20% <0%> (ø) ⬆️
cmd/minikube/cmd/config/enable.go 20% <0%> (ø) ⬆️
pkg/minikube/assets/addons.go 37.93% <0%> (+1.26%) ⬆️
pkg/minikube/config/config.go 56.66% <0%> (-8.72%) ⬇️
cmd/minikube/cmd/dashboard.go 1.76% <0%> (ø) ⬆️
cmd/minikube/cmd/start.go 22.73% <0%> (ø) ⬆️
... and 4 more

@minikube-bot
Copy link
Collaborator

All Times minikube: [ 122.713917 122.950899 121.816684]
All Times Minikube (PR 6124): [ 121.777733 111.863169 123.317285]

Average minikube: 122.493833
Average Minikube (PR 6124): 118.986062

Averages Time Per Log

+----------------------+-----------+--------------------+
|         LOG          | MINIKUBE  | MINIKUBE (PR 6124) |
+----------------------+-----------+--------------------+
| minikube v           |  0.245288 |           0.238411 |
| Creating kvm2        | 46.038524 |          45.489637 |
| Preparing Kubernetes | 49.949346 |          47.763775 |
| Pulling images       |  3.244870 |           2.600868 |
| Launching Kubernetes | 20.221802 |          19.911117 |
| Waiting for cluster  |  2.749545 |           2.939913 |
+----------------------+-----------+--------------------+

Copy link
Member

@medyagh medyagh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is really good work ! I would just add in the PR description a paste of the new config format just for clarity.

@medyagh
Copy link
Member

medyagh commented Dec 19, 2019

nicely done, just tested on my machine it works great.

here is the profile config after this PR.

{
	"Name": "minikube",
	"KeepContext": false,
	"EmbedCerts": false,
	"MinikubeISO": "https://storage.googleapis.com/minikube/iso/minikube-v1.6.0.iso",
	"Memory": 2000,
	"CPUs": 2,
	"DiskSize": 20000,
	"VMDriver": "hyperkit",
	"ContainerRuntime": "docker",
	"HyperkitVpnKitSock": "",
	"HyperkitVSockPorts": [],
	"DockerEnv": null,
	"InsecureRegistry": null,
	"RegistryMirror": null,
	"HostOnlyCIDR": "192.168.99.1/24",
	"HypervVirtualSwitch": "",
	"KVMNetwork": "default",
	"KVMQemuURI": "qemu:///system",
	"KVMGPU": false,
	"KVMHidden": false,
	"DockerOpt": null,
	"DisableDriverMounts": false,
	"NFSShare": [],
	"NFSSharesRoot": "/nfsshares",
	"UUID": "",
	"NoVTXCheck": false,
	"DNSProxy": false,
	"HostDNSResolver": true,
	"KubernetesConfig": {
		"KubernetesVersion": "v1.17.0",
		"NodeIP": "192.168.64.190",
		"NodePort": 8443,
		"NodeName": "minikube",
		"APIServerName": "minikubeCA",
		"APIServerNames": null,
		"APIServerIPs": null,
		"DNSDomain": "cluster.local",
		"ContainerRuntime": "docker",
		"CRISocket": "",
		"NetworkPlugin": "",
		"FeatureGates": "",
		"ServiceCIDR": "10.96.0.0/12",
		"ImageRepository": "",
		"ExtraOptions": null,
		"ShouldLoadCachedImages": true,
		"EnableDefaultCNI": false
	},
	"HostOnlyNicType": "virtio",
	"NatNicType": "virtio",
	"Addons": {
		"helm-tiller": true
	}
}

for the second profile without any addons:

{
    "Name": "p2",
    "KeepContext": false,
    "EmbedCerts": false,
    "MinikubeISO": "https://storage.googleapis.com/minikube/iso/minikube-v1.6.0.iso",
    "Memory": 2000,
    "CPUs": 2,
    "DiskSize": 20000,
    "VMDriver": "hyperkit",
    "ContainerRuntime": "docker",
    "HyperkitVpnKitSock": "",
    "HyperkitVSockPorts": [],
    "DockerEnv": null,
    "InsecureRegistry": null,
    "RegistryMirror": null,
    "HostOnlyCIDR": "192.168.99.1/24",
    "HypervVirtualSwitch": "",
    "KVMNetwork": "default",
    "KVMQemuURI": "qemu:///system",
    "KVMGPU": false,
    "KVMHidden": false,
    "DockerOpt": null,
    "DisableDriverMounts": false,
    "NFSShare": [],
    "NFSSharesRoot": "/nfsshares",
    "UUID": "",
    "NoVTXCheck": false,
    "DNSProxy": false,
    "HostDNSResolver": true,
    "KubernetesConfig": {
        "KubernetesVersion": "v1.17.0",
        "NodeIP": "192.168.64.191",
        "NodePort": 8443,
        "NodeName": "minikube",
        "APIServerName": "minikubeCA",
        "APIServerNames": null,
        "APIServerIPs": null,
        "DNSDomain": "cluster.local",
        "ContainerRuntime": "docker",
        "CRISocket": "",
        "NetworkPlugin": "",
        "FeatureGates": "",
        "ServiceCIDR": "10.96.0.0/12",
        "ImageRepository": "",
        "ExtraOptions": null,
        "ShouldLoadCachedImages": true,
        "EnableDefaultCNI": false
    },
    "HostOnlyNicType": "virtio",
    "NatNicType": "virtio",
    "Addons": null
}

@medyagh
Copy link
Member

medyagh commented Dec 19, 2019

one thing that could be for a PR is output of addons list
which can be converted to a table that
name, status, profile

it used to show list and status for all profiles, now we should be clear that if u wanna see addon list, u need to pass -p

I am totally okay with having it in following PRs.

Current output:

 ./out/minikube addons list
- addon-manager: enabled
- dashboard: disabled
- default-storageclass: enabled
- efk: disabled
- freshpod: disabled
- gvisor: disabled
- helm-tiller: enabled
- ingress: disabled
- ingress-dns: disabled
- logviewer: disabled
- metrics-server: disabled
- nvidia-driver-installer: disabled
- nvidia-gpu-device-plugin: disabled
- registry: disabled
- registry-creds: disabled
- storage-provisioner: enabled
- storage-provisioner-gluster: disabled

@priyawadhwa
Copy link
Author

@medyagh the output of addons list should also take the --profile flag and output for that profile!

In addons_list.go we call addonBundle.IsEnabled()

and in this PR I updated IsEnabled() to check for the specific profile here

@tstromberg
Copy link
Contributor

TestAddons failed:

  *  X addon enable failed: property name "ingress" not found

Check that the minikube start --addons feature still works.

@minikube-bot
Copy link
Collaborator

All Times Minikube (PR 6124): [ 122.765268 123.385578 122.021202]
All Times minikube: [ 122.649131 125.449881 125.029649]

Average Minikube (PR 6124): 122.724016
Average minikube: 124.376220

Averages Time Per Log

+----------------------+-----------+--------------------+
|         LOG          | MINIKUBE  | MINIKUBE (PR 6124) |
+----------------------+-----------+--------------------+
| minikube v           |  0.323457 |           0.328182 |
| Creating kvm2        | 46.152982 |          45.869669 |
| Preparing Kubernetes | 50.658453 |          51.201497 |
| Pulling images       |  3.245423 |           3.228833 |
| Launching Kubernetes | 21.557947 |          19.149234 |
| Waiting for cluster  |  2.394528 |           2.902497 |
+----------------------+-----------+--------------------+

@minikube-bot
Copy link
Collaborator

All Times minikube: [ 122.818927 121.265982 125.513654]
All Times Minikube (PR 6124): [ 125.194121 110.433621 125.694816]

Average minikube: 123.199521
Average Minikube (PR 6124): 120.440852

Averages Time Per Log

+----------------------+-----------+--------------------+
|         LOG          | MINIKUBE  | MINIKUBE (PR 6124) |
+----------------------+-----------+--------------------+
| minikube v           |  0.319480 |           0.315579 |
| Creating kvm2        | 46.105013 |          46.533377 |
| Preparing Kubernetes | 50.194957 |          47.894563 |
| Pulling images       |  3.218688 |           3.273204 |
| Launching Kubernetes | 20.548198 |          19.827113 |
| Waiting for cluster  |  2.769527 |           2.553055 |
+----------------------+-----------+--------------------+

@minikube-bot
Copy link
Collaborator

All Times minikube: [ 127.417149 124.653175 123.807327]
All Times Minikube (PR 6124): [ 110.037753 122.129579 124.660763]

Average minikube: 125.292550
Average Minikube (PR 6124): 118.942698

Averages Time Per Log

+----------------------+-----------+--------------------+
|         LOG          | MINIKUBE  | MINIKUBE (PR 6124) |
+----------------------+-----------+--------------------+
| minikube v           |  0.317820 |           0.322194 |
| Creating kvm2        | 46.491447 |          45.382078 |
| Preparing Kubernetes | 51.614356 |          48.386439 |
| Pulling images       |  2.824622 |           2.564738 |
| Launching Kubernetes | 21.278028 |          19.497639 |
| Waiting for cluster  |  2.721681 |           2.744802 |
+----------------------+-----------+--------------------+

@minikube-bot
Copy link
Collaborator

All Times minikube: [ 122.351220 123.394254 123.076054]
All Times Minikube (PR 6124): [ 122.260247 122.276750 111.016107]

Average minikube: 122.940509
Average Minikube (PR 6124): 118.517701

Averages Time Per Log

+----------------------+-----------+--------------------+
|         LOG          | MINIKUBE  | MINIKUBE (PR 6124) |
+----------------------+-----------+--------------------+
| minikube v           |  0.421465 |           0.352073 |
| Creating kvm2        | 45.786780 |          45.422706 |
| Preparing Kubernetes | 50.350999 |          47.528230 |
| Pulling images       |  3.199479 |           2.708104 |
| Launching Kubernetes | 20.392861 |          19.568914 |
| Waiting for cluster  |  2.743340 |           2.895133 |
+----------------------+-----------+--------------------+

@minikube-bot
Copy link
Collaborator

All Times minikube: [ 120.435942 122.476699 120.827028]
All Times Minikube (PR 6124): [ 123.841106 121.037110 125.952855]

Average minikube: 121.246556
Average Minikube (PR 6124): 123.610357

Averages Time Per Log

+----------------------+-----------+--------------------+
|         LOG          | MINIKUBE  | MINIKUBE (PR 6124) |
+----------------------+-----------+--------------------+
| minikube v           |  0.391875 |           0.358301 |
| Creating kvm2        | 46.803015 |          46.018932 |
| Preparing Kubernetes | 49.236097 |          51.514454 |
| Pulling images       |  2.854926 |           2.949324 |
| Launching Kubernetes | 19.196229 |          19.993486 |
| Waiting for cluster  |  2.717795 |           2.732150 |
+----------------------+-----------+--------------------+

@medyagh
Copy link
Member

medyagh commented Dec 21, 2019

weird to see "verify_cache_inside_node" test fail ! I wonder how could that be related to this PR

@medyagh
Copy link
Member

medyagh commented Dec 22, 2019

/retest this please

@kubernetes kubernetes deleted a comment from minikube-bot Dec 30, 2019
@kubernetes kubernetes deleted a comment from minikube-bot Dec 30, 2019
@kubernetes kubernetes deleted a comment from minikube-bot Dec 30, 2019
@minikube-bot
Copy link
Collaborator

All Times minikube: [ 124.561060 125.265181 116.068183]
All Times Minikube (PR 6124): [ 124.597357 126.837043 127.511870]

Average minikube: 121.964808
Average Minikube (PR 6124): 126.315424

Averages Time Per Log

+----------------------+-----------+--------------------+
|         LOG          | MINIKUBE  | MINIKUBE (PR 6124) |
+----------------------+-----------+--------------------+
| minikube v           |  0.452285 |           0.441915 |
| Creating kvm2        | 47.931920 |          47.599189 |
| Preparing Kubernetes | 46.662609 |          50.303966 |
| Pulling images       |  2.897964 |           2.959194 |
| Launching Kubernetes | 20.906854 |          22.052783 |
| Waiting for cluster  |  3.066555 |           2.913203 |
|                      |  0.046620 |           0.045173 |
+----------------------+-----------+--------------------+

@minikube-bot
Copy link
Collaborator

All Times minikube: [ 128.123514 127.219793 126.353198]
All Times Minikube (PR 6124): [ 126.060278 124.316983 125.552679]

Average minikube: 127.232168
Average Minikube (PR 6124): 125.309980

Averages Time Per Log

+----------------------+-----------+--------------------+
|         LOG          | MINIKUBE  | MINIKUBE (PR 6124) |
+----------------------+-----------+--------------------+
| minikube v           |  0.458726 |           0.476462 |
| Creating kvm2        | 47.469278 |          47.733205 |
| Preparing Kubernetes | 51.421075 |          50.794529 |
| Pulling images       |  3.181153 |           2.995977 |
| Launching Kubernetes | 21.900162 |          20.636154 |
| Waiting for cluster  |  2.754874 |           2.626117 |
| Done                 |           |                    |
|                      |  0.050523 |           0.047650 |
+----------------------+-----------+--------------------+

@minikube-bot
Copy link
Collaborator

All Times Minikube (PR 6124): [ 122.857389 126.527433 124.566560]
All Times minikube: [ 125.063490 122.220541 111.575859]

Average minikube: 119.619963
Average Minikube (PR 6124): 124.650460

Averages Time Per Log

+----------------------+-----------+--------------------+
|         LOG          | MINIKUBE  | MINIKUBE (PR 6124) |
+----------------------+-----------+--------------------+
| minikube v           |  0.417583 |           0.415036 |
| Creating kvm2        | 46.268140 |          46.839915 |
| Preparing Kubernetes | 47.329257 |          50.622934 |
| Pulling images       |  2.739873 |           2.935740 |
| Launching Kubernetes | 19.906832 |          20.892658 |
| Waiting for cluster  |  2.914819 |           2.900447 |
| Done                 |           |                    |
|                      |  0.043419 |           0.042436 |
+----------------------+-----------+--------------------+

@@ -33,7 +33,7 @@ import (
"github.com/pkg/errors"
"github.com/spf13/cobra"
"github.com/spf13/viper"
configcmd "k8s.io/minikube/cmd/minikube/cmd/config"
pkgaddons "k8s.io/minikube/pkg/addons"
Copy link
Contributor

@tstromberg tstromberg Jan 6, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can this be made just "addons" (here, and elsewhere when possible)?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so the compiler complains that there's a constant named addons in the same package :(

pkg/addons/addons.go Outdated Show resolved Hide resolved
pkg/addons/addons.go Outdated Show resolved Hide resolved
test/integration/addons_test.go Show resolved Hide resolved
add better error handling and remove duplication in enableOrDisableAddonsInternal
@minikube-bot
Copy link
Collaborator

All Times minikube: [ 127.516988 129.103127 128.002131]
All Times Minikube (PR 6124): [ 126.875179 125.690755 126.002569]

Average minikube: 128.207416
Average Minikube (PR 6124): 126.189501

Averages Time Per Log

+----------------------+-----------+--------------------+
|         LOG          | MINIKUBE  | MINIKUBE (PR 6124) |
+----------------------+-----------+--------------------+
| minikube v           |  0.455948 |           0.466136 |
| Creating kvm2        | 47.809100 |          47.732777 |
| Preparing Kubernetes | 52.238137 |          51.543340 |
| Pulling images       |  2.980345 |           2.526765 |
| Launching Kubernetes | 21.929662 |          20.798696 |
| Waiting for cluster  |  2.744674 |           3.071499 |
| Done                 |           |                    |
|                      |  0.049101 |           0.052452 |
+----------------------+-----------+--------------------+

@priyawadhwa
Copy link
Author

/retest

@priyawadhwa
Copy link
Author

/ok-to-test

@tstromberg tstromberg merged commit af4ee5c into kubernetes:master Jan 7, 2020
@tstromberg
Copy link
Contributor

Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. ok-to-test Indicates a non-member PR verified by an org member that is safe to test. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Addon enablement should be per-profile
6 participants