Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add cache command to cache non-minikube images #2203

Merged
merged 4 commits into from
Nov 29, 2017

Conversation

priyawadhwa
Copy link

The cache command adds and deletes images through the cache add and cache delete subcommands. The list of cached images is stored as a string array in the config file, so that it can be loaded into the docker daemon on minikube start.

#2065

@k8s-ci-robot k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Nov 17, 2017
@codecov-io
Copy link

codecov-io commented Nov 17, 2017

Codecov Report

Merging #2203 into master will decrease coverage by 0.62%.
The diff coverage is 2.77%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #2203      +/-   ##
==========================================
- Coverage   27.75%   27.12%   -0.63%     
==========================================
  Files          82       83       +1     
  Lines        5538     5684     +146     
==========================================
+ Hits         1537     1542       +5     
- Misses       3806     3946     +140     
- Partials      195      196       +1
Impacted Files Coverage Δ
pkg/minikube/machine/cache_images.go 7.76% <0%> (-2.3%) ⬇️
cmd/minikube/cmd/config/config.go 15.94% <0%> (-23.35%) ⬇️
cmd/minikube/cmd/start.go 9.62% <0%> (-0.16%) ⬇️
cmd/minikube/cmd/config/util.go 18.27% <0%> (-2.46%) ⬇️
cmd/minikube/cmd/cache.go 11.11% <11.11%> (ø)
pkg/util/kubeconfig/config.go 47.61% <0%> (-0.69%) ⬇️
pkg/minikube/service/service.go 28.95% <0%> (-0.14%) ⬇️
pkg/minikube/cluster/cluster.go 31.52% <0%> (+0.46%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update bc20281...6344d0b. Read the comment docs.

Copy link
Contributor

@r2d4 r2d4 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, just a few comments around the config file.

Run: func(cmd *cobra.Command, args []string) {
// Cache and load images into docker daemon
err := cacheAndLoadImages(args)
if err != nil {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For functions that just return an error like this its preferable to collapse them into a one liner

if err := cacheAndLoadImages(args); err != nil {
...
}

return nil, err
}
return bootstrapper.NewSSHRunner(client), nil

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

delete newline

}

func cacheAndLoadImages(images []string) error {

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

delete newline

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that this is generic enough to put in machine/cache_images.go and export it as CacheAndLoadImages(images []string) error

Short: "Delete an image from the local cache.",
Long: "Delete an image from the local cache.",
Run: func(cmd *cobra.Command, args []string) {

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

delete newline

@@ -98,6 +97,27 @@ func LoadImages(cmd bootstrapper.CommandRunner, images []string, cacheDir string
return nil
}

// DeleteImages deletes images from local daemon
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Whats the use case for deleting them from the minikube daemon? I think the delete function should just clean up the cache folder on the host if anything.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

my bad, i thought it was meant to delete from the daemon as well. sounds good!

@@ -213,6 +218,79 @@ func configurableFields() string {
return strings.Join(fields, "\n")
}

// AddToConfigArray adds entries to an array in the config file
func AddToConfigArray(name string, images []string) error {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe a map is a better data structure here if you want to maintain a list of keys without duplication.


// Add images to currently existing values in config file
if values != nil {
for _, v := range values.([]interface{}) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cast values to a []string, so you don't have to do the casts below.

if values != nil {
// Add images that are in config file but not in images to finalImages
// These are the images that should remain in the config file after deletion
for _, v := range values.([]interface{}) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[]string instead of []interface

return WriteConfig(configFile)
}

// DeleteFromConfigArray deletes entries in an array in the config file
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe map[string]interface{} or map[string]struct{} so then you can just

map[key] = struct{}{} for adding
and delete(map, key) for deleting

I think either is fine though. Using a map makes it more complicated for users to edit the config file themselves (offhand, I'm not sure how the marshalled json would look)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yah I switched it to a map[string]interface{} and it doesn't look too bad in the json so will probably go with that

if err != nil {
return err
}
values := configFile[name]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think you want a val, ok := configFile[name]. ok is a boolean that is true if the key exists in the map, and false if not.

Copy link
Contributor

@r2d4 r2d4 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two last comments but I tried it out locally and it looks good!

@@ -17,18 +17,19 @@ limitations under the License.
package machine
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I supposed this never worked, but we never cared about checking the error since it was done on a best-effort basis. But deleting the temp image tarballs in the VM requires sudo

diff --git a/pkg/minikube/machine/cache_images.go b/pkg/minikube/machine/cache_images.go
index ec26c0230..63cf4e40a 100644
--- a/pkg/minikube/machine/cache_images.go
+++ b/pkg/minikube/machine/cache_images.go
@@ -17,7 +17,6 @@ limitations under the License.
 package machine

 import (
-       "golang.org/x/sync/errgroup"
        "io/ioutil"
        "os"
        "os/exec"
@@ -25,6 +24,8 @@ import (
        "runtime"
        "strings"

+       "golang.org/x/sync/errgroup"
+
        "k8s.io/minikube/pkg/minikube/assets"
        "k8s.io/minikube/pkg/minikube/bootstrapper"
        "k8s.io/minikube/pkg/minikube/config"
@@ -212,7 +213,7 @@ func LoadFromCacheBlocking(cmd bootstrapper.CommandRunner, src string) error {
                return errors.Wrapf(err, "loading docker image: %s", dst)
        }

-       if err := cmd.Run("rm -rf " + dst); err != nil {
+       if err := cmd.Run("sudo rm -rf " + dst); err != nil {
                return errors.Wrap(err, "deleting temp docker image location")
        }

@@ -193,6 +194,10 @@ var settings = []Setting{
name: "disable-driver-mounts",
set: SetBool,
},
{
name: "cache",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This also needs a set method, since thats what minikube config set uses. Otherwise, it will panic!

diff --git a/cmd/minikube/cmd/config/config.go b/cmd/minikube/cmd/config/config.go
index 15bad2c56..582027d5a 100644
--- a/cmd/minikube/cmd/config/config.go
+++ b/cmd/minikube/cmd/config/config.go
@@ -196,6 +196,7 @@ var settings = []Setting{
        },
        {
                name:   "cache",
+               set:    SetConfigMap,
                setMap: SetMap,
        },
 }
diff --git a/cmd/minikube/cmd/config/util.go b/cmd/minikube/cmd/config/util.go
index 5a2db1d7a..212771ae3 100644
--- a/cmd/minikube/cmd/config/util.go
+++ b/cmd/minikube/cmd/config/util.go
@@ -20,6 +20,7 @@ import (
        "fmt"
        "os"
        "strconv"
+       "strings"

        "github.com/pkg/errors"
        "k8s.io/minikube/pkg/minikube/assets"
@@ -65,6 +66,16 @@ func SetMap(m config.MinikubeConfig, name string, val map[string]interface{}) er
        return nil
 }

+func SetConfigMap(m config.MinikubeConfig, name string, val string) error {
+       list := strings.Split(val, ",")
+       v := make(map[string]interface{})
+       for _, s := range list {
+               v[s] = nil
+       }
+       m[name] = v
+       return nil
+}
+
 func SetInt(m config.MinikubeConfig, name string, val string) error {
        i, err := strconv.Atoi(val)
        if err != nil {

Copy link
Contributor

@r2d4 r2d4 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@r2d4 r2d4 merged commit 43e4b0c into kubernetes:master Nov 29, 2017
@Morriz
Copy link

Morriz commented Nov 30, 2017

Can we get some documentation with this?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants