Fast context switch: commands #1501

simonferquel · 2018-11-07T10:51:42Z

Depends on #1500
This brings a context subcommands to manage contexts stored in the context store, as well as modifying docker info / docker version to show current context

silvin-lubecki · 2018-11-09T16:23:15Z

cli/command/system/version.go

@@ -22,6 +22,7 @@ import (

 var versionTemplate = `{{with .Client -}}
 Client:{{if ne .Platform.Name ""}} {{.Platform.Name}}{{end}}
+ Context: {{.Context}}


I wonder if we want the context here in the version command or in the info ? 🤔
@thaJeztah any thought?

cli/command/context/cmd.go

cli/command/context/create.go

vdemeester

Really not fan of the dockerEndpointOptions and kubernetesEndpointOptions struct…
Not sure for the command either : docker context set-docker-endpoint … vs docker context set docker-endpoint …, same with other set-* options… we may want to have more subcommands… This would allow docker context set defaultStackOrchestrator commands (targeting current context if no context is provided via a flag 👼 )

cli/command/context/create.go

cli/command/context/export.go

cli/command/context/setkubernetes.go

cli/command/context/setoptions.go

cli/command/context/use.go

silvin-lubecki · 2018-11-20T13:15:14Z

Linter complains

cli/command/context/setoptions.go:44::warning: line is 213 characters (lll)

silvin-lubecki · 2018-11-20T13:16:53Z

One test is failing:

--- FAIL: TestVersionAlign (0.00s)
    version_test.go:112: assertion failed: 
        --- expected
        +++ actual
        @@ -1,4 +1,4 @@
         Client:
        -·Context:···········test-context
        +·Context:·test-context
         ·Version:···········18.99.5-ce
         ·API·version:·······1.38

codecov-io · 2018-12-17T16:21:16Z

Codecov Report

Merging #1501 into master will decrease coverage by 0.15%.
The diff coverage is 51.51%.

@@            Coverage Diff             @@
##           master    #1501      +/-   ##
==========================================
- Coverage   55.32%   55.16%   -0.16%     
==========================================
  Files         283      301      +18     
  Lines       19330    20359    +1029     
==========================================
+ Hits        10694    11232     +538     
- Misses       7939     8323     +384     
- Partials      697      804     +107

andrewhsu · 2018-12-19T10:27:08Z

@simonferquel when you get the chance, can you update the description with any user interaction changes? just a simple example of what is the new command or option or output from docker version or docker info or whatnot would help describe to the end user what this PR will bring it because there's a lot of code changes in this one.

andrewhsu · 2018-12-19T14:44:13Z

i talked with @simonferquel @silvin-lubecki @thaJeztah about this and opened up an issue to describe the bigger idea and reasoning and track outstanding issues #1586

thaJeztah · 2019-01-08T15:29:07Z

cli/command/cli.go

+	if config != nil && config.CurrentContext != "" {
+		_, err := contextstore.GetContextMetadata(config.CurrentContext)
+		if store.IsErrContextDoesNotExist(err) {
+			warnCurrentContextNotFound(config.CurrentContext, stderr)


Just noticed this one: I don't think we should do the automatic fallback, but instead make it a hard fail. Falling back can be dangerous, for example:

docker --context=develop container rm -f somecontainer WARNING: current context "develop" is not found on filesystem. Falling back to default Docker endpoint

And now you removed the container from "whatever happened to be in DOCKER_HOST

Erf, you are right, but this one will be tricky: docker context use implicitly requires an initialized dockerCli.
As this error can occur only if the user messes directly with the store dir, is it ok if I do a hard fail, indicating that he should fix his configfile manually?

Trying to grasp your comment;

All the docker context commands are local operations, so (technically) should not need to care about the current context. i.e, if the "current" context uses unix:///no-such-thing), it should not be an error until I try to connect to dockerd or the k8s API, so docker context use <a valid context> should work (as it doesn't require a context to perform that action).

simonferquel · 2019-01-08T15:32:52Z

For the not-a-context context name, I suggest that for printing, we could re-introduce the <DOCKER_HOST> name (which is now invalid as a regular context name).

Also thinking that we need a way to switch back to the default/docker_host/unset situation (not a big fan of <DOCKER_HOST> as a name; it's very "shouty"); I'd be ok with a reserved name though, perhaps "default", "none" or "unset" (😅 I know) ? That way, I could switch between context X and back to the default situation.

I will go for default, as I don't like the idea of docker context ls printing a context named none.

For the import from kubeconfig, the problem is that you can't create a context without configuring a docker endpoint

Ah, right, gotcha

Maybe should I remove the --kubeconfig flag and make that feature into a separate subcommand?

So, if we can't import the exported --kubeconfig, what is the exact use-case? If the "regular" export contains the full configuration, is there a need to export just a subset of that information in a format that cannot be imported by the docker cli?

The idea is to be able to use it with other tools like kubectl etc.

For the ID, I really don't want to introduce an index of names to id, and this SHA based stuff seems good to me. However, I think that I should not call this an ID, because it is actually a SHA of the real identifier which is the name (and thus, it is confusing)

Right, so I'm wondering what we gained by introducing it (also see #1501 (comment)), as the original reason for it was to get a) consistency (have an ID as unique identifier and b) possibly allow name to be optional and/or allow renaming.

The only thing it brought is a "safe" name for the path (no chance on path-traversing and such), although we still use the name as default for export

What about naming it context_dir, and in the inspect function, I would report a metadata_dir and tls_dir with the path to the related storage ?

Perhaps xx_path (or CamelCase ..Path), as that's used elsewhere (see docker container inspect)

Are metadata and tls always relative to that path? If so, perhaps there's no need to include them separately

ok for _path suffixed fields

silvin-lubecki · 2019-01-08T16:29:13Z

cli/command/context/options.go

@@ -2,6 +2,7 @@ package context

 import (
 	"fmt"
+	"github.com/docker/docker/client"


nit: imports reordering

thaJeztah

Thanks! Getting close; left some comments inline 🤗

thaJeztah · 2019-01-10T12:44:35Z

cli/command/cli.go

+	if config != nil && config.CurrentContext != "" {
+		_, err := contextstore.GetContextMetadata(config.CurrentContext)
+		if store.IsErrContextDoesNotExist(err) {
+			warnCurrentContextNotFound(config.CurrentContext, stderr)


Trying to grasp your comment;

All the docker context commands are local operations, so (technically) should not need to care about the current context. i.e, if the "current" context uses unix:///no-such-thing), it should not be an error until I try to connect to dockerd or the k8s API, so docker context use <a valid context> should work (as it doesn't require a context to perform that action).

cli/command/context/create.go

thaJeztah · 2019-01-10T12:57:17Z

cli/command/context/export-import_test.go

 	defer cleanup()
 	createTestContextWithKube(t, cli)
+	withPipeInOut(closeChan)(cli)


Why was this needed?

edit: ah, looks like you updated in a follow-up commit

yep it is not needed anymore (and removed)

cli/command/context/update.go

cli/command/context/import.go

thaJeztah · 2019-01-10T15:13:11Z

cli/command/context/list_test.go

+	golden.Assert(t, cli.OutBuffer().String(), "quiet-list.golden")
+}
+
+func TestInspect(t *testing.T) {


Looks like this one is in the wrong file (should probably be in inspect_test.go)

thaJeztah · 2019-01-10T15:20:27Z

cli/command/context/list.go

+	}
+
+	flags := cmd.Flags()
+	flags.StringVar(&opts.format, "format", formatter.TableFormatKey, "Pretty-print contexts using a Go template")


nit (for a follow-up); for other commands we use "" as default, and in the code use something like;

format := opts.format if len(format) == 0 { format = formatter.TableFormatKey }

e.g.

cli/cli/command/system/df.go

Lines 45 to 48 in a900ba8

format := opts.format

if len(format) == 0 {

format = formatter.TableFormatKey

}

We can do a follow-up to decide which approach to take for all (either way makes some sense)

ok for a followup

thaJeztah · 2019-01-10T15:23:01Z

cli/command/context/list_test.go

+	golden.Assert(t, cli.OutBuffer().String(), "list.unset.golden")
+}
+
+func TestListNoContext(t *testing.T) {


I noticed the default context still disappears after selecting a context. Can we make the default context always visible in docker context ls?

docker context ls NAME DESCRIPTION DOCKER ENDPOINT KUBERNETES ENDPOINT ORCHESTRATOR default * Current DOCKER_HOST based configuration unix:///var/run/docker.sock swarm foobar unix:///var/run/docker.sock docker context use foobar foobar Current context is now "foobar" docker context ls NAME DESCRIPTION DOCKER ENDPOINT KUBERNETES ENDPOINT ORCHESTRATOR foobar * unix:///var/run/docker.sock docker context use default default Current context is now "default" NAME DESCRIPTION DOCKER ENDPOINT KUBERNETES ENDPOINT ORCHESTRATOR default * Current DOCKER_HOST based configuration unix:///var/run/docker.sock swarm foobar unix:///var/run/docker.sock

thaJeztah · 2019-01-10T15:25:27Z

cli/command/context/cmd.go

+		return errors.New("context name cannot be empty")
+	}
+	if name == "default" {
+		return errors.New(`"default" is a reserved context name`)


nit (ok for a follow-up): we should probably handle this with for other situations as well. Perhaps when using inspect; return the information that we obtained from the default (DOCKER_HOST). Currently;

docker context inspect default [] context "default" does not exist docker context rm default default: context "default" does not exist

Not sure about that: for now, when a context is set and I list the contexts, the default context appears with no docker/kubernetes endpoint. Not sure if Inspect makes sense.

Maybe indicate that the default context cannot be inspected ?

Yes, I think it's ok to produce a better /helpful error for now (then we can revisit in future)

It is in fact part of the latest commit :)

thaJeztah · 2019-01-10T15:32:09Z

cli/context/store/store.go

+}
+
+// IsErrContextDoesNotExist checks if the given error is a "context does not exist" condition
+func IsErrContextDoesNotExist(err error) bool {


Also for a follow-up; use an interface for these (like the errdefs package (or use that package); https://github.com/moby/moby/blob/d48392a35b157612d28c48eb8e4fad1272fa1442/errdefs/defs.go#L3-L6

// ErrNotFound signals that the requested object doesn't exist type ErrNotFound interface { NotFound() }

https://github.com/moby/moby/blob/d48392a35b157612d28c48eb8e4fad1272fa1442/errdefs/is.go#L32-L36

// IsNotFound returns if the passed in error is an ErrNotFound func IsNotFound(err error) bool { _, ok := getImplementer(err).(ErrNotFound) return ok }

https://github.com/moby/moby/blob/d48392a35b157612d28c48eb8e4fad1272fa1442/errdefs/helpers.go#L13-L19

// NotFound is a helper to create an error of the class with the same name from any error type func NotFound(err error) error { if err == nil { return nil } return errNotFound{err} }

It would make these errors consistent with other handling, and also preserve the original error in some cases where it may be relevant.

All the docker context commands are local operations, so (technically) should not need to care about the current context. i.e, if the "current" context uses unix:///no-such-thing), it should not be an error until I try to connect to dockerd or the k8s API, so docker context use should work (as it doesn't require a context to perform that action).

We don't need to do any request, but the initialization of the CLI still occur. And thus it parses the Host to create the WithHost option (without issuing request). so unix:///some-not-existing-file work (it is valid syntactically), but unix///malformed (note the missing :) fails with a parsing error

Arf, yes.

We'll have to think of a solution for that (e.g., if we can delay evaluating that option until it's actually needed 🤔 )

Actually, I now validate the Host on create/update so it should not be a problem anymore. Only case this error can happen, is if the user delete its store content manually but keep the currentContext set in its config file

Signed-off-by: Simon Ferquel <[email protected]>

simonferquel · 2019-01-14T11:13:34Z

ping @thaJeztah, can you PTAL the last commit ?

thaJeztah · 2019-01-15T16:32:09Z

The idea is to be able to use it with other tools like kubectl etc.

If I do a full docker context export, can I find the kubernetes configuration in the tar?
Still some questions about that, but ok to discuss after this one

Giving this another spin

docker context ls
NAME                DESCRIPTION                               DOCKER ENDPOINT               KUBERNETES ENDPOINT   ORCHESTRATOR
default *           Current DOCKER_HOST based configuration   unix:///var/run/docker.sock                         swarm

docker context ls --format '{{json .}}' | jq .
{
  "Current": true,
  "Description": "Current DOCKER_HOST based configuration",
  "DockerEndpoint": "unix:///var/run/docker.sock",
  "KubernetesEndpoint": "",
  "Name": "default",
  "StackOrchestrator": "swarm"
}

Formatting placeholders work as expected 👍 :

docker context ls --format '{{json .Name}}'
"default"

Validation works

docker context create --docker host=unix:/bla local
unable to create docker endpoint config: unable to apply docker endpoint options: unable to parse docker host `unix:/bla`

docker context create --docker host=unix:///var/run/docker.sock default
"default" is a reserved context name

docker context create --docker host=unix:///var/run/docker.sock local
local
Successfully created context "local"

docker context create --docker host=unix:///var/run/docker.sock local
context "local" already exists

We could even consider showing the same error message when trying to create a context named default;

context "local" already exists

docker context ls

NAME                DESCRIPTION                               DOCKER ENDPOINT               KUBERNETES ENDPOINT   ORCHESTRATOR
default *           Current DOCKER_HOST based configuration   unix:///var/run/docker.sock                         swarm
local                                                         unix:///var/run/docker.sock   

docker context use local
local
Current context is now "local"

docker context ls
NAME                DESCRIPTION                               DOCKER ENDPOINT               KUBERNETES ENDPOINT   ORCHESTRATOR
default             Current DOCKER_HOST based configuration                                                       
local *                                                       unix:///var/run/docker.sock                     

docker context use default
default
Current context is now "default"

🎉 the default context is still there: and I can switch back- and forth awesome!

Inspecting;

docker context inspect local
[
    {
        "Name": "local",
        "Metadata": {},
        "Endpoints": {
            "docker": {
                "Host": "unix:///var/run/docker.sock",
                "SkipTLSVerify": false
            }
        },
        "TLSMaterial": {},
        "Storage": {
            "MetadataPath": "/root/.docker/contexts/meta/25bf8e1a2393f1108d37029b3df5593236c755742ec93465bbafa9b290bddcf6",
            "TLSPath": "/root/.docker/contexts/tls/25bf8e1a2393f1108d37029b3df5593236c755742ec93465bbafa9b290bddcf6"
        }
    }
]

And custom format;

docker context inspect local --format '{{ .Storage.MetadataPath }}'
/root/.docker/contexts/meta/25bf8e1a2393f1108d37029b3df5593236c755742ec93465bbafa9b290bddcf6

"Pretty print" is still missing (but not a blocker for me; it's a "nice to have")

docker context inspect --format=pretty local
pretty

Exporting:

docker context export default
"default" is a reserved context name

docker context export local -
cowardly refusing to export to a terminal, please specify a file path

docker context export local - > ./bla.tar

docker context export local
Written file "local.dockercontext"

docker context export local
open local.dockercontext: file exists

Importing:

docker context import
"docker context import" requires exactly 2 arguments

docker context import bla ./bla.tar
bla
Successfully imported context "bla"

cat bla.tar | docker context import piped -
piped
Successfully imported context "piped"

docker context import imported-context ./local.dockercontext 
imported-context
Successfully imported context "imported-context"

(we could mention from file ... / from stdin as a future enhancement)

docker context import imported-context ./local.dockercontext 
context "imported-context" already exists

docker context import default ./local.dockercontext 
"default" is a reserved context name

Handling non existing / non-reachable contexts

docker context create nosuchhost --docker host=http://nosuchhost.nosuchdomain
nosuchhost
Successfully created context "nosuchhost"

docker context use nosuchhost

docker ps
error during connect: Get http://nosuchhost.nosuchdomain/v1.40/containers/json: dial tcp: lookup nosuchhost.nosuchdomain on 192.168.65.1:53: no such host

docker ps --help

Usage:	docker ps [OPTIONS]

...

Looks good!

thaJeztah

LGTM

Got some code nits / suggestions, but nothing blocking for now

can you cleanup the commits a bit?

simonferquel · 2019-01-15T16:37:08Z

The full export does not contain a kubeconfig file, no.
I'll squash the commits then, thanks a lot!

thaJeztah · 2019-01-15T17:09:04Z

The full export does not contain a kubeconfig file, no.

That's something we should think about doing, so that we can export / backup

thaJeztah · 2019-01-15T17:09:40Z

@silvin-lubecki @vdemeester still LGTY?

vdemeester

still LGTM 😉

simonferquel · 2019-01-15T17:12:07Z

Actually the full export contains everything we need to construct a kubeconfig file, but the layout is different (partly because we thought it would be nice to make it mandatory to put TLS material in a separate dir)

simonferquel requested review from silvin-lubecki and vdemeester as code owners November 7, 2018 10:51

GordonTheTurtle added the status/0-triage label Nov 7, 2018

ijc mentioned this pull request Nov 8, 2018

[do not merge] Fast context switch: foundation #1500

Closed

simonferquel force-pushed the use-context-commands branch from 8bf1f8b to a08fd72 Compare November 9, 2018 14:11