Fail early when an unsupported docker API version is found

[amogh09]

Summary

DockerClient's WithVersion method and sdkclientfactory's NewFactory function both deal with versioned docker clients. This PR improves the handling of unsupported API versions by detecting and reporting unsupported API versions sooner than later.

WithVersion method now checks if the requested API version is unsupported and returns an error to signal that to the consumer. Before this change the consumer would get back a versioned client even if the version is unsupported and any subsequent API calls on the returned client will fail later. Existing consumers of this method are *DockerTaskEngine.createContainer and *DockerTaskEngine.startContainer methods. Both are updated to ignore the returned error like they already do. The error is then dealt with later.

sdkclientfactory.NewFactory function already takes care of partitioning supported and unsupported API versions. It does so by checking for any errors during versioned client initialization. This is not enough for older Docker engine versions such as 17.03 as those versions do not fail client initialization when the engine's API version is lower than the client's API version. This PR updates the client initialization logic to ensure that the server's API version is not lower than client's API version.

For example, with Docker engine 17.12 (which has API version 1.35), sdkclientfactory.NewFactory is able to detect that clients with API version >1.35 are not supported and prints the log below.

[Info] Unable to get Docker client for version 1.36: Error response from daemon: client version 1.36 is too new. Maximum supported API version is 1.35

However, this does not happen with Docker engine 17.03 and sdkclientfactory.NewFactory does not detect newer client versions as unsupported. With the change in this PR, unsupported client versions are detected and the following log is printed.

[Info] Unable to get Docker client for version 1.28: server's API version is lower at 1.27

Testing

In addition to new and updated unit tests, I performed the following manual test.

Added the following temporary test to initialize an sdkclientfactory.Factory and print the number of supported and known API versions.

func TestTemp(t *testing.T) {
    f := NewFactory(context.Background(), "unix:///var/run/docker.sock")
    supported := f.FindSupportedAPIVersions()
    fmt.Println("supported length", len(supported))
    fmt.Println("known length", len(dockerclient.GetKnownAPIVersions()))
}

Without the changes in this PR, in an environment with Docker engine 17.03, the test prints the following output.

$ docker version
Client:
 Version:      17.03.1-ce
 API version:  1.27
 Go version:   go1.7.5
 Git commit:   7392c3b/17.03.1-ce
 Built:        Tue May 30 17:59:44 2017
 OS/Arch:      linux/amd64

Server:
 Version:      17.03.1-ce
 API version:  1.27 (minimum version 1.12)
 Go version:   go1.7.5
 Git commit:   7392c3b/17.03.1-ce
 Built:        Tue May 30 17:59:44 2017
 OS/Arch:      linux/amd64
 Experimental: false

$ go test -tags unit -count 1 -run TestTemp -v ./dockerclient/sdkclientfactory
=== RUN   TestTemp
supported length 28
known length 28
--- PASS: TestTemp (0.01s)
PASS
ok  	github.com/aws/amazon-ecs-agent/agent/dockerclient/sdkclientfactory	0.017s

As can be seen from the test output, no unsupported API versions were detected and the count of known and supported API versions is the same at 28.

The behavior is different on an environment with Docker engine 17.12 as unsupported versions are detected.

$ docker version
Client:
 Version:	17.12.1-ce
 API version:	1.35
 Go version:	go1.9.4
 Git commit:	3dfb8343b139d6342acfd9975d7f1068b5b1c3d3
 Built:	Tue Apr  3 23:37:44 2018
 OS/Arch:	linux/amd64

Server:
 Engine:
  Version:	17.12.1-ce
  API version:	1.35 (minimum version 1.12)
  Go version:	go1.9.4
  Git commit:	7390fc6/17.12.1-ce
  Built:	Tue Apr  3 23:38:52 2018
  OS/Arch:	linux/amd64
  Experimental:	false

$ go test -tags unit -count 1 -run TestTemp -v ./dockerclient/sdkclientfactory
=== RUN   TestTemp
1713214846856658786 [Info] Unable to get Docker client for version 1.36: Error response from daemon: client version 1.36 is too new. Maximum supported API version is 1.35
1713214846857237689 [Info] Unable to get Docker client for version 1.37: Error response from daemon: client version 1.37 is too new. Maximum supported API version is 1.35
1713214846857680024 [Info] Unable to get Docker client for version 1.38: Error response from daemon: client version 1.38 is too new. Maximum supported API version is 1.35
1713214846858262244 [Info] Unable to get Docker client for version 1.39: Error response from daemon: client version 1.39 is too new. Maximum supported API version is 1.35
1713214846858771809 [Info] Unable to get Docker client for version 1.40: Error response from daemon: client version 1.40 is too new. Maximum supported API version is 1.35
1713214846859300560 [Info] Unable to get Docker client for version 1.41: Error response from daemon: client version 1.41 is too new. Maximum supported API version is 1.35
1713214846859856829 [Info] Unable to get Docker client for version 1.42: Error response from daemon: client version 1.42 is too new. Maximum supported API version is 1.35
1713214846860340689 [Info] Unable to get Docker client for version 1.43: Error response from daemon: client version 1.43 is too new. Maximum supported API version is 1.35
1713215523176512865 [Info] Unable to get Docker client for version 1.44: Error response from daemon: client version 1.44 is too new. Maximum supported API version is 1.35
supported length 19
known length 28
--- PASS: TestTemp (0.01s)
PASS
ok  	github.com/aws/amazon-ecs-agent/agent/dockerclient/sdkclientfactory	0.019s

With the changes in this PR, in an environment with Docker engine 17.03, the test prints the following output.

$ docker version
Client:
 Version:      17.03.1-ce
 API version:  1.27
 Go version:   go1.7.5
 Git commit:   7392c3b/17.03.1-ce
 Built:        Tue May 30 17:59:44 2017
 OS/Arch:      linux/amd64

Server:
 Version:      17.03.1-ce
 API version:  1.27 (minimum version 1.12)
 Go version:   go1.7.5
 Git commit:   7392c3b/17.03.1-ce
 Built:        Tue May 30 17:59:44 2017
 OS/Arch:      linux/amd64
 Experimental: false
$ go test -tags unit -count 1 -run TestTemp -v ./dockerclient/sdkclientfactory
=== RUN   TestTemp
1713215663563111370 [Info] Unable to get Docker client for version 1.28: server's API version is lower at 1.27
1713215663563667606 [Info] Unable to get Docker client for version 1.29: server's API version is lower at 1.27
1713215663564209334 [Info] Unable to get Docker client for version 1.30: server's API version is lower at 1.27
1713215663564736273 [Info] Unable to get Docker client for version 1.31: server's API version is lower at 1.27
1713215663565224162 [Info] Unable to get Docker client for version 1.32: server's API version is lower at 1.27
1713215663565622049 [Info] Unable to get Docker client for version 1.33: server's API version is lower at 1.27
1713215663566019987 [Info] Unable to get Docker client for version 1.34: server's API version is lower at 1.27
1713215663566379721 [Info] Unable to get Docker client for version 1.35: server's API version is lower at 1.27
1713215663566766184 [Info] Unable to get Docker client for version 1.36: server's API version is lower at 1.27
1713215663567165491 [Info] Unable to get Docker client for version 1.37: server's API version is lower at 1.27
1713215663567640368 [Info] Unable to get Docker client for version 1.38: server's API version is lower at 1.27
1713215663568085713 [Info] Unable to get Docker client for version 1.39: server's API version is lower at 1.27
1713215663568686781 [Info] Unable to get Docker client for version 1.40: server's API version is lower at 1.27
1713215663569153313 [Info] Unable to get Docker client for version 1.41: server's API version is lower at 1.27
1713215663569644143 [Info] Unable to get Docker client for version 1.42: server's API version is lower at 1.27
1713215663570087936 [Info] Unable to get Docker client for version 1.43: server's API version is lower at 1.27
supported length 11
known length 28
1713215663570558683 [Info] Unable to get Docker client for version 1.44: server's API version is lower at 1.27
--- PASS: TestTemp (1.01s)
PASS

As is seen in the logs above, unsupported API versions are now detected.

There is no difference in behavior in environment with Docker engine 17.12.

[ec2-user@ip-172-31-5-155 agent]$ go test -tags unit -count 1 -run TestTemp -v ./dockerclient/sdkclientfactory
=== RUN   TestTemp
1713221182349481476 [Info] Unable to get Docker client for version 1.36: Error response from daemon: client version 1.36 is too new. Maximum supported API version is 1.35
1713221182349986840 [Info] Unable to get Docker client for version 1.37: Error response from daemon: client version 1.37 is too new. Maximum supported API version is 1.35
1713221182350493402 [Info] Unable to get Docker client for version 1.38: Error response from daemon: client version 1.38 is too new. Maximum supported API version is 1.35
1713221182351057696 [Info] Unable to get Docker client for version 1.39: Error response from daemon: client version 1.39 is too new. Maximum supported API version is 1.35
1713221182351545589 [Info] Unable to get Docker client for version 1.40: Error response from daemon: client version 1.40 is too new. Maximum supported API version is 1.35
1713221182352005426 [Info] Unable to get Docker client for version 1.41: Error response from daemon: client version 1.41 is too new. Maximum supported API version is 1.35
1713221182352503717 [Info] Unable to get Docker client for version 1.42: Error response from daemon: client version 1.42 is too new. Maximum supported API version is 1.35
1713221182353035227 [Info] Unable to get Docker client for version 1.43: Error response from daemon: client version 1.43 is too new. Maximum supported API version is 1.35
supported length 19
known length 28
1713221182353520542 [Info] Unable to get Docker client for version 1.44: Error response from daemon: client version 1.44 is too new. Maximum supported API version is 1.35
--- PASS: TestTemp (1.01s)
PASS

New tests cover the changes: yes

Description for the changelog

Bugfix: Fail early when an unsupported docker API version is detected

Does this PR include breaking model changes? If so, Have you added transformation functions?

Licensing

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.