V1.14.5 release

.gitignore

@@ -13,3 +13,5 @@ _bin/
 *.sublime-*
 .DS_Store
 /misc/windows-iam/devcon*
+/misc/pause-container/pause
+*-stamp

.gitmodules

@@ -0,0 +1,3 @@
+[submodule "amazon-ecs-cni-plugins"]
+	path = amazon-ecs-cni-plugins
+	url = https://github.com/aws/amazon-ecs-cni-plugins.git

CHANGELOG.md

@@ -1,5 +1,20 @@
 # Changelog
 
+## UNRELEASED
+* Feature - Support for provisioning Tasks with ENIs
+
+## 1.14.5
+* Enhancement - Retry failed container image pull operations [#975](https://github.com/aws/amazon-ecs-agent/pull/975)
+* Enhancement - Set read and write timeouts for websocket connectons [#993](https://github.com/aws/amazon-ecs-agent/pull/993)
+* Enhancement - Add support for the SumoLogic Docker log driver plugin
+  [#992](https://github.com/aws/amazon-ecs-agent/pull/992) 
+* Bug - Fixed a memory leak issue when submitting the task state change [#967](https://github.com/aws/amazon-ecs-agent/pull/967)
+* Bug - Fixed a race condition where a container can be created twice when agent restarts. [#939](https://github.com/aws/amazon-ecs-agent/pull/939)
+* Bug - Fixed an issue where `microsoft/windowsservercore:latest` was not
+  pulled on Windows under certain conditions.
+  [#990](https://github.com/aws/amazon-ecs-agent/pull/990)
+* Bug - Fixed an issue where task IAM role credentials could be logged to disk. [#998](https://github.com/aws/amazon-ecs-agent/pull/998)
+
 ## 1.14.4
 * Enhancement - Batch container state change events. [#867](https://github.com/aws/amazon-ecs-agent/pull/867)
 * Enhancement - Improve the error message when reserved memory is larger than the available memory. [#897](https://github.com/aws/amazon-ecs-agent/pull/897)
@@ -8,14 +23,14 @@
 * Enhancement - Allow instance attributes to be provided from config file
   by [@ejholmes](https://github.com/ejholmes). [#908](https://github.com/aws/amazon-ecs-agent/pull/908)
 * Enhancement - Reduce the disconnection period to the backend for idle connections. [#912](https://github.com/aws/amazon-ecs-agent/pull/912)
-* Bug - Fix data race where a pointer was returned in Getter. [#889](https://github.com/aws/amazon-ecs-agent/pull/899)
+* Bug - Fixed data race where a pointer was returned in Getter. [#889](https://github.com/aws/amazon-ecs-agent/pull/899)
 * Bug - Reset agent state if the instance id changed on agent restart. [#892](https://github.com/aws/amazon-ecs-agent/pull/892)
-* Bug - Fix a situation in which containers may be falsely reported as STOPPED
+* Bug - Fixed a situation in which containers may be falsely reported as STOPPED
   in the case of a Docker "stop" API failure. [#910](https://github.com/aws/amazon-ecs-agent/pull/910)
-* Bug - Fix typo in log string by [@sharuzzaman](https://github.com/sharuzzaman). [#930](https://github.com/aws/amazon-ecs-agent/pull/930)
+* Bug - Fixed typo in log string by [@sharuzzaman](https://github.com/sharuzzaman). [#930](https://github.com/aws/amazon-ecs-agent/pull/930)
 
 ## 1.14.3
-* Bug - Fix a deadlock that was caused by the ImageCleanup and Image Pull. [#836](https://github.com/aws/amazon-ecs-agent/pull/836)
+* Bug - Fixed a deadlock that was caused by the ImageCleanup and Image Pull. [#836](https://github.com/aws/amazon-ecs-agent/pull/836)
 
 ## 1.14.2
 * Enhancement - Added introspection API for querying tasks by short docker ID, by [@aaronwalker](https://github.com/aaronwalker). [#813](https://github.com/aws/amazon-ecs-agent/pull/813)

LICENSE

@@ -22,6 +22,8 @@ AWS SDK for Go, Copyright 2014-2015 Stripe, Inc. and Amazon.com, Inc. or its aff
 libcontainer, Copyright 2014 Docker, Inc.
 uuid, Copyright (c) 2009,2014 Google Inc.
 runc, Copyright 2012-2015 Docker, Inc.
+Kubernetes, Copyright 2016 The Kubernetes Authors.
+kubernetes/build/pause Copyright 2016 The Kubernetes Authors.
 
 each of which are licensed under the Apache License, Version 2.0 (the
 "License"); you may not use this file except in compliance with the
@@ -62,7 +64,7 @@ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
 3. You must retain, in the Source form of any Derivative Works that You distribute, all copyright, patent, trademark, and attribution notices from the Source form of the Work, excluding those notices that do not pertain to any part of the Derivative Works; and
 4. If the Work includes a "NOTICE" text file as part of its distribution, then any Derivative Works that You distribute must include a readable copy of the attribution notices contained within such NOTICE file, excluding those notices that do not pertain to any part of the Derivative Works, in at least one of the following places: within a NOTICE text file distributed as part of the Derivative Works; within the Source form or documentation, if provided along with the Derivative Works; or, within a display generated by the Derivative Works, if and wherever such third-party notices normally appear. The contents of the NOTICE file are for informational purposes only and do not modify the License. You may add Your own attribution notices within Derivative Works that You distribute, alongside or as an addendum to the NOTICE text from the Work, provided that such additional attribution notices cannot be construed as modifying the License.
 
-You may add Your own copyright statement to Your modifications and may provide additional or different license terms and conditions for use, reproduction, or distribution of Your modifications, or for any such Derivative Works as a whole, provided Your use, reproduction, and distribution of the Work otherwise complies with the conditions stated in this License. 
+You may add Your own copyright statement to Your modifications and may provide additional or different license terms and conditions for use, reproduction, or distribution of Your modifications, or for any such Derivative Works as a whole, provided Your use, reproduction, and distribution of the Work otherwise complies with the conditions stated in this License.
 5. Submission of Contributions. Unless You explicitly state otherwise, any Contribution intentionally submitted for inclusion in the Work by You to the Licensor shall be under the terms and conditions of this License, without any additional terms or conditions. Notwithstanding the above, nothing herein shall supersede or modify the terms of any separate license agreement you may have executed with Licensor regarding such Contributions.
 6. Trademarks. This License does not grant permission to use the trade names, trademarks, service marks, or product names of the Licensor, except as required for reasonable and customary use in describing the origin of the Work and reproducing the content of the NOTICE file.
 7. Disclaimer of Warranty. Unless required by applicable law or agreed to in writing, Licensor provides the Work (and each Contributor provides its Contributions) on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied, including, without limitation, any warranties or conditions of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A PARTICULAR PURPOSE. You are solely responsible for determining the appropriateness of using or redistributing the Work and assume any risks associated with Your exercise of permissions under this License.
@@ -185,9 +187,9 @@ THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
 OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 
 ***
-Golang archive/tar, Copyright (c) 2012 The Go Authors 
-Golang net, Copyright (c) 2009 The Go Authors 
-Golang x/sys, Copyright (c) 2009 The Go Authors. 
+Golang archive/tar, Copyright (c) 2012 The Go Authors
+Golang net, Copyright (c) 2009 The Go Authors
+Golang x/sys, Copyright (c) 2009 The Go Authors.
 
 each of which are licensed under the following terms and conditions:
 
@@ -699,3 +701,25 @@ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
 LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
 OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
 SOFTWARE.
+
+***
+github.com/pkg/errors
+Copyright (c) 2015, Dave Cheney <[email protected]>
+All rights reserved.
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions are met:
+* Redistributions of source code must retain the above copyright notice, this
+  list of conditions and the following disclaimer.
+* Redistributions in binary form must reproduce the above copyright notice,
+  this list of conditions and the following disclaimer in the documentation
+  and/or other materials provided with the distribution.
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
+AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
+IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
+FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
+DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
+SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
+CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
+OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

Makefile

@@ -11,7 +11,18 @@
 # ANY KIND, either express or implied. See the License for the specific
 # language governing permissions and limitations under the License.
 
-.PHONY: all gobuild static docker release certs test clean netkitten test-registry run-functional-tests gremlin benchmark-test gogenerate run-integ-tests image-cleanup-test-images
+PAUSE_CONTAINER_IMAGE = "amazon/amazon-ecs-pause"
+PAUSE_CONTAINER_TAG = "0.1.0"
+PAUSE_CONTAINER_TARBALL = "amazon-ecs-pause.tar"
+
+# Variable to determine branch/tag of amazon-ecs-cni-plugins
+ECS_CNI_REPOSITORY_REVISION=master
+
+# Variable to override cni repository location
+ECS_CNI_REPOSITORY_SRC_DIR=$(PWD)/amazon-ecs-cni-plugins
+
+
+.PHONY: all gobuild static docker release certs test clean netkitten test-registry run-functional-tests gremlin benchmark-test gogenerate run-integ-tests image-cleanup-test-images pause-container get-cni-sources cni-plugins
 
 all: docker
 
@@ -28,24 +39,32 @@ static:
 # directory
 build-in-docker:
 	@docker build -f scripts/dockerfiles/Dockerfile.build -t "amazon/amazon-ecs-agent-build:make" .
-	@docker run --net=none -e TARGET_OS="${TARGET_OS}" \
-	  -v "$(shell pwd)/out:/out" \
-	  -v "$(shell pwd):/go/src/github.com/aws/amazon-ecs-agent" \
+	@docker run --net=none \
+	  -e TARGET_OS="${TARGET_OS}" \
+	  -e LDFLAGS="-X github.com/aws/amazon-ecs-agent/agent/config.DefaultPauseContainerTag=$(PAUSE_CONTAINER_TAG) \
+	  -X github.com/aws/amazon-ecs-agent/agent/config.DefaultPauseContainerImageName=$(PAUSE_CONTAINER_IMAGE)" \
+	  -v "$(PWD)/out:/out" \
+	  -v "$(PWD):/go/src/github.com/aws/amazon-ecs-agent" \
 	  "amazon/amazon-ecs-agent-build:make"
 
 # 'docker' builds the agent dockerfile from the current sourcecode tree, dirty
 # or not
-docker: certs build-in-docker
+docker: certs build-in-docker pause-container-release cni-plugins
 	@cd scripts && ./create-amazon-ecs-scratch
 	@docker build -f scripts/dockerfiles/Dockerfile.release -t "amazon/amazon-ecs-agent:make" .
 	@echo "Built Docker image \"amazon/amazon-ecs-agent:make\""
 
 # 'docker-release' builds the agent from a clean snapshot of the git repo in
 # 'RELEASE' mode
-docker-release:
+docker-release: pause-container-release cni-plugins
 	@docker build -f scripts/dockerfiles/Dockerfile.cleanbuild -t "amazon/amazon-ecs-agent-cleanbuild:make" .
-	@docker run --net=none -e TARGET_OS="${TARGET_OS}" -v "$(shell pwd)/out:/out" \
-	  -v "$(shell pwd):/src/amazon-ecs-agent" "amazon/amazon-ecs-agent-cleanbuild:make"
+	@docker run --net=none \
+	  -e TARGET_OS="${TARGET_OS}" \
+	  -v "$(PWD)/out:/out" \
+	  -e LDFLAGS="-X github.com/aws/amazon-ecs-agent/agent/config.DefaultPauseContainerTag=$(PAUSE_CONTAINER_TAG) \
+	  -X github.com/aws/amazon-ecs-agent/agent/config.DefaultPauseContainerImageName=$(PAUSE_CONTAINER_IMAGE)" \
+	  -v "$(PWD):/src/amazon-ecs-agent" \
+	  "amazon/amazon-ecs-agent-cleanbuild:make"
 
 # Release packages our agent into a "scratch" based dockerfile
 release: certs docker-release
@@ -65,6 +84,9 @@ misc/certs/ca-certificates.crt:
 test:
 	. ./scripts/shared_env && go test -race -timeout=25s -v -cover $(shell go list ./agent/... | grep -v /vendor/)
 
+test-silent:
+	. ./scripts/shared_env && go test -timeout=25s -cover $(shell go list ./agent/... | grep -v /vendor/)
+
 benchmark-test:
 	. ./scripts/shared_env && go test -run=XX -bench=. $(shell go list ./agent/... | grep -v /vendor/)
 
@@ -75,56 +97,88 @@ test-registry: netkitten volumes-test squid awscli image-cleanup-test-images flu
 test-in-docker:
 	docker build -f scripts/dockerfiles/Dockerfile.test -t "amazon/amazon-ecs-agent-test:make" .
 	# Privileged needed for docker-in-docker so integ tests pass
-	docker run -v "$(shell pwd):/go/src/github.com/aws/amazon-ecs-agent" --privileged "amazon/amazon-ecs-agent-test:make"
+	docker run --net=none -v "$(PWD):/go/src/github.com/aws/amazon-ecs-agent" --privileged "amazon/amazon-ecs-agent-test:make"
 
 run-functional-tests: testnnp test-registry
 	. ./scripts/shared_env && go test -tags functional -timeout=30m -v ./agent/functional_tests/...
 
 testnnp:
-	cd misc/testnnp; $(MAKE) $(MFLAGS)
+	$(MAKE) -C misc/testnnp $(MFLAGS)
+
+pause-container:
+	@docker build -f scripts/dockerfiles/Dockerfile.buildPause -t "amazon/amazon-ecs-build-pause-bin:make" .
+	@docker run --net=none \
+		-v "$(PWD)/misc/pause-container:/out" \
+		-v "$(PWD)/misc/pause-container/buildPause:/usr/src/buildPause" \
+		"amazon/amazon-ecs-build-pause-bin:make"
+
+	$(MAKE) -C misc/pause-container $(MFLAGS)
+	@docker rmi -f "amazon/amazon-ecs-build-pause-bin:make"
+
+pause-container-release: pause-container
+	mkdir -p "$(PWD)/out"
+	@docker save ${PAUSE_CONTAINER_IMAGE}:${PAUSE_CONTAINER_TAG} > "$(PWD)/out/${PAUSE_CONTAINER_TARBALL}"
+
+get-cni-sources:
+	git submodule update --init --checkout
+
+cni-plugins: get-cni-sources
+	@docker build -f scripts/dockerfiles/Dockerfile.buildCNIPlugins -t "amazon/amazon-ecs-build-cniplugins:make" .
+	mkdir -p out/cni-plugins
+	docker run --rm --net=none \
+		-e GIT_SHORT_HASH=$(shell cd $(ECS_CNI_REPOSITORY_SRC_DIR) && git rev-parse --short HEAD) \
+		-e GIT_PORCELAIN=$(shell cd $(ECS_CNI_REPOSITORY_SRC_DIR) && git status --porcelain 2> /dev/null | wc -l | sed 's/^ *//') \
+		-v "$(PWD)/out/cni-plugins:/go/src/github.com/aws/amazon-ecs-cni-plugins/bin/plugins" \
+		-v "$(ECS_CNI_REPOSITORY_SRC_DIR):/go/src/github.com/aws/amazon-ecs-cni-plugins" \
+		"amazon/amazon-ecs-build-cniplugins:make"
+	@echo "Built amazon-ecs-cni-plugins successfully."
 
 run-integ-tests: test-registry gremlin
 	. ./scripts/shared_env && go test -race -tags integration -timeout=5m -v ./agent/engine/... ./agent/stats/... ./agent/app/...
 
 netkitten:
-	cd misc/netkitten; $(MAKE) $(MFLAGS)
+	$(MAKE) -C misc/netkitten $(MFLAGS)
 
 volumes-test:
-	cd misc/volumes-test; $(MAKE) $(MFLAGS)
+	$(MAKE) -C misc/volumes-test $(MFLAGS)
 
 # TODO, replace this with a build on dockerhub or a mechanism for the
 # functional tests themselves to build this
 .PHONY: squid awscli fluentd
 squid:
-	cd misc/squid; $(MAKE) $(MFLAGS)
+	$(MAKE) -C misc/squid $(MFLAGS)
 
 gremlin:
-	cd misc/gremlin; $(MAKE) $(MFLAGS)
+	$(MAKE) -C misc/gremlin $(MFLAGS)
 
 awscli:
-	cd misc/awscli; $(MAKE) $(MFLAGS)
+	$(MAKE) -C misc/awscli $(MFLAGS)
 
 fluentd:
-	cd misc/fluentd; $(MAKE) $(MFLAGS)
+	$(MAKE) -C misc/fluentd $(MFLAGS)
 
 image-cleanup-test-images:
-	cd misc/image-cleanup-test-images; $(MAKE) $(MFLAGS)
+	$(MAKE) -C misc/image-cleanup-test-images $(MFLAGS)
 
-get-deps:
+.get-deps-stamp:
 	go get github.com/tools/godep
 	go get golang.org/x/tools/cmd/cover
 	go get github.com/golang/mock/mockgen
 	go get golang.org/x/tools/cmd/goimports
+	touch .get-deps-stamp
 
+get-deps: .get-deps-stamp
 
 clean:
-        # ensure docker is running and we can talk to it, abort if not:
+	# ensure docker is running and we can talk to it, abort if not:
 	docker ps > /dev/null
 	rm -f misc/certs/ca-certificates.crt &> /dev/null
-	rm -f out/amazon-ecs-agent out/amazon-ecs-agent.exe &> /dev/null
+	rm -rf out/*
+	$(MAKE) -C $(ECS_CNI_REPOSITORY_SRC_DIR) clean
 	rm -rf agent/Godeps/_workspace/pkg/
 	-$(MAKE) -C misc/netkitten $(MFLAGS) clean
 	-$(MAKE) -C misc/volumes-test $(MFLAGS) clean
 	-$(MAKE) -C misc/gremlin $(MFLAGS) clean
 	-$(MAKE) -C misc/testnnp $(MFLAGS) clean
 	-$(MAKE) -C misc/image-cleanup-test-images $(MFLAGS) clean
+	-rm -f .get-deps-stamp

NOTICE

@@ -26,3 +26,6 @@ violate applicable laws.
 For more information, please see https://www.bis.doc.gov
 
 See also https://www.apache.org/dev/crypto.html and/or seek legal counsel.
+
+Kubernetes, Copyright 2016 The Kubernetes Authors.
+kubernetes/build/pause Copyright 2016 The Kubernetes Authors.

README.md

@@ -175,6 +175,10 @@ configure them as something other than the defaults.
 | `ECS_IMAGE_MINIMUM_CLEANUP_AGE` | 30m | The minimum time interval between when an image is pulled and when it can be considered for automated image cleanup. | 1h | 1h |
 | `ECS_NUM_IMAGES_DELETE_PER_CYCLE` | 5 | The maximum number of images to delete in a single automated image cleanup cycle. If set to less than 1, the value is ignored. | 5 | 5 |
 | `ECS_INSTANCE_ATTRIBUTES` | `{"stack": "prod"}` | These attributes take effect only during initial registration. After the agent has joined an ECS cluster, use the PutAttributes API action to add additional attributes. For more information, see [Amazon ECS Container Agent Configuration](http://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs-agent-config.html) in the Amazon ECS Developer Guide.| `{}` | `{}` |
+| `ECS_ENABLE_TASK_ENI` | `false` | Whether to enable task networking for task to be launched with its own network interface | `false` | Not applicable |
+| `ECS_CNI_PLUGINS_PATH` | `/ecs/cni` | The path where the cni binary file is located | `/amazon-ecs-cni-plugins` | Not applicable |
+| `ECS_AWSVPC_BLOCK_IMDS` | `true` | Whether to block access to [Instance Metdata](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-metadata.html) for Tasks started with `awsvpc` network mode | `false` | Not applicable |
+| `ECS_AWSVPC_ADDITIONAL_LOCAL_ROUTES` | `["10.0.15.0/24"]` | In `awsvpc` network mode, traffic to these prefixes will be routed via the host bridge instead of the task ENI | `[]` | Not applicable |
 
 ### Persistence
 

VERSION

@@ -1 +1 @@
-1.14.4
+1.14.5