feat(ecs): support private registry authentication (#1737)

Non-ECR registries now accept a secretsmanager Secret with a username and password to use for registry authentication.

Fixes #1698.

BREAKING CHANGE: `ContainerImage.fromDockerHub` has been renamed to `ContainerImage.fromRegistry`.

design/aws-ecs-priv-registry-support.md

@@ -0,0 +1,125 @@
+# AWS ECS - Support for Private Registry Authentication
+
+To address issue [#1698](https://github.com/awslabs/aws-cdk/issues/1698), the ECS construct library should provide a way for customers to specify [`repositoryCredentials`](https://docs.aws.amazon.com/AmazonECS/latest/APIReference/API_ContainerDefinition.html#ECS-Type-ContainerDefinition-repositoryCredentials) on their container. 
+
+Minimally, this would mean adding a new string field on `ContainerDefinition`, however this doesn't provide any added value in terms of logical grouping or resource creation. We can instead modify the existing ECS CDK construct [`ContainerImage`](https://github.com/awslabs/aws-cdk/blob/master/packages/%40aws-cdk/aws-ecs/lib/container-image.ts) so that repository credentials are specified along with the image they're meant to access.
+
+## General approach
+
+The [`ecs.ContainerImage`](https://github.com/awslabs/aws-cdk/blob/master/packages/%40aws-cdk/aws-ecs/lib/container-image.ts) class already includes constructs for 3 types of images:
+
+* DockerHubImage
+* EcrImage
+* AssetImage
+
+DockerHub images are assumed public, however DockerHub also provides private repositories and there's currently no way to specify credentials for them in the CDK.
+
+There's also no explicit way to specify images hosted outside of DockerHub, AWS, or your local machine. Customers hosting their own registries or using another registry host, like Quay.io or JFrog Artifactory, would need to be able to specify both the image URI and the registry credentials in order to pull their images down for ECS tasks.
+
+Fundamentally, specifying images hosted in DockerHub or elsewhere works the same, because when passed an image URI vs. a plain (or namespaced) image name + tag, the Docker daemon does the right thing and tries to pull the image from the specified registery. 
+
+Therefore, we should rename the existing `DockerHubImage` type be more generic and add the ability to optionally specify credentials.
+
+
+## Code changes
+
+Given the above, we should make the following changes to support private registry authentication:
+
+1. Define `RepositoryCredentials` interface & class, add to `IContainerImage` 
+2. Rename `DockerHubImage` construct to be more generic, optionally accept and set `RepositoryCreds` 
+
+
+# Part 1: How to define registry credentials 
+
+For extensibility, we can define a new `IRepositoryCreds` interface to house the AWS Secrets Manager secret with the creds and a new `RepositoryCreds` class which satisfies it using specific constructs (e.g., "fromSecret").
+
+```ts
+export interface IRepositoryCreds {
+    readonly secret: secretsManager.Secret;
+}
+
+export class RepositoryCreds {
+  public readonly secret: secretsManager.Secret;
+
+  public static fromSecret(secret: secretsManager.Secret) {
+    this.secret = secret;
+  }
+
+  public bind(containerDefinition: ContainerDefinition): void {
+    // grant the execution role read access so the secret can be read prior to image pull
+    this.secret.grantRead(containerDefinition.taskDefinition.obtainExecutionRole());
+  }
+}
+```
+
+The `IContainerImage` interface will be updated to include an optional `repositoryCredentials` property:
+```ts
+export interface IContainerImage {
+  // ...
+  readonly imageName: string;
+
+  /**
+   * NEW: The credentials required to access the image
+   */
+  readonly repositoryCredentials?: IRepositoryCreds;
+
+  // ...
+  bind(containerDefinition: ContainerDefinition): void;
+}
+```
+
+
+# Part 2: Rename `DockerHubImage` class to `WebHostedImage`, add optional creds
+
+The `DockerHubImage` construct will be renamed to `WebHostedImage`, and augmented to take in optional "credentials" via keyword props:
+```ts
+// define props
+export interface WebHostedImageProps {
+    credentials: RepositoryCreds;
+}
+
+// "DockerHubImage" class --> "WebHostedImage"
+export class WebHostedImage implements IContainerImage {
+  public readonly imageName: string;
+  public readonly credentials: IRepositoryCreds;
+
+  // add credentials to constructor
+  constructor(imageName: string, props: WebHostedImageProps) {
+    this.imageName = imageName
+    this.credentials = props.credentials
+  }
+
+  public bind(_containerDefinition: ContainerDefinition): void {
+    // bind repositoryCredentials to ContainerDefinition
+    this.repositoryCredentials.bind();
+  }
+}
+```
+
+We will also update the API on `ContainerImage` to match:
+```ts
+export class ContainerImage {
+  //...
+  public static fromInternet(imageName: string, props: WebHostedImageProps) {
+    return new WebHostedImage(imageName, props);
+  }
+  // ...
+
+}
+```
+
+Example use:
+```ts
+const taskDefinition = new ecs.Ec2TaskDefinition(stack, 'TaskDef');
+
+const secret = secretsManager.Secret.import(stack, 'myRepoSecret', {
+  secretArn: 'arn:aws:secretsmanager:.....'
+})
+
+taskDefinition.AddContainer('myPrivateContainer', {
+  image: ecs.ContainerImage.fromInternet('userx/test', {
+      credentials: ecs.RepositoryCreds.fromSecret(secret)
+  });
+});
+
+```

packages/@aws-cdk/aws-ecs/README.md

@@ -30,7 +30,7 @@ cluster.addDefaultAutoScalingGroupCapacity('Capacity', {
 const ecsService = new ecs.LoadBalancedEc2Service(this, 'Service', {
   cluster,
   memoryLimitMiB: 512,
-  image: ecs.ContainerImage.fromDockerHub("amazon/amazon-ecs-sample"),
+  image: ecs.ContainerImage.fromRegistry("amazon/amazon-ecs-sample"),
 });
 ```
 
@@ -134,7 +134,7 @@ To add containers to a task definition, call `addContainer()`:
 ```ts
 const container = fargateTaskDefinition.addContainer("WebContainer", {
   // Use an image from DockerHub
-  image: ecs.ContainerImage.fromDockerHub("amazon/amazon-ecs-sample"),
+  image: ecs.ContainerImage.fromRegistry("amazon/amazon-ecs-sample"),
   // ... other options here ...
 });
 ```
@@ -148,7 +148,7 @@ const ec2TaskDefinition = new ecs.Ec2TaskDefinition(this, 'TaskDef', {
 
 const container = ec2TaskDefinition.addContainer("WebContainer", {
   // Use an image from DockerHub
-  image: ecs.ContainerImage.fromDockerHub("amazon/amazon-ecs-sample"),
+  image: ecs.ContainerImage.fromRegistry("amazon/amazon-ecs-sample"),
   memoryLimitMiB: 1024
   // ... other options here ...
 });
@@ -183,8 +183,8 @@ const taskDefinition = new ecs.TaskDefinition(this, 'TaskDef', {
 Images supply the software that runs inside the container. Images can be
 obtained from either DockerHub or from ECR repositories, or built directly from a local Dockerfile.
 
-* `ecs.ContainerImage.fromDockerHub(imageName)`: use a publicly available image from
-  DockerHub.
+* `ecs.ContainerImage.fromRegistry(imageName)`: use a public image.
+* `ecs.ContainerImage.fromRegistry(imageName, { credentials: mySecret })`: use a private image that requires credentials.
 * `ecs.ContainerImage.fromEcrRepository(repo, tag)`: use the given ECR repository as the image
   to start. If no tag is provided, "latest" is assumed.
 * `ecs.ContainerImage.fromAsset(this, 'Image', { directory: './image' })`: build and upload an

packages/@aws-cdk/aws-ecs/lib/container-definition.ts

@@ -379,7 +379,7 @@ export class ContainerDefinition extends cdk.Construct {
       portMappings: this.portMappings.map(renderPortMapping),
       privileged: this.props.privileged,
       readonlyRootFilesystem: this.props.readonlyRootFilesystem,
-      repositoryCredentials: undefined, // FIXME
+      repositoryCredentials: this.props.image.toRepositoryCredentialsJson(),
       ulimits: this.ulimits.map(renderUlimit),
       user: this.props.user,
       volumesFrom: this.volumesFrom.map(renderVolumeFrom),

packages/@aws-cdk/aws-ecs/lib/container-image.ts

@@ -1,17 +1,17 @@
 import ecr = require('@aws-cdk/aws-ecr');
 import cdk = require('@aws-cdk/cdk');
-
 import { ContainerDefinition } from './container-definition';
+import { CfnTaskDefinition } from './ecs.generated';
 
 /**
  * Constructs for types of container images
  */
 export abstract class ContainerImage {
   /**
-   * Reference an image on DockerHub
+   * Reference an image on DockerHub or another online registry
    */
-  public static fromDockerHub(name: string) {
-    return new DockerHubImage(name);
+  public static fromRegistry(name: string, props: RepositoryImageProps = {}) {
+    return new RepositoryImage(name, props);
   }
 
   /**
@@ -37,8 +37,13 @@ export abstract class ContainerImage {
    * Called when the image is used by a ContainerDefinition
    */
   public abstract bind(containerDefinition: ContainerDefinition): void;
+
+  /**
+   * Render the Repository credentials to the CloudFormation object
+   */
+  public abstract toRepositoryCredentialsJson(): CfnTaskDefinition.RepositoryCredentialsProperty | undefined;
 }
 
 import { AssetImage, AssetImageProps } from './images/asset-image';
-import { DockerHubImage } from './images/dockerhub';
 import { EcrImage } from './images/ecr';
+import { RepositoryImage, RepositoryImageProps } from './images/repository';

packages/@aws-cdk/aws-ecs/lib/images/asset-image.ts

@@ -2,6 +2,7 @@ import { DockerImageAsset } from '@aws-cdk/assets-docker';
 import cdk = require('@aws-cdk/cdk');
 import { ContainerDefinition } from '../container-definition';
 import { ContainerImage } from '../container-image';
+import { CfnTaskDefinition } from '../ecs.generated';
 
 export interface AssetImageProps {
   /**
@@ -15,6 +16,7 @@ export interface AssetImageProps {
  */
 export class AssetImage extends ContainerImage {
   private readonly asset: DockerImageAsset;
+
   constructor(scope: cdk.Construct, id: string, props: AssetImageProps) {
     super();
     this.asset = new DockerImageAsset(scope, id, { directory: props.directory });
@@ -24,6 +26,10 @@ export class AssetImage extends ContainerImage {
     this.asset.repository.grantPull(containerDefinition.taskDefinition.obtainExecutionRole());
   }
 
+  public toRepositoryCredentialsJson(): CfnTaskDefinition.RepositoryCredentialsProperty | undefined {
+      return undefined;
+  }
+
   public get imageName() {
     return this.asset.imageUri;
   }

packages/@aws-cdk/aws-ecs/lib/images/ecr.ts

@@ -1,6 +1,7 @@
 import ecr = require('@aws-cdk/aws-ecr');
 import { ContainerDefinition } from '../container-definition';
 import { ContainerImage } from '../container-image';
+import { CfnTaskDefinition } from '../ecs.generated';
 
 /**
  * An image from an ECR repository
@@ -18,4 +19,8 @@ export class EcrImage extends ContainerImage {
   public bind(containerDefinition: ContainerDefinition): void {
     this.repository.grantPull(containerDefinition.taskDefinition.obtainExecutionRole());
   }
+
+  public toRepositoryCredentialsJson(): CfnTaskDefinition.RepositoryCredentialsProperty | undefined {
+    return undefined;
+  }
 }

packages/@aws-cdk/aws-ecs/lib/images/repository.ts

@@ -0,0 +1,39 @@
+import secretsmanager = require('@aws-cdk/aws-secretsmanager');
+import { ContainerDefinition } from "../container-definition";
+import { ContainerImage } from "../container-image";
+import { CfnTaskDefinition } from '../ecs.generated';
+
+export interface RepositoryImageProps {
+    /**
+     * Optional secret that houses credentials for the image registry
+     */
+    credentials?: secretsmanager.ISecret;
+}
+
+/**
+ * A container image hosted on DockerHub or another online registry
+ */
+export class RepositoryImage extends ContainerImage {
+  public readonly imageName: string;
+
+  private credentialsSecret?: secretsmanager.ISecret;
+
+  constructor(imageName: string, props: RepositoryImageProps = {}) {
+    super();
+    this.imageName = imageName;
+    this.credentialsSecret = props.credentials;
+  }
+
+  public bind(containerDefinition: ContainerDefinition): void {
+    if (this.credentialsSecret) {
+      this.credentialsSecret.grantRead(containerDefinition.taskDefinition.obtainExecutionRole());
+    }
+  }
+
+  public toRepositoryCredentialsJson(): CfnTaskDefinition.RepositoryCredentialsProperty | undefined {
+    if (!this.credentialsSecret) { return undefined; }
+    return {
+      credentialsParameter: this.credentialsSecret.secretArn
+    };
+  }
+}

packages/@aws-cdk/aws-ecs/lib/index.ts

@@ -21,7 +21,7 @@ export * from './load-balanced-ecs-service';
 export * from './load-balanced-fargate-service-applet';
 
 export * from './images/asset-image';
-export * from './images/dockerhub';
+export * from './images/repository';
 export * from './images/ecr';
 
 export * from './log-drivers/aws-log-driver';

packages/@aws-cdk/aws-ecs/lib/load-balanced-fargate-service-applet.ts

@@ -132,7 +132,7 @@ export class LoadBalancedFargateServiceApplet extends cdk.Stack {
       memoryMiB: props.memoryMiB,
       publicLoadBalancer: props.publicLoadBalancer,
       publicTasks: props.publicTasks,
-      image: ContainerImage.fromDockerHub(props.image),
+      image: ContainerImage.fromRegistry(props.image),
       desiredCount: props.desiredCount,
       environment: props.environment,
       certificate,

packages/@aws-cdk/aws-ecs/package.json

@@ -79,6 +79,7 @@
     "@aws-cdk/aws-logs": "^0.25.3",
     "@aws-cdk/aws-route53": "^0.25.3",
     "@aws-cdk/aws-sns": "^0.25.3",
+    "@aws-cdk/aws-secretsmanager": "^0.25.3",
     "@aws-cdk/cdk": "^0.25.3",
     "@aws-cdk/cx-api": "^0.25.3"
   },
@@ -97,6 +98,7 @@
     "@aws-cdk/aws-iam": "^0.25.3",
     "@aws-cdk/aws-logs": "^0.25.3",
     "@aws-cdk/aws-route53": "^0.25.3",
+    "@aws-cdk/aws-secretsmanager": "^0.25.3",
     "@aws-cdk/cdk": "^0.25.3"
   },
   "engines": {

packages/@aws-cdk/aws-ecs/test/ec2/integ.lb-awsvpc-nw.ts

@@ -19,7 +19,7 @@ const taskDefinition = new ecs.Ec2TaskDefinition(stack, 'TaskDef', {
 });
 
 const container = taskDefinition.addContainer('web', {
-  image: ecs.ContainerImage.fromDockerHub("amazon/amazon-ecs-sample"),
+  image: ecs.ContainerImage.fromRegistry("amazon/amazon-ecs-sample"),
   memoryLimitMiB: 256,
 });
 

packages/@aws-cdk/aws-ecs/test/ec2/integ.lb-bridge-nw.ts

@@ -21,7 +21,7 @@ const taskDefinition = new ecs.Ec2TaskDefinition(stack, 'TaskDef', {
 });
 
 const container = taskDefinition.addContainer('web', {
-  image: ecs.ContainerImage.fromDockerHub("amazon/amazon-ecs-sample"),
+  image: ecs.ContainerImage.fromRegistry("amazon/amazon-ecs-sample"),
   memoryLimitMiB: 256,
 });
 container.addPortMappings({

packages/@aws-cdk/aws-ecs/test/ec2/test.ec2-event-rule-target.ts

@@ -17,7 +17,7 @@ export = {
 
     const taskDefinition = new ecs.Ec2TaskDefinition(stack, 'TaskDef');
     taskDefinition.addContainer('TheContainer', {
-      image: ecs.ContainerImage.fromDockerHub('henk'),
+      image: ecs.ContainerImage.fromRegistry('henk'),
       memoryLimitMiB: 256
     });