feat(ecs-patterns): remove default desiredCount to align with cfn behaviour (under feature flag) (#13130)

AWS CloudFormation previously required a desiredCount to be specified to create an Amazon ECS Service. They have recently removed that required field, and now provide the following default behaviour, if the desired count is not specified.
* when creating a service, the service is created and starts up with a desired count of 1.
* when updating a service, the service is updated and starts up with the same desired count as it had prior to the deployment.

The AWS-CDK currently adds a default desiredCount of 1 when creating or updating an Amazon ECS service, and this could lead to issues when a service has scaled up due to autoscaling. When updating a service using the AWS-CDK, the service would then be scaled down to the original desiredCount, leading to potential outages.

This change introduces a new feature flag `REMOVE_DEFAULT_DESIRED_COUNT` to flip the default behaviour of setting the desiredCount for any service created using any of the following ECS Patterns constructs. Without the feature flag set, the desiredCount of any service created is automatically defaulted to 1 (which is the current behaviour). With the feature flag set (to true), the default will be removed and the desiredCount will only be passed to the service if it is passed in as input.

* ApplicationLoadBalancedEc2Service
* ApplicationLoadBalancedFargateService
* NetworkLoadBalancedEc2Service
* NetworkLoadBalancedFargateService
* QueueProcessingEc2Service
* QueueProcessingFargateService

**BREAKING CHANGE:** the desiredCount property stored on the above constructs will be optional, allowing them to be undefined. This is enabled through the `@aws-cdk/aws-ecs-patterns:removeDefaultDesiredCount` feature flag. We would recommend all aws-cdk users to set the `REMOVE_DEFAULT_DESIRED_COUNT` flag to true for all of their existing applications. 

Fixes: #12990

----

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*

Author: piradeepk

packages/@aws-cdk/aws-ecs-patterns/README.md

@@ -466,3 +466,41 @@ const scheduledFargateTask = new ScheduledFargateTask(stack, 'ScheduledFargateTa
   platformVersion: ecs.FargatePlatformVersion.VERSION1_4,
 });
 ```
+
+### Use the REMOVE_DEFAULT_DESIRED_COUNT feature flag
+
+The REMOVE_DEFAULT_DESIRED_COUNT feature flag is used to override the default desiredCount that is autogenerated by the CDK. This will set the desiredCount of any service created by any of the following constructs to be undefined. 
+
+* ApplicationLoadBalancedEc2Service
+* ApplicationLoadBalancedFargateService
+* NetworkLoadBalancedEc2Service
+* NetworkLoadBalancedFargateService
+* QueueProcessingEc2Service
+* QueueProcessingFargateService
+
+If a desiredCount is not passed in as input to the above constructs, CloudFormation will either create a new service to start up with a desiredCount of 1, or update an existing service to start up with the same desiredCount as prior to the update.
+
+To enable the feature flag, ensure that the REMOVE_DEFAULT_DESIRED_COUNT flag within an application stack context is set to true, like so:
+
+```ts
+stack.node.setContext(cxapi.ECS_REMOVE_DEFAULT_DESIRED_COUNT, true);
+```
+
+The following is an example of an application with the REMOVE_DEFAULT_DESIRED_COUNT feature flag enabled:
+
+```ts
+const app = new App();
+
+const stack = new Stack(app, 'aws-ecs-patterns-queue');
+stack.node.setContext(cxapi.ECS_REMOVE_DEFAULT_DESIRED_COUNT, true);
+
+const vpc = new ec2.Vpc(stack, 'VPC', {
+  maxAzs: 2,
+});
+
+new QueueProcessingFargateService(stack, 'QueueProcessingService', {
+  vpc,
+  memoryLimitMiB: 512,
+  image: new ecs.AssetImage(path.join(__dirname, '..', 'sqs-reader')),
+});
+```

packages/@aws-cdk/aws-ecs-patterns/lib/base/application-load-balanced-service-base.ts

@@ -78,7 +78,9 @@ export interface ApplicationLoadBalancedServiceBaseProps {
    * The desired number of instantiations of the task definition to keep running on the service.
    * The minimum value is 1
    *
-   * @default 1
+   * @default - If the feature flag, ECS_REMOVE_DEFAULT_DESIRED_COUNT is false, the default is 1;
+   * if true, the default is 1 for all new services and uses the existing services desired count
+   * when updating an existing service.
    */
   readonly desiredCount?: number;
 
@@ -311,12 +313,19 @@ export interface ApplicationLoadBalancedTaskImageOptions {
  * The base class for ApplicationLoadBalancedEc2Service and ApplicationLoadBalancedFargateService services.
  */
 export abstract class ApplicationLoadBalancedServiceBase extends CoreConstruct {
-
   /**
    * The desired number of instantiations of the task definition to keep running on the service.
+   * @deprecated - Use `internalDesiredCount` instead.
    */
   public readonly desiredCount: number;
 
+  /**
+   * The desired number of instantiations of the task definition to keep running on the service.
+   * The default is 1 for all new services and uses the existing services desired count
+   * when updating an existing service if one is not provided.
+   */
+  public readonly internalDesiredCount?: number;
+
   /**
    * The Application Load Balancer for the service.
    */
@@ -368,7 +377,9 @@ export abstract class ApplicationLoadBalancedServiceBase extends CoreConstruct {
     if (props.desiredCount !== undefined && props.desiredCount < 1) {
       throw new Error('You must specify a desiredCount greater than 0');
     }
+
     this.desiredCount = props.desiredCount || 1;
+    this.internalDesiredCount = props.desiredCount;
 
     const internetFacing = props.publicLoadBalancer ?? true;
 

packages/@aws-cdk/aws-ecs-patterns/lib/base/application-multiple-target-groups-service-base.ts

@@ -47,7 +47,9 @@ export interface ApplicationMultipleTargetGroupsServiceBaseProps {
   /**
    * The desired number of instantiations of the task definition to keep running on the service.
    *
-   * @default 1
+   * @default - If the feature flag, ECS_REMOVE_DEFAULT_DESIRED_COUNT is false, the default is 1;
+   * if true, the default is 1 for all new services and uses the existing services desired count
+   * when updating an existing service.
    */
   readonly desiredCount?: number;
 
@@ -329,12 +331,19 @@ export interface ApplicationListenerProps {
  * The base class for ApplicationMultipleTargetGroupsEc2Service and ApplicationMultipleTargetGroupsFargateService classes.
  */
 export abstract class ApplicationMultipleTargetGroupsServiceBase extends CoreConstruct {
-
   /**
    * The desired number of instantiations of the task definition to keep running on the service.
+   * @deprecated - Use `internalDesiredCount` instead.
    */
   public readonly desiredCount: number;
 
+  /**
+   * The desired number of instantiations of the task definition to keep running on the service.
+   * The default is 1 for all new services and uses the existing services desired count
+   * when updating an existing service, if one is not provided.
+   */
+  public readonly internalDesiredCount?: number;
+
   /**
    * The default Application Load Balancer for the service (first added load balancer).
    */
@@ -365,7 +374,10 @@ export abstract class ApplicationMultipleTargetGroupsServiceBase extends CoreCon
     this.validateInput(props);
 
     this.cluster = props.cluster || this.getDefaultCluster(this, props.vpc);
+
     this.desiredCount = props.desiredCount || 1;
+    this.internalDesiredCount = props.desiredCount;
+
     if (props.taskImageOptions) {
       this.logDriver = this.createLogDriver(props.taskImageOptions.enableLogging, props.taskImageOptions.logDriver);
     }

packages/@aws-cdk/aws-ecs-patterns/lib/base/network-load-balanced-service-base.ts

@@ -67,7 +67,9 @@ export interface NetworkLoadBalancedServiceBaseProps {
    * The desired number of instantiations of the task definition to keep running on the service.
    * The minimum value is 1
    *
-   * @default 1
+   * @default - If the feature flag, ECS_REMOVE_DEFAULT_DESIRED_COUNT is false, the default is 1;
+   * if true, the default is 1 for all new services and uses the existing services desired count
+   * when updating an existing service.
    */
   readonly desiredCount?: number;
 
@@ -263,9 +265,17 @@ export interface NetworkLoadBalancedTaskImageOptions {
 export abstract class NetworkLoadBalancedServiceBase extends CoreConstruct {
   /**
    * The desired number of instantiations of the task definition to keep running on the service.
+   * @deprecated - Use `internalDesiredCount` instead.
    */
   public readonly desiredCount: number;
 
+  /**
+   * The desired number of instantiations of the task definition to keep running on the service.
+   * The default is 1 for all new services and uses the existing services desired count
+   * when updating an existing service, if one is not provided.
+   */
+  public readonly internalDesiredCount?: number;
+
   /**
    * The Network Load Balancer for the service.
    */
@@ -306,7 +316,9 @@ export abstract class NetworkLoadBalancedServiceBase extends CoreConstruct {
     if (props.desiredCount !== undefined && props.desiredCount < 1) {
       throw new Error('You must specify a desiredCount greater than 0');
     }
+
     this.desiredCount = props.desiredCount || 1;
+    this.internalDesiredCount = props.desiredCount;
 
     const internetFacing = props.publicLoadBalancer ?? true;
 

packages/@aws-cdk/aws-ecs-patterns/lib/base/network-multiple-target-groups-service-base.ts

@@ -45,7 +45,9 @@ export interface NetworkMultipleTargetGroupsServiceBaseProps {
    * The desired number of instantiations of the task definition to keep running on the service.
    * The minimum value is 1
    *
-   * @default 1
+   * @default - If the feature flag, ECS_REMOVE_DEFAULT_DESIRED_COUNT is false, the default is 1;
+   * if true, the default is 1 for all new services and uses the existing services desired count
+   * when updating an existing service.
    */
   readonly desiredCount?: number;
 
@@ -264,9 +266,17 @@ export interface NetworkTargetProps {
 export abstract class NetworkMultipleTargetGroupsServiceBase extends CoreConstruct {
   /**
    * The desired number of instantiations of the task definition to keep running on the service.
+   * @deprecated - Use `internalDesiredCount` instead.
    */
   public readonly desiredCount: number;
 
+  /**
+   * The desired number of instantiations of the task definition to keep running on the service.
+   * The default is 1 for all new services and uses the existing services desired count
+   * when updating an existing service, if one is not provided.
+   */
+  public readonly internalDesiredCount?: number;
+
   /**
    * The Network Load Balancer for the service.
    */
@@ -297,7 +307,10 @@ export abstract class NetworkMultipleTargetGroupsServiceBase extends CoreConstru
     this.validateInput(props);
 
     this.cluster = props.cluster || this.getDefaultCluster(this, props.vpc);
+
     this.desiredCount = props.desiredCount || 1;
+    this.internalDesiredCount = props.desiredCount;
+
     if (props.taskImageOptions) {
       this.logDriver = this.createLogDriver(props.taskImageOptions.enableLogging, props.taskImageOptions.logDriver);
     }

packages/@aws-cdk/aws-ecs-patterns/lib/base/queue-processing-service-base.ts

@@ -3,6 +3,7 @@ import { IVpc } from '@aws-cdk/aws-ec2';
 import { AwsLogDriver, BaseService, Cluster, ContainerImage, DeploymentController, ICluster, LogDriver, PropagatedTagSource, Secret } from '@aws-cdk/aws-ecs';
 import { IQueue, Queue } from '@aws-cdk/aws-sqs';
 import { CfnOutput, Duration, Stack } from '@aws-cdk/core';
+import * as cxapi from '@aws-cdk/cx-api';
 import { Construct } from 'constructs';
 
 // v2 - keep this import as a separate section to reduce merge conflict when forward merging with the v2 branch.
@@ -53,7 +54,10 @@ export interface QueueProcessingServiceBaseProps {
   /**
    * The desired number of instantiations of the task definition to keep running on the service.
    *
-   * @default 1
+   * @default - If the feature flag, ECS_REMOVE_DEFAULT_DESIRED_COUNT is false, the default is 1;
+   * if true, the minScalingCapacity is 1 for all new services and uses the existing services desired count
+   * when updating an existing service.
+   * @deprecated - Use `minScalingCapacity` or a literal object instead.
    */
   readonly desiredTaskCount?: number;
 
@@ -109,10 +113,17 @@ export interface QueueProcessingServiceBaseProps {
   /**
    * Maximum capacity to scale to.
    *
-   * @default (desiredTaskCount * 2)
+   * @default - If the feature flag, ECS_REMOVE_DEFAULT_DESIRED_COUNT is false, the default is (desiredTaskCount * 2); if true, the default is 2.
    */
   readonly maxScalingCapacity?: number
 
+  /**
+   * Minimum capacity to scale to.
+   *
+   * @default - If the feature flag, ECS_REMOVE_DEFAULT_DESIRED_COUNT is false, the default is the desiredTaskCount; if true, the default is 1.
+   */
+  readonly minScalingCapacity?: number
+
   /**
    * The intervals for scaling based on the SQS queue's ApproximateNumberOfMessagesVisible metric.
    *
@@ -214,6 +225,7 @@ export abstract class QueueProcessingServiceBase extends CoreConstruct {
 
   /**
    * The minimum number of tasks to run.
+   * @deprecated - Use `minCapacity` instead.
    */
   public readonly desiredCount: number;
 
@@ -222,6 +234,11 @@ export abstract class QueueProcessingServiceBase extends CoreConstruct {
    */
   public readonly maxCapacity: number;
 
+  /**
+   * The minimum number of instances for autoscaling to scale down to.
+   */
+  public readonly minCapacity: number;
+
   /**
    * The scaling interval for autoscaling based off an SQS Queue size.
    */
@@ -272,9 +289,21 @@ export abstract class QueueProcessingServiceBase extends CoreConstruct {
     this.environment = { ...(props.environment || {}), QUEUE_NAME: this.sqsQueue.queueName };
     this.secrets = props.secrets;
 
-    // Determine the desired task count (minimum) and maximum scaling capacity
     this.desiredCount = props.desiredTaskCount ?? 1;
-    this.maxCapacity = props.maxScalingCapacity || (2 * this.desiredCount);
+
+    // Determine the desired task count (minimum) and maximum scaling capacity
+    if (!this.node.tryGetContext(cxapi.ECS_REMOVE_DEFAULT_DESIRED_COUNT)) {
+      this.minCapacity = props.minScalingCapacity || this.desiredCount;
+      this.maxCapacity = props.maxScalingCapacity || (2 * this.desiredCount);
+    } else {
+      if (props.desiredTaskCount != null) {
+        this.minCapacity = props.minScalingCapacity || this.desiredCount;
+        this.maxCapacity = props.maxScalingCapacity || (2 * this.desiredCount);
+      } else {
+        this.minCapacity = props.minScalingCapacity || 1;
+        this.maxCapacity = props.maxScalingCapacity || 2;
+      }
+    }
 
     if (!this.desiredCount && !this.maxCapacity) {
       throw new Error('maxScalingCapacity must be set and greater than 0 if desiredCount is 0');
@@ -290,7 +319,7 @@ export abstract class QueueProcessingServiceBase extends CoreConstruct {
    * @param service the ECS/Fargate service for which to apply the autoscaling rules to
    */
   protected configureAutoscalingForService(service: BaseService) {
-    const scalingTarget = service.autoScaleTaskCount({ maxCapacity: this.maxCapacity, minCapacity: this.desiredCount });
+    const scalingTarget = service.autoScaleTaskCount({ maxCapacity: this.maxCapacity, minCapacity: this.minCapacity });
     scalingTarget.scaleOnCpuUtilization('CpuScaling', {
       targetUtilizationPercent: 50,
     });

packages/@aws-cdk/aws-ecs-patterns/lib/ecs/application-load-balanced-ecs-service.ts

@@ -1,4 +1,5 @@
 import { Ec2Service, Ec2TaskDefinition } from '@aws-cdk/aws-ecs';
+import * as cxapi from '@aws-cdk/cx-api';
 import { Construct } from 'constructs';
 import { ApplicationLoadBalancedServiceBase, ApplicationLoadBalancedServiceBaseProps } from '../base/application-load-balanced-service-base';
 
@@ -117,9 +118,11 @@ export class ApplicationLoadBalancedEc2Service extends ApplicationLoadBalancedSe
       throw new Error('You must specify one of: taskDefinition or image');
     }
 
+    const desiredCount = this.node.tryGetContext(cxapi.ECS_REMOVE_DEFAULT_DESIRED_COUNT) ? this.internalDesiredCount : this.desiredCount;
+
     this.service = new Ec2Service(this, 'Service', {
       cluster: this.cluster,
-      desiredCount: this.desiredCount,
+      desiredCount: desiredCount,
       taskDefinition: this.taskDefinition,
       assignPublicIp: false,
       serviceName: props.serviceName,

packages/@aws-cdk/aws-ecs-patterns/lib/ecs/application-multiple-target-groups-ecs-service.ts

@@ -1,5 +1,6 @@
 import { Ec2Service, Ec2TaskDefinition } from '@aws-cdk/aws-ecs';
 import { ApplicationTargetGroup } from '@aws-cdk/aws-elasticloadbalancingv2';
+import * as cxapi from '@aws-cdk/cx-api';
 import { Construct } from 'constructs';
 import {
   ApplicationMultipleTargetGroupsServiceBase,
@@ -136,9 +137,11 @@ export class ApplicationMultipleTargetGroupsEc2Service extends ApplicationMultip
   }
 
   private createEc2Service(props: ApplicationMultipleTargetGroupsEc2ServiceProps): Ec2Service {
+    const desiredCount = this.node.tryGetContext(cxapi.ECS_REMOVE_DEFAULT_DESIRED_COUNT) ? this.internalDesiredCount : this.desiredCount;
+
     return new Ec2Service(this, 'Service', {
       cluster: this.cluster,
-      desiredCount: this.desiredCount,
+      desiredCount: desiredCount,
       taskDefinition: this.taskDefinition,
       assignPublicIp: false,
       serviceName: props.serviceName,

packages/@aws-cdk/aws-ecs-patterns/lib/ecs/network-load-balanced-ecs-service.ts

@@ -1,4 +1,5 @@
 import { Ec2Service, Ec2TaskDefinition } from '@aws-cdk/aws-ecs';
+import * as cxapi from '@aws-cdk/cx-api';
 import { Construct } from 'constructs';
 import { NetworkLoadBalancedServiceBase, NetworkLoadBalancedServiceBaseProps } from '../base/network-load-balanced-service-base';
 
@@ -115,9 +116,11 @@ export class NetworkLoadBalancedEc2Service extends NetworkLoadBalancedServiceBas
       throw new Error('You must specify one of: taskDefinition or image');
     }
 
+    const desiredCount = this.node.tryGetContext(cxapi.ECS_REMOVE_DEFAULT_DESIRED_COUNT) ? this.internalDesiredCount : this.desiredCount;
+
     this.service = new Ec2Service(this, 'Service', {
       cluster: this.cluster,
-      desiredCount: this.desiredCount,
+      desiredCount: desiredCount,
       taskDefinition: this.taskDefinition,
       assignPublicIp: false,
       serviceName: props.serviceName,

packages/@aws-cdk/aws-ecs-patterns/lib/ecs/network-multiple-target-groups-ecs-service.ts

@@ -1,5 +1,6 @@
 import { Ec2Service, Ec2TaskDefinition } from '@aws-cdk/aws-ecs';
 import { NetworkTargetGroup } from '@aws-cdk/aws-elasticloadbalancingv2';
+import * as cxapi from '@aws-cdk/cx-api';
 import { Construct } from 'constructs';
 import {
   NetworkMultipleTargetGroupsServiceBase,
@@ -136,9 +137,11 @@ export class NetworkMultipleTargetGroupsEc2Service extends NetworkMultipleTarget
   }
 
   private createEc2Service(props: NetworkMultipleTargetGroupsEc2ServiceProps): Ec2Service {
+    const desiredCount = this.node.tryGetContext(cxapi.ECS_REMOVE_DEFAULT_DESIRED_COUNT) ? this.internalDesiredCount : this.desiredCount;
+
     return new Ec2Service(this, 'Service', {
       cluster: this.cluster,
-      desiredCount: this.desiredCount,
+      desiredCount: desiredCount,
       taskDefinition: this.taskDefinition,
       assignPublicIp: false,
       serviceName: props.serviceName,