Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: Encapsulate overriding of logicalId in a single place #364

Merged
merged 1 commit into from
Apr 7, 2021

Conversation

akash1810
Copy link
Member

@akash1810 akash1810 commented Mar 30, 2021

What does this change?

A resource's logicalId maps to the name of the created resource.

For example, take this CloudFormation YAML for an ASG:

DataNodeAutoScalingGroup:
  Type: AWS::AutoScaling::AutoScalingGroup
  Properties:
    AvailabilityZones: !GetAZs ""
    LaunchConfigurationName: !Ref DataNodeLaunchConfig
    MinSize: 3
    MaxSize: 4
    Cooldown: 180
    HealthCheckType: EC2
    HealthCheckGracePeriod: 300
    VPCZoneIdentifier: !Ref PrivateVpcSubnets
    Tags:
    - { Key: Stack, Value: !Ref Stack, PropagateAtLaunch: true }
    - { Key: App, Value: "elk-es-data", PropagateAtLaunch: true }
    - { Key: Stage, Value: !Ref Stage, PropagateAtLaunch: true }
    - { Key: ElasticSearchCluster, Value: !Ref ClusterName, PropagateAtLaunch: true }

This creates an ASG called DataNodeAutoScalingGroup. This pattern is widely used in YAML definitions of CloudFormation as YAML files are typically hand-written.

CDK changes this as it auto-generates the logicalIds, as there one will gain more understanding about the resource from looking at the CDK template.

In order to enable a no-op migration between a YAML stack a CDK stack, we allow the logicalId to be overridden.
Our constructs seem to follow different rules regarding when to override the logicalId of a CloudFormation resource.

For example, we have this:

if (props.overrideId || (scope.migratedFromCloudFormation && props.overrideId !== false))

And this:

if (props.overrideId) {

This change aims to bring consistency and simplicity across our constructs by applying a universal rule that,
migratedFromCloudFormation on the GuStack must be true and existingLogicalId to be set for the logicalId to be overridden.

If these conditions are not met, we let CDK auto-generate the logicalId.

It's worth noting that this doesn't implement the change on the constructs, it's simply to define the behaviour.
This is mainly to keep the diff down - we'll migrate the constructs in follow-up PRs.

Usage will roughly look like this:

const MyComponentProps extends GuMigratingResource { }

class MyComponent extends SomeGuConstruct {
  constructor(scope: GuStack, id: string, props: MyComponentProps) {
    super(scope, id, props);
    GuMigratingResource.setLogicalId(this, scope, props);
  }
}

Does this change require changes to existing projects or CDK CLI?

No.

How to test

See added tests.

How can we measure success?

Consistency in behaviour across constructs.

Have we considered potential risks?

n/a

@akash1810 akash1810 changed the title fix: encapsulate local id logic in an interface refactor: encapsulate local id logic in an interface Mar 31, 2021
@akash1810 akash1810 changed the title refactor: encapsulate local id logic in an interface refactor: encapsulate logicalId logic in an interface Mar 31, 2021
@akash1810 akash1810 marked this pull request as ready for review March 31, 2021 15:37
@akash1810 akash1810 requested a review from a team March 31, 2021 15:37
jacobwinch
jacobwinch previously approved these changes Mar 31, 2021
Copy link
Contributor

@jacobwinch jacobwinch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me 👍

@akash1810 akash1810 dismissed jacobwinch’s stale review April 1, 2021 23:51

more changes have been made

@akash1810 akash1810 marked this pull request as draft April 6, 2021 09:44
@akash1810 akash1810 force-pushed the aa-migrating-interface branch 2 times, most recently from c84498f to 8ec1861 Compare April 6, 2021 13:01
@akash1810 akash1810 changed the title refactor: encapsulate logicalId logic in an interface refactor: Encapsulate overriding of logicalId in a single place Apr 6, 2021
@akash1810 akash1810 marked this pull request as ready for review April 6, 2021 13:03
@akash1810
Copy link
Member Author

akash1810 commented Apr 6, 2021

@stephengeller @jacobwinch I've re-worked and re-worded this to make the logic more assertive, principally migratedFromCloudFormation has to be set - the function is greatly simplified and is more readable as a result. I'm hoping the commit makes sense.

A review would be welcomed.

} else {
if (existingLogicalId) {
console.warn(
`GuStack has 'migratedFromCloudFormation' set to false. ${id} has an 'existingLogicalId' set to ${existingLogicalId}. This will have no effect - the logicalId will be auto-generated. Set 'migratedFromCloudFormation' to true for 'existingLogicalId' to be observed.`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Much clearer 👍

Copy link
Contributor

@jacobwinch jacobwinch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work - thanks for taking the time to document and simplify this 💯

@akash1810
Copy link
Member Author

Now requires #388.

A resource's logicalId maps to the name of the created resource.

For example, take this CloudFormation YAML for an ASG:

```yaml
DataNodeAutoScalingGroup:
  Type: AWS::AutoScaling::AutoScalingGroup
  Properties:
    AvailabilityZones: !GetAZs ""
    LaunchConfigurationName: !Ref DataNodeLaunchConfig
    MinSize: 3
    MaxSize: 4
    Cooldown: 180
    HealthCheckType: EC2
    HealthCheckGracePeriod: 300
    VPCZoneIdentifier: !Ref PrivateVpcSubnets
    Tags:
    - { Key: Stack, Value: !Ref Stack, PropagateAtLaunch: true }
    - { Key: App, Value: "elk-es-data", PropagateAtLaunch: true }
    - { Key: Stage, Value: !Ref Stage, PropagateAtLaunch: true }
    - { Key: ElasticSearchCluster, Value: !Ref ClusterName, PropagateAtLaunch: true }
```

This creates an ASG called `DataNodeAutoScalingGroup`. This pattern is widely used in YAML definitions of CloudFormation as YAML files are typically hand-written.

CDK changes this as it auto-generates the logicalIds, as there one will gain more understanding about the resource from looking at the CDK template.

In order to enable a no-op migration between a YAML stack a CDK stack, we allow the logicalId to be overridden.
Our constructs seem to follow different rules regarding when to override the logicalId of a CloudFormation resource.

For example, we have [this](https://github.com/guardian/cdk/blob/60cbf21698b3ddbb8cfd23556fd2af886e9551c7/src/constructs/rds/instance.ts#L38-L40):

```typescript
if (props.overrideId || (scope.migratedFromCloudFormation && props.overrideId !== false))
```

And [this](https://github.com/guardian/cdk/blob/60cbf21698b3ddbb8cfd23556fd2af886e9551c7/src/constructs/iam/policies/base-policy.ts#L15-L18):

```typescript
if (props.overrideId) {
```

This change aims to bring consistency and simplicity across our constructs by applying a universal rule that,
`migratedFromCloudFormation` on the `GuStack` must be `true` and `existingLogicalId` to be set for the logicalId to be overridden.

If these conditions are not met, we let CDK auto-generate the logicalId.

It's worth noting that this doesn't implement the change on the constructs, it's simply to define the behaviour.
This is mainly to keep the diff down - we'll migrate the constructs in follow-up PRs.

Usage will roughly look like this:

```typescript
const MyComponentProps extends GuMigratingResource { }

class MyComponent extends SomeGuConstruct {
  constructor(scope: GuStack, id: string, props: MyComponentProps) {
    super(scope, id, props);
    GuMigratingResource.setLogicalId(this, scope, props);
  }
}
```
akash1810 added a commit that referenced this pull request Apr 12, 2021
In #364 we placed the logic of overriding a construct's logicalId into a single place.

In this change, we're updating the `GuPolicy` construct to adopt the new logic. As of #418 it's as simple as using the GuStatefulMigratableConstruct mixin!
akash1810 added a commit that referenced this pull request Apr 12, 2021
In #364 we placed the logic of overriding a construct's logicalId into a single place.

In this change, we're updating the `GuRole` construct to adopt the new logic. As of #418 it's as simple as using the GuStatefulMigratableConstruct mixin!

As a by-product, `GuInstanceRole` also gets updated; the snapshot tests are updated to reflect the fact that `GuInstanceRole` doesn't set `existingLogicalId` when calling `GuRole`, therefore the logicalId will always be autp-generated.
akash1810 added a commit that referenced this pull request Apr 12, 2021
In #364 we placed the logic of overriding a construct's logicalId into a single place.

In this change, we're updating the `GuRole` construct to adopt the new logic. As of #418 it's as simple as using the `GuStatefulMigratableConstruct` mixin!

As a by-product, `GuInstanceRole`'s tests need updating. The snapshot tests are updated to reflect the fact that `GuInstanceRole` doesn't set `existingLogicalId` when calling `GuRole`, therefore the logicalId will always be auto-generated.
akash1810 added a commit that referenced this pull request Apr 12, 2021
In #364 we placed the logic of overriding a construct's logicalId into a single place.

In this change, we're updating the `GuBaseSecurityGroup` construct to adopt the new logic. As of #418 it's as simple as using the `GuStatefulMigratableConstruct` mixin!
akash1810 added a commit that referenced this pull request Apr 12, 2021
In #364 we placed the logic of overriding a construct's logicalId into a single place.

In this change, we're updating the `GuKinesisStream` construct to adopt the new logic. As of #418 it's as simple as using the `GuStatefulMigratableConstruct` mixin!
akash1810 added a commit that referenced this pull request Apr 12, 2021
In #364 we placed the logic of overriding a construct's logicalId into a single place.

In this change, we're updating the `GuBaseSecurityGroup` construct to adopt the new logic. As of #418 it's as simple as using the `GuMigratableConstruct` mixin!
akash1810 added a commit that referenced this pull request Apr 12, 2021
In #364 we placed the logic of overriding a construct's logicalId into a single place.

In this change, we're updating the `GuRole` construct to adopt the new logic. As of #418 it's as simple as using the `GuMigratableConstruct` mixin!

As a by-product, `GuInstanceRole`'s tests need updating. The snapshot tests are updated to reflect the fact that `GuInstanceRole` doesn't set `existingLogicalId` when calling `GuRole`, therefore the logicalId will always be auto-generated.
akash1810 added a commit that referenced this pull request Apr 12, 2021
In #364 we placed the logic of overriding a construct's logicalId into a single place.

In this change, we're updating the `GuPolicy` construct to adopt the new logic. As of #418 it's as simple as using the `GuMigratableConstruct` mixin!
akash1810 added a commit that referenced this pull request Apr 12, 2021
In #364 we placed the logic of overriding a construct's logicalId into a single place.

In this change, we're updating the `GuSnsTopic` construct to adopt the new logic. As of #418 it's as simple as using the `GuStatefulMigratableConstruct` mixin!
akash1810 added a commit that referenced this pull request Apr 12, 2021
In #364 we placed the logic of overriding a construct's logicalId into a single place.

In this change, we're updating the `GuDatabaseInstance` construct to adopt the new logic. As of #418 it's as simple as using the `GuStatefulMigratableConstruct` mixin!
akash1810 added a commit that referenced this pull request Apr 12, 2021
When a stack is synthesized, AWS CDK will:
  - print the resulting CloudFormation template as YAML to the console
  - save the resulting CloudFormation template as JSON to disk

The AWS CDK library does not support synthesizing a YAML template to disk. The only way to achieve this is is to redirect the result of `cdk synth` to disk.

For example:

```console
cdk synth > cloudformation.yaml
```

Since #364 we have started printing important messages to the console. This breaks the saving YAML disk trick as the file on disk is not valid YAML.

For example:

```yaml
GuStack has 'migratedFromCloudFormation' set to false. MyDatabase is a stateful construct, it's logicalId will be auto-generated and AWS will create a new resource.
Parameters:
  Stage:
    Type: String
    Default: CODE
    AllowedValues:
      - CODE
      - PROD
    Description: Stage name
Resources:
  MyDatabaseA1B2C3D4:
    Type: AWS::RDS::DBInstance
```

For this reason, prefix console messages with "# ". This results in `cloudformation.yaml` still being valid YAML.

For example:

```yaml
\# GuStack has 'migratedFromCloudFormation' set to false. MyDatabase is a stateful construct, it's logicalId will be auto-generated and AWS will create a new resource.
Parameters:
  Stage:
    Type: String
    Default: CODE
    AllowedValues:
      - CODE
      - PROD
    Description: Stage name
Resources:
  MyDatabaseA1B2C3D4:
    Type: AWS::RDS::DBInstance
```
akash1810 added a commit that referenced this pull request Apr 13, 2021
When a stack is synthesized, AWS CDK will:
  - print the resulting CloudFormation template as YAML to the console
  - save the resulting CloudFormation template as JSON to disk

The AWS CDK library does not support synthesizing a YAML template to disk. The only way to achieve this is is to redirect the result of `cdk synth` to disk.

For example:

```console
cdk synth > cloudformation.yaml
```

Since #364 we have started printing important messages to the console. This breaks the saving YAML to disk trick as the file on disk is not valid YAML.

For example:

```yaml
GuStack has 'migratedFromCloudFormation' set to false. MyDatabase is a stateful construct, it's logicalId will be auto-generated and AWS will create a new resource.
Parameters:
  Stage:
    Type: String
    Default: CODE
    AllowedValues:
      - CODE
      - PROD
    Description: Stage name
Resources:
  MyDatabaseA1B2C3D4:
    Type: AWS::RDS::DBInstance
```

For this reason, prefix console messages with "# ". This results in `cloudformation.yaml` still being valid YAML.

For example:

```yaml
\# GuStack has 'migratedFromCloudFormation' set to false. MyDatabase is a stateful construct, it's logicalId will be auto-generated and AWS will create a new resource.
Parameters:
  Stage:
    Type: String
    Default: CODE
    AllowedValues:
      - CODE
      - PROD
    Description: Stage name
Resources:
  MyDatabaseA1B2C3D4:
    Type: AWS::RDS::DBInstance
```
akash1810 added a commit that referenced this pull request Apr 13, 2021
In #364 we placed the logic of overriding a construct's logicalId into a single place.

In this change, we're updating the `GuApplicationTargetGroup` construct to adopt the new logic. As of #418 it's as simple as using the `GuStatefulMigratableConstruct` mixin!
akash1810 added a commit that referenced this pull request Apr 13, 2021
In #364 we placed the logic of overriding a construct's logicalId into a single place.

In this change, we're updating the `GuApplicationLoadBalancer` construct to adopt the new logic. As of #418 it's as simple as using the `GuStatefulMigratableConstruct` mixin!
akash1810 added a commit that referenced this pull request Apr 13, 2021
In #364 we placed the logic of overriding a construct's logicalId into a single place.

In this change, we're updating the `GuClassicLoadBalancer` construct to adopt the new logic. As of #418 it's as simple as using the `GuStatefulMigratableConstruct` mixin!
akash1810 added a commit that referenced this pull request Apr 13, 2021
In #364 we placed the logic of overriding a construct's logicalId into a single place.

In this change, we're updating the `GuAutoScalingGroup` construct to adopt the new logic. As of #418 it's as simple as using the `GuStatefulMigratableConstruct` mixin!
akash1810 added a commit that referenced this pull request Apr 13, 2021
In #364 we placed the logic of overriding a construct's logicalId into a single place.

In this change, we're updating the `GuClassicLoadBalancer` construct to adopt the new logic. As of #418 it's as simple as using the `GuStatefulMigratableConstruct` mixin!
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants