(eks): Managed node with LaunchTemplate failed to set amiType

[vandreykiv]

After update to the latest cdk found that npm run build fails with the next error:

    at new Nodegroup (./aws-infra/node_modules/@aws-cdk/aws-eks/lib/managed-nodegroup.ts:303:13)
    at Cluster.addNodegroupCapacity (./aws-infra/node_modules/@aws-cdk/aws-eks/lib/cluster.ts:1203:12)
    at new EKSStack (./aws-infra/lib/eks-stack.ts:383:37)
    at Object.<anonymous> (./aws-infra/bin/aws-infra.ts:51:13)
    at Module._compile (node:internal/modules/cjs/loader:1108:14)
    at Module.m._compile (./aws-infra/node_modules/ts-node/src/index.ts:1056:23)
    at Module._extensions..js (node:internal/modules/cjs/loader:1137:10)
    at Object.require.extensions.<computed> [as .ts] (./aws-infra/node_modules/ts-node/src/index.ts:1059:12)
    at Module.load (node:internal/modules/cjs/loader:973:32)
    at Function.Module._load (node:internal/modules/cjs/loader:813:14)

Nodegroup config:

this.cluster.addNodegroupCapacity('gpu', {
            amiType: eks.NodegroupAmiType.AL2_X86_64_GPU,
            minSize: this.config.eks.gpuNodeGroupMinCapacity,
            desiredSize: this.config.eks.gpuNodeGroupMinCapacity,
            maxSize: this.config.eks.gpuNodeGroupMaxCapacity,
            launchTemplateSpec: {
                id: GPUlaunchTemplate.ref,
                version: GPUlaunchTemplate.attrLatestVersionNumber
            },
            nodeRole: eksInstanceRole,
            labels: {
                instance_type: 'gpu'
            },
            subnets: {
                subnetGroupName: 'Kube Private'
            }
        });

LaunchTemplate:

        const GPUlaunchTemplate = new ec2.CfnLaunchTemplate(this, 'GPUNodeGroupLaunchTemplate', {
            launchTemplateData: {
                instanceType: new ec2.InstanceType(this.config.eks.gpuNodeGroupInstanceType).toString(),
                userData: userData,
                securityGroupIds: [this.vpcStack.eksSecurityGroup.securityGroupId, this.vpcStack.esSecurityGroup.securityGroupId, this.vpcStack.rdsSecurityGroup.securityGroupId],
                blockDeviceMappings: [
                    {
                        deviceName: "/dev/xvda",
                        ebs: {
                            volumeSize: this.config.eks.gpuNodeGroupDiskSize
                        }
                    }
                ],
                monitoring: { enabled: true }
            }
        });

According to docs we can't set instanceType for LaunchTemplate and NodeGroup at the same time. If I remove instance type from LaunchTemplate and add it to NodeGroup amiType would be removed from NodeGroup and this cause NodeGroup recreation.

Resources
[~] AWS::EC2::LaunchTemplate GPUNodeGroupLaunchTemplate GPUNodeGroupLaunchTemplate
 └─ [~] LaunchTemplateData
     └─ [-] Removed: .InstanceType
[~] AWS::EC2::LaunchTemplate CPUNodeGroupLaunchTemplate CPUNodeGroupLaunchTemplate
 └─ [~] LaunchTemplateData
     └─ [-] Removed: .InstanceType
[~] AWS::EC2::LaunchTemplate SSDNodeGroupLaunchTemplate SSDNodeGroupLaunchTemplate
 └─ [~] LaunchTemplateData
     └─ [-] Removed: .InstanceType
[~] Custom::AWSCDK-EKS-KubernetesResource eks-cluster/AwsAuth/manifest/Resource eksclusterAwsAuthmanifest769BDE7D
 └─ [+] Overwrite
     └─ true
[~] AWS::EKS::Nodegroup eks-cluster/Nodegroupgpu eksclusterNodegroupgpu2C0AC521 replace
 ├─ [-] AmiType (requires replacement)
 │   └─ AL2_x86_64_GPU
 └─ [+] InstanceTypes (requires replacement)
     └─ ["g4dn.4xlarge"]
[~] AWS::EKS::Nodegroup eks-cluster/Nodegroupcpu eksclusterNodegroupcpu16E5F90A replace
 ├─ [-] AmiType (requires replacement)
 │   └─ AL2_x86_64
 └─ [+] InstanceTypes (requires replacement)
     └─ ["m5d.4xlarge"]
[~] AWS::EKS::Nodegroup eks-cluster/Nodegroupssd eksclusterNodegroupssd5E0EB251 replace
 ├─ [-] AmiType (requires replacement)
 │   └─ AL2_x86_64
 └─ [+] InstanceTypes (requires replacement)
     └─ ["c5d.xlarge"]

Reproduction Steps

        const GPUlaunchTemplate = new ec2.CfnLaunchTemplate(this, 'GPUNodeGroupLaunchTemplate', {
            launchTemplateData: {
                instanceType: new ec2.InstanceType(this.config.eks.gpuNodeGroupInstanceType).toString(),
                userData: userData,
                securityGroupIds: [this.vpcStack.eksSecurityGroup.securityGroupId, this.vpcStack.esSecurityGroup.securityGroupId, this.vpcStack.rdsSecurityGroup.securityGroupId],
                blockDeviceMappings: [
                    {
                        deviceName: "/dev/xvda",
                        ebs: {
                            volumeSize: this.config.eks.gpuNodeGroupDiskSize
                        }
                    }
                ],
                monitoring: { enabled: true }
            }
        });

        this.cluster.addNodegroupCapacity('gpu', {
            amiType: eks.NodegroupAmiType.AL2_X86_64_GPU,
            minSize: this.config.eks.gpuNodeGroupMinCapacity,
            desiredSize: this.config.eks.gpuNodeGroupMinCapacity,
            maxSize: this.config.eks.gpuNodeGroupMaxCapacity,
            launchTemplateSpec: {
                id: GPUlaunchTemplate.ref,
                version: GPUlaunchTemplate.attrLatestVersionNumber
            },
            nodeRole: eksInstanceRole,
            labels: {
                instance_type: 'gpu'
            },
            subnets: {
                subnetGroupName: 'Kube Private'
            }
        });

Environment

• CDK CLI Version : 1.83.0 (build 827c5f4)
• Framework Version:
• Node.js Version: v15.5.1
• OS : macOS 11.1
• Language (Version): TypeScript (4.1.3)

---

This is 🐛 Bug Report

[iliapolo]

@vandreykiv Thanks for reporting this.

Fix coming up: #12441

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

[iliapolo]

@vandreykiv When the next version of the CDK comes, you should be able to use with the original code you had and it won't cause any errors or replacements.