(core): Templating error on custom-resource introduced with version 1.92

[mzakarian]

for (let index = 0; index < existingVpc.availabilityZones.length; index++) {
      let getEndpointIp = new AwsCustomResource(this, idPrefix + 'GetEndpointIp' + index, {
        policy: AwsCustomResourcePolicy.fromSdkCalls({resources: AwsCustomResourcePolicy.ANY_RESOURCE}),
        onUpdate: {
          action: 'describeNetworkInterfaces',
          service: 'EC2',
          parameters: {NetworkInterfaceIds: vpcendpoint.vpcEndpointNetworkInterfaceIds},
          physicalResourceId: PhysicalResourceId.of('DirectoryId'),
          outputPath: 'NetworkInterfaces.' + index + '.PrivateIpAddress'
        }
      })
      targetGroup.addTarget(new IpTarget(Token.asString(getEndpointIp.getResponseField('NetworkInterfaces.' + index + '.PrivateIpAddress'))));
    }

We are doing this, because we need to put an api gateway behind a load balancer (company policy). If there is a better solution to do this, feedback is welcome.

Reproduction Steps

this is the output with aws-cdk/core 1.91

...
  announceGetEndpointIp0D5B4E67A:
    Type: Custom::AWS
    Properties:
      ServiceToken:
        Fn::GetAtt:
          - AWS12345678
          - Arn
      Create:
        action: describeNetworkInterfaces
        service: EC2
        parameters:
          NetworkInterfaceIds:
            Fn::GetAtt:
              - announcevpcannouncevpcendpoint55991BB5
              - NetworkInterfaceIds
        physicalResourceId:
          id: DirectoryId
        outputPath: NetworkInterfaces.0.PrivateIpAddress
      Update:
        action: describeNetworkInterfaces
        service: EC2
        parameters:
          NetworkInterfaceIds:
            Fn::GetAtt:
              - announcevpcannouncevpcendpoint55991BB5
              - NetworkInterfaceIds
        physicalResourceId:
          id: DirectoryId
        outputPath: NetworkInterfaces.0.PrivateIpAddress
      InstallLatestAwsSdk: true
    DependsOn:
      - announceGetEndpointIp0CustomResourcePolicyC5EFFE47
    UpdateReplacePolicy: Delete
    DeletionPolicy: Delete
    Metadata:
      aws:cdk:path: announce-dev-local/announce-GetEndpointIp0/Resource/Default
...

this is the output with aws-cdk/core 1.92

...
  announceGetEndpointIp0D5B4E67A:
    Type: Custom::AWS
    Properties:
      ServiceToken:
        Fn::GetAtt:
          - AWS12345678
          - Arn
      Create:
        Fn::Join:
          - ""
          - - '{"action":"describeNetworkInterfaces","service":"EC2","parameters":{"NetworkInterfaceIds":"'
            - Fn::GetAtt:
                - announcevpcannouncevpcendpoint55991BB5
                - NetworkInterfaceIds
            - '"},"physicalResourceId":{"id":"DirectoryId"},"outputPath":"NetworkInterfaces.0.PrivateIpAddress"}'
      Update:
        Fn::Join:
          - ""
          - - '{"action":"describeNetworkInterfaces","service":"EC2","parameters":{"NetworkInterfaceIds":"'
            - Fn::GetAtt:
                - announcevpcannouncevpcendpoint55991BB5
                - NetworkInterfaceIds
            - '"},"physicalResourceId":{"id":"DirectoryId"},"outputPath":"NetworkInterfaces.0.PrivateIpAddress"}'
      InstallLatestAwsSdk: true
    DependsOn:
      - announceGetEndpointIp0CustomResourcePolicyC5EFFE47
    UpdateReplacePolicy: Delete
    DeletionPolicy: Delete
    Metadata:
      aws:cdk:path: announce-dev-local/announce-GetEndpointIp0/Resource/Default
...

What did you expect to happen?

After using cdk deploy we expected the same behavior as in version 1.91 with successful deployment.

What actually happened?

The error only occurs during a cdk deploy with version 1.92.

26/44 | 8:32:42 AM | CREATE_FAILED        | Custom::AWS                               | announce-GetEndpointIp2/Resource/Default (announceGetEndpointIp20B447BE7) Template error: every Fn::Join object requires two parameters, (1) a string delimiter and (2) a list of strings to be joined or a function that returns a list of strings (such as Fn::GetAZs) to be joined.
	new CustomResource (/builds/marceldarvis/toolbox/services/announce-pipeline-modules-service/node_modules/@aws-cdk/core/lib/custom-resource.ts:119:21)
	\_ new AwsCustomResource (/builds/marceldarvis/toolbox/services/announce-pipeline-modules-service/node_modules/@aws-cdk/custom-resources/lib/aws-custom-resource/aws-custom-resource.ts:374:27)

Environment

• CDK CLI Version :1.92
• Framework Version:None
• Node.js Version:v14.16.0
• OS :linux
• Language (Version):TypeScript (4.2.3)

Other

We assume that this issue is related to this change:

• 3cb3104

---

This is 🐛 Bug Report

[eladb]

I think this is actually an issue with how toJsonString() encodes non-string attributes.

The change in #13074 encodes the API requests as JSON strings so it will be possible to use tokens in e.g. map keys. The stack.toJsonString() uses Fn::Join in order to concatenate tokens into a single JSON string.

However, in this use case vpcendpoint.vpcEndpointNetworkInterfaceIds resolves to an Fn::GetAtt that returns array instead of a string (very rare for attributes) and Fn::Join cannot concatenate it.

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.

[mzakarian]

@rix0rrr @eladb thank you for resolving this issue ❤️

[jogold]

@eladb @rix0rrr I'm not sure I understand how is this issue fixed with the refactor of CloudFormationLang.toJSON()? You are still going to get a Fn::GetAtt returning an array after the change, no?

Just tested this:

[rix0rrr]

🤦‍♂️

@jogold you are correct. I feel silly now, I was solving the wrong problem :(

[jogold]

@rix0rrr the only way to solve this right now for the user is this:

{
  NetworkInterfaceIds: [cdk.Token.asList(cdk.Fn.join(',', endpoint.vpcEndpointNetworkInterfaceIds))]
}

which will give the following correct CF:

{
  "Fn::Join": [
    "",
    [
      "{\"action\":\"describeNetworkInterfaces\",\"service\":\"EC2\",\"parameters\":{\"NetworkInterfaceIds\":[",
      {
        "Fn::Join": [
          ",",
          {
            "Fn::GetAtt": [
              "EndpointEEF1FD8F",
              "NetworkInterfaceIds"
            ]
          }
        ]
      },
      "]},\"physicalResourceId\":{\"id\":\"physical-od\"},\"outputPath\":\"NetworkInterfaces.0.PrivateIpAddress\"}"
    ]
  ]
}

[jogold]

@rix0rrr actually the code above doesn't work because the items in the vpcEndpointNetworkInterfaceIds aren't quoted.

This is very ugly but it works:

{
  NetworkInterfaceIds: [cdk.Token.asList(cdk.Stack.of(stack).toJsonString(cdk.Fn.join('","', endpoint.vpcEndpointNetworkInterfaceIds)))]
}

{
  "Fn::Join": [
    "",
    [
      "{\"action\":\"describeNetworkInterfaces\",\"service\":\"EC2\",\"parameters\":{\"NetworkInterfaceIds\":[\"",
      {
        "Fn::Join": [
          "\\\",\\\"",
          {
            "Fn::GetAtt": [
              "EndpointEEF1FD8F",
              "NetworkInterfaceIds"
            ]
          }
        ]
      },
      "\"]},\"physicalResourceId\":{\"id\":\"physical-od\"},\"outputPath\":\"NetworkInterfaces.0.PrivateIpAddress\"}"
    ]
  ]
}

[rix0rrr]

Oh wow.

I hadn't even thought that far ahead. I was thinking we'll resort to CfnJson but you're right, this ugly hackery might also just work.

Not sure it's worth it though, as it's probably going to break in further edge cases.