opensearch,elasticsearch,events-targets: Custom Resources provided in these modules always attempt to install latest version of AWS SDK v2

[mrgrain]

Describe the bug

The following built-in Custom Resources force the install of the latest AWS SDK version in their Lambda which will fail in environments with restricted internet access (e.g. China Regions or when internet access is disabled in Lambdas):

• OpenSearchAccessPolicy in aws-elasticsearch
• ElasticsearchAccessPolicy in aws-opensearch
• LogGroupResourcePolicy in aws-elasticsearch, aws-opensearch and aws-events-targets

There is no way to disable this behavior, because the resources extend AwsCustomResource which defaults installLatestAwsSdk to true. They do not provide an option to disable this.

Expected Behavior

The affected resources do not attempt to install the latest SDK version. All API request for these resources are known so it's not required to have the latest version available.

Current Behavior

They always attempt to install the latest SDK version.

Reproduction Steps

class TestStack extends Stack {
  constructor(scope: Construct, id: string, props?: StackProps) {
    super(scope, id, props);

    const domainProps: opensearch.DomainProps = {
      removalPolicy: RemovalPolicy.DESTROY,
      version: opensearch.EngineVersion.ELASTICSEARCH_7_1,
      ebs: {
        volumeSize: 10,
        volumeType: EbsDeviceVolumeType.GENERAL_PURPOSE_SSD,
      },
      logging: {
        slowSearchLogEnabled: true,
        appLogEnabled: true,
      },
      nodeToNodeEncryption: true,
      encryptionAtRest: {
        enabled: true,
      },
      advancedOptions: {
        'rest.action.multi.allow_explicit_index': 'false',
        'indices.fielddata.cache.size': '25',
        'indices.query.bool.max_clause_count': '2048',
      },
      // test the access policies custom resource works
      accessPolicies: [
        new iam.PolicyStatement({
          effect: iam.Effect.ALLOW,
          actions: ['es:ESHttp*'],
          principals: [new iam.AccountRootPrincipal()],
          resources: ['*'],
        }),
      ],
    };

    // create 2 domains to ensure that Cloudwatch Log Group policy names dont conflict
    new opensearch.Domain(this, 'Domain1', domainProps);
    new opensearch.Domain(this, 'Domain2', domainProps);
  }
}

Possible Solution

• Check if we can default the setting to false for these custom resources
  It should be possible to ascertain if the used APIs are available in the default SDK
  Then add installLatestAwsSdk: false to here
• Use a layer to provide the latest version of the SDK
  Similar to how we provide the AWS CLI already

Additional Information/Context

Workaround:

declare stack: cdk.Stack;

// Create an Aspect to stop installing the latest SDK version
class AwsCustomResourceUseDefaultAwsSdk implements cdk.IAspect {
  public readonly resourceTypes: string[];

  public constructor(resourceTypes: string[] = ['Custom::AWS']) {
    this.resourceTypes = resourceTypes;
  }

  public visit(node: IConstruct): void {
    if (node instanceof cdk.CfnResource && this.resourceTypes.includes(node.cfnResourceType)) {
      node.addPropertyOverride('InstallLatestAwsSdk', false);
    }
  }
}

// Apply this Aspect to any Stack
// Note that specific resource types have to be provided in the constructor call
cdk.Aspects.of(stack).add(new AwsCustomResourceUseDefaultAwsSdk(['Custom::OpenSearchAccessPolicy']));

CDK CLI Version

2.52.20

Framework Version

2.52.0

Node.js Version

any

OS

macos

Language

Typescript, Python, .NET, Java, Go

Language Version

No response

Other information

These Custom Resources currently run in nodejs14.x which defaults the AWS SDK for JS to version 2.1055.0 (source).

I have checked and confirm that the API for OpenSearchAccessPolicy & ElasticSearchAccessPolicy is available in this particular version of the SDK.

[rix0rrr]

installLatestAwsSdk: true should never have been the default.

Let's:

• Explicitly configure it as false for all custom resources we instantiate
• Create a feature flag to make false the new default

[github-actions]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see.
If you need more assistance, please either tag a team member or open a new issue that references this one.
If you wish to keep having a conversation with other community members under this issue feel free to do so.