custom-resource: better log customization design for custom resource logs

[pahud]

Describe the feature

Challenges

In AWS CDK, some constructs implicitly create custom resources with associated providers that generate CloudWatch log groups. Customers currently have no way to customize these log groups, either by:

1. Using their own existing log groups instead of the generated ones.
2. Specifying the log retention period for the generated log groups.

As a result, when customers deploy stacks containing these implicit custom resources, the generated log groups are left orphaned, requiring manual cleanup by the customer.

Affected Issues

The issues related to this challenge are:

• [aws-ec2] Accidental Log Group creation when creating a VPC #33025
• CustomResourceConfig: Logging retention not set for CR when "autoDeleteObjects: true" is added to S3 Bucket #33116

Affected Modules

This challenge affects all CDK modules that implicitly create custom resources with lambda function providers.

Use Case

deploy any L2 constructs that have custom resources implicitly created.

Proposed Solution

Expected Experience

To improve the customer experience, it would be desirable to allow customers more freedom in customizing the log groups associated with these implicit custom resources, such as:

1. Enabling customers to specify their own existing log groups for the custom resource providers to use, instead of generating new ones.
2. Allowing customers to set a default log retention period for the generated log groups, to avoid orphaned log groups.
3. Exploring other ideas that give customers more control over the log group configuration for these implicit custom resources.

Other Information

No response

Acknowledgements

• I may be able to implement this feature request
• This feature might incur a breaking change

CDK version used

2.177.0

Environment details (OS name and version, etc.)

all

[samson-keung]

Adding some more info there with what I know. There was an attempt to implement this feature with the CustomResourceConfig class: https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.custom_resources.CustomResourceConfig.html

But as pointed out in #33116, CustomResourceConfig does not work for some of the custom resource providers. This should be fixed. Contributions are welcome.