Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

r/s3_bucket_replication_configuration: Add token parameter for x-amz-bucket-object-lock-token #23624

Merged
merged 4 commits into from
Mar 11, 2022

Conversation

alexb-dd
Copy link
Contributor

@alexb-dd alexb-dd commented Mar 10, 2022

This allows one to enable replication on an object-lock enabled bucket, OR enabling object lock on an existing bucket. See https://docs.aws.amazon.com/AmazonS3/latest/userguide/object-lock-managing.html#object-lock-managing-replication for details.

Community Note

  • Please vote on this pull request by adding a 👍 reaction to the original pull request comment to help the community and maintainers prioritize this request
  • Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for pull request followers and do not help prioritize the request

Closes #14061.

Output from acceptance testing:

% make testacc TESTS='TestAccS3BucketReplication.*' PKG=s3
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./internal/service/s3/... -v -count 1 -parallel 20 -run='TestAccS3BucketReplication.*'  -timeout 180m
=== RUN   TestAccS3BucketReplicationConfiguration_basic
=== PAUSE TestAccS3BucketReplicationConfiguration_basic
=== RUN   TestAccS3BucketReplicationConfiguration_disappears
=== PAUSE TestAccS3BucketReplicationConfiguration_disappears
=== RUN   TestAccS3BucketReplicationConfiguration_multipleDestinationsEmptyFilter
=== PAUSE TestAccS3BucketReplicationConfiguration_multipleDestinationsEmptyFilter
=== RUN   TestAccS3BucketReplicationConfiguration_multipleDestinationsNonEmptyFilter
=== PAUSE TestAccS3BucketReplicationConfiguration_multipleDestinationsNonEmptyFilter
=== RUN   TestAccS3BucketReplicationConfiguration_twoDestination
=== PAUSE TestAccS3BucketReplicationConfiguration_twoDestination
=== RUN   TestAccS3BucketReplicationConfiguration_configurationRuleDestinationAccessControlTranslation
=== PAUSE TestAccS3BucketReplicationConfiguration_configurationRuleDestinationAccessControlTranslation
=== RUN   TestAccS3BucketReplicationConfiguration_configurationRuleDestinationAddAccessControlTranslation
=== PAUSE TestAccS3BucketReplicationConfiguration_configurationRuleDestinationAddAccessControlTranslation
=== RUN   TestAccS3BucketReplicationConfiguration_replicationTimeControl
=== PAUSE TestAccS3BucketReplicationConfiguration_replicationTimeControl
=== RUN   TestAccS3BucketReplicationConfiguration_replicaModifications
=== PAUSE TestAccS3BucketReplicationConfiguration_replicaModifications
=== RUN   TestAccS3BucketReplicationConfiguration_withoutStorageClass
=== PAUSE TestAccS3BucketReplicationConfiguration_withoutStorageClass
=== RUN   TestAccS3BucketReplicationConfiguration_schemaV2
=== PAUSE TestAccS3BucketReplicationConfiguration_schemaV2
=== RUN   TestAccS3BucketReplicationConfiguration_schemaV2SameRegion
=== PAUSE TestAccS3BucketReplicationConfiguration_schemaV2SameRegion
=== RUN   TestAccS3BucketReplicationConfiguration_schemaV2DestinationMetrics
=== PAUSE TestAccS3BucketReplicationConfiguration_schemaV2DestinationMetrics
=== RUN   TestAccS3BucketReplicationConfiguration_existingObjectReplication
    bucket_replication_configuration_test.go:714: skipping test: AWS Technical Support request required to allow ExistingObjectReplication
--- SKIP: TestAccS3BucketReplicationConfiguration_existingObjectReplication (0.00s)
=== RUN   TestAccS3BucketReplicationConfiguration_filter_tagFilter
=== PAUSE TestAccS3BucketReplicationConfiguration_filter_tagFilter
=== RUN   TestAccS3BucketReplicationConfiguration_filter_andOperator
=== PAUSE TestAccS3BucketReplicationConfiguration_filter_andOperator
=== RUN   TestAccS3BucketReplicationConfiguration_withoutPrefix
=== PAUSE TestAccS3BucketReplicationConfiguration_withoutPrefix
=== CONT  TestAccS3BucketReplicationConfiguration_basic
=== CONT  TestAccS3BucketReplicationConfiguration_replicaModifications
=== CONT  TestAccS3BucketReplicationConfiguration_withoutPrefix
=== CONT  TestAccS3BucketReplicationConfiguration_twoDestination
=== CONT  TestAccS3BucketReplicationConfiguration_replicationTimeControl
=== CONT  TestAccS3BucketReplicationConfiguration_filter_andOperator
=== CONT  TestAccS3BucketReplicationConfiguration_filter_tagFilter
=== CONT  TestAccS3BucketReplicationConfiguration_configurationRuleDestinationAccessControlTranslation
=== CONT  TestAccS3BucketReplicationConfiguration_schemaV2DestinationMetrics
=== CONT  TestAccS3BucketReplicationConfiguration_schemaV2SameRegion
=== CONT  TestAccS3BucketReplicationConfiguration_schemaV2
=== CONT  TestAccS3BucketReplicationConfiguration_withoutStorageClass
=== CONT  TestAccS3BucketReplicationConfiguration_disappears
=== CONT  TestAccS3BucketReplicationConfiguration_configurationRuleDestinationAddAccessControlTranslation
=== CONT  TestAccS3BucketReplicationConfiguration_multipleDestinationsEmptyFilter
=== CONT  TestAccS3BucketReplicationConfiguration_multipleDestinationsNonEmptyFilter
--- PASS: TestAccS3BucketReplicationConfiguration_disappears (59.40s)
--- PASS: TestAccS3BucketReplicationConfiguration_schemaV2SameRegion (171.63s)
--- PASS: TestAccS3BucketReplicationConfiguration_filter_tagFilter (305.94s)
--- PASS: TestAccS3BucketReplicationConfiguration_replicaModifications (307.00s)
--- PASS: TestAccS3BucketReplicationConfiguration_replicationTimeControl (307.06s)
--- PASS: TestAccS3BucketReplicationConfiguration_schemaV2 (307.94s)
--- PASS: TestAccS3BucketReplicationConfiguration_withoutStorageClass (309.00s)
--- PASS: TestAccS3BucketReplicationConfiguration_withoutPrefix (309.10s)
--- PASS: TestAccS3BucketReplicationConfiguration_multipleDestinationsNonEmptyFilter (309.21s)
--- PASS: TestAccS3BucketReplicationConfiguration_twoDestination (309.21s)
--- PASS: TestAccS3BucketReplicationConfiguration_multipleDestinationsEmptyFilter (309.34s)
--- PASS: TestAccS3BucketReplicationConfiguration_schemaV2DestinationMetrics (309.42s)
--- PASS: TestAccS3BucketReplicationConfiguration_filter_andOperator (331.59s)
--- PASS: TestAccS3BucketReplicationConfiguration_configurationRuleDestinationAddAccessControlTranslation (336.38s)
--- PASS: TestAccS3BucketReplicationConfiguration_configurationRuleDestinationAccessControlTranslation (336.73s)
--- PASS: TestAccS3BucketReplicationConfiguration_basic (347.10s)
PASS
ok  	github.com/hashicorp/terraform-provider-aws/internal/service/s3	353.025s
...

make test passed, and tested this on an ad-hoc TF module and was able to successfully enable replication on an object-lock enabled bucket. Before:

│ Error: error creating S3 replication configuration for bucket (<redacted>): InvalidRequest: Replication configuration cannot be applied to an Object Lock enabled bucket

After:

aws_s3_bucket_replication_configuration.replication: Creating...
aws_s3_bucket_replication_configuration.replication: Creation complete after 1s [id=<redacted>]

Apply complete! Resources: 1 added, 0 changed, 0 destroyed.

This allows one to enable replication on an object-lock enabled bucket.
@github-actions github-actions bot added needs-triage Waiting for first response or review from a maintainer. service/s3 Issues and PRs that pertain to the s3 service. size/XS Managed by automation to categorize the size of a PR. labels Mar 10, 2022
Copy link

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Welcome @alexb-dd 👋

It looks like this is your first Pull Request submission to the Terraform AWS Provider! If you haven’t already done so please make sure you have checked out our CONTRIBUTING guide and FAQ to make sure your contribution is adhering to best practice and has all the necessary elements in place for a successful approval.

Also take a look at our FAQ which details how we prioritize Pull Requests for inclusion.

Thanks again, and welcome to the community! 😃

@anGie44 anGie44 added enhancement Requests to existing resources that expand the functionality or scope. and removed needs-triage Waiting for first response or review from a maintainer. labels Mar 11, 2022
Copy link
Contributor

@anGie44 anGie44 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @alexb-dd , thank you for this PR! Overall it's looking great, just a couple comments to address and if you could please add the token argument to the documentation (website/docs/r/s3_bucket_replication_configuration.html) similar to the description provided in the aws_s3_bucket_object_lock_configuration resource's token arg, then we should be good to go, e.g.

https://github.com/hashicorp/terraform-provider-aws/blob/main/website/docs/r/s3_bucket_object_lock_configuration.html.markdown#argument-reference

I'm curious, did AWS support first recommend enabling bucket versioning to then get the token that get's generated behind the scenes? (that's currently what is noted in the aws_s3_bucket_object_lock_configurationdocs)

@anGie44 anGie44 self-assigned this Mar 11, 2022
@alexb-dd
Copy link
Contributor Author

I'm curious, did AWS support first recommend enabling bucket versioning to then get the token that get's generated behind the scenes? (that's currently what is noted in the aws_s3_bucket_object_lock_configurationdocs)

They asked me for the bucket for which to generate the token. Thus the bucket needed to exist. I think since object lock being enabled on a bucket (whether at creation time or after the fact) implies versioning is on, they did not need to ask for versioning explicitly.

1) Copied the existing implementation from bucket_object_lock_configuration.go
which allowed us to make it sensitive, per below:

      + token  = (sensitive value)

2) Fixed optional handling to not initialize token in constructor
   expression but only if it is set.

3) Added docs for new field.
@github-actions github-actions bot added the documentation Introduces or discusses updates to documentation. label Mar 11, 2022
@alexb-dd alexb-dd requested a review from anGie44 March 11, 2022 15:11
@@ -216,6 +216,8 @@ The following arguments are supported:

* `bucket` - (Required) The name of the source S3 bucket you want Amazon S3 to monitor.
* `role` - (Required) The ARN of the IAM role for Amazon S3 to assume when replicating the objects.
* `token` - (Optional) A token to allow replication to be enabled on an Object Lock-enabled bucket. You must contact AWS support for the bucket's "Object Lock token".
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very minor nit: do you mind moving token just below rule to maintain the order?

Copy link
Contributor

@anGie44 anGie44 Mar 11, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Going to edit this as we're going to do a release today and would love to get this in as it's ready to go 🚀

@anGie44 anGie44 changed the title Add token parameter for x-amz-bucket-object-lock-token r/s3_bucket_replication_configuration: Add token parameter for x-amz-bucket-object-lock-token Mar 11, 2022
@anGie44
Copy link
Contributor

anGie44 commented Mar 11, 2022

Output of acceptance tests:

--- SKIP: TestAccS3BucketReplicationConfiguration_existingObjectReplication (0.00s)
--- PASS: TestAccS3BucketReplicationConfiguration_replicaModifications (270.65s)
--- PASS: TestAccS3BucketReplicationConfiguration_twoDestination (272.33s)
--- PASS: TestAccS3BucketReplicationConfiguration_replicationTimeControl (272.38s)
--- PASS: TestAccS3BucketReplicationConfiguration_configurationRuleDestinationAddAccessControlTranslation (296.10s)
--- PASS: TestAccS3BucketReplicationConfiguration_basic (314.84s)
--- PASS: TestAccS3BucketReplicationConfiguration_multipleDestinationsEmptyFilter (269.47s)
--- PASS: TestAccS3BucketReplicationConfiguration_multipleDestinationsNonEmptyFilter (269.48s)
--- PASS: TestAccS3BucketReplicationConfiguration_configurationRuleDestinationAccessControlTranslation (291.62s)
--- PASS: TestAccS3BucketReplicationConfiguration_schemaV2DestinationMetrics (268.07s)
--- PASS: TestAccS3BucketReplicationConfiguration_withoutPrefix (267.35s)
--- PASS: TestAccS3BucketReplicationConfiguration_disappears (25.03s)
--- PASS: TestAccS3BucketReplicationConfiguration_schemaV2SameRegion (146.25s)
--- PASS: TestAccS3BucketReplicationConfiguration_filter_tagFilter (267.76s)
--- PASS: TestAccS3BucketReplicationConfiguration_filter_andOperator (289.40s)
--- PASS: TestAccS3BucketReplicationConfiguration_withoutStorageClass (268.14s)
--- PASS: TestAccS3BucketReplicationConfiguration_schemaV2 (267.31s)

Copy link
Contributor

@anGie44 anGie44 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks again @alexb-dd , LGTM 🚀

@anGie44 anGie44 added this to the v4.5.0 milestone Mar 11, 2022
@anGie44 anGie44 merged commit 538c633 into hashicorp:main Mar 11, 2022
@alexb-dd alexb-dd deleted the f-replconfig-token branch March 11, 2022 18:20
@github-actions
Copy link

This functionality has been released in v4.5.0 of the Terraform AWS Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!

@github-actions
Copy link

I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators May 11, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
documentation Introduces or discusses updates to documentation. enhancement Requests to existing resources that expand the functionality or scope. service/s3 Issues and PRs that pertain to the s3 service. size/XS Managed by automation to categorize the size of a PR.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Support PutBucketReplication x-amz-bucket-object-lock-token
2 participants