Lake Formation Resource #13267

gmazelier · 2020-05-11T20:52:21Z

Community Note

Please vote on this pull request by adding a 👍 reaction to the original pull request comment to help the community and maintainers prioritize this request
Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for pull request followers and do not help prioritize the request

Relates #9700
Depends on #13250

AWS Lake Formation support: manages the data (Amazon S3 buckets and folders) that is being registered with AWS Lake Formation.

Release note for CHANGELOG:

* **New Resource**: `aws_lakeformation_resource` (#13267)

Output from acceptance testing:

$ make testacc TEST=./aws TESTARGS='-run=TestAccAWSLakeFormationResource'
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./aws -v -count 1 -parallel 20 -run=TestAccAWSLakeFormationResource -timeout 120m
=== RUN   TestAccAWSLakeFormationResource_basic
=== PAUSE TestAccAWSLakeFormationResource_basic
=== RUN   TestAccAWSLakeFormationResource_withRole
=== PAUSE TestAccAWSLakeFormationResource_withRole
=== RUN   TestAccAWSLakeFormationResource_update
=== PAUSE TestAccAWSLakeFormationResource_update
=== CONT  TestAccAWSLakeFormationResource_basic
=== CONT  TestAccAWSLakeFormationResource_update
=== CONT  TestAccAWSLakeFormationResource_withRole
--- PASS: TestAccAWSLakeFormationResource_basic (59.96s)
--- PASS: TestAccAWSLakeFormationResource_withRole (68.19s)
--- PASS: TestAccAWSLakeFormationResource_update (105.42s)
PASS
ok  	github.com/terraform-providers/terraform-provider-aws/aws	106.869s

aws/resource_aws_lakeformation_resource.go

robertstettner · 2020-05-19T10:07:49Z

This is really great! I started working on this as well, and managed to get the data sources for data_aws_lakeformation_resource, data_aws_lakeformation_datalake_settings, and data_aws_lakeformation_permissions to some good state. Maybe we should collaborate on this, and get the resource sorted for permissions? 😄

gmazelier · 2020-05-19T17:59:45Z

@robertstettner Yes it would be great! Good new for data sources support. In a draft pull request #13396 for permissions support. Work well for my first local tests, I need to improve acceptance tests but I will keep you posted in a few days to get your feedback.

teamterraform · 2020-08-18T21:08:44Z

Notification of Recent and Upcoming Changes to Contributions

Thank you for this contribution! There have been a few recent development changes that affect this pull request. We apologize for the inconvenience, especially if there have been long review delays up until now. Please note that this is automated message from an unmonitored account. See the FAQ for additional information on the maintainer team and review prioritization.

If you are unable to complete these updates, please leave a comment for the community and maintainers so someone can potentially continue the work. The maintainers will encourage other contributors to use the existing contribution as the base for additional changes as appropriate. Otherwise, contributions that do not receive updated code or comments from the original contributor may be closed in the future so the maintainers can focus on active items.

For the most up to date information about Terraform AWS Provider development, see the Contributing Guide. Additional technical debt changes can be tracked with the technical-debt label on issues.

As part of updating a pull request with these changes, the most current unit testing and linting will run. These may report issues that were not previously reported.

Action Required: Terraform 0.12 Syntax

Reference: #8950
Reference: #14417

Version 3 and later of the Terraform AWS Provider, which all existing contributions would potentially be added, only supports Terraform 0.12 and later. Certain syntax elements of Terraform 0.11 and earlier show deprecation warnings during runs with Terraform 0.12. Documentation and test configurations, such as those including deprecated string interpolations (some_attribute = "${aws_service_thing.example.id}") should be updated to the newer syntax (some_attribute = aws_service_thing.example.id). Contribution testing will automatically fail on older syntax in the near future. Please see the referenced issues for additional information.

Action Required: Terraform Plugin SDK Version 2

Reference: #14551

The Terraform AWS Provider has been upgraded to the latest version of the Terraform Plugin SDK. Generally, most changes to contributions should only involve updating Go import paths in source code files. Please see the referenced issue for additional information.

Action Required: Removal of website/aws.erb File

Reference: #14712

Any changes to the website/aws.erb file are no longer necessary and should be removed from this contribution to prevent merge issues in the near future when the file is removed from the repository. Please see the referenced issue for additional information.

Upcoming Change of Git Branch Naming

Reference: #14292

Development environments will need their upstream Git branch updated from master to main in the near future. Please see the referenced issue for additional information and scheduling.

Upcoming Change of GitHub Organization

Reference: #14715

This repository will be migrating from https://github.com/terraform-providers/terraform-provider-aws to https://github.com/hashicorp/terraform-provider-aws. No practitioner or developer action is anticipated and most GitHub functionality will automatically redirect to the new location. Go import paths including terraform-providers can remain for now. Please see the referenced issue for additional information and scheduling.

…er PR)

YakDriver · 2020-12-09T18:32:33Z

@gmazelier @ewbankkit I think this is in a pretty good place. I'm waiting to get the other PRs in shape, and add a few data sources before merging. If you have any feedback on this after the changes, please let me know!

Acceptance tests, `us-gov-west-1`

--- PASS: TestAccAWSLakeFormationResource_disappears (27.96s)
--- PASS: TestAccAWSLakeFormationResource_serviceLinkedRole (27.97s)
--- PASS: TestAccAWSLakeFormationResource_basic (28.78s)
--- PASS: TestAccAWSLakeFormationResource_update (51.93s)

Acceptance tests, `us-west-2`

Account without service-linked role:

    provider_test.go:777: skipping tests; missing IAM service-linked role /aws-service-role/lakeformation.amazonaws.com. Please create the role and retry
--- SKIP: TestAccAWSLakeFormationResource_serviceLinkedRole (1.50s)
--- PASS: TestAccAWSLakeFormationResource_disappears (26.91s)
--- PASS: TestAccAWSLakeFormationResource_basic (26.95s)
--- PASS: TestAccAWSLakeFormationResource_update (49.62s)

aws/resource_aws_lakeformation_resource.go

ewbankkit · 2020-12-09T19:15:24Z

aws/resource_aws_lakeformation_resource.go

+	if _, ok := d.GetOk("role_arn"); !ok {
+		return resourceAwsLakeFormationResourceCreate(d, meta)
+	}


Maybe clearer to call RegisterResource() here directly rather than call resourceAwsLakeFormationResourceCreate().
My head initially went to Why is role_arn not ForceNew: true?.

After more investigating, I agree with the original approach of @gmazelier. ForceNew is better. The UpdateResource() API call is quirky and only works for IAM role to IAM role updates. I added new acceptance tests, docs to show the limits.

Co-authored-by: Kit Ewbank <[email protected]>

YakDriver · 2020-12-09T21:08:36Z

Latest:

Acceptance tests, `us-gov-west-1`

--- PASS: TestAccAWSLakeFormationResource_disappears (28.22s)
--- PASS: TestAccAWSLakeFormationResource_serviceLinkedRole (28.24s)
--- PASS: TestAccAWSLakeFormationResource_basic (28.62s)
--- PASS: TestAccAWSLakeFormationResource_updateSLRToRole (50.82s)
--- PASS: TestAccAWSLakeFormationResource_updateRoleToRole (52.89s)

Acceptance tests, `us-west-2`

--- PASS: TestAccAWSLakeFormationResource_basic (27.39s)
--- PASS: TestAccAWSLakeFormationResource_disappears (28.05s)
--- PASS: TestAccAWSLakeFormationResource_serviceLinkedRole (29.05s)
--- PASS: TestAccAWSLakeFormationResource_updateRoleToRole (50.81s)
--- PASS: TestAccAWSLakeFormationResource_updateSLRToRole (50.88s)

ewbankkit · 2020-12-09T22:01:08Z

Verified acceptance tests (us-west-2, without service-linked role):

$ make testacc TEST=./aws/ TESTARGS='-run=TestAccAWSLakeFormationResource_'
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./aws -v -count 1 -parallel 20 -run=TestAccAWSLakeFormationResource_ -timeout 120m
=== RUN   TestAccAWSLakeFormationResource_basic
=== PAUSE TestAccAWSLakeFormationResource_basic
=== RUN   TestAccAWSLakeFormationResource_disappears
=== PAUSE TestAccAWSLakeFormationResource_disappears
=== RUN   TestAccAWSLakeFormationResource_serviceLinkedRole
=== PAUSE TestAccAWSLakeFormationResource_serviceLinkedRole
=== RUN   TestAccAWSLakeFormationResource_updateRoleToRole
=== PAUSE TestAccAWSLakeFormationResource_updateRoleToRole
=== RUN   TestAccAWSLakeFormationResource_updateSLRToRole
=== PAUSE TestAccAWSLakeFormationResource_updateSLRToRole
=== CONT  TestAccAWSLakeFormationResource_basic
=== CONT  TestAccAWSLakeFormationResource_updateRoleToRole
=== CONT  TestAccAWSLakeFormationResource_updateSLRToRole
=== CONT  TestAccAWSLakeFormationResource_serviceLinkedRole
=== CONT  TestAccAWSLakeFormationResource_disappears
=== CONT  TestAccAWSLakeFormationResource_updateSLRToRole
    provider_test.go:777: skipping tests; missing IAM service-linked role /aws-service-role/lakeformation.amazonaws.com. Please create the role and retry
--- SKIP: TestAccAWSLakeFormationResource_updateSLRToRole (0.91s)
=== CONT  TestAccAWSLakeFormationResource_serviceLinkedRole
    provider_test.go:777: skipping tests; missing IAM service-linked role /aws-service-role/lakeformation.amazonaws.com. Please create the role and retry
--- SKIP: TestAccAWSLakeFormationResource_serviceLinkedRole (0.92s)
--- PASS: TestAccAWSLakeFormationResource_basic (26.26s)
--- PASS: TestAccAWSLakeFormationResource_disappears (26.30s)
--- PASS: TestAccAWSLakeFormationResource_updateRoleToRole (47.26s)
PASS
ok  	github.com/terraform-providers/terraform-provider-aws/aws	47.385s

gmazelier · 2020-12-11T18:29:12Z

@YakDriver great!

ghost · 2020-12-18T22:26:24Z

This has been released in version 3.22.0 of the Terraform AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template for triage. Thanks!

ghost · 2021-01-11T17:10:02Z

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!

gmazelier requested a review from a team May 11, 2020 20:52

ewbankkit reviewed May 12, 2020

View reviewed changes

aws/resource_aws_lakeformation_resource.go Outdated Show resolved Hide resolved

ewbankkit reviewed May 12, 2020

View reviewed changes

aws/resource_aws_lakeformation_resource.go Outdated Show resolved Hide resolved

gmazelier mentioned this pull request May 19, 2020

Lake Formation Permissions #13396

Merged

anGie44 force-pushed the master branch from 54db70d to 48ba2f7 Compare July 21, 2020 18:26

YakDriver self-assigned this Nov 17, 2020

YakDriver added the service/lakeformation Issues and PRs that pertain to the lakeformation service. label Nov 17, 2020

YakDriver added partition/aws-us-gov Pertains to the aws-us-gov partition. and removed needs-triage Waiting for first response or review from a maintainer. labels Dec 4, 2020

gmazelier added 13 commits December 8, 2020 09:29

New resource: aws_lakeformation_datalake_settings

abddf85

Check resource state on destroy

9d54d24

Add documentation

e7f3f7d

Add AWS region in settings identifier

c4e9992

Add missing website LakeFormation subcategory

5b52ecf

Rename expand and flatten functions for admins list

e62033e

Make data lake admins list a set

efbe990

Use Lake Formation official spelling

6673cd5

Cleanup

92cb24a

New resource: aws_lakeformation_resource

5ed60ca

Add resource documentation

acc1152

Cleanup

6ba35a3

Better validations

75d25e8

YakDriver added 5 commits December 9, 2020 10:49

tests/resource/lakeformation_resource: Add exists and disappears

09123f6

resource/lakeformation: Remove lakeformation_datalake_settings

8f7614b

docs/lakeformation_resource: Fix formatting

896f026

docs/lakeformation_datalake_settings: Remove from branch (part of lat…

8555491

…er PR)

tests/lakeformation_resource: Fix to use ARN check func

7455563

YakDriver force-pushed the lakeformation_resource branch from 7f583ec to 7455563 Compare December 9, 2020 17:30

resource/lakeformation_resource: Remove go.mod, go.sum

c30f5fb

YakDriver changed the title ~~Lakeformation resource~~ Lake Formation Resource Dec 9, 2020

tests/lakeformation_resource: Add precheck

70b9151

ewbankkit reviewed Dec 9, 2020

View reviewed changes

aws/resource_aws_lakeformation_resource.go Outdated Show resolved Hide resolved

ewbankkit reviewed Dec 9, 2020

View reviewed changes

aws/resource_aws_lakeformation_resource.go Outdated Show resolved Hide resolved

ewbankkit reviewed Dec 9, 2020

View reviewed changes

aws/resource_aws_lakeformation_resource.go Outdated Show resolved Hide resolved

ewbankkit reviewed Dec 9, 2020

View reviewed changes

YakDriver and others added 2 commits December 9, 2020 13:26

resource/lakeformation_resource: Improve error messages

7e920ba

Co-authored-by: Kit Ewbank <[email protected]>

resource/lakeformation_resource: Use ForceNew for more update options

870c35d

YakDriver force-pushed the lakeformation_resource branch from 58a01e6 to 870c35d Compare December 9, 2020 21:03

YakDriver approved these changes Dec 11, 2020

View reviewed changes

YakDriver added this to the v3.22.0 milestone Dec 11, 2020

YakDriver merged commit 29fcdc9 into hashicorp:master Dec 11, 2020

This was referenced Dec 11, 2020

New service: AWS Lake Formation #9700

Closed

Lake Formation Data Lake Settings #13250

Merged

This was referenced Dec 18, 2020

ds/lakeformation_resources: New data source #16845

Closed

ds/lakeformation_effective_permissions: New data source #16846

Closed

ghost locked as resolved and limited conversation to collaborators Jan 11, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Lake Formation Resource #13267

Lake Formation Resource #13267

gmazelier commented May 11, 2020 •

edited

Loading

robertstettner commented May 19, 2020

gmazelier commented May 19, 2020

teamterraform commented Aug 18, 2020

YakDriver commented Dec 9, 2020 •

edited

Loading

ewbankkit Dec 9, 2020

YakDriver Dec 9, 2020

YakDriver commented Dec 9, 2020

ewbankkit commented Dec 9, 2020

gmazelier commented Dec 11, 2020

ghost commented Dec 18, 2020

ghost commented Jan 11, 2021

Lake Formation Resource #13267

Lake Formation Resource #13267

Conversation

gmazelier commented May 11, 2020 • edited Loading

Community Note

robertstettner commented May 19, 2020

gmazelier commented May 19, 2020

teamterraform commented Aug 18, 2020

Notification of Recent and Upcoming Changes to Contributions

Action Required: Terraform 0.12 Syntax

Action Required: Terraform Plugin SDK Version 2

Action Required: Removal of website/aws.erb File

Upcoming Change of Git Branch Naming

Upcoming Change of GitHub Organization

YakDriver commented Dec 9, 2020 • edited Loading

Acceptance tests, us-gov-west-1

Acceptance tests, us-west-2

ewbankkit Dec 9, 2020

Choose a reason for hiding this comment

YakDriver Dec 9, 2020

Choose a reason for hiding this comment

YakDriver commented Dec 9, 2020

Acceptance tests, us-gov-west-1

Acceptance tests, us-west-2

ewbankkit commented Dec 9, 2020

gmazelier commented Dec 11, 2020

ghost commented Dec 18, 2020

ghost commented Jan 11, 2021

gmazelier commented May 11, 2020 •

edited

Loading

YakDriver commented Dec 9, 2020 •

edited

Loading

Acceptance tests, `us-gov-west-1`

Acceptance tests, `us-west-2`

Acceptance tests, `us-gov-west-1`

Acceptance tests, `us-west-2`