Added the aws resource glue resource policy #10361

stijndehaes · 2019-10-03T05:51:57Z

Community Note

Please vote on this pull request by adding a 👍 reaction to the original pull request comment to help the community and maintainers prioritize this request
Please do not leave "+1" comments, they generate extra noise for pull request followers and do not help prioritize the request

Release note for CHANGELOG:

new resource: aws_glue_resource_policy

Output from acceptance testing:

$ make testacc TEST=./aws TESTARGS='-run=TestAccAWSGlueResourcePolicy_Basic'
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./aws -v -count 1 -parallel 20 -run=TestAccAWSGlueResourcePolicy_Basic -timeout 120m
=== RUN   TestAccAWSGlueResourcePolicy_Basic
2019/10/03 07:49:43 [DEBUG] Unable to read account ID from test provider: unconfigured provider
=== PAUSE TestAccAWSGlueResourcePolicy_Basic
=== CONT  TestAccAWSGlueResourcePolicy_Basic
--- PASS: TestAccAWSGlueResourcePolicy_Basic (27.04s)
PASS
ok      github.com/terraform-providers/terraform-provider-aws/aws       27.095s



$ make testacc TEST=./aws TESTARGS='-run=TestAccAWSGlueResourcePolicy_Update'
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./aws -v -count 1 -parallel 20 -run=TestAccAWSGlueResourcePolicy_Update -timeout 120m
=== RUN   TestAccAWSGlueResourcePolicy_Update
2019/10/03 07:42:22 [DEBUG] Unable to read account ID from test provider: unconfigured provider
2019/10/03 07:42:22 [DEBUG] Unable to read account ID from test provider: unconfigured provider
=== PAUSE TestAccAWSGlueResourcePolicy_Update
=== CONT  TestAccAWSGlueResourcePolicy_Update
--- PASS: TestAccAWSGlueResourcePolicy_Update (48.10s)
PASS
ok      github.com/terraform-providers/terraform-provider-aws/aws       48.154s

If someone could help me with the test setup of the acceptance tests that would be nice. I added the environment variable AWS_PROFILE and the acceptance tests get access to aws. However it can not read the account id from the test provider. So I manually added in the policy to run the test and removed it after account. Anyone can help me with my setup?

Also since this is my first PR please help me with anything I forgot to add.

lucienfregosi · 2020-03-26T17:25:01Z

Hi @stijndehaes
Do you plan to fix Travis Integration and merge this PR ? Actually we would love to use this feature.
Maybe we can help you ?
Many Thanks

stijndehaes · 2020-03-26T20:17:22Z

@lucienfregosi Thank you for reminding me :) Rebased and will follow up the PR tomorrow and see if I can fix the Travis issues. I can't get this PR merged though we would have to get the attention form someone from Hashicorp for that. Feel free to review my PR in the mean time :)

stijndehaes · 2020-03-27T08:20:39Z

Build got fixed ready for review :)

bflad

Hi @stijndehaes 👋 Thank you for submitting this. Overall this is shaping up, please see the below for information about fixing the testing and getting this ready for merge. 👍

website/docs/r/glue_resource_policy.html.markdown

aws/resource_aws_glue_resource_policy_test.go

website/docs/r/glue_resource_policy.html.markdown

stijndehaes · 2020-03-27T15:49:31Z

@bflad I updated the code an reran the acceptance tests:

TF_ACC=1 go test ./aws -v -count 1 -parallel 20 -run=TestAccAWSGlueResourcePolicy_Basic -timeout 120m
=== RUN   TestAccAWSGlueResourcePolicy_Basic
=== PAUSE TestAccAWSGlueResourcePolicy_Basic
=== CONT  TestAccAWSGlueResourcePolicy_Basic
--- PASS: TestAccAWSGlueResourcePolicy_Basic (39.43s)
PASS
ok      github.com/terraform-providers/terraform-provider-aws/aws       40.713s

make testacc TEST=./aws TESTARGS='-run=TestAccAWSGlueResourcePolicy_Update'
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./aws -v -count 1 -parallel 20 -run=TestAccAWSGlueResourcePolicy_Update -timeout 120m
=== RUN   TestAccAWSGlueResourcePolicy_Update
=== PAUSE TestAccAWSGlueResourcePolicy_Update
=== CONT  TestAccAWSGlueResourcePolicy_Update
--- PASS: TestAccAWSGlueResourcePolicy_Update (60.31s)
PASS
ok      github.com/terraform-providers/terraform-provider-aws/aws       61.593s

Everything seems to be working now :) Thank you very much for the pointers on the tests and thanks for the review.

stijndehaes · 2020-04-05T12:42:51Z

Rebased again, and also fixed the formatting :)

stijndehaes · 2020-08-06T14:34:37Z

Rebased this branch to the latest master commit

stijndehaes · 2020-08-06T14:40:20Z

Also reran acceptance tests:

➜  terraform-provider-aws git:(feature/aws-glue-data-catalog-resource-policy) ✗ TF_ACC=1 make testacc TEST=./aws TESTARGS='-run=TestAccAWSGlueResourcePolicy_Basic'
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./aws -v -count 1 -parallel 20 -run=TestAccAWSGlueResourcePolicy_Basic -timeout 120m
=== RUN   TestAccAWSGlueResourcePolicy_Basic
=== PAUSE TestAccAWSGlueResourcePolicy_Basic
=== CONT  TestAccAWSGlueResourcePolicy_Basic
--- PASS: TestAccAWSGlueResourcePolicy_Basic (27.86s)
PASS
ok      github.com/terraform-providers/terraform-provider-aws/aws       32.897s
➜  terraform-provider-aws git:(feature/aws-glue-data-catalog-resource-policy) ✗ make testacc TEST=./aws TESTARGS='-run=TestAccAWSGlueResourcePolicy_Update'
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./aws -v -count 1 -parallel 20 -run=TestAccAWSGlueResourcePolicy_Update -timeout 120m
=== RUN   TestAccAWSGlueResourcePolicy_Update
=== PAUSE TestAccAWSGlueResourcePolicy_Update
=== CONT  TestAccAWSGlueResourcePolicy_Update
--- PASS: TestAccAWSGlueResourcePolicy_Update (45.73s)
PASS
ok      github.com/terraform-providers/terraform-provider-aws/aws       49.653s

teamterraform · 2020-08-18T20:17:22Z

Notification of Recent and Upcoming Changes to Contributions

Thank you for this contribution! There have been a few recent development changes that affect this pull request. We apologize for the inconvenience, especially if there have been long review delays up until now. Please note that this is automated message from an unmonitored account. See the FAQ for additional information on the maintainer team and review prioritization.

If you are unable to complete these updates, please leave a comment for the community and maintainers so someone can potentially continue the work. The maintainers will encourage other contributors to use the existing contribution as the base for additional changes as appropriate. Otherwise, contributions that do not receive updated code or comments from the original contributor may be closed in the future so the maintainers can focus on active items.

For the most up to date information about Terraform AWS Provider development, see the Contributing Guide. Additional technical debt changes can be tracked with the technical-debt label on issues.

As part of updating a pull request with these changes, the most current unit testing and linting will run. These may report issues that were not previously reported.

Terraform 0.12 Syntax

Reference: #8950
Reference: #14417

Version 3 and later of the Terraform AWS Provider, which all existing contributions would potentially be added, only supports Terraform 0.12 and later. Certain syntax elements of Terraform 0.11 and earlier show deprecation warnings during runs with Terraform 0.12. Documentation and test configurations, such as those including deprecated string interpolations (some_attribute = "${aws_service_thing.example.id}") should be updated to the newer syntax (some_attribute = aws_service_thing.example.id). Contribution testing will automatically fail on older syntax in the near future. Please see the referenced issues for additional information.

Action Required: Terraform Plugin SDK Version 2

Reference: #14551

The Terraform AWS Provider has been upgraded to the latest version of the Terraform Plugin SDK. Generally, most changes to contributions should only involve updating Go import paths in source code files. Please see the referenced issue for additional information.

Action Required: Removal of website/aws.erb File

Reference: #14712

Any changes to the website/aws.erb file are no longer necessary and should be removed from this contribution to prevent merge issues in the near future when the file is removed from the repository. Please see the referenced issue for additional information.

Upcoming Change of Git Branch Naming

Reference: #14292

Development environments will need their upstream Git branch updated from master to main in the near future. Please see the referenced issue for additional information and scheduling.

Upcoming Change of GitHub Organization

Reference: #14715

This repository will be migrating from https://github.com/terraform-providers/terraform-provider-aws to https://github.com/hashicorp/terraform-provider-aws. No practitioner or developer action is anticipated and most GitHub functionality will automatically redirect to the new location. Go import paths including terraform-providers can remain for now. Please see the referenced issue for additional information and scheduling.

DrFaust92 · 2020-08-23T17:50:59Z

Hey @stijndehaes, do you mind rebasing again and fixing things up according to the bot message above?

on another note, can you add the ResourceArn to set the policy for a specific resource? optionally support for EnableHybrid can also be added

stijndehaes · 2020-08-24T07:12:06Z

ResourceArn

Hey I will update my PR and I'll have to look into ResourceArn didn't know that that existed.

DrFaust92 · 2020-09-23T21:03:18Z

Hey @stijndehaes, are you still lokking into this?

stijndehaes · 2020-10-01T09:03:06Z

Hey @stijndehaes, are you still lokking into this?

I will make the time to do this next week :) It's not hard to add I just have to make sure I make the right integration test

stijndehaes · 2020-10-01T17:53:38Z

@DrFaust92 I had a look at adding ResourceArn support tonight however it looks like this is actually unsupported for glue. For example this aws cli command (pleas don't forget to replace ACCOUNTID):

aws glue put-resource-policy --region us-west-2 --resource-arn arn:aws:glue:us-west-2:ACCOUNTID:catalog --policy-in-json '{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Principal": {
        "AWS": ["*"]
      },
      "Effect": "Allow",
      "Action": [
        "glue:CreateTable"
      ],
      "Resource": [
        "arn:aws:glue:us-west-2:ACCOUNTID:catalog"
      ]
    }
  ]
}'

results in the following error:
An error occurred (InvalidInputException) when calling the PutResourcePolicy operation: Glue resource level policy is not currently supported

So I don't think this field is supported yet for glue. I also don't find any mention anywere for setting resource policies on database or tables in the documentation.

DrFaust92 · 2020-10-16T18:20:04Z

Hey @stijndehaes, if so whats the use case for the general policy?

ferrouswheel · 2020-10-16T18:48:11Z

@DrFaust92 I've been watching this PR for a wee while as I need to allow glue catalog access from another AWS account.

Thiago-Dantas · 2020-10-16T19:49:03Z

@DrFaust92 same as @ferrouswheel here

aws/resource_aws_glue_resource_policy_test.go

stijndehaes · 2020-10-21T17:28:22Z

Hey @stijndehaes, if so whats the use case for the general policy?

The general policy just allows you to set access to the catalog and all the databases/tables that are under the catalog.
So it allows you to say another account can read a table of this catalog. Or all tables of this catalog.
For more information you can have a look here: https://docs.aws.amazon.com/glue/latest/dg/glue-resource-policies.html

DrFaust92 · 2020-10-21T17:52:46Z

Hey @stijndehaes, when you have time I just noticed a disappears test is missing. see https://github.com/terraform-providers/terraform-provider-aws/blob/master/docs/contributing/running-and-writing-acceptance-tests.md#disappears-acceptance-tests

DrFaust92

some more comments,

After running tests I realised we need to serialize tests: see example

aws/resource_aws_glue_resource_policy.go

stijndehaes · 2020-10-21T18:42:31Z

Hey @stijndehaes, when you have time I just noticed a disappears test is missing. see https://github.com/terraform-providers/terraform-provider-aws/blob/master/docs/contributing/running-and-writing-acceptance-tests.md#disappears-acceptance-tests

some more comments,

After running tests I realised we need to serialize tests: see example

Created the serialize tests, also changed the tests to resource.Test instead of paralleltest since they are editing the same resource.

Output of integration tests:

➜  terraform-provider-aws git:(feature/aws-glue-data-catalog-resource-policy) ✗ make testacc TESTARGS='-run=TestAccAWSGlue_serial'
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./aws -v -count 1 -parallel 20 -run=TestAccAWSGlue_serial -timeout 120m
=== RUN   TestAccAWSGlue_serial
=== RUN   TestAccAWSGlue_serial/ResourcePolicy
=== RUN   TestAccAWSGlue_serial/ResourcePolicy/basic
=== RUN   TestAccAWSGlue_serial/ResourcePolicy/update
=== RUN   TestAccAWSGlue_serial/ResourcePolicy/disappears
--- PASS: TestAccAWSGlue_serial (103.45s)
    --- PASS: TestAccAWSGlue_serial/ResourcePolicy (103.45s)
        --- PASS: TestAccAWSGlue_serial/ResourcePolicy/basic (29.37s)
        --- PASS: TestAccAWSGlue_serial/ResourcePolicy/update (48.87s)
        --- PASS: TestAccAWSGlue_serial/ResourcePolicy/disappears (25.21s)
PASS
ok      github.com/terraform-providers/terraform-provider-aws/aws       105.155s

stijndehaes · 2020-10-22T07:11:00Z

Fixed the last linting error everything is green now :)

DrFaust92

LGTM

--- PASS: TestAccAWSGlue_serial (141.95s)    --- PASS: TestAccAWSGlue_serial/ResourcePolicy (141.95s)
        --- PASS: TestAccAWSGlue_serial/ResourcePolicy/basic (39.27s)
        --- PASS: TestAccAWSGlue_serial/ResourcePolicy/update (69.54s)
        --- PASS: TestAccAWSGlue_serial/ResourcePolicy/disappears (33.13s)

aws/resource_aws_glue_resource_policy_test.go

breathingdust · 2020-10-22T22:15:36Z

LGTM 🚀 Thanks @stijndehaes!

Verified Acceptance Tests in Commercial (us-west-2)

make testacc TEST=./aws TESTARGS='-run=TestAccAWSGlue_serial'
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./aws -v -count 1 -parallel 20 -run=TestAccAWSGlue_serial -timeout 120m
=== RUN   TestAccAWSGlue_serial
=== RUN   TestAccAWSGlue_serial/ResourcePolicy
=== RUN   TestAccAWSGlue_serial/ResourcePolicy/disappears
=== RUN   TestAccAWSGlue_serial/ResourcePolicy/basic
=== RUN   TestAccAWSGlue_serial/ResourcePolicy/update
--- PASS: TestAccAWSGlue_serial (59.41s)
    --- PASS: TestAccAWSGlue_serial/ResourcePolicy (59.41s)
        --- PASS: TestAccAWSGlue_serial/ResourcePolicy/disappears (14.40s)
        --- PASS: TestAccAWSGlue_serial/ResourcePolicy/basic (15.77s)
        --- PASS: TestAccAWSGlue_serial/ResourcePolicy/update (29.25s)
PASS
ok  	github.com/terraform-providers/terraform-provider-aws/aws	60.858s

Verified Acceptance Tests in GovCloud (us-gov-west-1)

make testacc TEST=./aws TESTARGS='-run=TestAccAWSGlue_serial'
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./aws -v -count 1 -parallel 20 -run=TestAccAWSGlue_serial -timeout 120m
=== RUN   TestAccAWSGlue_serial
=== RUN   TestAccAWSGlue_serial/ResourcePolicy
=== RUN   TestAccAWSGlue_serial/ResourcePolicy/disappears
=== RUN   TestAccAWSGlue_serial/ResourcePolicy/basic
=== RUN   TestAccAWSGlue_serial/ResourcePolicy/update
--- PASS: TestAccAWSGlue_serial (37.45s)
    --- PASS: TestAccAWSGlue_serial/ResourcePolicy (37.45s)
        --- PASS: TestAccAWSGlue_serial/ResourcePolicy/disappears (9.15s)
        --- PASS: TestAccAWSGlue_serial/ResourcePolicy/basic (10.65s)
        --- PASS: TestAccAWSGlue_serial/ResourcePolicy/update (17.65s)
PASS
ok  	github.com/terraform-providers/terraform-provider-aws/aws	38.817s

ghost · 2020-10-29T22:20:57Z

This has been released in version 3.13.0 of the Terraform AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template for triage. Thanks!

ghost · 2020-11-22T17:10:36Z

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!

stijndehaes requested a review from a team October 3, 2019 05:51

stijndehaes mentioned this pull request Oct 3, 2019

Support for setting AWS Glue Data Catalog Encryption/settings #9736

Closed

ghost added the documentation Introduces or discusses updates to documentation. label Oct 3, 2019

IslamAzab approved these changes Mar 27, 2020

View reviewed changes

bflad self-assigned this Mar 27, 2020

bflad added the new-resource Introduces a new resource. label Mar 27, 2020

bflad suggested changes Mar 27, 2020

View reviewed changes

stijndehaes requested a review from bflad March 27, 2020 15:50

anGie44 force-pushed the master branch from 6c6566f to 081bf57 Compare June 29, 2020 16:14

DrFaust92 reviewed Oct 21, 2020

View reviewed changes

aws/resource_aws_glue_resource_policy_test.go Outdated Show resolved Hide resolved

DrFaust92 assigned DrFaust92 and unassigned bflad Oct 21, 2020

DrFaust92 suggested changes Oct 21, 2020

View reviewed changes

aws/resource_aws_glue_resource_policy.go Outdated Show resolved Hide resolved

aws/resource_aws_glue_resource_policy.go Outdated Show resolved Hide resolved

ghost added size/XL Managed by automation to categorize the size of a PR. and removed size/L Managed by automation to categorize the size of a PR. labels Oct 21, 2020

stijndehaes requested a review from DrFaust92 October 21, 2020 18:44

DrFaust92 approved these changes Oct 22, 2020

View reviewed changes

YakDriver requested changes Oct 22, 2020

View reviewed changes

aws/resource_aws_glue_resource_policy_test.go Outdated Show resolved Hide resolved

Added the aws resource glue resource policy

f3099aa

stijndehaes requested a review from YakDriver October 22, 2020 17:45

breathingdust added this to the v3.13.0 milestone Oct 22, 2020

breathingdust merged commit bd05f3b into hashicorp:master Oct 22, 2020

breathingdust added a commit that referenced this pull request Oct 22, 2020

Update CHANGELOG.md for #10361

9d55412

ghost locked as resolved and limited conversation to collaborators Nov 22, 2020

stijndehaes deleted the feature/aws-glue-data-catalog-resource-policy branch November 22, 2020 17:22

DrFaust92 removed their assignment Jun 6, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Added the aws resource glue resource policy #10361

Added the aws resource glue resource policy #10361

stijndehaes commented Oct 3, 2019 •

edited

Loading

lucienfregosi commented Mar 26, 2020

stijndehaes commented Mar 26, 2020

stijndehaes commented Mar 27, 2020

bflad left a comment

stijndehaes commented Mar 27, 2020 •

edited

Loading

stijndehaes commented Apr 5, 2020 •

edited

Loading

stijndehaes commented Aug 6, 2020

stijndehaes commented Aug 6, 2020

teamterraform commented Aug 18, 2020

DrFaust92 commented Aug 23, 2020

stijndehaes commented Aug 24, 2020

DrFaust92 commented Sep 23, 2020

stijndehaes commented Oct 1, 2020

stijndehaes commented Oct 1, 2020

DrFaust92 commented Oct 16, 2020

ferrouswheel commented Oct 16, 2020

Thiago-Dantas commented Oct 16, 2020

stijndehaes commented Oct 21, 2020

DrFaust92 commented Oct 21, 2020

DrFaust92 left a comment

stijndehaes commented Oct 21, 2020

stijndehaes commented Oct 22, 2020

DrFaust92 left a comment

breathingdust commented Oct 22, 2020

ghost commented Oct 29, 2020

ghost commented Nov 22, 2020

Added the aws resource glue resource policy #10361

Added the aws resource glue resource policy #10361

Conversation

stijndehaes commented Oct 3, 2019 • edited Loading

Community Note

lucienfregosi commented Mar 26, 2020

stijndehaes commented Mar 26, 2020

stijndehaes commented Mar 27, 2020

bflad left a comment

Choose a reason for hiding this comment

stijndehaes commented Mar 27, 2020 • edited Loading

stijndehaes commented Apr 5, 2020 • edited Loading

stijndehaes commented Aug 6, 2020

stijndehaes commented Aug 6, 2020

teamterraform commented Aug 18, 2020

Notification of Recent and Upcoming Changes to Contributions

Terraform 0.12 Syntax

Action Required: Terraform Plugin SDK Version 2

Action Required: Removal of website/aws.erb File

Upcoming Change of Git Branch Naming

Upcoming Change of GitHub Organization

DrFaust92 commented Aug 23, 2020

stijndehaes commented Aug 24, 2020

DrFaust92 commented Sep 23, 2020

stijndehaes commented Oct 1, 2020

stijndehaes commented Oct 1, 2020

DrFaust92 commented Oct 16, 2020

ferrouswheel commented Oct 16, 2020

Thiago-Dantas commented Oct 16, 2020

stijndehaes commented Oct 21, 2020

DrFaust92 commented Oct 21, 2020

DrFaust92 left a comment

Choose a reason for hiding this comment

stijndehaes commented Oct 21, 2020

stijndehaes commented Oct 22, 2020

DrFaust92 left a comment

Choose a reason for hiding this comment

breathingdust commented Oct 22, 2020

ghost commented Oct 29, 2020

ghost commented Nov 22, 2020

stijndehaes commented Oct 3, 2019 •

edited

Loading

stijndehaes commented Mar 27, 2020 •

edited

Loading

stijndehaes commented Apr 5, 2020 •

edited

Loading