providers/aws: Add support for policy on S3 bucket

[justincampbell]

Adds support for policies on S3 buckets: http://docs.aws.amazon.com/AmazonS3/latest/dev/example-bucket-policies.html

resource "aws_s3_bucket" "default" {
  bucket = "hc-jc-tf-test"
  policy = "${file("policy.json")}"
}

[justincampbell]

This currently fails due to aws/aws-sdk-go#236

I believe @m-s-austin also ran into this in iJoinSolutions@cd45e45

[m-s-austin]

That does indeed appear to be bugged in the sdk, I had not had a chance to test it directly yet.

[justincampbell]

This has been resolved in aws/aws-sdk-go@c37b3cb

[justincampbell]

--- PASS: TestAccAWSS3Bucket_basic (3.80s)
--- PASS: TestAccAWSS3Bucket_Policy (4.22s)
--- PASS: TestAccAWSS3Bucket_Website (7.14s)
--- PASS: TestAccAWSS3Bucket_WebsiteRedirect (4.78s)

[m-s-austin]

@justincampbell I have been testing your code today, and one thing I noticed, in the version I was working on with a separate resource for the policy, I could delete the policy resource from the tf file, and then terraform apply would also delete the policy from the bucket. It seems that in this version I can not achieve the same functionality? Deleting the argument has no action and if I try to explicitly set policy = "" I get an error policy: "" => "Error parsing JSON: unexpected end of JSON input"

leaving me with no way to remove a policy from tf.

[catsby]

TestAccAWSS3Bucket_Policy is not passing for me:

provider.aws
--- FAIL: TestAccAWSS3Bucket_Policy (3.11s)
    testing.go:131: Step 0 error: Check failed: bad policy, found nil, expected: { "Version": "2008-10-17", "Statement": [ { "Sid": "", "Effect": "Allow", "Principal": { "AWS": "*" }, "Action": "s3:GetObject", "Resource": "arn:aws:s3:::tf-test-bucket-7742106432384881169/*" } ] }
FAIL
exit status 1
FAIL    github.com/hashicorp/terraform/builtin/providers/aws    3.126s

[justincampbell]

@m-s-austin Are you using the latest code from this PR? It seems to remove properly for me.

The "" => "Error parsing JSON: unexpected end of JSON input" behavior is unfortunate with invalid JSON. We're using normalizeJson as a SchemaStateFunc, which only returns a string: https://github.com/hashicorp/terraform/blob/master/helper/schema/schema.go#L174

I'm not sure what to do here. We could change SchemaStateFunc to also return an error. Or, just return an empty string, or possibly the original string passed in, on invalid JSON.

[justincampbell]

@catsby You may need to update aws-sdk-go

[catsby]

@justincampbell that did help, thanks 😄

It seems though that I have invalid JSON:

+ aws_s3_bucket.bucket
    acl:              "" => "public-read"
    bucket:           "" => "tf-test-bucket-14"
    hosted_zone_id:   "" => "<computed>"
    policy:           "" => "Error parsing JSON: invalid character '\\'' looking for beginning of object key string"
    region:           "" => "<computed>"
    website_endpoint: "" => "<computed>"

I do not want that policy applied 😸

[catsby]

That issue aside, I used a valid policy JSON doc (from a file) and it seemed to apply / remove as expected.

@justincampbell did you test this with in-line policies? That's where I got the parsing error, I wonder if Go or HCL did something there. If you anticipate most people using from a file, we should provide show the "${file("./policy.json")}" example in the docs

Other than that, 👍

[m-s-austin]

@justincampbell sorry that was my bad I forgot to update sdk.

Seems to be working now

[justincampbell]

@catsby Does a heredoc work? Otherwise you would have to escape all of the quotes

[m-s-austin]

I have tested with this format it works fine:

    policy = <<EOF
{
  "Id": "Policy1431447664360",
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "Stmt1431447659874",
      "Action": [
        "s3:GetObject"
      ],
      "Effect": "Allow",
      "Resource": "arn:aws:s3:::my_tf_test_bucket/*",
      "Principal": "*"
    }
  ]
}
EOF

[justincampbell]

@m-s-austin Thanks for testing!

[justincampbell]

@catsby Added policy = "#{file("policy.json")}" to the website example

[catsby]

LGTM 👍

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.