Error during terraform apply after importing manually created private S3 bucket using TF v1.0.11

[abjavaprof]

Hello Team,

---

name: Error: Error putting S3 ACL: AccessControlListNotSupported: The bucket does not allow ACLs
about: We are facing an issue when importing a manually created S3 bucket using Terraform v1.0.11 (hashicorp/aws v3.69.0).

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform CLI and Terraform AWS Provider Version

TF Version - Terraform v1.0.11
Provider AWS - hashicorp/aws v3.69.0

Affected Resource(s)

• aws_cloudfront_origin_access_identity
• aws_s3_bucket
• aws_s3_bucket_public_access_block
• aws_s3_bucket_policy

Terraform Configuration Files

1. backend.tf

terraform {
  backend "s3" {
    region  = "us-east-1"
    profile = "xxxxx-develop-profile"
    key     = "terraform-statefile"
    bucket  = "xxxx_xxx_bucket"
  }
}

2. providers.tf

provider "aws" {
  profile = var.profile
  region  = "us-east-1"
}

3. variables.tf

variable "profile" {
  type    = string
  default = "xxxxx-develop-profile"
}

variable "bucket_name" {
  type    = string
  default = "my-test-bucket-82943572"
}

4. deps.tf

data "aws_iam_policy_document" "s3_bucket_policy" {
  statement {
    actions   = ["s3:GetObject"]
    resources = ["${aws_s3_bucket.cf-s3.arn}/*"]
    principals {
      type        = "AWS"
      identifiers = [aws_cloudfront_origin_access_identity.cf-oai.iam_arn]
    }
  }
}

5. resources.tf

resource "aws_cloudfront_origin_access_identity" "cf-oai" {
  comment = "my-test-OAI"
}

# Creating S3 Bucket
resource "aws_s3_bucket" "cf-s3" {
  bucket = var.bucket_name
  acl    = "private"
  tags   = var.tags
}

# Block all public access to the new bucket.
resource "aws_s3_bucket_public_access_block" "cf-s3" {
  bucket                  = aws_s3_bucket.cf-s3.id
  block_public_acls       = true
  block_public_policy     = true
  ignore_public_acls      = true
  restrict_public_buckets = true
}

resource "aws_s3_bucket_policy" "cf-s3-bucket-policy" {
  # avoiding 2 simultaneous conflicting operations on the bucket
  depends_on  = [ aws_s3_bucket_public_access_block.cf-s3 ]
  bucket      = aws_s3_bucket.cf-s3.id
  policy      = data.aws_iam_policy_document.s3_bucket_policy.json
}

Debug Output

 Error: Error putting S3 ACL: AccessControlListNotSupported: The bucket does not allow ACLs
	status code: 400, request id: VM1XPX6VZDWGD51F, host id: D0Bu4J9yl0HiRs7nQomGqaklvqsUShUb/MBT0RuEyGiconmqZHMXqE1K2eG/v9UTbSF/FlkLeIc=
 
   with aws_s3_bucket.cf-s3,
   on resources.tf line 2, in resource "aws_s3_bucket" "cf-s3":
    2: resource "aws_s3_bucket" "cf-s3" {

Panic Output

N/A

Expected Behavior

1. Import offollowing resources is successful
   a. aws_cloudfront_origin_access_identity
   b. aws_s3_bucket
   c. aws_s3_bucket_public_access_block
   d. aws_s3_bucket_policy
2. terraform apply command runs successfully after importing S3 bucket resources.

Actual Behavior

1. Import offollowing resources is successful
   a. aws_s3_bucket
   b. aws_s3_bucket_public_access_block
   c. aws_s3_bucket_policy
2. Running terraform apply --auto-approve gives following error:
   a. Error: Error putting S3 ACL: AccessControlListNotSupported: The bucket does not allow ACLs

Steps to Reproduce

1. terraform import aws_cloudfront_origin_access_identitycf-oai my-test-OAI
2. terraform import aws_s3_bucket_policy.cf-s3-bucket-policy my-test-bucket-82943572
3. terraform import aws_s3_bucket_public_access_block.cf-s3 my-test-bucket-82943572
4. terraform import aws_s3_bucket.cf-s3 my-test-bucket-82943572
5. terraform apply --auto-approve

Important Factoids

N/A

[justinretzolk]

Hey @abjavaprof 👋 Thank you for taking the time to submit this issue! So that we have all of the necessary information in order to investigate this, can you update the issue description with the rest of the information requested in the bug template, including the community note and (scrubbed as necessary) debug logs?

[abjavaprof]

@justinretzolk I have updated the bug description as per the template. Please let me know if you need more info from my end.

Looking forward to hearing your response.

Thank you!

[KyleKotowick]

This issue is a result of the recent S3 feature to disable ACLs altogether. The aws_s3_bucket resource will also need an update to support this new feature.

[ghost]

Hey @abjavaprof, I have a workaround for this. You need to create an additional resource:

 resource "aws_s3_bucket_ownership_controls" "cf-s3-ownership" {
  bucket = aws_s3_bucket.cf-s3.id
  rule {
    object_ownership = "BucketOwnerEnforced"
  }
}

and run the terraform import command terraform import aws_s3_bucket_ownership_controls.cf-s3-ownership my-test-bucket-82943572
Once you have this resource imported, you can perform the terraform apply without issues

[jlambert121]

This fix doesn't seem to work for me (upgrading from aws provider 3 to 4.11.0)

I have these blocks:

resource "aws_s3_bucket_ownership_controls" "mybucket" {
  bucket = aws_s3_bucket.mybucket.id

  rule {
    object_ownership = "BucketOwnerEnforced"
  }
}

resource "aws_s3_bucket_acl" "mybucket" {
  bucket = aws_s3_bucket.mybucket.id
  acl    = "private"
}

I have imported both, tried not importing both or neither, no change. I keep getting AccessControlListNotSupported: The bucket does not allow ACLs status code: 400. Am I missing something?

[gerardokaztro]

@hashicorp-cloud #22069 @justinretzolk
For those of you who got this error message, when importing an S3 bucket created from the AWS console into terraform:

Error: Error putting S3 ACL: AccessControlListNotSupported: The bucket does not allow ACLs

You need add this resource on your “file”.tf (s3.tf on my case)

resource “aws_s3_bucket” “mybucket2” {
  bucket = “mybucket-awsconsole-to-tf”
}

resource “aws_s3_bucket_acl” “mybucket2-acl” {
  bucket = aws_s3_bucket.mybucket2.id
  acl    = “private”
}

resource “aws_s3_bucket_ownership_controls” “mybucket2-acl-ownership” {
  bucket = aws_s3_bucket.mybucket2.id
  rule {
    object_ownership = “BucketOwnerEnforced”
  }
}

After this, you must first go to the S3 console bucket and in the “Permissions” tab look for the “Object Ownership” feature, which by default will have the value of “Object Writer”. You should change this to “Bucket owner enforced”

Now, you should use the import command:
terraform import aws_s3_bucket_ownership_controls.mybucket2-acl-ownership mybucket-awsconsole-to-tf

Once this import is applied, you will be able to do terraform plan and apply without any issue.

The reason is because With this feature, AWS disables access control lists (ACLs). When this setting is applied at the bucket level, all objects in an S3 bucket are owned by the bucket owner, and ACLs are no longer used to grant permissions. ACLs were the original way to control access on S3. Subsequently, IAM and policies were introduced to control permissions on AWS resources. Now by enabling the S3 Object Ownership feature you can change the way S3 does access control to a bucket so that only IAM policies are used and this is exactly where using Terraform, which by the way uses IAM permissions to authenticate and take administration of our environment in AWS (either by AK / SK, Role, etc) the error mentioned at the beginning of this comment is obtained because AWS S3 by default configuration has the Object Ownership at the Object Writer level only which keeps ACLs enabled and in use to grant access to the bucket and its objects, making it impossible for terraform to take the necessary administration to apply imported changes.

[Pabloin]

@hashicorp-cloud #22069 @justinretzolk For those of you who got this error message, when importing an S3 bucket created from the AWS console into terraform:

Error: Error putting S3 ACL: AccessControlListNotSupported: The bucket does not allow ACLs

You need add this resource on your “file”.tf (s3.tf on my case)

resource “aws_s3_bucket” “mybucket2” {
  bucket = “mybucket-awsconsole-to-tf”
}

resource “aws_s3_bucket_acl” “mybucket2-acl” {
  bucket = aws_s3_bucket.mybucket2.id
  acl    = “private”
}

resource “aws_s3_bucket_ownership_controls” “mybucket2-acl-ownership” {
  bucket = aws_s3_bucket.mybucket2.id
  rule {
    object_ownership = “BucketOwnerEnforced”
  }
}

After this, you must first go to the S3 console bucket and in the “Permissions” tab look for the “Object Ownership” feature, which by default will have the value of “Object Writer”. You should change this to “Bucket owner enforced”

Now, you should use the import command: terraform import aws_s3_bucket_ownership_controls.mybucket2-acl-ownership mybucket-awsconsole-to-tf

Once this import is applied, you will be able to do terraform plan and apply without any issue.

The reason is because With this feature, AWS disables access control lists (ACLs). When this setting is applied at the bucket level, all objects in an S3 bucket are owned by the bucket owner, and ACLs are no longer used to grant permissions. ACLs were the original way to control access on S3. Subsequently, IAM and policies were introduced to control permissions on AWS resources. Now by enabling the S3 Object Ownership feature you can change the way S3 does access control to a bucket so that only IAM policies are used and this is exactly where using Terraform, which by the way uses IAM permissions to authenticate and take administration of our environment in AWS (either by AK / SK, Role, etc) the error mentioned at the beginning of this comment is obtained because AWS S3 by default configuration has the Object Ownership at the Object Writer level only which keeps ACLs enabled and in use to grant access to the bucket and its objects, making it impossible for terraform to take the necessary administration to apply imported changes.

Thanks!

[jlambert121]

@Pabloin Thanks for the tips. I did every combination of importing/deleting stuff from TF and updating things in the console trying to make TF happy last week (including setting the bucket owner in the console and them importing it) without success. It seems like there's a larger TF bug to work around here.

[ghost]

@jlambert121 the solution I posted above works for aws provider v3, v4 is a whole different topic.
My advice would to be import it with version 3 and then follow the hashicorp guide to upgrade buckets from v3 to v4.

[Shubham-tvlk]

Try rerunning terraform apply. It will solve the issue.

[amazingguni]

I found better solution 😄 Adding depends_on!
I wish it can be helpful to someone

resource “aws_s3_bucket” “mybucket2” {
  bucket = “mybucket-awsconsole-to-tf”
}

resource “aws_s3_bucket_acl” “mybucket2-acl” {
  bucket = aws_s3_bucket.mybucket2.id
  acl    = “private”
}

resource “aws_s3_bucket_ownership_controls” “mybucket2-acl-ownership” {
  bucket = aws_s3_bucket.mybucket2.id
  rule {
    object_ownership = “BucketOwnerEnforced”
  }
  # Add just this depends_on condition
  depends_on = [aws_s3_bucket_acl.mybucket2-acl]
}

[justinretzolk]

Hey y'all 👋 It looks like this is resolved by the addition of the aws_s3_bucket_acl resource. Given that, I'll mark this issue as closed. If you feel I've done this in error, please do let me know.

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.