aws_s3_bucket resource causes unexpected drift

[AnthonyLaiuppa]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform CLI and Terraform AWS Provider Version

Terraform v1.1.8
on darwin_amd64

• provider registry.terraform.io/hashicorp/aws v4.10.0

Affected Resource(s)

• aws_s3_bucket

In conjunction with

• aws_s3_bucket_acl
• aws_s3_bucket_lifecycle_configuration
• aws_s3_bucket_public_access_block
• aws_s3_bucket_server_side_encryption_configuration
• aws_s3_bucket_versioning

Terraform Configuration Files

Please include all Terraform configurations required to reproduce the bug. Bug reports without a functional reproduction may be closed without investigation.

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = ">= 4.10.0"
    }
  }
}

provider "aws" {
  region = "us-east-1"
}

resource "aws_default_vpc" "default" {
  tags = {
    Name = "Default VPC"
  }
}

resource "random_pet" "bucket" {}

resource "aws_s3_bucket" "vpc_flow_logs" {
  bucket = "vpc-flow-log-${random_pet.bucket.id}"
}

resource "aws_s3_bucket_server_side_encryption_configuration" "vpc_flow_logs" {
  bucket = aws_s3_bucket.vpc_flow_logs.id

  rule {
    apply_server_side_encryption_by_default {
      sse_algorithm = "aws:kms"
    }
    bucket_key_enabled = true
  }
}

resource "aws_s3_bucket_versioning" "vpc_flow_logs" {
  bucket = aws_s3_bucket.vpc_flow_logs.id
  versioning_configuration {
    status = "Enabled"
  }
}

resource "aws_s3_bucket_acl" "vpc_flow_logs" {
  bucket = aws_s3_bucket.vpc_flow_logs.id
  acl    = "private"
}

resource "aws_s3_bucket_public_access_block" "vpc_flow_logs" {
  bucket                  = aws_s3_bucket.vpc_flow_logs.id
  block_public_acls       = true
  block_public_policy     = true
  restrict_public_buckets = true
  ignore_public_acls      = true
}

resource "aws_s3_bucket_lifecycle_configuration" "vpc_flow_logs" {
  bucket = aws_s3_bucket.vpc_flow_logs.bucket

  rule {
    status = "Enabled"
    expiration {
      days = 180
    }
    id = "archive"
  }
}

resource "aws_flow_log" "vpc" {
  log_destination          = aws_s3_bucket.vpc_flow_logs.arn
  log_destination_type     = "s3"
  max_aggregation_interval = 600
  traffic_type             = "ALL"
  vpc_id                   = aws_default_vpc.default.id
  destination_options {
    file_format        = "parquet"
    per_hour_partition = true
  }
}

Debug Output

Debug Output GIST

Expected Behavior

With the aws_s3_bucket bucket resource solely managing the name, we would expect the auxiliary resources to manage the state of the other bucket attributes.

No changes. Your infrastructure matches the configuration.

Terraform has compared your real infrastructure against your configuration and found no differences, so no changes are needed.

Actual Behavior

Note: Objects have changed outside of Terraform

Terraform detected the following changes made outside of Terraform since the last "terraform apply":

  # aws_flow_log.vpc has changed
  ~ resource "aws_flow_log" "vpc" {
        id                       = "fl-23e5403"
      + tags                     = {}
        # (8 unchanged attributes hidden)

        # (1 unchanged block hidden)
    }

  # aws_s3_bucket.vpc_flow_logs has changed
  ~ resource "aws_s3_bucket" "vpc_flow_logs" {
        id                          = "vpc-flow-log-renewed-mongrel"
      + policy                      = jsonencode(
            {
              + Id        = "AWSLogDeliveryWrite20150319"
              + Statement = [
                  + {
                      + Action    = "s3:PutObject"
                      + Condition = {
                          + ArnLike      = {
                              + "aws:SourceArn" = "arn:aws:logs:us-east-1:<ACCOUNTID>:*"
                            }
                          + StringEquals = {
                              + "aws:SourceAccount" = "<ACCOUNTID>"
                              + "s3:x-amz-acl"      = "bucket-owner-full-control"
                            }
                        }
                      + Effect    = "Allow"
                      + Principal = {
                          + Service = "delivery.logs.amazonaws.com"
                        }
                      + Resource  = "arn:aws:s3:::vpc-flow-log-renewed-mongrel/AWSLogs/<ACCOUNTID>/*"
                      + Sid       = "AWSLogDeliveryWrite"
                    },
                  + {
                      + Action    = "s3:GetBucketAcl"
                      + Condition = {
                          + ArnLike      = {
                              + "aws:SourceArn" = "arn:aws:logs:us-east-1:<ACCOUNTID>:*"
                            }
                          + StringEquals = {
                              + "aws:SourceAccount" = "<ACCOUNTID>"
                            }
                        }
                      + Effect    = "Allow"
                      + Principal = {
                          + Service = "delivery.logs.amazonaws.com"
                        }
                      + Resource  = "arn:aws:s3:::vpc-flow-log-renewed-mongrel"
                      + Sid       = "AWSLogDeliveryAclCheck"
                    },
                ]
              + Version   = "2012-10-17"
            }
        )
      + tags                        = {}
        # (10 unchanged attributes hidden)


      + lifecycle_rule {
          + abort_incomplete_multipart_upload_days = 0
          + enabled                                = true
          + id                                     = "archive"
          + tags                                   = {}

          + expiration {
              + days                         = 180
              + expired_object_delete_marker = false
            }
        }

      + server_side_encryption_configuration {
          + rule {
              + bucket_key_enabled = true

              + apply_server_side_encryption_by_default {
                  + sse_algorithm = "aws:kms"
                }
            }
        }

      ~ versioning {
          ~ enabled    = false -> true
            # (1 unchanged attribute hidden)
        }
        # (1 unchanged block hidden)
    }


Unless you have made equivalent changes to your configuration, or ignored the relevant attributes using ignore_changes, the following plan may include actions to undo or
respond to these changes.

─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────

No changes. Your infrastructure matches the configuration.

Steps to Reproduce

1. terraform init
2. terraform apply
3. terraform plan

Important Factoids

References

• #0000

[Howard3]

I'm uncertain if this is related but I find that aws_s3_bucket_server_side_encryption_configuration and aws_s3_bucket_logging are on occasion getting disabled, when I run checks for drift it finds the drift but does not seem to fix it.

[thiagolsfortunato]

Same issue here.

resource "aws_s3_bucket" "test" {
  bucket = "test-sse-bucket-drift"
}

resource "aws_s3_bucket_server_side_encryption_configuration" "test" {
  bucket = aws_s3_bucket.test.id

  rule {
    bucket_key_enabled = false

    apply_server_side_encryption_by_default {
      sse_algorithm = "AES256"
    }
  }
}

After applied if I run again terraform plan returns:

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  ~ update in-place

Terraform will perform the following actions:

# aws_s3_bucket.test will be updated in-place
  ~ resource "aws_s3_bucket" "test" {
        id                          = "test-sse-bucket-drift"
        tags                        = {}
        # (11 unchanged attributes hidden)

      + server_side_encryption_configuration {
          + rule {
              + bucket_key_enabled = false -> null

              + apply_server_side_encryption_by_default {
                  + sse_algorithm = "AES256" -> null
                }
            }
        }

        # (1 unchanged block hidden)
    }

Plan: 0 to add, 1 to change, 0 to destroy.

I apply again (second time) and now Terraform wants to remove it when I run terraform plan for the third time:

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  ~ update in-place

Terraform will perform the following actions:

  # aws_s3_bucket.test will be updated in-place
  ~ resource "aws_s3_bucket" "test" {
        id                          = "test-sse-bucket-drift"
        tags                        = {}
        # (11 unchanged attributes hidden)

      - server_side_encryption_configuration {
          - rule {
              - bucket_key_enabled = false -> null

              - apply_server_side_encryption_by_default {
                  - sse_algorithm = "AES256" -> null
                }
            }
        }

        # (1 unchanged block hidden)
    }

Plan: 0 to add, 1 to change, 0 to destroy.

To fix it, I need to add lifecycle ignore_chages, but it's not the typical behavior that I expected.

[m1mohamad]

I'm using aws terraform provider 3.56 and all of sudden I see drift changes on s3 bucket policies , bucket encryption settings like the one mentioned above , in addition to that policies that report changes to removed ( while my code base and terraform state seems to match )

[lordgordon]

I'm using aws terraform provider 3.56 and all of sudden I see drift changes on s3 bucket policies , bucket encryption settings like the one mentioned above , in addition to that policies that report changes to removed ( while my code base and terraform state seem to match )

Same! To work around this, I must add the lifecycle ignore_chages everywhere. How painful!

[lordgordon]

Quick update: I just ran terraform init --upgrade, and all the drifts are now solved 🪄.

[thiagolsfortunato]

@lordgordon Can you share which version you are using?

[lordgordon]

@lordgordon Can you share which version you are using?

Unfortunately, I no longer have access to the repo with that configuration. I think the module's version was up-to-date with the 3.x series.

[thiagolsfortunato]

After updating my module adding the ignore_changes, when I run terraform validate I receive a ⚠️

$ tf version 
Terraform v1.4.4
on darwin_arm64
+ provider registry.terraform.io/hashicorp/aws v4.58.0

$ tf validate
╷
│ Warning: Deprecated attribute
│ 
│   on .terraform/modules/example/main.tf line 34, in resource "aws_s3_bucket" "bucket":
│   34:       acl,
│ 
│ The attribute "acl" is deprecated. Refer to the provider documentation for details.
│ 
│ (and one more similar warning elsewhere)
╵
Success! The configuration is valid, but there were some validation warnings as shown above.

The same warning happens for acl, policy, versioning, and server_side_encryption_configuration 😖

[lordgordon]

@thiagolsfortunato I think yours is due to #28353.