RDS instance got recreated after another apply without any change

[snowsky]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform CLI and Terraform AWS Provider Version

Terraform v0.13.5

• provider registry.terraform.io/hashicorp/aws v3.23.0
• provider registry.terraform.io/hashicorp/local v1.4.0
• provider registry.terraform.io/hashicorp/null v2.1.2
• provider registry.terraform.io/hashicorp/template v2.2.0

Affected Resource(s)

• RDS instance

Terraform Configuration Files

The output of terraform plan:

-/+ resource "aws_db_instance" "default" {
      ~ address                               = "a-staging-test-rds.xxxx.us-east-1.rds.amazonaws.com" -> (known after apply)
        allocated_storage                     = 20
        allow_major_version_upgrade           = false
        apply_immediately                     = true
      ~ arn                                   = "arn:aws:rds:us-east-1:xxx:db:a-staging-test-rds" -> (known after apply)
        auto_minor_version_upgrade            = true
      ~ availability_zone                     = "us-east-1b" -> (known after apply)
        backup_retention_period               = 0
        backup_window                         = "22:00-03:00"
        ca_cert_identifier                    = "rds-ca-2019"
      + character_set_name                    = (known after apply)
        copy_tags_to_snapshot                 = true
        db_subnet_group_name                  = "a-staging-test-rds"
        delete_automated_backups              = true
        deletion_protection                   = false
      - enabled_cloudwatch_logs_exports       = [] -> null
      ~ endpoint                              = "a-staging-test-rds.xxx.us-east-1.rds.amazonaws.com:3306" -> (known after apply)
        engine                                = "mysql"
        engine_version                        = "8.0.20"
        final_snapshot_identifier             = "a-staging-test-rds-final-snapshot"
      ~ hosted_zone_id                        = "xxx" -> (known after apply)
        iam_database_authentication_enabled   = false
      ~ id                                    = "a-staging-test-rds" -> (known after apply)
        identifier                            = "a-staging-test-rds"
      + identifier_prefix                     = (known after apply)
        instance_class                        = "db.t2.small"
        iops                                  = 0
      + kms_key_id                            = (known after apply)
      ~ latest_restorable_time                = "0001-01-01T00:00:00Z" -> (known after apply)
      ~ license_model                         = "general-public-license" -> (known after apply)
        maintenance_window                    = "mon:03:00-mon:04:00"
        max_allocated_storage                 = 1000
        monitoring_interval                   = 0
      + monitoring_role_arn                   = (known after apply)
        multi_az                              = false
      + name                                  = "teststage" # forces replacement
        option_group_name                     = "a-staging-test-rds"
        parameter_group_name                  = "a-staging-test-rds"
        password                              = (sensitive value)
        performance_insights_enabled          = false
      + performance_insights_kms_key_id       = (known after apply)
      ~ performance_insights_retention_period = 0 -> (known after apply)
        port                                  = 3306
        publicly_accessible                   = false
      ~ replicas                              = [] -> (known after apply)
      ~ resource_id                           = "db-xxx" -> (known after apply)
      - security_group_names                  = [] -> null
        skip_final_snapshot                   = true
        snapshot_identifier                   = "orig-stage-database"
      ~ status                                = "available" -> (known after apply)
        storage_encrypted                     = false
        storage_type                          = "gp2"
        tags                                  = {
            "Attributes"  = "rds"
            "Environment" = "staging"
            "Name"        = "a-staging-test-rds"
            "Namespace"   = "a"
        }
      + timezone                              = (known after apply)
        username                              = "admin"
        vpc_security_group_ids                = [
            "sg-xxx",
        ]
    }

Debug Output

Expected Behavior

RDS instance wouldn't be recreated if no change on tf files

Actual Behavior

RDS instance got recreated after another apply without any change

Steps to Reproduce

1. Use snapshot_identifier for create a RDS instance
2. terraform apply
3. RDS instance got created
4. terraform apply without any change
5. RDS instance got recreated

[snowsky]

Should we pass DBName to this function?

terraform-provider-aws/aws/resource_aws_db_instance.go

Lines 893 to 902 in c84391e

	opts := rds.RestoreDBInstanceFromDBSnapshotInput{
	AutoMinorVersionUpgrade: aws.Bool(d.Get("auto_minor_version_upgrade").(bool)),
	CopyTagsToSnapshot: aws.Bool(d.Get("copy_tags_to_snapshot").(bool)),
	DBInstanceClass: aws.String(d.Get("instance_class").(string)),
	DBInstanceIdentifier: aws.String(d.Get("identifier").(string)),
	DBSnapshotIdentifier: aws.String(d.Get("snapshot_identifier").(string)),
	DeletionProtection: aws.Bool(d.Get("deletion_protection").(bool)),
	PubliclyAccessible: aws.Bool(d.Get("publicly_accessible").(bool)),
	Tags: tags,
	}

[luong-komorebi]

The same problem just happened to us this morning. We were trying to update a DNS record then out of nowhere it was asking to recreate the whole database. Not sure if this change is from amazon's side causing mismatch with the way tf handles it or there are some glitches in the codes

[luong-komorebi]

These are the logs in our case. Some identifying information has been redacted

An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
~ update in-place
-/+ destroy and then create replacement

Terraform will perform the following actions:

  # aws_backup_selection.production1 must be replaced
-/+ resource "aws_backup_selection" "production1" {
        iam_role_arn = "arn:aws:iam:REDACTED:role/service-role/AWSBackupDefaultServiceRole"
      ~ id           = "f513f5ae-03a1-4a4c-940d-2d61f17217c2" -> (known after apply)
        name         = "production1"
        plan_id      = "2a03b079-009c-4809-9f25-5e7ebe5592b4"
      ~ resources    = [
          - "arn:aws:rds:ap-southeast-2REDACTED:db:production1",
        ] -> (known after apply) # forces replacement
    }

  # aws_db_instance.production1 must be replaced
-/+ resource "aws_db_instance" "production1" {
      ~ address                               = "REDACTED_RDS_LINK" -> (known after apply)
        allocated_storage                     = 1100
      + apply_immediately                     = (known after apply)
      ~ arn                                   = "arn:aws:rds:ap-southeast-2REDACTED:db:production1" -> (known after apply)
        auto_minor_version_upgrade            = true
      ~ availability_zone                     = "ap-southeast-2a" -> "ap-southeast-2c" # forces replacement
        backup_retention_period               = 35
        backup_window                         = "14:00-14:30"
      ~ ca_cert_identifier                    = "rds-ca-2019" -> (known after apply)
      + character_set_name                    = (known after apply)
        copy_tags_to_snapshot                 = false
        db_subnet_group_name                  = "default-vpc-b903e9dd"
        delete_automated_backups              = true
      - deletion_protection                   = false -> null
      - enabled_cloudwatch_logs_exports       = [] -> null
      ~ endpoint                              = "REDACTED_RDS_LINK:5432" -> (known after apply)
        engine                                = "postgres"
        engine_version                        = "9.6.18"
      ~ hosted_zone_id                        = "Z32T0VRHXEXS0V" -> (known after apply)
      - iam_database_authentication_enabled   = false -> null
      ~ id                                    = "production1" -> (known after apply)
        identifier                            = "production1"
      + identifier_prefix                     = (known after apply)
        instance_class                        = "db.m4.xlarge"
      - iops                                  = 0 -> null
      + kms_key_id                            = (known after apply)
      ~ latest_restorable_time                = "2021-01-13T02:44:36Z" -> (known after apply)
      ~ license_model                         = "postgresql-license" -> (known after apply)
        maintenance_window                    = "sun:16:00-sun:16:30"
      - max_allocated_storage                 = 0 -> null
        monitoring_interval                   = 60
      ~ monitoring_role_arn                   = "arn:aws:iam:REDACTED:role/rds-monitoring-role" -> (known after apply)
        multi_az                              = true
      + name                                  = (known after apply)
      ~ option_group_name                     = "default:postgres-9-6" -> (known after apply)
        parameter_group_name                  = "production1"
        password                              = (sensitive value)
        performance_insights_enabled          = false
      + performance_insights_kms_key_id       = (known after apply)
      ~ performance_insights_retention_period = 0 -> (known after apply)
        port                                  = 5432
        publicly_accessible                   = false
      ~ replicas                              = [
          - "production1-replica",
        ] -> (known after apply)
      ~ resource_id                           = "db-YQCXDETVH5AHNOTL6Y4XCW42VI" -> (known after apply)
      - security_group_names                  = [] -> null
        skip_final_snapshot                   = true
      ~ status                                = "available" -> (known after apply)
      - storage_encrypted                     = false -> null
        storage_type                          = "gp2"
        tags                                  = {
            "workload-type" = "production"
        }
      + timezone                              = (known after apply)
        username                              = "production1"
        vpc_security_group_ids                = [
            "sg-008d8464",
        ]

      - timeouts {}
    }

  # aws_route53_record._dmarc-REDACTED-COMPANY-NAME-com-TXT will be updated in-place
~ resource "aws_route53_record" "_dmarc-REDACTED-COMPANY-NAME-com-TXT" {
        allow_overwrite = false
        fqdn            = "_dmarc.REDACTED-COMPANY-NAME.com"
        id              = "Z3AI2I4US71WBA__dmarc.REDACTED-COMPANY-NAME.com_TXT"
        name            = "_dmarc.REDACTED-COMPANY-NAME.com"
      ~ records         = [
          + "v=DMARC1; p=quarantine; pct=5; sp=none; rua=mailto:[email protected]",
          - "v=DMARC1; p=reject; sp=none; rua=mailto:REDACTED_EMAIL_ADDRESS.com",
        ]
        ttl             = 3600
        type            = "TXT"
        zone_id         = "Z3AI2I4US71WBA"
    }

[brent-au]

As per the AWS RDS API and commented in the code accordingly, the name attribute is ignored when restoring from a snapshot for MySQL, PostgreSQL or MariaDB.

terraform-provider-aws/aws/resource_aws_db_instance.go

Lines 896 to 905 in 95f94ad

	if attr, ok := d.GetOk("name"); ok {
	// "Note: This parameter [DBName] doesn't apply to the MySQL, PostgreSQL, or MariaDB engines."
	// https://docs.aws.amazon.com/AmazonRDS/latest/APIReference/API_RestoreDBInstanceFromDBSnapshot.html
	switch strings.ToLower(d.Get("engine").(string)) {
	case "mysql", "postgres", "mariadb":
	// skip
	default:
	opts.DBName = aws.String(attr.(string))
	}
	}

A similar case exists for the attribute username , which is also demonstrated in @luong-komorebi's example (I was mistaken here, in that example, the AZ is changing).

This appears to have been raised on the Terraform repository back in the day several times, here amongst other places:

• AWS RDS - Restoring from snapshot doesn't set DBname terraform#9988
• provider/aws: [question] DBName cannot be set when restoring from a MySQL, PostgreSQL, or MariaDB snapshot terraform#11278

To remove any ambiguity concerning expected behaviour, should the expected behaviour be to throw an error if the attribute name or username is set when restoring from snapshot for the affected databases? For example we could introduce:

return fmt.Errorf("Error restoring database from AWS DB Snapshot: %s: name attribute is not supported when engine is %s",d.Get("snapshot_identifier").(string), d.Get("engine").(string)) in the switch case above. Similar behaviour for username.

[snowsky]

Thanks @brent-au

[ghost]

This has been released in version 3.29.0 of the Terraform AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template for triage. Thanks!

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!