using id instead of self_link field in google_kms_crypto_key broke the plan

[hablislim]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request.
• Please do not leave +1 or me too comments, they generate extra noise for issue followers and do not help prioritize the request.
• If you are interested in working on this issue or have submitted a pull request, please leave a comment.
• If an issue is assigned to the modular-magician user, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If an issue is assigned to a user, that user is claiming responsibility for the issue. If an issue is assigned to hashibot, a community member has claimed the issue already.

Terraform Version

$ terraform -v
Terraform v1.0.4
on linux_amd64

Affected Resource(s)

• google_container_cluster

Terraform Configuration Files

resource "google_kms_crypto_key" "key" {
  name     = var.name
  key_ring = google_kms_key_ring.key.id

[...]
}

Debug Output

Terraform will perform the following actions:

  # module.kubernetes_cluster.google_container_cluster.master will be updated in-place
  ~ resource "google_container_cluster" "master" {
        # (27 unchanged attributes hidden)

      ~ database_encryption {
          ~ key_name = "[...]" -> (known after apply)
            # (1 unchanged attribute hidden)
        }

        # (24 unchanged blocks hidden)
    }

Plan: 0 to add, 1 to change, 0 to destroy.

Panic Output

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

module.kubernetes_cluster.google_container_cluster.master: Modifying... [id=projects/sp-hprod-wip-5fd7/locations/europe-west1/clusters/wip]
╷
│ Warning: Unsupported attribute:This object has no argument, nested block, or exported attribute named "self_link".
│ 
│ 
╵
╷
│ Error: configuration for module.kubernetes_cluster.google_container_cluster.master still contains unknown values during apply (this is a bug in Terraform; please report it!)
│ 
│ 
╵
[terragrunt] 2021/11/15 16:31:32 Hit multiple errors:
exit status 1

Expected Behavior

Changing self_link by id on google_kms_crypto_key when upgrading from 3.88.0 to 4.0.0, should return no changes

Actual Behavior

By running a terraform apply again , the change shows a new creation of the existing cluster

The whole terrafrom state was broken when running the plan again:

  # module.kubernetes_cluster.google_container_cluster.master will be created
  + resource "google_container_cluster" "master" {
      + cluster_ipv4_cidr           = (known after apply)
      + datapath_provider           = (known after apply)
      + default_max_pods_per_node   = (known after apply)
[...]

Steps to Reproduce

1. terraform apply

Important Factoids

NA

[edwardmedia]

@hablislim there is below breaking change in v4.0.0. You may need to update your code accordingly.

#8275

[parmus]

I'm seeing this issue as well. The problem seems ot be that the id field provides a different format than what self-link did, but the Cluster API expects the old format.

My configuration looks roughly like this:

resource "google_kms_crypto_key" "public-k8s" {
  name            = "public-k8s"
  key_ring        = google_kms_key_ring.k8s-secrets.id
  rotation_period = "7776000s" // 90 days, the default

  lifecycle {
    prevent_destroy = true
  }
}

resource "google_container_cluster" "public" {
  name       = "public"

  [...]
  
  database_encryption {
    state    = "ENCRYPTED"
    key_name = google_kms_crypto_key.public-k8s.id
  }
}

This gives me this suggested change:

  # google_container_cluster.public will be updated in-place
  ~ resource "google_container_cluster" "public" {
        id                          = "projects/[redacted]/locations/europe-west1/clusters/public"
        name                        = "public"
        # (25 unchanged attributes hidden)



      ~ database_encryption {
          ~ key_name = "projects/[redacted]/locations/europe-west1/keyRings/k8s-secrets/cryptoKeys/public-k8s" -> "europe-west1/k8s-secrets/public-k8s"
            # (1 unchanged attribute hidden)
        }

However, applying this change results in the following error:

Error: googleapi: Error 400: The supplied key is not in the expected format: europe-west1/k8s-secrets/public-k8s, must match regex: "^(projects/([^/]+)/locations/([^/]+)/keyRings/([^/]+)/cryptoKeys/([^/]+))(?:/grants/([^/]+))?(?:/cryptoKeyVersions/([^/]+))?$"., failedPrecondition
│ 
│   with google_container_cluster.public,
│   on public-k8s.tf line 31, in resource "google_container_cluster" "public":
│   31: resource "google_container_cluster" "public" {

[hablislim]

I'm seeing this issue as well. The problem seems ot be that the id field provides a different format than what self-link did, but the Cluster API expects the old format.

My configuration looks roughly like this:

resource "google_kms_crypto_key" "public-k8s" {
  name            = "public-k8s"
  key_ring        = google_kms_key_ring.k8s-secrets.id
  rotation_period = "7776000s" // 90 days, the default

  lifecycle {
    prevent_destroy = true
  }
}

resource "google_container_cluster" "public" {
  name       = "public"

  [...]
  
  database_encryption {
    state    = "ENCRYPTED"
    key_name = google_kms_crypto_key.public-k8s.id
  }
}

This gives me this suggested change:

  # google_container_cluster.public will be updated in-place
  ~ resource "google_container_cluster" "public" {
        id                          = "projects/[redacted]/locations/europe-west1/clusters/public"
        name                        = "public"
        # (25 unchanged attributes hidden)



      ~ database_encryption {
          ~ key_name = "projects/[redacted]/locations/europe-west1/keyRings/k8s-secrets/cryptoKeys/public-k8s" -> "europe-west1/k8s-secrets/public-k8s"
            # (1 unchanged attribute hidden)
        }

However, applying this change results in the following error:

Error: googleapi: Error 400: The supplied key is not in the expected format: europe-west1/k8s-secrets/public-k8s, must match regex: "^(projects/([^/]+)/locations/([^/]+)/keyRings/([^/]+)/cryptoKeys/([^/]+))(?:/grants/([^/]+))?(?:/cryptoKeyVersions/([^/]+))?$"., failedPrecondition
│ 
│   with google_container_cluster.public,
│   on public-k8s.tf line 31, in resource "google_container_cluster" "public":
│   31: resource "google_container_cluster" "public" {

In my case when I tried self_link and id I noticed the same format and my plan got applied.
Once I ran again, I notice terraform trying to deploy the whole cluster again (the tfstate was broken)

[djfinnoy]

Experiencing the same issue with google_storage_bucket. The plan wants to change projects/<project>/locations/<location>/keyRings/<keyRing>/cryptoKeys/<cryptoKey> to <location>/<keyRing>/<cryptoKey>. The plan fails with Error 400: Bad Cloud KMS crypto key.

Have to use the following workaround for now:

locals {
  kms_parts = split("/", var.kms_key)
  kms_key   = var.kms_key == "" ? "" : "projects/${var.project}/locations/${var.location}/keyRings/${local.kms_parts[1]}/cryptoKeys/${local.kms_parts[2]}"
}

[hablislim]

Thank you @djfinnoy

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.