Changing the body of a Secret manifest doesn't cause any changes

[joaocc]

We are deploying some Secrets via kubectl_manifest

resource "kubectl_manifest" "this" {
  for_each = local.create ? var.secrets : {}

  yaml_body = yamlencode(
    {
      apiVersion = "v1"
      kind = "Secret"

      metadata = {
        name = (each.value).name

        namespace   = coalesce((each.value).namespace, var.namespace)
        labels      = merge(
                        var.labels,
                        coalesce((each.value).labels, {})
                      )
        annotations = {}  # some annotations were here (removed for conciseness)
      }

      stringData = (each.value).data

      type = coalesce((each.value).type, var.type)

      immutable = coalesce((each.value).immutable, var.immutable, false)
    }
  )

  server_side_apply = true
  wait = true
}

In this case, immutable is set to false.

Everything goes well on first apply.
We then change the content of the manifest (some 14 normal sized fields and 1 large field (json)).
While plan does detect the need to change, and application runs without errors, the secret doesn't change in k8s.
Removing the secret in k8s and then reapplying terraform does yield the secret with the desired changes.

module.my_module[0].kubectl_manifest.this["_"]: Refreshing state... [id=/api/v1/namespaces/my-k8s-namespace/secrets/my-secret-name]

Terraform will perform the following actions:

  # module.my_module[0].kubectl_manifest.this["_"] will be updated in-place
  ~ resource "kubectl_manifest" "this" {
        id                      = "/api/v1/namespaces/my-k8s-namespace/secrets/my-secret-name"
        name                    = "my-secret-name"
      ~ yaml_body               = (sensitive value)
        # (16 unchanged attributes hidden)
    }

module.my_module[0].kubectl_manifest.this["_"]: Modifying... [id=/api/v1/namespaces/my-k8s-namespace/secrets/my-secret-name]
module.my_module[0].kubectl_manifest.this["_"]: Modifications complete after 1s [id=/api/v1/namespaces/my-k8s-namespace/secrets/my-secret-name]

Even with server_side_apply==false, the issue still happens.

Any idea of what may be causing this issue? Thx

[alekc]

Does it happen with the latest version of the provider? If yes, is there anything in a trace?

Tbh, with standard kubernetes objects (and especially secrets), I find that the kubernetes provider works a bit better for a change management.

[joaocc]

We are using 2.0.3.
We switched from kubernetes_manifests due to other issues - it tend to force recreation of resources instead of updating them, with catastrophic consequences in the cases of helm-releases and other resources.

[alekc]

I mean using the https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs/resources/secret resource, not the kubernetes_manifests.

Having said that, please try the latest 2.0.4, there has been some refactoring around secrets which might (or not) cure your issue.

[alekc]

Closing this off due to lack of response

[joaocc]

Hi
The kubernetes_secret had (?has) similar behaviour to kubernetes_manifest - it almost always detects changes of the kind that causes items to be recreated (instead of updated in place). This is the reason we changed to the kubectl provider.
However we encountered #120 today, which needed a workaround until a fix is implemented at the provider level.
And it is also interesting to understand that the root cause for the fix you did here (in v2.0.4) is in this small statement in stringData in the kubernetes API Secret spec (https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.28/#secret-v1-core):
stringData (...) It is provided as a write-only input field for convenience. All keys and values are merged into the data field on write, overwriting any existing values. The stringData field is never output when reading from the API.