Beta clusters don't automatically enable Workload Identity

[schostin]

When using a beta private cluster in version 8.1.0 identity namespace gets enabled by default, but not workload identity.

We delete the default node-pool after creation and use a node pool with the following basic settings:

node_pools = [
    {
      name         = "main"
      node_count   = 2
      image_type   = "COS_CONTAINERD"
      min_count    = 2
      max_count    = 4
      machine_type = "n1-standard-8"
    }
  ]

When looking at the GCloud UI, we can see, that identity namespace for the cluster is set. Looking at the node pool we can see, that Security -> Enable GKE Metadata Server is not enabled (which is required for Workload Identity).

Looking at the code it seems like we have to set node_metadata = "GKE_METADATA_SERVER" on the module on our own to enable that feature.

Since beta cluster should enable Identity Workload as default, this should also be the default for node_metadata. Also happing to issue a PR if you guys say that this is really a bug that I encounter :)

[morgante]

Good point. I think we can change the node pool default to GKE_METADATA_SERVER if workload identity is enabled.

[bharathkkb]

@sebastianneb Happy to review the PR if you are working on this.

[schostin]

Just got the tests working on my machine and will quickly adapt those tomorrow and issue a PR here.

[eng-atwijukire]

Great remark