Issue with CMEK key while creating management GKE Private cluster

[Rajchirag1993]

Hello Team,

I am trying to create kubeflow management private GKE cluster on Google Cloud, My organization doesn't allow to create a cluster without specifying CMEK key for cluster and nodepool.

I have checked the documentation : https://cloud.google.com/sdk/gcloud/reference/anthos/config/controller/create

It doesn't have any CMEK configurations to mention in the command. Can we bypass or any remediations to follow to avoid the below error.

`
$ make create-cluster

The management cluster name "kf-mgmt-cluster1" is valid.
gcloud services enable krmapihosting.googleapis.com
container.googleapis.com
cloudresourcemanager.googleapis.com
Operation "operations/acat.p2-XXXXX-0befa1b9-94b4-4ebe-a26a-bd99e399592a" finished successfully.
gcloud anthos config controller create kf-mgmt-cluster1 --location=us-central1
--cluster-ipv4-cidr-block="XXXXXXX"
--master-ipv4-cidr-block="XXXXXX"
--network="projects/XXXXX/global/networks/XXXXX"
--subnet="projects/XXXX/regions/us-central1/subnetworks/XXXXX"
--use-private-endpoint
--man-blocks="XXXXXX"
--full-management

Create request issued for: [kf-mgmt-cluster1]
Waiting for operation [projects/corp-slvr-shared3l/locations/us-central1/operations/operation-1713278218227-61637ab994d8
b-1f60966a-0aaa3744] to complete...failed.
ERROR: (gcloud.anthos.config.controller.create) unexpected error occurred while waiting for SLM operation [projects/krmap
ihosting-slm/locations/us-central1/operations/operation-1713278224490-61637abf8e008-6f837005-cdceaec4]: errored while wai
ting for operation: projects/krmapihosting-slm/locations/us-central1/operations/operation-1713278224490-61637abf8e008-6f8
37005-cdceaec4: Operation failed with error:
generic::invalid_argument: terraform apply failed, error: exit status 1, stderr:

Error: Error creating Cluster: googleapi: Error 400: Failed precondition: Constraint constraints/gcp.restrictNonCmekServ ices violated for projects/XXXXXX attempting to create a resource without specifying a KMS CryptoKey.
Details:
[
{
"@type": "type.googleapis.com/google.rpc.DebugInfo",
"detail": "FAILED_PRECONDITION: failed precondition: Constraint constraints/gcp.restrictNonCmekServices violated fo
r projects/XXXXX attempting to create a resource without specifying a KMS CryptoKey",
"stackEntries": [
"cloud/kubernetes/engine/common/error_desc.go:432 +0x26 google3/cloud/kubernetes/engine/common/errdesc.(*GKEErrorDe
scriptor).createErr(0xc0019ca5a0, {0x56526612cca8, 0x56526fc98d40})",
"cloud/kubernetes/engine/common/error_desc.go:298 +0x4c google3/cloud/kubernetes/engine/common/errdesc.(*GKEErrorDe
scriptor).WithMsgCtx(0xe576dbc6f9?, {0x56526612cca8?, 0x56526fc98d40?}, {0x565254327ee6, 0x93}, {0xc07d6ea458, 0x1, 0x1})
",
"cloud/kubernetes/engine/common/error_desc.go:290 google3/cloud/kubernetes/engine/common/errdesc.(*GKEErrorDescript
or).WithMsg(...)",
"cloud/kubernetes/server/patch/field/common_validation.go:899 +0x1fb google3/cloud/kubernetes/server/patch/field/co
mmonvalidation.validateCustomerManagedEncryptionKeyServiceRestriction({0x56526612cbc8, 0xc43fe33c80}, {0x5652661399f8, 0x
c06185e798}, 0xe576dbc6f9, {0x0?, 0xc4ca8e8390?})",
"cloud/kubernetes/server/patch/field/common_validation.go:815 +0xd2 google3/cloud/kubernetes/server/patch/field/com
monvalidation.ValidateCustomerManagedEncryptionKey({0x56526612cbc8, 0xc43fe33c80}, {0x5652661399f8, 0xc06185e798}, 0x1, {
0xc2bfc9cde6?, 0xc07d6ea5a0?}, 0xe576dbc6f9, {0x0, 0x0}, ...)",
"cloud/kubernetes/server/patch/field/node/node_pool_config.go:255 +0x3e5 google3/cloud/kubernetes/server/patch/fiel
d/node/config.(*nodeConfigValidator).Validate(0xc10070d2c0, {0x56526612cbc8, 0xc43fe33c80}, 0xc5552c5740, 0x0)",
"cloud/kubernetes/server/patch/common/field.go:125 +0x63 google3/cloud/kubernetes/server/patch/common/patchbase.(*F
ield).Validate(0xc065dfed00, {0x56526612cbc8, 0xc43fe33c80}, 0xc5552c5740, 0x0)",
"cloud/kubernetes/server/patch/patcher/field_list.go:225 +0x1c5 google3/cloud/kubernetes/server/patch/patcher/field
list.FieldList.Validate({0xc688760e00, 0x16, 0x20}, {0x56526612cbc8, 0xc43fe33c80}, 0xc5552c5740, 0x0)",
"cloud/kubernetes/server/patch/common/field_interfaces.go:404 +0x350 google3/cloud/kubernetes/server/patch/common/p
atchbase.ValidatePatchRequest({0x56526612cbc8, 0xc43fe33c80}, 0xc11e19b448, 0xc5552c5740, 0xc5552c5740?)",
"cloud/kubernetes/server/patch/patcher/node_pool_fields.go:126 +0x1bc google3/cloud/kubernetes/server/patch/patcher
/nodepool.(*patcher).Validate(0xc07d6ea950?, {0x56526612cbc8?, 0xc43fe33c80?}, 0xc11e19b448?, 0xc5552c5740?)",
"cloud/kubernetes/server/server_create.go:960 +0x75 google3/cloud/kubernetes/server/server.validateCreateNodePool({
0x56526612cbc8, 0xc43fe33c80}, 0xc11e19b448, 0xc5552c5740)",
"cloud/kubernetes/server/server_create.go:600 +0x32e7 google3/cloud/kubernetes/server/server.(*ClusterServer).Creat
eCluster(0xc0745b0e08, {0x56526612cbc8, 0xc43fe33c80}, {0xc2bfc9cde6, 0xb}, {0xc2bfc9cdc9, 0x12}, 0xc07bc99408, 0xc2ae599
de8, {0x56525412fb05, ...})",
"cloud/kubernetes/server/v1alpha1/server.go:150 +0x199 google3/cloud/kubernetes/server/v1alpha1/server.(*ClusterSer
ver).createCluster(0xc07adb0880, {0x56526612cbc8, 0xc293877710}, 0xc0bf32eaf0, 0xc2cc377400, 0xc2ae599de8)",
"cloud/kubernetes/server/v1alpha1/server.go:121 +0x331 google3/cloud/kubernetes/server/v1alpha1/server.(*ClusterSer
ver).CreateCluster(0xc07adb0880, {0x56526612cbc8, 0xc293877710}, 0xc0bf32eaf0, 0xc2cc377400)",
"cloud/kubernetes/engine/server/api/v1/server.go:38 +0xdd google3/cloud/kubernetes/engine/server/api/v1/server.(*Cl
usterServer).CreateCluster(0xc077902718, {0x56526612cbc8, 0xc293877710}, 0xc0bf32e310, 0xc2cc377300)",
"blaze-out/k8-opt/bin/google/container/v1/cluster_service.pb.go:34387 +0xe8 google3/google/container/v1/cluster_ser
vice_go_proto._ClusterManager_CreateCluster_Handler({0x565265ea4580, 0xc077902718}, 0xc1897dc008, {0x565266037d60?, 0xc0b
f32e310})",
"cloud/kubernetes/engine/common/interceptors/stubby_interceptor.go:149 +0x40c google3/cloud/kubernetes/engine/commo
n/interceptors/stubbyinterceptor.(*Hook).handleRPCWithCall(0xc07fedcea0, {0x56526612d008, 0xc5901c11a0}, 0xc081a052c0, 0x
c0bf32e5b0)",
"cloud/kubernetes/engine/common/interceptors/stubby_interceptor.go:99 +0xb2 google3/cloud/kubernetes/engine/common/
interceptors/stubbyinterceptor.(*Hook).handleRPC(0xc07fedcea0, {0x56526612d008, 0xc5901c11a0}, 0xc0bf32e5b0)"
]
},
{
"@type": "type.googleapis.com/google.rpc.RequestInfo",
"requestId": "0xf44a1bdf688b74f9"
}
]

on main_autopilot.tf line 32, in resource "google_container_cluster" "acp_cluster":
32: resource "google_container_cluster" "acp_cluster" {

, stdout:
google_container_cluster.acp_cluster: Creating...

Subsequent cleanup succeeded
make: *** [Makefile:146: create-cluster] Error 1
`