Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for FUSE CSI driver with GKE Autopilot #15817

Closed
chillerm opened this issue Sep 12, 2023 · 4 comments · Fixed by GoogleCloudPlatform/magic-modules#8998, hashicorp/terraform-provider-google-beta#6378 or #15996

Comments

@chillerm
Copy link

chillerm commented Sep 12, 2023

Community Note

  • Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
  • Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
  • If you are interested in working on this issue or have submitted a pull request, please leave a comment. If the issue is assigned to the "modular-magician" user, it is either in the process of being autogenerated, or is planned to be autogenerated soon. If the issue is assigned to a user, that user is claiming responsibility for the issue. If the issue is assigned to "hashibot", a community member has claimed the issue already.

Description

GKE supports adding FUSE mounts with both autopilot and non autopilot clusters. There is currently code in the Terraform provider that prevents this combination, even though it is supported by the underlying APIs.

As a work around for this, we have configured an autopilot cluster, and then run the additional step below to enable FUSE.

gcloud beta container clusters update ${CLUSTER} --update-addons GcsFuseCsiDriver=ENABLED --region=${REGION}

Provider details below:

Initializing provider plugins...
- Finding hashicorp/google versions matching ">= 4.43.0, >= 4.66.0, >= 4.81.0"...
- Finding hashicorp/http versions matching ">= 3.4.0"...
- Finding hashicorp/random versions matching ">= 3.5.1"...
- Finding poseidon/ct versions matching "0.10.0"...
- Using previously-installed hashicorp/google v4.82.0
- Using previously-installed hashicorp/http v3.4.0
- Using previously-installed hashicorp/random v3.5.1
- Using previously-installed poseidon/ct v0.10.0

Currently this results in the below error:

│ Error: Conflicting configuration arguments
│ 
│   with module.cluster.google_container_cluster.cluster,
│   on ../../../DH9514_thdgit/tf-mod-gke-cluster/main.tf line 44, in resource "google_container_cluster" "cluster":
│   44:     gcs_fuse_csi_driver_config {
│ 
│ "addons_config.0.gcs_fuse_csi_driver_config": conflicts with enable_autopilot

New or Affected Resource(s)

  • google_container_cluster

Potential Terraform Configuration

resource "google_container_cluster" "cluster" {
...  
  enable_autopilot = true

  addons_config {
    gcs_fuse_csi_driver_config {
      enabled = true
    }
  }
...
}

References

b/300237991

@github-actions github-actions bot added forward/review In review; remove label to forward service/container labels Sep 12, 2023
@trodge trodge removed the forward/review In review; remove label to forward label Sep 12, 2023
@toVersus
Copy link
Contributor

@chillerm
Hi, there's no need to explicitly set addons_config.gcs_fuse_csi_driver_config.enabled to true for Autopilot clusters, as mentioned in the official documentation. Additionally, I believe that when you upgrade your cluster to version 1.24, the Cloud Storage FUSE CSI driver feature is automatically enabled.

https://cloud.google.com/kubernetes-engine/docs/how-to/persistent-volumes/cloud-storage-fuse-csi-driver#enable

The Cloud Storage FUSE CSI driver is enabled by default for Autopilot clusters with version 1.24 or later. You can enable the feature by upgrading your Autopilot cluster to version 1.24 or later. If your clusters have Cloud Storage FUSE CSI driver enabled, skip to Configure access to Cloud Storage buckets using GKE Workload Identity.

I cannot create a new Autopilot cluster with version 1.23 and verify this behaviour because Autopilot cluster only supports creating clusters of version 1.24 or later at this moment.

When creating a new cluster with version 1.24 without specifying addons_config.gcs_fuse_csi_driver_config.enabled: true, I confirmed that the Cloud Storage FUSE CSI driver feature is enabled.

resource "google_container_cluster" "cluster" {
  name               = "test-cluster"
  location           = "us-central1"
  initial_node_count = 1
  enable_autopilot   = true
}

Retrieve information about the newly created Autopilot cluster:

gcloud container clusters describe test-cluster --region us-central1
addonsConfig:
  dnsCacheConfig:
    enabled: true
  gcePersistentDiskCsiDriverConfig:
    enabled: true
  gcpFilestoreCsiDriverConfig:
    enabled: true
  gcsFuseCsiDriverConfig:
    enabled: true
  kubernetesDashboard:
    disabled: true
  networkPolicyConfig:
    disabled: true
...

@melinath melinath added this to the Goals milestone Sep 18, 2023
@melinath
Copy link
Collaborator

melinath commented Sep 18, 2023

Fixing this issue is likely as simple as removing the "conflicts with" from the TF provider field

@toVersus
Copy link
Contributor

Thanks, I submitted the PR to relax the validation logic regarding conflicts with Autopilot clusters.

@github-actions
Copy link

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Oct 26, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.