Add Support for cluster_autoscaling

[lawliet89]

This is the node auto-provisioning feature that is supported by the cluster_autoscaling field.

[thecodejunkie]

I was just looking at this module for this feature as we need to create clusters that have node auto-provisioning enabled. Is there any planned ETA (or workaround) for this? Thanks!

[morgante]

It's not currently supported, but we should add it!

Expected interface:

cluster_autoscaling = {
  enabled = true
  max_cpu_cores = 20
  min_cpu_cores = 5
  max_memory_gb = 30
  min_memory_gb = 10
}

[thecodejunkie]

@morgante I know I'm asking a lot here, but any chance this could be added very soon? It's literally the only thing we're missing, in the module, to be able to migrate our current solution over to Terraform (this module looks too good to do it using the normal provider 😄)

[thecodejunkie]

By the way, we need to be able to enable auto-scaling on a per-node pool level

[morgante]

@thecodejunkie Can you clarify which kind of autoscaling you're looking for?

• Cluster autoscaler automatically scales existing node pools up and down (adding new nodes): this is already supported in the module. (In fact, it's our default behavior.)
• Node autoprovisioning will automatically provision entire new node pools as necessary: this is what we need to add in this issue

[thecodejunkie]

@morgante oh sorry, yes it's cluster autoscaling I was looking for. I've been unable to find any docs for the node_pools inputs. The only docs I've been able to find are those mentioned in this PR https://github.com/terraform-google-modules/terraform-google-kubernetes-engine/pull/297/files but the autoscaling property is not mentioned? Thanks!

[morgante]

You can do it like this:

  node_pools = [
    {
      ...
      autoscaling = true
      min_count = 10
      max_count = 100
    },
  ]

[thecodejunkie]

@morgante 👍 then perhaps the autoscaling property should also be documented in #297 ?

[morgante]

Absolutely, @neelesh9795 please make sure to document all autoscaling properties.

[kopachevsky]

it this format is fine?

variable "cluster_autoscaling" {
  type = object({
    enabled = bool
    resource_limits = map(number)
  })
  default = {
    enabled = false
    resource_limits = {}
  }
}

and example of value

  cluster_autoscaling = {
    enabled = true
    resource_limits = {
      max_cpu_cores = 20
      min_cpu_cores = 5
      max_memory_gb = 30
      min_memory_gb = 10
    }
  }

Can't set "enabled" field to same map as long has different type

[morgante]

Yes, that works.

[kopachevsky]

@morgante it seems both resource types and both limits (min and max) required on API level, on provider they are optional, for example this one will fail:

  cluster_autoscaling {
    enabled = true
    resource_limits {
      resource_type = "cpu"
      maximum = 30
    }
}

And more insetting this one as well:

  cluster_autoscaling {
    enabled = true
    resource_limits {
      resource_type = "cpu"
      minimum = 10
      maximum = 30
    }
}

The error output is:

Error: googleapi: Error 400: Resource limits for memory must be specified when enabling node autoprovisioning., badRequest
  on ../../../modules/beta-public-cluster/cluster.tf line 22, in resource "google_container_cluster" "primary":
  22: resource "google_container_cluster" "primary"

So my question here, should we switch from map to object for variable as long all fields seems to be mandatory?

[morgante]

@kopachevsky What happens if you only specify memory? From the response provided, it only calls out memory as required.

[kopachevsky]

@morgante Will send output later today, in a middle of other task now, but if specify only memory, without cpu - it also fails, only full set of args works

[kopachevsky]

Error: googleapi: Error 400: Resource limits for cpu must be specified when enabling node autoprovisioning., badRequest

[morgante]

@kopachevsky In that case, we should indeed make it an object since all params are apparently required.