diff --git a/deploy-manage/deploy/cloud-on-k8s/deploy-eck-on-gke-autopilot.md b/deploy-manage/deploy/cloud-on-k8s/deploy-eck-on-gke-autopilot.md index 3f02e5ec4e..3cbfb16f4c 100644 --- a/deploy-manage/deploy/cloud-on-k8s/deploy-eck-on-gke-autopilot.md +++ b/deploy-manage/deploy/cloud-on-k8s/deploy-eck-on-gke-autopilot.md @@ -19,15 +19,18 @@ This page shows how to run ECK on GKE Autopilot. 1. It is recommended that each Kubernetes host’s virtual memory kernel settings be modified. Refer to [Virtual memory](virtual-memory.md). 2. It is recommended that {{es}} Pods have an `initContainer` that waits for virtual memory settings to be in place. 3. For Elastic Agent/Beats there are storage limitations to be considered. -4. Ensure you are using a node class that is applicable for your workload by adding a `cloud.google.com/compute-class` label in a `nodeSelector`. Refer to [GKE Autopilot documentation.](https://cloud.google.com/kubernetes-engine/docs/concepts/autopilot-compute-classes). +4. Ensure you are using a node class that is applicable for your workload by adding a `cloud.google.com/compute-class` label in a `nodeSelector`. Refer to [GKE Autopilot documentation](https://cloud.google.com/kubernetes-engine/docs/concepts/autopilot-compute-classes). ## Ensuring virtual memory kernel settings [k8s-autopilot-setting-virtual-memory] -If you are intending to run production workloads on GKE Autopilot then `vm.max_map_count` should be set. The recommended way to set this kernel setting on the Autopilot hosts is with a `Daemonset` as described in the [Virtual memory](virtual-memory.md) section. You must be running at least version 1.25 when on the `regular` channel or using the `rapid` channel, which currently runs version 1.27. +If you are intending to run production workloads on GKE Autopilot then `vm.max_map_count` should be set. The recommended way to set this kernel setting on the Autopilot hosts depends on your GKE version: -::::{warning} -Only use the provided `Daemonset` exactly as specified or it could be rejected by the Autopilot control plane. -:::: +* **GKE 1.30.3-gke.1451000 or later**: [Use a custom ComputeClass](/deploy-manage/deploy/cloud-on-k8s/virtual-memory.md#k8s_using_a_computeclass_to_set_virtual_memory). Using a custom ComputeClass allows you to set a higher value for `vm.max_map_count`, avoiding the limitations of the `DaemonSet` approach. +* **Earlier versions**: [Use a DaemonSet](/deploy-manage/deploy/cloud-on-k8s/virtual-memory.md#k8s_using_a_daemonset_to_set_virtual_memory). You must be running at least version 1.25 when on the `regular` channel or using the `rapid` channel, which currently runs version 1.27. + + ::::{warning} + Use the provided `Daemonset` exactly as specified, with a `vm.max_map_count` value of `262144`, or it could be rejected by the Autopilot control plane. + :::: ## Install the ECK Operator [k8s-autopilot-deploy-the-operator] @@ -35,9 +38,38 @@ Refer to [*Install ECK*](install.md) for more information on installation option ## Deploy an {{es}} cluster [k8s-autopilot-deploy-elasticsearch] -Create an {{es}} cluster. If you are using the `Daemonset` described in the [Virtual memory](virtual-memory.md) section to set `max_map_count` you can add the `initContainer` below is also used to ensure the setting is set prior to starting {{es}}. +Create an {{es}} cluster. The information that you need to provide in your spec depends on whether you've increased your virtual memory kernel setting, and the method that you used. + +::::{tab-set} + +:::{tab-item} Using a custom ComputeClass +If you used a custom ComputeClass to set `vm.max_map_count`, then you need to reference the custom ComputeClass as part of your template spec. -```shell subs=true +```yaml subs=true +cat < /proc/sys/vm/max_map_count'] + command: ['/usr/local/bin/bash', '-e', '-c', 'echo 1048576 > /proc/sys/vm/max_map_count'] <1> containers: - name: sleep image: docker.io/bash:5.2.21 command: ['sleep', 'infinity'] EOF ``` +1. In GKE Autopilot environments, `vm.max_map_count` must be set to 262144 when using a DaemonSet. To run an {{es}} instance that waits for the kernel setting to be in place: @@ -122,8 +127,60 @@ spec: # Do not use this if setting config.node.store.allow_mmap: false initContainers: - name: max-map-count-check - command: ['sh', '-c', "while true; do mmc=$(cat /proc/sys/vm/max_map_count); if [ ${mmc} -eq 262144 ]; then exit 0; fi; sleep 1; done"] + command: ['sh', '-c', "while true; do mmc=$(cat /proc/sys/vm/max_map_count); if [ ${mmc} -eq 262144 ]; then exit 0; fi; sleep 1; done"] <1> EOF ``` +1. In GKE Autopilot environments, `vm.max_map_count` must be set to 262144 when using a DaemonSet. + + +## Using a custom ComputeClass to set virtual memory [k8s_using_a_computeclass_to_set_virtual_memory] +```{applies_to} +deployment: + eck: ga 3.2+ +``` + +If you're using GKE to run ECK, then you can use a [custom ComputeClass](https://docs.cloud.google.com/kubernetes-engine/docs/concepts/about-custom-compute-classes), rather than a DaemonSet, to increase the `vm.max_map_count` setting. On [GKE Autopilot](/deploy-manage/deploy/cloud-on-k8s/deploy-eck-on-gke-autopilot.md) this allows you to set a higher value, which is not possible with a DaemonSet. +1. Create a ComputeClass that changes the host kernel setting on all nodes: + ```yaml + cat < + nodePoolAutoCreation: + enabled: true + priorityDefaults: <2> + nodeSystemConfig: + linuxNodeConfig: + sysctls: + vm.max_map_count: 1048576 + priorities: + - machineFamily: n2 + EOF + ``` + 1. Default since GKE 1.33 + 2. `priorityDefaults` is available only since GKE 1.32.1-gke.1729000 + +2. Create your {{es}} instance using the custom ComputeClass: + + ```yaml subs=true + cat <<'EOF' | kubectl apply -f - + apiVersion: elasticsearch.k8s.elastic.co/v1 + kind: Elasticsearch + metadata: + name: elasticsearch + spec: + version: {{version.stack}} + nodeSets: + - name: default + count: 1 + podTemplate: + spec: + nodeSelector: + cloud.google.com/compute-class: "elasticsearch" + EOF + ``` \ No newline at end of file