diff --git a/deploy/helm-chart/README.md b/deploy/helm-chart/README.md index b428944bf..b6324958f 100644 --- a/deploy/helm-chart/README.md +++ b/deploy/helm-chart/README.md @@ -1,5 +1,24 @@ # NGINX Gateway Fabric Helm Chart +- [NGINX Gateway Fabric Helm Chart](#nginx-gateway-fabric-helm-chart) + - [Introduction](#introduction) + - [Prerequisites](#prerequisites) + - [Installing the Gateway API resources](#installing-the-gateway-api-resources) + - [Installing the Chart](#installing-the-chart) + - [Installing the Chart from the OCI Registry](#installing-the-chart-from-the-oci-registry) + - [Installing the Chart via Sources](#installing-the-chart-via-sources) + - [Pulling the Chart](#pulling-the-chart) + - [Installing the Chart](#installing-the-chart-1) + - [Upgrading the Chart](#upgrading-the-chart) + - [Upgrading the Gateway Resources](#upgrading-the-gateway-resources) + - [Upgrading the CRDs](#upgrading-the-crds) + - [Upgrading the Chart from the OCI Registry](#upgrading-the-chart-from-the-oci-registry) + - [Upgrading the Chart from the Sources](#upgrading-the-chart-from-the-sources) + - [Configure Delayed Termination for Zero Downtime Upgrades](#configure-delayed-termination-for-zero-downtime-upgrades) + - [Uninstalling the Chart](#uninstalling-the-chart) + - [Uninstalling the Gateway Resources](#uninstalling-the-gateway-resources) + - [Configuration](#configuration) + ## Introduction This chart deploys the NGINX Gateway Fabric in your Kubernetes cluster. @@ -59,6 +78,10 @@ helm install my-release . --create-namespace --wait -n nginx-gateway ## Upgrading the Chart +> **Note** +> See [below](#configure-delayed-termination-for-zero-downtime-upgrades) for instructions on how to configure delayed +> termination if required for zero downtime upgrades in your environment. + ### Upgrading the Gateway Resources Before you upgrade a release, ensure the Gateway API resources are the correct version as supported by the NGINX @@ -105,6 +128,55 @@ the release `my-release`, run: helm upgrade my-release . -n nginx-gateway ``` +### Configure Delayed Termination for Zero Downtime Upgrades + +To achieve zero downtime upgrades (meaning clients will not see any interruption in traffic while a rolling upgrade is +being performed on NGF), you may need to configure delayed termination on the NGF Pod, depending on your environment. + +> **Note** +> When proxying Websocket or any long-lived connections, NGINX will not terminate until that connection is closed +> by either the client or the backend. This means that unless all those connections are closed by clients/backends +> before or during an upgrade, NGINX will not terminate, which means Kubernetes will kill NGINX. As a result, the +> clients will see the connections abruptly closed and thus experience downtime. + +1. Add `lifecycle` to both the nginx and the nginx-gateway container definition. To do so, update your `values.yaml` + file to include the following (update the `sleep` values to what is required in your environment): + + ```yaml + nginxGateway: + <...> + lifecycle: + preStop: + exec: + command: + - /usr/bin/gateway + - sleep + - --duration=40s # This flag is optional, the default is 30s + + nginx: + <...> + lifecycle: + preStop: + exec: + command: + - /bin/sleep + - "40" + ``` + +2. Ensure the `terminationGracePeriodSeconds` matches or exceeds the `sleep` value from the `preStopHook` (the default + is 30). This is to ensure Kubernetes does not terminate the Pod before the `preStopHook` is complete. To do so, + update your `values.yaml` file to include the following (update the value to what is required in your environment): + + ```yaml + terminationGracePeriodSeconds: 50 + ``` + +> **Note** +> More information on container lifecycle hooks can be found +> [here](https://kubernetes.io/docs/concepts/containers/container-lifecycle-hooks/#container-hooks) and a detailed +> description of Pod termination behavior can be found in +> [Termination of Pods](https://kubernetes.io/docs/concepts/workloads/Pods/Pod-lifecycle/#Pod-termination). + ## Uninstalling the Chart To uninstall/delete the release `my-release`: diff --git a/docs/installation.md b/docs/installation.md index 6cb3c8254..9ce6a0fde 100644 --- a/docs/installation.md +++ b/docs/installation.md @@ -2,6 +2,23 @@ This guide walks you through how to install NGINX Gateway Fabric on a generic Kubernetes cluster. +- [Installation](#installation) + - [Prerequisites](#prerequisites) + - [Deploy NGINX Gateway Fabric using Helm](#deploy-nginx-gateway-fabric-using-helm) + - [Deploy NGINX Gateway Fabric from Manifests](#deploy-nginx-gateway-fabric-from-manifests) + - [Expose NGINX Gateway Fabric](#expose-nginx-gateway-fabric) + - [Create a NodePort Service](#create-a-nodeport-service) + - [Create a LoadBalancer Service](#create-a-loadbalancer-service) + - [Upgrading NGINX Gateway Fabric](#upgrading-nginx-gateway-fabric) + - [Upgrade NGINX Gateway Fabric from Manifests](#upgrade-nginx-gateway-fabric-from-manifests) + - [Upgrade NGINX Gateway Fabric using Helm](#upgrade-nginx-gateway-fabric-using-helm) + - [Configure Delayed Termination for Zero Downtime Upgrades](#configure-delayed-termination-for-zero-downtime-upgrades) + - [Configure Delayed Termination Using Manifests](#configure-delayed-termination-using-manifests) + - [Configure Delayed Termination Using Helm](#configure-delayed-termination-using-helm) + - [Uninstalling NGINX Gateway Fabric](#uninstalling-nginx-gateway-fabric) + - [Uninstall NGINX Gateway Fabric from Manifests](#uninstall-nginx-gateway-fabric-from-manifests) + - [Uninstall NGINX Gateway Fabric using Helm](#uninstall-nginx-gateway-fabric-using-helm) + ## Prerequisites - [kubectl](https://kubernetes.io/docs/tasks/tools/) @@ -131,6 +148,10 @@ Create a Service with type `LoadBalancer` using the appropriate manifest for you ## Upgrading NGINX Gateway Fabric +> **Note** +> See [below](#configure-delayed-termination-for-zero-downtime-upgrades) for instructions on how to configure delayed +> termination if required for zero downtime upgrades in your environment. + ### Upgrade NGINX Gateway Fabric from Manifests 1. Upgrade the Gateway Resources @@ -168,6 +189,60 @@ Create a Service with type `LoadBalancer` using the appropriate manifest for you To upgrade NGINX Gateway Fabric when the deployment method is Helm, please follow the instructions [here](/deploy/helm-chart/README.md#upgrading-the-chart). +### Configure Delayed Termination for Zero Downtime Upgrades + +To achieve zero downtime upgrades (meaning clients will not see any interruption in traffic while a rolling upgrade is +being performed on NGF), you may need to configure delayed termination on the NGF Pod, depending on your environment. + +> **Note** +> When proxying Websocket or any long-lived connections, NGINX will not terminate until that connection is closed +> by either the client or the backend. This means that unless all those connections are closed by clients/backends +> before or during an upgrade, NGINX will not terminate, which means Kubernetes will kill NGINX. As a result, the +> clients will see the connections abruptly closed and thus experience downtime. + +#### Configure Delayed Termination Using Manifests + +Edit the `deploy/manifests/nginx-gateway.yaml` to include the following: + +1. Add `lifecycle` prestop hooks to both the nginx and the nginx-gateway container definitions: + + ```yaml + <...> + name: nginx-gateway + <...> + lifecycle: + preStop: + exec: + command: + - /usr/bin/gateway + - sleep + - --duration=40s # This flag is optional, the default is 30s + <...> + name: nginx + <...> + lifecycle: + preStop: + exec: + command: + - /bin/sleep + - "40" + <...> + ``` + +2. Ensure the `terminationGracePeriodSeconds` matches or exceeds the `sleep` value from the `preStopHook` (the default + is 30). This is to ensure Kubernetes does not terminate the Pod before the `preStopHook` is complete. + +> **Note** +> More information on container lifecycle hooks can be found +> [here](https://kubernetes.io/docs/concepts/containers/container-lifecycle-hooks/#container-hooks) and a detailed +> description of Pod termination behavior can be found in +> [Termination of Pods](https://kubernetes.io/docs/concepts/workloads/Pods/Pod-lifecycle/#Pod-termination). + +#### Configure Delayed Termination Using Helm + +To configure delayed termination on the NGF Pod when the deployment method is Helm, please follow the instructions +[here](/deploy/helm-chart/README.md#configure-delayed-termination-for-zero-downtime-upgrades). + ## Uninstalling NGINX Gateway Fabric ### Uninstall NGINX Gateway Fabric from Manifests