From c94aa8922102993c70a4c8efcb6857deb9b89a1d Mon Sep 17 00:00:00 2001 From: sudipto baral Date: Fri, 21 Mar 2025 01:26:45 -0400 Subject: [PATCH 1/9] docs: enhance Proxy Metrics documentation. Signed-off-by: sudipto baral --- .../tasks/observability/proxy-metric.md | 122 ++++++++++++++++-- 1 file changed, 110 insertions(+), 12 deletions(-) diff --git a/site/content/en/latest/tasks/observability/proxy-metric.md b/site/content/en/latest/tasks/observability/proxy-metric.md index 404b558926..f615a607c0 100644 --- a/site/content/en/latest/tasks/observability/proxy-metric.md +++ b/site/content/en/latest/tasks/observability/proxy-metric.md @@ -2,38 +2,136 @@ title: "Proxy Metrics" --- -Envoy Gateway provides observability for the ControlPlane and the underlying EnvoyProxy instances. -This task show you how to config proxy metrics. +Envoy Gateway offers observability for both the Control Plane and the underlying Envoy Proxy instances. +This task shows you how to configure proxy metrics. ## Prerequisites -{{< boilerplate o11y_prerequisites >}} +### Install Envoy Gateway + +{{< boilerplate prerequisites >}} + +### Install Add-ons + +Envoy Gateway provides an add-ons Helm chart to simplify the installation of observability components. +The documentation for the add-ons chart can be found +[here](https://gateway.envoyproxy.io/docs/install/gateway-addons-helm-api/). + +Follow the instructions below to install the add-ons Helm chart. + +```shell +helm install eg-addons oci://docker.io/envoyproxy/gateway-addons-helm --version {{< helm-version >}} -n monitoring --create-namespace +``` + +By default, the [OpenTelemetry Collector](https://opentelemetry.io/docs/collector/) is **disabled.** +To install add-ons with OpenTelemetry Collector enabled, use the following command. + +```shell +helm install eg-addons oci://docker.io/envoyproxy/gateway-addons-helm --version {{< helm-version >}} --set opentelemetry-collector.enabled=true -n monitoring --create-namespace +``` ## Metrics -By default, Envoy Gateway expose metrics with prometheus endpoint. +### Prometheus Metrics + +To query metrics using Prometheus API, follow the steps below. + +```shell +export PROMETHEUS_PORT=$(kubectl get service prometheus -n monitoring -o jsonpath='{.spec.ports[0].port}') +kubectl port-forward service/prometheus -n monitoring 19001:$PROMETHEUS_PORT +``` + +Query metrics using Prometheus API: + +```shell +curl -s 'http://localhost:19001/api/v1/query?query=topk(1,envoy_cluster_upstream_cx_connect_ms_sum)' | jq . +``` -Verify metrics: +To directly view the metrics in Prometheus format from the Envoy's `/stats/prometheus` +[admin endpoint](https://www.envoyproxy.io/docs/envoy/latest/operations/admin), follow the steps below. ```shell export ENVOY_POD_NAME=$(kubectl get pod -n envoy-gateway-system --selector=gateway.envoyproxy.io/owning-gateway-namespace=default,gateway.envoyproxy.io/owning-gateway-name=eg -o jsonpath='{.items[0].metadata.name}') kubectl port-forward pod/$ENVOY_POD_NAME -n envoy-gateway-system 19001:19001 +``` -# check metrics +View the metrics: + +```shell curl localhost:19001/stats/prometheus | grep "default/backend/rule/0" ``` -You can disable metrics by setting the `telemetry.metrics.prometheus.disable` to `true` in the `EnvoyProxy` CRD. +To disable the Prometheus metric endpoint, set the `telemetry.metrics.prometheus.disable` to true in the +_EnvoyProxy CRD_ as shown in the following command. -```shell -kubectl apply -f https://raw.githubusercontent.com/envoyproxy/gateway/latest/examples/kubernetes/metric/disable-prometheus.yaml +```yaml +cat < Date: Fri, 21 Mar 2025 13:40:06 -0400 Subject: [PATCH 2/9] docs: update wording. Signed-off-by: sudipto baral --- site/content/en/latest/tasks/observability/proxy-metric.md | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/site/content/en/latest/tasks/observability/proxy-metric.md b/site/content/en/latest/tasks/observability/proxy-metric.md index f615a607c0..9d273a4072 100644 --- a/site/content/en/latest/tasks/observability/proxy-metric.md +++ b/site/content/en/latest/tasks/observability/proxy-metric.md @@ -61,8 +61,8 @@ View the metrics: curl localhost:19001/stats/prometheus | grep "default/backend/rule/0" ``` -To disable the Prometheus metric endpoint, set the `telemetry.metrics.prometheus.disable` to true in the -_EnvoyProxy CRD_ as shown in the following command. +If you are only using the OpenTelemetry sink, you might want to set the `telemetry.metrics.prometheus.disable` to `true` +in the _EnvoyProxy CRD_ as shown in the following command. ```yaml cat < Date: Wed, 26 Mar 2025 19:32:26 -0400 Subject: [PATCH 3/9] docs: Viewing otel metrics in pod logs. Signed-off-by: sudipto baral --- .../tasks/observability/proxy-metric.md | 20 ++++++++++++------- 1 file changed, 13 insertions(+), 7 deletions(-) diff --git a/site/content/en/latest/tasks/observability/proxy-metric.md b/site/content/en/latest/tasks/observability/proxy-metric.md index 9d273a4072..86c947e731 100644 --- a/site/content/en/latest/tasks/observability/proxy-metric.md +++ b/site/content/en/latest/tasks/observability/proxy-metric.md @@ -27,7 +27,7 @@ By default, the [OpenTelemetry Collector](https://opentelemetry.io/docs/collecto To install add-ons with OpenTelemetry Collector enabled, use the following command. ```shell -helm install eg-addons oci://docker.io/envoyproxy/gateway-addons-helm --version {{< helm-version >}} --set opentelemetry-collector.enabled=true -n monitoring --create-namespace +helm install eg-addons oci://docker.io/envoyproxy/gateway-addons-helm --version {{< helm-version >}} --set opentelemetry-collector.enabled=true --set opentelemetry-collector.config.service.pipelines.metrics.exporters='{debug,prometheus}' -n monitoring --create-namespace ``` ## Metrics @@ -94,7 +94,7 @@ EOF To completely remove Prometheus resources from the cluster, set the `prometheus.enabled` Helm value to `false`. ```shell -helm upgrade eg-addons oci://docker.io/envoyproxy/gateway-addons-helm --version v0.0.0-latest -n monitoring --set prometheus.enabled=false +helm upgrade eg-addons oci://docker.io/envoyproxy/gateway-addons-helm --version {{< helm-version >}} -n monitoring --set prometheus.enabled=false ``` ### OpenTelemetry Metrics @@ -133,12 +133,18 @@ spec: EOF ``` -Verify OTel-Collector metrics: +Temporarily enable the debug exporter in the OpenTelemetry Collector +to view metrics in the pod logs using the following commands: ```shell -export OTEL_POD_NAME=$(kubectl get pod -n monitoring --selector=app.kubernetes.io/name=opentelemetry-collector -o jsonpath='{.items[0].metadata.name}') -kubectl port-forward pod/$OTEL_POD_NAME -n monitoring 19001:19001 +helm upgrade eg-addons oci://docker.io/envoyproxy/gateway-addons-helm --version {{< helm-version >}} -n monitoring --reuse-values --set opentelemetry-collector.config.service.pipelines.metrics.exporters='{debug,prometheus}' -# check metrics -curl localhost:19001/metrics | grep "default/backend/rule/0" ``` + +To view the logs of the OpenTelemetry Collector, use the following command: + +```shell +export OTEL_POD_NAME=$(kubectl get pod -n monitoring --selector=app.kubernetes.io/name=opentelemetry-collector -o jsonpath='{.items[0].metadata.name}') +kubectl logs -n monitoring -f $OTEL_POD_NAME --tail=100 + +``` \ No newline at end of file From 85ba3033a240ba1d2f635f9f599d3b263dcf6f35 Mon Sep 17 00:00:00 2001 From: sudipto baral Date: Wed, 26 Mar 2025 19:39:50 -0400 Subject: [PATCH 4/9] docs: use boilerplate Signed-off-by: sudipto baral --- .../latest/boilerplates/o11y_prerequisites.md | 20 ++++++++++++--- .../tasks/observability/proxy-metric.md | 25 ++----------------- 2 files changed, 18 insertions(+), 27 deletions(-) diff --git a/site/content/en/latest/boilerplates/o11y_prerequisites.md b/site/content/en/latest/boilerplates/o11y_prerequisites.md index 81ef67bd18..09ec524e14 100644 --- a/site/content/en/latest/boilerplates/o11y_prerequisites.md +++ b/site/content/en/latest/boilerplates/o11y_prerequisites.md @@ -1,13 +1,25 @@ --- --- +### Install Envoy Gateway + {{< boilerplate prerequisites >}} -Envoy Gateway provides an add-ons Helm Chart, which includes all the needing components for observability. -By default, the [OpenTelemetry Collector](https://opentelemetry.io/docs/collector/) is disabled. +### Install Add-ons + +Envoy Gateway provides an add-ons Helm chart to simplify the installation of observability components. +The documentation for the add-ons chart can be found +[here](https://gateway.envoyproxy.io/docs/install/gateway-addons-helm-api/). -Install the add-ons Helm Chart: +Follow the instructions below to install the add-ons Helm chart. ```shell -helm install eg-addons oci://docker.io/envoyproxy/gateway-addons-helm --version {{< helm-version >}} --set opentelemetry-collector.enabled=true -n monitoring --create-namespace +helm install eg-addons oci://docker.io/envoyproxy/gateway-addons-helm --version {{< helm-version >}} -n monitoring --create-namespace ``` + +By default, the [OpenTelemetry Collector](https://opentelemetry.io/docs/collector/) is **disabled.** +To install add-ons with OpenTelemetry Collector enabled, use the following command. + +```shell +helm install eg-addons oci://docker.io/envoyproxy/gateway-addons-helm --version {{< helm-version >}} --set opentelemetry-collector.enabled=true --set opentelemetry-collector.config.service.pipelines.metrics.exporters='{debug,prometheus}' -n monitoring --create-namespace +``` \ No newline at end of file diff --git a/site/content/en/latest/tasks/observability/proxy-metric.md b/site/content/en/latest/tasks/observability/proxy-metric.md index 86c947e731..15116c9808 100644 --- a/site/content/en/latest/tasks/observability/proxy-metric.md +++ b/site/content/en/latest/tasks/observability/proxy-metric.md @@ -7,28 +7,7 @@ This task shows you how to configure proxy metrics. ## Prerequisites -### Install Envoy Gateway - -{{< boilerplate prerequisites >}} - -### Install Add-ons - -Envoy Gateway provides an add-ons Helm chart to simplify the installation of observability components. -The documentation for the add-ons chart can be found -[here](https://gateway.envoyproxy.io/docs/install/gateway-addons-helm-api/). - -Follow the instructions below to install the add-ons Helm chart. - -```shell -helm install eg-addons oci://docker.io/envoyproxy/gateway-addons-helm --version {{< helm-version >}} -n monitoring --create-namespace -``` - -By default, the [OpenTelemetry Collector](https://opentelemetry.io/docs/collector/) is **disabled.** -To install add-ons with OpenTelemetry Collector enabled, use the following command. - -```shell -helm install eg-addons oci://docker.io/envoyproxy/gateway-addons-helm --version {{< helm-version >}} --set opentelemetry-collector.enabled=true --set opentelemetry-collector.config.service.pipelines.metrics.exporters='{debug,prometheus}' -n monitoring --create-namespace -``` +{{< boilerplate o11y_prerequisites >}} ## Metrics @@ -133,7 +112,7 @@ spec: EOF ``` -Temporarily enable the debug exporter in the OpenTelemetry Collector +Temporarily enable the `debug` exporter in the OpenTelemetry Collector to view metrics in the pod logs using the following commands: ```shell From f42b6eceb0ee36d4b63427a7204959d993bc99a9 Mon Sep 17 00:00:00 2001 From: sudipto baral Date: Wed, 26 Mar 2025 20:08:39 -0400 Subject: [PATCH 5/9] docs: enable debug exporter in otel section. Signed-off-by: sudipto baral --- site/content/en/latest/boilerplates/o11y_prerequisites.md | 2 +- site/content/en/latest/tasks/observability/proxy-metric.md | 3 ++- 2 files changed, 3 insertions(+), 2 deletions(-) diff --git a/site/content/en/latest/boilerplates/o11y_prerequisites.md b/site/content/en/latest/boilerplates/o11y_prerequisites.md index 09ec524e14..5c2e1e3e09 100644 --- a/site/content/en/latest/boilerplates/o11y_prerequisites.md +++ b/site/content/en/latest/boilerplates/o11y_prerequisites.md @@ -21,5 +21,5 @@ By default, the [OpenTelemetry Collector](https://opentelemetry.io/docs/collecto To install add-ons with OpenTelemetry Collector enabled, use the following command. ```shell -helm install eg-addons oci://docker.io/envoyproxy/gateway-addons-helm --version {{< helm-version >}} --set opentelemetry-collector.enabled=true --set opentelemetry-collector.config.service.pipelines.metrics.exporters='{debug,prometheus}' -n monitoring --create-namespace +helm install eg-addons oci://docker.io/envoyproxy/gateway-addons-helm --version {{< helm-version >}} --set opentelemetry-collector.enabled=true -n monitoring --create-namespace ``` \ No newline at end of file diff --git a/site/content/en/latest/tasks/observability/proxy-metric.md b/site/content/en/latest/tasks/observability/proxy-metric.md index 15116c9808..c2e9f5028b 100644 --- a/site/content/en/latest/tasks/observability/proxy-metric.md +++ b/site/content/en/latest/tasks/observability/proxy-metric.md @@ -113,7 +113,8 @@ EOF ``` Temporarily enable the `debug` exporter in the OpenTelemetry Collector -to view metrics in the pod logs using the following commands: +to view metrics in the pod logs using the following command. Debug exporter is enabled for demonstration purposes and +should not be used in production. ```shell helm upgrade eg-addons oci://docker.io/envoyproxy/gateway-addons-helm --version {{< helm-version >}} -n monitoring --reuse-values --set opentelemetry-collector.config.service.pipelines.metrics.exporters='{debug,prometheus}' From 7c274155eb900d08df25044eb4ca475faa19236b Mon Sep 17 00:00:00 2001 From: sudipto baral Date: Wed, 26 Mar 2025 20:18:22 -0400 Subject: [PATCH 6/9] docs: Add next steps. Signed-off-by: sudipto baral --- site/content/en/latest/tasks/observability/proxy-metric.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/site/content/en/latest/tasks/observability/proxy-metric.md b/site/content/en/latest/tasks/observability/proxy-metric.md index c2e9f5028b..f2940ff60a 100644 --- a/site/content/en/latest/tasks/observability/proxy-metric.md +++ b/site/content/en/latest/tasks/observability/proxy-metric.md @@ -127,4 +127,8 @@ To view the logs of the OpenTelemetry Collector, use the following command: export OTEL_POD_NAME=$(kubectl get pod -n monitoring --selector=app.kubernetes.io/name=opentelemetry-collector -o jsonpath='{.items[0].metadata.name}') kubectl logs -n monitoring -f $OTEL_POD_NAME --tail=100 -``` \ No newline at end of file +``` + +## Next Steps + +Checkout [Visualising metrics using Grafana](./grafana-integration.md) to visualise the metrics. \ No newline at end of file From 5c9cc5148ab7c413c919f1abeedc88cd870cdbf3 Mon Sep 17 00:00:00 2001 From: sudipto baral Date: Wed, 26 Mar 2025 20:29:59 -0400 Subject: [PATCH 7/9] docs: Use tab Signed-off-by: sudipto baral --- .../tasks/observability/proxy-metric.md | 69 ++++++++++++++++++- 1 file changed, 67 insertions(+), 2 deletions(-) diff --git a/site/content/en/latest/tasks/observability/proxy-metric.md b/site/content/en/latest/tasks/observability/proxy-metric.md index f2940ff60a..b7f860cfcc 100644 --- a/site/content/en/latest/tasks/observability/proxy-metric.md +++ b/site/content/en/latest/tasks/observability/proxy-metric.md @@ -43,7 +43,9 @@ curl localhost:19001/stats/prometheus | grep "default/backend/rule/0" If you are only using the OpenTelemetry sink, you might want to set the `telemetry.metrics.prometheus.disable` to `true` in the _EnvoyProxy CRD_ as shown in the following command. -```yaml +{{< tabpane text=true >}} +{{% tab header="Apply from stdin" %}} +```shell cat <}} + To completely remove Prometheus resources from the cluster, set the `prometheus.enabled` Helm value to `false`. @@ -82,7 +113,9 @@ Envoy Gateway can export metrics to an OpenTelemetry sink. Use the following com OpenTelemetry Collector. Ensure that the OpenTelemetry components are enabled, as mentioned in the [Prerequisites](#prerequisites). -```yaml +{{< tabpane text=true >}} +{{% tab header="Apply from stdin" %}} +```shell cat <}} + Temporarily enable the `debug` exporter in the OpenTelemetry Collector to view metrics in the pod logs using the following command. Debug exporter is enabled for demonstration purposes and From fcebab34d5b874398fb58b062f33418a8a109152 Mon Sep 17 00:00:00 2001 From: sudipto baral Date: Thu, 27 Mar 2025 01:43:33 -0400 Subject: [PATCH 8/9] docs: Link Envoyproxy with Gateway instead of GatewayClass. Signed-off-by: sudipto baral --- .../tasks/observability/proxy-metric.md | 78 ++++++++++++------- 1 file changed, 49 insertions(+), 29 deletions(-) diff --git a/site/content/en/latest/tasks/observability/proxy-metric.md b/site/content/en/latest/tasks/observability/proxy-metric.md index b7f860cfcc..9290d02c69 100644 --- a/site/content/en/latest/tasks/observability/proxy-metric.md +++ b/site/content/en/latest/tasks/observability/proxy-metric.md @@ -48,16 +48,21 @@ in the _EnvoyProxy CRD_ as shown in the following command. ```shell cat <}} -n monitoring --set prometheus.enabled=false +helm upgrade eg-addons oci://docker.io/envoyproxy/gateway-addons-helm --version {{< helm-version >}} -n monitoring --reuse-values --set prometheus.enabled=false ``` ### OpenTelemetry Metrics @@ -118,16 +128,21 @@ as mentioned in the [Prerequisites](#prerequisites). ```shell cat < Date: Thu, 27 Mar 2025 02:01:44 -0400 Subject: [PATCH 9/9] docs: Update wording. Signed-off-by: sudipto baral --- site/content/en/latest/tasks/observability/proxy-metric.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/site/content/en/latest/tasks/observability/proxy-metric.md b/site/content/en/latest/tasks/observability/proxy-metric.md index 9290d02c69..08b87df22b 100644 --- a/site/content/en/latest/tasks/observability/proxy-metric.md +++ b/site/content/en/latest/tasks/observability/proxy-metric.md @@ -216,4 +216,4 @@ kubectl logs -n monitoring -f $OTEL_POD_NAME --tail=100 ## Next Steps -Checkout [Visualising metrics using Grafana](./grafana-integration.md) to visualise the metrics. \ No newline at end of file +Check out the [Visualising metrics using Grafana](./grafana-integration.md) section to learn more about how you can observe all the metrics in one place. \ No newline at end of file