Skip to content

Add emit_tags_as_labels setting to envoy metrics service cfg#184

Merged
jjti merged 3 commits intomainfrom
jjtimmons/add-envoy-emit-tags
Jun 29, 2023
Merged

Add emit_tags_as_labels setting to envoy metrics service cfg#184
jjti merged 3 commits intomainfrom
jjtimmons/add-envoy-emit-tags

Conversation

@jjti
Copy link
Contributor

@jjti jjti commented Jun 27, 2023

This syncs the change from consul for the same: hashicorp/consul#17888

We need this to get envoy service metrics tagged with source and destination service meta. Without it, we get metrics from Envoy that do not have the upstream service name, partition, etc (it's just the FQDN). Eg:

cluster_ef15b5b5_consul_telemetry_collector_default_otlp_single_node_cluster_internal_d1779413_9e75_e538_94cc_8a6b0e6a5175_consul_upstream_cx_total

I made this by:

  1. make copy-bootstrap-config sync'ing changes
  2. manually updating pkg/consuldp/testdata/TestBootstrapConfig/hcp-metrics.golden that I guess isn't sync'ed

Testing & Reproduction steps

I made this change directly to consul-dataplane, deployed it to a personal Docker Hub repo, and ran consul-k8s install with it:

consul-k8s install -preset cloud -hcp-resource-id "$HCP_RESOURCE_ID" -demo \
    --set="global.enterpriseLicense.secretKey=key" \
    --set="global.enterpriseLicense.secretName=consul-license" \
    --set="global.logLevel=debug" \
    --set="global.imageConsulDataplane=jjtimmons/consul-dataplane:latest" \
    --set="metrics.enableTelemetryCollector=true" \
    --set="peering.enabled=true" \
    --set="server.replicas=3"

Checked Prometheus and we're getting all the labels:

{name="http_downstream_rq_active", cluster="josh-test-envoy-emit-tags-labels", consul_source_datacenter="josh-test-envoy-emit-tags-labels", consul_source_namespace="default", consul_source_partition="default", consul_source_service="frontend", consul_upstream_datacenter="josh-test-envoy-emit-tags-labels", consul_upstream_namespace="default", consul_upstream_partition="default", consul_upstream_service="consul-telemetry-collector", envoy_cluster="frontend", envoy_http_conn_manager_prefix="upstream", hcp_internal_id="00000000-e387-6f3e-286b-bc6f1e1b4f84", hcp_organization_id="00000000-06f2-45b3-96ba-e4b0ced62717", hcp_project_id="0000000-0087-4a1b-a9d6-b6030d55ea5c", local_cluster="frontend", namespace="default", node_id="frontend-f74f5f4d4-cw72z-frontend-sidecar-proxy", partition="default"}

Links

Envoy docs: https://www.envoyproxy.io/docs/envoy/latest/api-v3/config/metrics/v3/metrics_service.proto

If true, metrics will have their tags emitted as labels on the metrics objects sent to the MetricsService, and the tag extracted name will be used instead of the full name, which may contain values used by the tag extractor or additional tags added during stats creation.

josh added 2 commits June 27, 2023 12:36
Signed-off-by: josh <josh.timmons@hashicorp.com>
Signed-off-by: josh <josh.timmons@hashicorp.com>
@jjti jjti marked this pull request as ready for review June 27, 2023 16:39
@jjti jjti requested a review from a team as a code owner June 27, 2023 16:39
Signed-off-by: josh <josh.timmons@hashicorp.com>
@jjti jjti added backport/1.1 Changes are backported to 1.1 backport/1.2 labels Jun 27, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport/1.1 Changes are backported to 1.1

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants