Skip to content

Backport of Add emit_tags_as_labels setting to envoy metrics service cfg into release/1.0.x#196

Merged
hc-github-team-consul-core merged 3 commits intorelease/1.0.xfrom
backport/jjtimmons/add-envoy-emit-tags/luckily-winning-bat
Jun 30, 2023
Merged

Backport of Add emit_tags_as_labels setting to envoy metrics service cfg into release/1.0.x#196
hc-github-team-consul-core merged 3 commits intorelease/1.0.xfrom
backport/jjtimmons/add-envoy-emit-tags/luckily-winning-bat

Conversation

@hc-github-team-consul-core
Copy link
Collaborator

Backport

This PR is auto-generated from #184 to be assessed for backporting due to the inclusion of the label backport/1.0.

The below text is copied from the body of the original PR.


This syncs the change from consul for the same: hashicorp/consul#17888

We need this to get envoy service metrics tagged with source and destination service meta. Without it, we get metrics from Envoy that do not have the upstream service name, partition, etc (it's just the FQDN). Eg:

cluster_ef15b5b5_consul_telemetry_collector_default_otlp_single_node_cluster_internal_d1779413_9e75_e538_94cc_8a6b0e6a5175_consul_upstream_cx_total

I made this by:

  1. make copy-bootstrap-config sync'ing changes
  2. manually updating pkg/consuldp/testdata/TestBootstrapConfig/hcp-metrics.golden that I guess isn't sync'ed

Testing & Reproduction steps

I made this change directly to consul-dataplane, deployed it to a personal Docker Hub repo, and ran consul-k8s install with it:

consul-k8s install -preset cloud -hcp-resource-id "$HCP_RESOURCE_ID" -demo \
    --set="global.enterpriseLicense.secretKey=key" \
    --set="global.enterpriseLicense.secretName=consul-license" \
    --set="global.logLevel=debug" \
    --set="global.imageConsulDataplane=jjtimmons/consul-dataplane:latest" \
    --set="metrics.enableTelemetryCollector=true" \
    --set="peering.enabled=true" \
    --set="server.replicas=3"

Checked Prometheus and we're getting all the labels:

{name="http_downstream_rq_active", cluster="josh-test-envoy-emit-tags-labels", consul_source_datacenter="josh-test-envoy-emit-tags-labels", consul_source_namespace="default", consul_source_partition="default", consul_source_service="frontend", consul_upstream_datacenter="josh-test-envoy-emit-tags-labels", consul_upstream_namespace="default", consul_upstream_partition="default", consul_upstream_service="consul-telemetry-collector", envoy_cluster="frontend", envoy_http_conn_manager_prefix="upstream", hcp_internal_id="00000000-e387-6f3e-286b-bc6f1e1b4f84", hcp_organization_id="00000000-06f2-45b3-96ba-e4b0ced62717", hcp_project_id="0000000-0087-4a1b-a9d6-b6030d55ea5c", local_cluster="frontend", namespace="default", node_id="frontend-f74f5f4d4-cw72z-frontend-sidecar-proxy", partition="default"}

Links

Envoy docs: https://www.envoyproxy.io/docs/envoy/latest/api-v3/config/metrics/v3/metrics_service.proto

If true, metrics will have their tags emitted as labels on the metrics objects sent to the MetricsService, and the tag extracted name will be used instead of the full name, which may contain values used by the tag extractor or additional tags added during stats creation.


Overview of commits

@hc-github-team-consul-core hc-github-team-consul-core requested a review from a team as a code owner June 29, 2023 13:26
@hc-github-team-consul-core hc-github-team-consul-core force-pushed the backport/jjtimmons/add-envoy-emit-tags/luckily-winning-bat branch from af668eb to a99c421 Compare June 29, 2023 13:26
@hc-github-team-consul-core hc-github-team-consul-core enabled auto-merge (squash) June 29, 2023 13:26
@hc-github-team-consul-core hc-github-team-consul-core merged commit 8e2b209 into release/1.0.x Jun 30, 2023
@hc-github-team-consul-core hc-github-team-consul-core deleted the backport/jjtimmons/add-envoy-emit-tags/luckily-winning-bat branch June 30, 2023 16:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants