Consul Kubernetes Datadog Integration Docs Update

[natemollica-nm]

Description

Adding in an Observability piece to supplement hashicorp/consul-k8s#3407 which added
helm configuration options for configuring Datadog metrics integration with consul-k8s.

Testing & Reproduction steps

Worked with the information here to update Consul documentation website with details outlining the changes made in the PR.

Links

• Consul Integration (Datadog Checks)
• Consul Integration Source Code
• DogstatsD over UDP
• DogstatsD over Unix Domain Socket

PR Checklist

• external facing docs updated
• appropriate backport labels added
• not a security concern

[boruszak]

Thanks for this docs contribution!

There are some style issues that need to be sorted before I can finish reviewing the page and approve it.

Refer to the following style guides to help you format:

https://github.com/hashicorp/tutorials/blob/main/STYLE_GUIDE.md

https://github.com/hashicorp/engineering-docs/blob/main/writing/markdown.md

[boruszak]

Can you reformat this table using Markdown instead of HTML? Link to style guide: Markdown vs. HTML tables

[natemollica-nm]

Done!

[boruszak]

1. Highlight is not appropriate here. Refer to the Alerts section in the style guide for more info.
2. Even if you use an alert, this one is too long. So format it without the alert and we can adjust from there.
3. Use Markdown instead of HTML.

[natemollica-nm]

Done!

[boruszak]

Can you add a sentence between these headings to describe the section? And a sentence after "Helm CHart configuration?

[natemollica-nm]

Done!

[boruszak]

Suggested change

	Use of this method will map to Datadog as described in [Mapping Prometheus Metrics to Datadog Metrics](https://docs.datadoghq.com/integrations/guide/prometheus-metrics/?tab=latestversion) and summarized below.
	Use of this method maps to Datadog as described in [Mapping Prometheus Metrics to Datadog Metrics](https://docs.datadoghq.com/integrations/guide/prometheus-metrics/?tab=latestversion). The following table summarizing how these metrics map to each other.

[natemollica-nm]

updated!

[boruszak]

Suggested change

	| OpenMetrics Metric Type | Datadog Metric Type |
	|-------------------------|------------------------------------|
	| `Gauge` | `gauge` |
	| `Counter` | `count` |
	| Histogram: `_count ` | `count.count` |
	| Histogram: `_sum` | `count.sum` |
	| Histogram: `_bucket` | `count.bucket` \|\| `distribution` |
	| Summary: `_count` | `count.count` |
	| Summary: `_sum` | `count.sum` |
	| Summary: `sample` | `gauge.quantile` |
	| OpenMetrics metric type | Datadog metric type |
	| :------------------------ | :----------------------------------- |
	| `Gauge` | `gauge` |
	| `Counter` | `count` |
	| Histogram: `_count ` | `count.count` |
	| Histogram: `_sum` | `count.sum` |
	| Histogram: `_bucket` | `count.bucket` \|\| `distribution` |
	| Summary: `_count` | `count.count` |
	| Summary: `_sum` | `count.sum` |
	| Summary: `sample` | `gauge.quantile` |

[natemollica-nm]

Updated!

[boruszak]

I'm not sure what your intent is with this section? The list isn't described so I can't tell what info you want to communicate to the user.

[natemollica-nm]

You're right. Don't know why this was as a list. Thank you! Updated!

[david-yu]

@natemollica-nm Thank you for doing this and I think a huge boost to what we can do with DataDog. I have a high level question around placement of this doc. It's currently place under Service Mesh -> Observabililty. Are these metrics more for gathering Consul server metrics via DataDog or are these for collecting metrics for all service mesh components?

[natemollica-nm]

@david-yu -- Good point to bring up on the placement. This is more Consul specific metrics collection from the servers and nothing regarding Envoy metrics. That's a separate set of annotations that should be placed on Kube pods that have dataplane containers with them. Any recommendations on better placement or where you'd like to see it?

[david-yu]

I think moving it under Deployment Configs makes the most sense for me perhaps: https://developer.hashicorp.com/consul/docs/k8s/deployment-configurations/datadog-metrics

I would put it under Vault Secrets Backend in the nav bar.

[natemollica-nm]

@david-yu -- Done! Just moved it over.

[david-yu]

Awesome, great work!

[natemollica-nm]

@david-yu @boruszak -- Sorry for this last commit and push 😬. I added some additional amplifying information in regard to the Consul Datadog Checks and OpenMetrics sections (Notes) in regard to how Consul RPC TLS certificates are handles by the automated integration. Let me know if these changes are OK with you guys and I'll go ahead and merge.

[david-yu]

Changes LGTM