Clean up K8s deployment methods

[hjet]

• Updates README directing users to Graf Cloud docs for full instructions
• Removes generation of agent.yaml and agent-sigv4.yaml from build script
• Removes generated agent.yaml and agent-sigv4.yaml
• Updates logs Tanka to remove ConfigMap (moved to Graf Cloud docs)
• Removes install scripts

PR Checklist

• CHANGELOG updated
• Documentation added
• Tests updated

[CLAassistant]

All committers have signed the CLA.

[hjet]

@rfratto also tagging you on the logs docs PR

[hjet]

Dead link is a chicken-egg with https://github.com/grafana/website/pull/5248

[rfratto]

Thanks!

[hjet]

Dead links will be fixed by https://github.com/grafana/website/pull/5258, I guess we'll wait for that to get in first on Monday (cc @matthewhelmke)

[rfratto]

Looks like the links are still broken, does this need to be updated again?

[hjet]

Yeah just pushed a fix, the manifests were referenced in another spot

[hjet]

Should have tracing done by EOD (but doesn't block this PR)

[rfratto]

Thanks!

[flokli]

I noticed this stops deploying node-exporter, and having custom scrape configs from node-exporter and kube-state-metrics.

The Grafana Agent Metrics Kubernetes Quickstart doesn't seem to include a mention about this - as well as the requirement of a port named -metrics. At least for the latter, we had that remark in https://github.com/grafana/agent/pull/749/files#diff-4369de20767a72f00e221e1f2c5dab1f1b282af26d355ab3ce8289a840c64a26L64 - and should probably take it over to the website.

[hjet]

Hey @flokli, thanks for your message and interest in the Grafana Agent!

I noticed this stops deploying node-exporter, and having custom scrape configs from node-exporter and kube-state-metrics.

Yeah, that agent.yaml (with node-exporter, kube-state-metrics, etc.) manifest was confusing users more than it was helping them so we've decided to temporarily keep things simple with 3 "getting started" manifests:

• agent-bare.yaml for metrics (deployment)
• agent-loki.yaml for logs (daemonset)
• agent-tempo.yaml for traces (deployment)

With quickstarts available here.

Running the Agent as a DaemonSet for scraping metrics (and using the Agent node-exporter integration) is still possible, however the configuration and setup is a little more involved, so we've chosen to adopt a simpler approach for "quickstart" purposes. You can learn more about that architecture in this community forums post.

As @rfratto noted, we are focussing on the Agent Operator (note only supports metrics at the moment) and Helm charts, as well as first-class integrations support, which will streamline and simplify deploying Agent (and node-exporter) into your clusters. We are also working on improvements to the Agent's clustering and scraping service, which will also help with this.

as well as the requirement of a port named -metrics.

The new scrape config in the quickstart doesn't have this clause:

- action: keep
  regex: .*-metrics
  source_labels:
    - __meta_kubernetes_pod_container_port_name

So there is no requirement for a -metrics port name anymore. The default config only scrapes the /metrics/cadvisor and /metrics endpoints on all of your cluster Nodes.

[flokli]

I realized the agent-bare.yaml doesn't scrape pods at all anymore - so yes, the -metrics requirement is gone as well as with that whole scraping. I guess the recommended way to bring this back is to run other instances of grafana-agent, with (some) pod scraping config, to get those values back?

[hjet]

This guide shows you how to bring up an Agent Deployment to scrape metrics in your cluster.

The default config does not configure the Agent to scrape Pod metrics. A sample config stanza to scrape Pod metrics can be found in the old manifest.

For convenience I am pasting the relevant section here:

- job_name: kubernetes-pods
  kubernetes_sd_configs:
    - role: pod
  relabel_configs:
    - action: drop
      regex: "false"
      source_labels:
        - __meta_kubernetes_pod_annotation_prometheus_io_scrape
    - action: keep
      regex: .*-metrics
      source_labels:
        - __meta_kubernetes_pod_container_port_name
    - action: replace
      regex: (https?)
      replacement: $1
      source_labels:
        - __meta_kubernetes_pod_annotation_prometheus_io_scheme
      target_label: __scheme__
    - action: replace
      regex: (.+)
      replacement: $1
      source_labels:
        - __meta_kubernetes_pod_annotation_prometheus_io_path
      target_label: __metrics_path__
    - action: replace
      regex: (.+?)(\:\d+)?;(\d+)
      replacement: $1:$3
      source_labels:
        - __address__
        - __meta_kubernetes_pod_annotation_prometheus_io_port
      target_label: __address__
    - action: drop
      regex: ""
      source_labels:
        - __meta_kubernetes_pod_label_name
    - action: replace
      replacement: $1
      separator: /
      source_labels:
        - __meta_kubernetes_namespace
        - __meta_kubernetes_pod_label_name
      target_label: job
    - action: replace
      source_labels:
        - __meta_kubernetes_namespace
      target_label: namespace
    - action: replace
      source_labels:
        - __meta_kubernetes_pod_name
      target_label: pod
    - action: replace
      source_labels:
        - __meta_kubernetes_pod_container_name
      target_label: container
    - action: replace
      separator: ':'
      source_labels:
        - __meta_kubernetes_pod_name
        - __meta_kubernetes_pod_container_name
        - __meta_kubernetes_pod_container_port_name
      target_label: instance
    - action: labelmap
      regex: __meta_kubernetes_pod_annotation_prometheus_io_param_(.+)
      replacement: __param_$1
    - action: drop
      regex: Succeeded|Failed
      source_labels:
        - __meta_kubernetes_pod_phase

Note that you may need to modify this depending on your use case. If you are using this block, note that the old requirements also apply:

• The pod must not have an annotation matching prometheus.io/scrape: "false" (this wouldn't be there unless you explicitly add it or if you deploy a Helm chart that has it).
• The pod must have a port with a name ending in -metrics. This is the port that will be scraped by the Agent. A lot of people using Helm struggle with this, since Helm charts don't usually follow this. You would need to add a new scrape config to scrape helm charts or find a way to tweak the Helm chart to follow this rules.
• The pod must have a label named name with any non-empty value. Helm usually lets you add extra labels, so this is less of a problem for Helm users.
• The pod must currently be running. (i.e., Kubernetes must not report it having a phase of Succeeded or Failed).

TLDR you can use the same manifest / Deployment, just add the above config block to the Agent's ConfigMap to configure it to scrape Pod metrics.

[flokli]

Thanks! Yeah, I realized these were gone when switching from agent.yaml to agent-bare.yaml.

Instead of resurrecting them and writing a lot of scraping rules by myself, I went ahead deploying your operator, and have it generate that yaml for me :-)