cAdvisor Integration

CHANGELOG.md

@@ -3,6 +3,8 @@
 - [FEATURE] (beta) Enable experimental config urls for fetching remote configs. Currently,
    only HTTP/S is supported. Use `-experiment.config-urls.enable` flag to turn this on. (@rlankfo)
 
+- [FEATURE] Added [cAdvisor](https://github.com/google/cadvisor) integration. (@rgeyer)
+
 - [ENHANCEMENT] Traces: Improved pod association in PromSD processor (@mapno)
 
 - [BUGFIX] Fix usage of POSTGRES_EXPORTER_DATA_SOURCE_NAME when using postgres_exporter integration (@f11r)

docs/configuration/integrations/cadvisor-config.md

@@ -0,0 +1,112 @@
++++
+title = "cadvisor_config"
++++
+
+# cadvisor_config
+
+The `cadvisor_config` block configures the `cadvisor` integration,
+which is an embedded version of
+[`cadvisor`](https://github.com/google/cadvisor). This allows for the collection of container utilization metrics.
+
+The cAdvisor integration requires some broad privileged permissions to the host. Without these permissions the metrics will not be accessible. This means that the agent must *also* have those elevated permissions.
+
+A good example of the required file, and system permissions can be found in the docker run command published in the [cAdvisor docs](https://github.com/google/cadvisor#quick-start-running-cadvisor-in-a-docker-container).
+
+Full reference of options:
+
+```yaml
+  # Enables the cadvisor integration, allowing the Agent to automatically
+  # collect metrics for the specified github objects.
+  [enabled: <boolean> | default = false]
+
+  # Sets an explicit value for the instance label when the integration is
+  # self-scraped. Overrides inferred values.
+  [instance: <string> | default = <integrations_config.instance>]
+
+  # Automatically collect metrics from this integration. If disabled,
+  # the cadvisor integration will be run but not scraped and thus not
+  # remote-written. Metrics for the integration will be exposed at
+  # /integrations/cadvisor/metrics and can be scraped by an external
+  # process.
+  [scrape_integration: <boolean> | default = <integrations_config.scrape_integrations>]
+
+  # How often should the metrics be collected? Defaults to
+  # prometheus.global.scrape_interval.
+  [scrape_interval: <duration> | default = <global_config.scrape_interval>]
+
+  # The timeout before considering the scrape a failure. Defaults to
+  # prometheus.global.scrape_timeout.
+  [scrape_timeout: <duration> | default = <global_config.scrape_timeout>]
+
+  # Allows for relabeling labels on the target.
+  relabel_configs:
+    [- <relabel_config> ... ]
+
+  # Relabel metrics coming from the integration, allowing to drop series
+  # from the integration that you don't care about.
+  metric_relabel_configs:
+    [ - <relabel_config> ... ]
+
+  # How frequent to truncate the WAL for this integration.
+  [wal_truncate_frequency: <duration> | default = "60m"]
+
+  #
+  # cAdvisor-specific configuration options
+  #
+
+  # Convert container labels and environment variables into labels on prometheus metrics for each container. If false, then only metrics exported are container name, first alias, and image name.
+  [store_container_labels: <boolean> | default = true]
+
+  # List of container labels to be converted to labels on prometheus metrics for each container. store_container_labels must be set to false for this to take effect.
+  allowlisted_container_labels:
+    [ - <string> ]
+
+  # List of environment variable keys matched with specified prefix that needs to be collected for containers, only support containerd and docker runtime for now.
+  env_metadata_allowlist:
+    [ - <string> ]
+
+  # List of cgroup path prefix that needs to be collected even when docker_only is specified.
+  raw_cgroup_prefix_allowlist:
+    [ - <string> ]
+
+  # Path to a JSON file containing configuration of perf events to measure. Empty value disabled perf events measuring.
+  [perf_events_config: <boolean>]
+
+  # resctrl mon groups updating interval. Zero value disables updating mon groups.
+  [resctrl_interval: <int> | default = 0]
+
+  # List of `metrics` to be disabled.
+  disabled_metrics:
+    [ - <string> ]
+
+  # List of `metrics` to be enabled. If set, overrides disabled_metrics
+  enabled_metrics:
+    [ - <string> ]
+
+  # Length of time to keep data stored in memory
+  [storage_duration: <duration> | default = "2m"]
+
+  # Containerd endpoint
+  [containerd: <string> | default = "/run/containerd/containerd.sock"]
+
+  # Containerd namespace
+  [containerd_namespace: <string> | default = "k8s.io"]
+
+  # Docker endpoint
+  [docker: <string> | default = "unix:///var/run/docker.sock"]
+
+  # Use TLS to connect to docker
+  [docker_tls: <boolean> | default = false]
+
+  # Path to client certificate for TLS connection to docker
+  [docker_tls_cert: <string> | default = "cert.pem"]
+
+  # Path to private key for TLS connection to docker
+  [docker_tls_key: <string> | default = "key.pem"]
+
+  # Path to a trusted CA for TLS connection to docker
+  [docker_tls_ca: <string> | default = "ca.pem"]
+
+  # Only report docker containers in addition to root stats
+  [docker_only: <boolean> | default = false]
+```

example/docker-compose/agent/config/agent-local.yaml

@@ -85,4 +85,10 @@ integrations:
       replacement: 'mongodb'   
     - source_labels: [__address__]
       target_label: mongodb_cluster
-      replacement: 'mongodb-cluster'  
+      replacement: 'mongodb-cluster'
+  cadvisor:
+    enabled: true
+    disabled_metrics:
+    - disk
+    enabled_metrics:
+    - percpu

go.mod

@@ -17,6 +17,7 @@ require (
 	github.com/go-sql-driver/mysql v1.6.0
 	github.com/gogo/protobuf v1.3.2
 	github.com/golang/protobuf v1.5.2
+	github.com/google/cadvisor v0.43.0 // indirect
 	github.com/google/dnsmasq_exporter v0.0.0-00010101000000-000000000000
 	github.com/google/go-jsonnet v0.17.0
 	github.com/gorilla/mux v1.8.0
@@ -84,6 +85,9 @@ require (
 	k8s.io/api v0.22.3
 	k8s.io/apimachinery v0.22.3
 	k8s.io/client-go v12.0.0+incompatible
+	k8s.io/klog v1.0.0 // indirect
+	k8s.io/klog/v2 v2.20.0 // indirect
+	k8s.io/utils v0.0.0-20211116205334-6203023598ed // indirect
 	sigs.k8s.io/controller-runtime v0.9.0-beta.5
 	sigs.k8s.io/yaml v1.2.0
 )