Skip to content

feat: Populate metric unit of cpu metrics #274

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

feat: Populate metric unit of cpu metrics #274

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion charts/agent/Chart.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ apiVersion: v2
name: agent
description: Chart to install K8s collection stack based on Observe Agent
type: application
version: 0.30.6
version: 0.31.0
appVersion: "1.1.0"
dependencies:
- name: opentelemetry-collector
Expand Down
2 changes: 1 addition & 1 deletion charts/agent/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# agent

![Version: 0.30.6](https://img.shields.io/badge/Version-0.30.6-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square) ![AppVersion: 1.1.0](https://img.shields.io/badge/AppVersion-1.1.0-informational?style=flat-square)
![Version: 0.31.0](https://img.shields.io/badge/Version-0.31.0-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square) ![AppVersion: 1.1.0](https://img.shields.io/badge/AppVersion-1.1.0-informational?style=flat-square)

> [!CAUTION]
> This chart is under active development and is not meant to be installed yet.
Expand Down
9 changes: 6 additions & 3 deletions charts/agent/templates/_cluster-metrics-config.tpl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -151,6 +151,8 @@ processors:

{{- include "config.processors.resource.observe_common" . | nindent 2 }}

{{- include "config.processors.transform.metrics" . | nindent 2 }}

# attributes to append to objects
attributes/debug_source_cluster_metrics:
actions:
Expand All @@ -163,19 +165,20 @@ processors:
action: insert
value: pod_metrics


service:
extensions: [health_check]
pipelines:
metrics:
receivers: [k8s_cluster]
processors: [memory_limiter, k8sattributes, batch, resource/observe_common, attributes/debug_source_cluster_metrics]
processors: [memory_limiter, k8sattributes, batch, resource/observe_common, attributes/debug_source_cluster_metrics, transform/metrics]
exporters: [prometheusremotewrite, debug/override]
{{- if .Values.application.prometheusScrape.enabled }}
metrics/pod_metrics:
receivers: [prometheus/pod_metrics]
processors: [memory_limiter, k8sattributes, batch, resource/observe_common, attributes/debug_source_pod_metrics]
processors: [memory_limiter, k8sattributes, batch, resource/observe_common, attributes/debug_source_pod_metrics, transform/metrics]
exporters: [prometheusremotewrite, debug/override]
{{ end -}}
{{- include "config.service.telemetry" . | nindent 2 }}

{{- end }}
{{- end }}
10 changes: 10 additions & 0 deletions charts/agent/templates/_config-processors.tpl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -87,3 +87,13 @@ memory_limiter:
{{- define "config.processors.attributes.observek8sattributes" -}}
observek8sattributes:
{{- end -}}

{{- define "config.processors.transform.metrics" -}}
transform/metrics:
metric_statements:
- context: metric
statements:
- set(unit, "test")
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These are my various attempts at making the change show up in o2. All of them have failed so far.
The docs say that I can access the "unit" field from the "metric" context, but apparently it's not the case, unless I am missing something.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for some reason even the metrics transform processor doesn't mention being able to modify a metric's unit. i wonder if its prohibited for some reason

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

found this example
https://opentelemetry.io/blog/2023/any-metric-receiver/#fine-tuning-with-the-transform-processor

processors:
  transform:
    metric_statements:
      - context: datapoint
        statements:
          - set(metric.unit, attributes["unit"])
          - delete_key(attributes, "unit")

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok so this uses a lower level context. This will be more expensive for sure, I'll try it out. Thanks!

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I will open an issue regarding the metric context btw, because it explicitly says that you can access the unit, but it clearly doesn't work.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wait but your example also doesn't work, right? Have you tried it? I don't see any metrics showing up with the "unit" label

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't tried it, just found it in the opentelemetry docs

# - set(unit, "cores") where unit == ""
# - set(unit, "cores") where IsMatch(name, ".*cpu.*")
{{- end -}}