diff --git a/.chloggen/fix_container_cpu_mode.yaml b/.chloggen/fix_container_cpu_mode.yaml new file mode 100644 index 0000000000..f010ac0482 --- /dev/null +++ b/.chloggen/fix_container_cpu_mode.yaml @@ -0,0 +1,22 @@ +# Use this changelog template to create an entry for release notes. +# +# If your change doesn't affect end users you should instead start +# your pull request title with [chore] or use the "Skip Changelog" label. + +# One of 'breaking', 'deprecation', 'new_component', 'enhancement', 'bug_fix' +change_type: bug_fix + +# The name of the area of concern in the attributes-registry, (e.g. http, cloud, db) +component: container + +# A brief description of the change. Surround your text with quotes ("") if it needs to start with a backtick (`). +note: Remove kernel from cpu.mode enum and clarify container cpu states + +# Mandatory: One or more tracking issues related to the change. You can use the PR number here if no issue exists. +# The values here must be integers. +issues: [3700] + +# (Optional) One or more lines of additional information to render under the primary note. +# These lines will be padded with 2 spaces and then inserted directly into the document. +# Use pipe (|) for multiline entries. +subtext: diff --git a/docs/registry/attributes/cpu.md b/docs/registry/attributes/cpu.md index 8c559fa969..3a3b48fa7e 100644 --- a/docs/registry/attributes/cpu.md +++ b/docs/registry/attributes/cpu.md @@ -23,7 +23,6 @@ Attributes specific to a cpu instance. | `idle` | Idle | ![Development](https://img.shields.io/badge/-development-blue) | | `interrupt` | Interrupt | ![Development](https://img.shields.io/badge/-development-blue) | | `iowait` | IO Wait | ![Development](https://img.shields.io/badge/-development-blue) | -| `kernel` | Kernel | ![Development](https://img.shields.io/badge/-development-blue) | | `nice` | Nice | ![Development](https://img.shields.io/badge/-development-blue) | | `steal` | Steal | ![Development](https://img.shields.io/badge/-development-blue) | | `system` | System | ![Development](https://img.shields.io/badge/-development-blue) | diff --git a/docs/runtime/dotnet-metrics.md b/docs/runtime/dotnet-metrics.md index bfe08dcf07..1e3371d2b1 100644 --- a/docs/runtime/dotnet-metrics.md +++ b/docs/runtime/dotnet-metrics.md @@ -96,7 +96,6 @@ This metric reports the same values as accessing the corresponding processor tim | `idle` | Idle | ![Development](https://img.shields.io/badge/-development-blue) | | `interrupt` | Interrupt | ![Development](https://img.shields.io/badge/-development-blue) | | `iowait` | IO Wait | ![Development](https://img.shields.io/badge/-development-blue) | -| `kernel` | Kernel | ![Development](https://img.shields.io/badge/-development-blue) | | `nice` | Nice | ![Development](https://img.shields.io/badge/-development-blue) | | `steal` | Steal | ![Development](https://img.shields.io/badge/-development-blue) | | `system` | System | ![Development](https://img.shields.io/badge/-development-blue) | diff --git a/docs/system/container-metrics.md b/docs/system/container-metrics.md index 27d578dd97..3a43aff317 100644 --- a/docs/system/container-metrics.md +++ b/docs/system/container-metrics.md @@ -61,9 +61,9 @@ This metric is [recommended][MetricRecommended]. | Name | Instrument Type | Unit (UCUM) | Description | Stability | Entity Associations | | -------- | --------------- | ----------- | -------------- | --------- | ------ | -| `container.cpu.time` | Counter | `s` | Total CPU time consumed. [1] | ![Release Candidate](https://img.shields.io/badge/-rc-mediumorchid) | [`container`](/docs/registry/entities/container.md#container) | +| `container.cpu.time` | Counter | `s` | CPU time consumed. [1] | ![Release Candidate](https://img.shields.io/badge/-rc-mediumorchid) | [`container`](/docs/registry/entities/container.md#container) | -**[1]:** Total CPU time consumed by the specific container on all available CPU cores +**[1]:** CPU time consumed by the specific container on all available CPU cores **Attributes:** @@ -71,9 +71,12 @@ This metric is [recommended][MetricRecommended]. | --- | --- | --- | --- | --- | --- | | [`cpu.mode`](/docs/registry/attributes/cpu.md) | ![Development](https://img.shields.io/badge/-development-blue) | `Conditionally Required` [1] | string | The CPU mode for this data point. A container's CPU metric SHOULD be characterized _either_ by data points with no `mode` labels, _or only_ data points with `mode` labels. [2] | `user`; `system` | -**[1] `cpu.mode`:** Required if mode is available, i.e. metrics coming from the Docker Stats API. +**[1] `cpu.mode`:** Required if mode is available, i.e. metrics coming from the Docker Stats API, containerd stats or cAdvisor. -**[2] `cpu.mode`:** Following states SHOULD be used: `user`, `system`, `kernel` +**[2] `cpu.mode`:** Following modes SHOULD be used for containers if available: `user`, `system`. +In implementations like [`opencontainers/cgroup`](https://pkg.go.dev/github.com/opencontainers/cgroups@v0.0.6#CpuUsage), [Moby/Docker](https://pkg.go.dev/github.com/Moby/Moby@v28.5.2+incompatible/api/types/container#CPUUsage) or [cAdvisor](https://pkg.go.dev/github.com/google/cadvisor@v0.56.2/info/v1#CpuUsage), the respective states are retrieved from the cgroup stats directly. In cgroup v1 the user/system modes come from the [cpuacct.stat](https://www.kernel.org/doc/html/latest/admin-guide/cgroup-v1/cpuacct.html#cpu-accounting-controller) file and the `user`/`system` stats respectively. In cgroup v2 the user/system modes come from the [cpu.stat](https://docs.kernel.org/admin-guide/cgroup-v2.html#cpu-interface-files) file and the `user_usec`/`system_usec` stats respectively. +Kubernetes only exposes the total CPU time ([cpu.stat.usage_usec](https://docs.kernel.org/admin-guide/cgroup-v2.html#cpu-interface-files)) through Kubelet's stats API, hence the metric should be reported without the mode attribute implying it is the total of all modes. +In pure container environments the CPU's mode is usually available and cpu.mode should be reported. In that case summarising the CPU time over the different modes gives the total CPU time. --- @@ -84,7 +87,6 @@ This metric is [recommended][MetricRecommended]. | `idle` | Idle | ![Development](https://img.shields.io/badge/-development-blue) | | `interrupt` | Interrupt | ![Development](https://img.shields.io/badge/-development-blue) | | `iowait` | IO Wait | ![Development](https://img.shields.io/badge/-development-blue) | -| `kernel` | Kernel | ![Development](https://img.shields.io/badge/-development-blue) | | `nice` | Nice | ![Development](https://img.shields.io/badge/-development-blue) | | `steal` | Steal | ![Development](https://img.shields.io/badge/-development-blue) | | `system` | System | ![Development](https://img.shields.io/badge/-development-blue) | @@ -115,9 +117,9 @@ This metric is [opt-in][MetricOptIn]. | --- | --- | --- | --- | --- | --- | | [`cpu.mode`](/docs/registry/attributes/cpu.md) | ![Development](https://img.shields.io/badge/-development-blue) | `Conditionally Required` [1] | string | The CPU mode for this data point. A container's CPU metric SHOULD be characterized _either_ by data points with no `mode` labels, _or only_ data points with `mode` labels. [2] | `user`; `system` | -**[1] `cpu.mode`:** Required if mode is available, i.e. metrics coming from the Docker Stats API. +**[1] `cpu.mode`:** Required if mode is available, i.e. metrics coming from the Docker Stats API, containerd stats or cAdvisor. -**[2] `cpu.mode`:** Following states SHOULD be used: `user`, `system`, `kernel` +**[2] `cpu.mode`:** Following modes SHOULD be used for containers if available: `user`, `system`. --- @@ -128,7 +130,6 @@ This metric is [opt-in][MetricOptIn]. | `idle` | Idle | ![Development](https://img.shields.io/badge/-development-blue) | | `interrupt` | Interrupt | ![Development](https://img.shields.io/badge/-development-blue) | | `iowait` | IO Wait | ![Development](https://img.shields.io/badge/-development-blue) | -| `kernel` | Kernel | ![Development](https://img.shields.io/badge/-development-blue) | | `nice` | Nice | ![Development](https://img.shields.io/badge/-development-blue) | | `steal` | Steal | ![Development](https://img.shields.io/badge/-development-blue) | | `system` | System | ![Development](https://img.shields.io/badge/-development-blue) | diff --git a/docs/system/process-metrics.md b/docs/system/process-metrics.md index bd12e9a8c9..eb3c7ff079 100644 --- a/docs/system/process-metrics.md +++ b/docs/system/process-metrics.md @@ -76,7 +76,6 @@ This metric is [recommended][MetricRecommended]. | `idle` | Idle | ![Development](https://img.shields.io/badge/-development-blue) | | `interrupt` | Interrupt | ![Development](https://img.shields.io/badge/-development-blue) | | `iowait` | IO Wait | ![Development](https://img.shields.io/badge/-development-blue) | -| `kernel` | Kernel | ![Development](https://img.shields.io/badge/-development-blue) | | `nice` | Nice | ![Development](https://img.shields.io/badge/-development-blue) | | `steal` | Steal | ![Development](https://img.shields.io/badge/-development-blue) | | `system` | System | ![Development](https://img.shields.io/badge/-development-blue) | @@ -116,7 +115,6 @@ This metric is [opt-in][MetricOptIn]. | `idle` | Idle | ![Development](https://img.shields.io/badge/-development-blue) | | `interrupt` | Interrupt | ![Development](https://img.shields.io/badge/-development-blue) | | `iowait` | IO Wait | ![Development](https://img.shields.io/badge/-development-blue) | -| `kernel` | Kernel | ![Development](https://img.shields.io/badge/-development-blue) | | `nice` | Nice | ![Development](https://img.shields.io/badge/-development-blue) | | `steal` | Steal | ![Development](https://img.shields.io/badge/-development-blue) | | `system` | System | ![Development](https://img.shields.io/badge/-development-blue) | diff --git a/docs/system/system-metrics.md b/docs/system/system-metrics.md index 51e698f31e..624ecfcfb4 100644 --- a/docs/system/system-metrics.md +++ b/docs/system/system-metrics.md @@ -175,7 +175,6 @@ This metric is [recommended][MetricRecommended]. | `idle` | Idle | ![Development](https://img.shields.io/badge/-development-blue) | | `interrupt` | Interrupt | ![Development](https://img.shields.io/badge/-development-blue) | | `iowait` | IO Wait | ![Development](https://img.shields.io/badge/-development-blue) | -| `kernel` | Kernel | ![Development](https://img.shields.io/badge/-development-blue) | | `nice` | Nice | ![Development](https://img.shields.io/badge/-development-blue) | | `steal` | Steal | ![Development](https://img.shields.io/badge/-development-blue) | | `system` | System | ![Development](https://img.shields.io/badge/-development-blue) | @@ -239,7 +238,6 @@ This metric is [opt-in][MetricOptIn]. | `idle` | Idle | ![Development](https://img.shields.io/badge/-development-blue) | | `interrupt` | Interrupt | ![Development](https://img.shields.io/badge/-development-blue) | | `iowait` | IO Wait | ![Development](https://img.shields.io/badge/-development-blue) | -| `kernel` | Kernel | ![Development](https://img.shields.io/badge/-development-blue) | | `nice` | Nice | ![Development](https://img.shields.io/badge/-development-blue) | | `steal` | Steal | ![Development](https://img.shields.io/badge/-development-blue) | | `system` | System | ![Development](https://img.shields.io/badge/-development-blue) | diff --git a/model/container/metrics.yaml b/model/container/metrics.yaml index 56e948dd6c..a49d4197ac 100644 --- a/model/container/metrics.yaml +++ b/model/container/metrics.yaml @@ -23,9 +23,9 @@ groups: code_generation: metric_value_type: double stability: release_candidate - brief: "Total CPU time consumed." + brief: "CPU time consumed." note: > - Total CPU time consumed by the specific container on all available CPU cores + CPU time consumed by the specific container on all available CPU cores instrument: counter unit: "s" entity_associations: @@ -33,9 +33,30 @@ groups: attributes: - ref: cpu.mode brief: "The CPU mode for this data point. A container's CPU metric SHOULD be characterized _either_ by data points with no `mode` labels, _or only_ data points with `mode` labels." - note: "Following states SHOULD be used: `user`, `system`, `kernel`" + note: > + Following modes SHOULD be used for containers if available: `user`, `system`. + + In implementations like [`opencontainers/cgroup`](https://pkg.go.dev/github.com/opencontainers/cgroups@v0.0.6#CpuUsage), + [Moby/Docker](https://pkg.go.dev/github.com/Moby/Moby@v28.5.2+incompatible/api/types/container#CPUUsage) + or [cAdvisor](https://pkg.go.dev/github.com/google/cadvisor@v0.56.2/info/v1#CpuUsage), the respective states + are retrieved from the cgroup stats directly. + In cgroup v1 the user/system modes come from the + [cpuacct.stat](https://www.kernel.org/doc/html/latest/admin-guide/cgroup-v1/cpuacct.html#cpu-accounting-controller) + file and the `user`/`system` stats respectively. + In cgroup v2 the user/system modes come from the + [cpu.stat](https://docs.kernel.org/admin-guide/cgroup-v2.html#cpu-interface-files) file + and the `user_usec`/`system_usec` stats respectively. + + Kubernetes only exposes the total CPU time + ([cpu.stat.usage_usec](https://docs.kernel.org/admin-guide/cgroup-v2.html#cpu-interface-files)) + through Kubelet's stats API, hence the metric should be + reported without the mode attribute implying it is the total of all modes. + + In pure container environments the CPU's mode is usually available and cpu.mode should be reported. + In that case summarising the CPU time over the different modes gives the total CPU time. requirement_level: - conditionally_required: Required if mode is available, i.e. metrics coming from the Docker Stats API. + conditionally_required: > + Required if mode is available, i.e. metrics coming from the Docker Stats API, containerd stats or cAdvisor. - id: metric.container.cpu.usage type: metric @@ -54,9 +75,11 @@ groups: attributes: - ref: cpu.mode brief: "The CPU mode for this data point. A container's CPU metric SHOULD be characterized _either_ by data points with no `mode` labels, _or only_ data points with `mode` labels." - note: "Following states SHOULD be used: `user`, `system`, `kernel`" + note: > + Following modes SHOULD be used for containers if available: `user`, `system`. requirement_level: - conditionally_required: Required if mode is available, i.e. metrics coming from the Docker Stats API. + conditionally_required: > + Required if mode is available, i.e. metrics coming from the Docker Stats API, containerd stats or cAdvisor. # container.memory.* metrics and attribute group - id: metric.container.memory.usage diff --git a/model/cpu/registry.yaml b/model/cpu/registry.yaml index 9d6e527b0e..d459cee6d1 100644 --- a/model/cpu/registry.yaml +++ b/model/cpu/registry.yaml @@ -39,8 +39,11 @@ groups: stability: development - id: kernel value: 'kernel' - brief: Kernel stability: development + brief: "Deprecated. Use `system` instead." + deprecated: + reason: renamed + renamed_to: "system" stability: development examples: [ "user", "system" ] - id: cpu.logical_number