Skip to content

[metricbeat]kubernetes: add persistentvolumeclaim from kube-state-metrics#15066

Merged
odacremolbap merged 11 commits intoelastic:masterfrom
odacremolbap:task/ksm-persistentvolumeclaims
Dec 18, 2019
Merged

[metricbeat]kubernetes: add persistentvolumeclaim from kube-state-metrics#15066
odacremolbap merged 11 commits intoelastic:masterfrom
odacremolbap:task/ksm-persistentvolumeclaims

Conversation

@odacremolbap
Copy link
Copy Markdown
Contributor

kubernetes persistent volume claim metrics for kube-state-metrics.
refer to:

https://github.com/kubernetes/kube-state-metrics/blob/release-1.8/docs/persistentvolumeclaim-metrics.md

Note: master branch ksm document above adds a metric family that is not supported yet, and won't be until next ksm is released.

related to #14821

@odacremolbap odacremolbap requested a review from a team as a code owner December 11, 2019 18:09
@odacremolbap odacremolbap self-assigned this Dec 11, 2019
@odacremolbap odacremolbap added Metricbeat Metricbeat Team:Integrations Label for the Integrations team containers Related to containers use case in progress Pull request is currently in progress. labels Dec 11, 2019
@odacremolbap odacremolbap added review and removed in progress Pull request is currently in progress. labels Dec 11, 2019
- name: storage_class
type: keyword
description: Storage class for the PVC.
- name: labels.*
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do you know if this labels are exactly Kubernetes labels? If that's the case we could map them under the common kubernetes.labels

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is.

I moved it to the module location.
Will need to do so with PV and Services (on a different PR)

Copy link
Copy Markdown
Contributor

@exekias exekias left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is looking good, just left a question on labels

func (m *persistentvolumeclaimMetricSet) Fetch(reporter mb.ReporterV2) {
events, err := m.prometheus.GetProcessedMetrics(m.mapping)
if err != nil {
m.Logger().Error(err)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can write the signature of the method like func (m *persistentvolumeclaimMetricSet) Fetch(reporter mb.ReporterV2) error and return err here directly. The under the hood implementation will do the m.Logger().Error(err) and the reporter.Error(err).

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks a lot @sayden
will use that func signature from now on.

Copy link
Copy Markdown
Contributor

@sayden sayden left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. If you want to change the thing on the comment is also fine :)

description: >
Information and statistics of pods managed by kubernetes.
fields:
- name: labels.*
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

to clarify, that fields.yml is included in any beat using the processor, including Metricbeat

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh, for some reason beyond my comprehension I tested without adding that field and it appeared as non mapped at kibana.

I'll re-check and remove if the field is already there.

thanks!

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done @exekias
please, re-review if you will

kube_persistentvolumeclaim_status_phase{namespace="default",persistentvolumeclaim="mysql-data",phase="Lost"} 0
kube_persistentvolumeclaim_status_phase{namespace="default",persistentvolumeclaim="mysql-data",phase="Pending"} 0
kube_persistentvolumeclaim_resource_requests_storage_bytes{namespace="default",persistentvolumeclaim="mysql-data"} 1.073741824e+09
kube_persistentvolumeclaim_labels{label_app="mysql-server",namespace="default",persistentvolumeclaim="mysql-data"} 1
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it's interesting how they included labels here when it's normally not part of the info we get. Do you know why?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(not sure if I follow)
that labels pattern seems to be common to all main kube-state-metrics objects. Not seeing it anywhere else, nor for annotations, but it looks like the label_keyname="value" is here to stay.

Copy link
Copy Markdown
Contributor

@exekias exekias left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, could you remove the labels field? It should not be needed and can cause issues

@odacremolbap odacremolbap requested a review from exekias December 16, 2019 18:20
@odacremolbap
Copy link
Copy Markdown
Contributor Author

jenkins test this

@odacremolbap
Copy link
Copy Markdown
Contributor Author

jenkins test this

failed tests look unrelated

@odacremolbap odacremolbap merged commit 9b48f2b into elastic:master Dec 18, 2019
@odacremolbap odacremolbap deleted the task/ksm-persistentvolumeclaims branch December 18, 2019 10:44
odacremolbap pushed a commit to odacremolbap/beats that referenced this pull request Dec 24, 2019
…rics (elastic#15066)

kubernetes.persistentvolumeclaim from kube-state-metrics

(cherry picked from commit 9b48f2b)
odacremolbap pushed a commit that referenced this pull request Dec 27, 2019
…rics (#15066) (#15253)

kubernetes.persistentvolumeclaim from kube-state-metrics

(cherry picked from commit 9b48f2b)
@odacremolbap
Copy link
Copy Markdown
Contributor Author

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

containers Related to containers use case Metricbeat Metricbeat review Team:Integrations Label for the Integrations team v7.6.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants