Skip to content

[Metricbeat] Add new page statistics to system/memory metricset#15492

Merged
fearful-symmetry merged 11 commits intoelastic:masterfrom
fearful-symmetry:add-new-vmstat-data
Jan 14, 2020
Merged

[Metricbeat] Add new page statistics to system/memory metricset#15492
fearful-symmetry merged 11 commits intoelastic:masterfrom
fearful-symmetry:add-new-vmstat-data

Conversation

@fearful-symmetry
Copy link
Copy Markdown
Contributor

@fearful-symmetry fearful-symmetry commented Jan 13, 2020

See #13744

This adds a new set of page scan and reclaim metrics to the system/memory metricset, as requested in the original issue. I'm a tad hesitant do dump the entire /proc/vmstat metrics in here, since this metricset is relatively clean and has a a fairly consistent format that I'd rather not dirty with a vmstat.* group.

These are fairly high-level metrics, so this adds a new page_stats group for them.

Metrics added:

  • pgscan metrics, measuring pages scanned
  • pgsteal metrics, measuring pages reclaimed
  • pgfree metric, measuring pages freed by the system
  • two efficiency metrics, measuring the efficiency of page reclaim.

With regards to efficiency, I was hesitant to to copy the vmeff metric from sar, as the pgscan and pgsteal metrics we get are much more detailed than what sar is collecting, so I instead copied the vmeff metric into groups, that measure the same thing (reclaim efficiency) at a more detailed level.

Test Plan

  • Start metricbeat with the system/memory metricset enabled on a linux system
  • Look for this field in the data:
           "page_stats": {
                "direct_efficiency": {
                    "pct": 0.9976
                },
                "kswapd_efficiency": {
                    "pct": 0.6213
                },
                "pgfree": {
                    "pages": 4382105954
                },
                "pgscan_direct": {
                    "pages": 485820
                },
                "pgscan_kswapd": {
                    "pages": 77390925
                },
                "pgsteal_direct": {
                    "pages": 484631
                },
                "pgsteal_kswapd": {
                    "pages": 48081976
                }
            },

Copy link
Copy Markdown
Contributor

@mtojek mtojek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, but I believe this level knowledge deserves a short paragraph in README.

Comment thread metricbeat/module/system/memory/_meta/fields.yml Outdated
@unixsurfer
Copy link
Copy Markdown

@fearful-symmetry Thanks a lot for your work! It looks great, I only had on comment about the description for the efficiency field.

@fearful-symmetry
Copy link
Copy Markdown
Contributor Author

Updated data.json and field descriptions.

@fearful-symmetry fearful-symmetry merged commit 0fd2250 into elastic:master Jan 14, 2020
fearful-symmetry added a commit to fearful-symmetry/beats that referenced this pull request Jan 14, 2020
…tic#15492)

* add new page statistics to system/memory

* update field descriptions

* make update

* update data.json

* update system tests

* update changelog

* fix conflicts

* try to fix system python tests

(cherry picked from commit 0fd2250)
fearful-symmetry added a commit that referenced this pull request Jan 14, 2020
…etricset (#15527)

* [Metricbeat] Add new page statistics to system/memory metricset (#15492)

* add new page statistics to system/memory

* update field descriptions

* make update

* update data.json

* update system tests

* update changelog

* fix conflicts

* try to fix system python tests

(cherry picked from commit 0fd2250)

* update fields

* fix readme
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants