Skip to content

[DOCS] Backports monitoring configuration reorg #42645

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
lcawl merged 1 commit into from
May 28, 2019
Merged

[DOCS] Backports monitoring configuration reorg #42645

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
106 changes: 48 additions & 58 deletions docs/reference/monitoring/configuring-metricbeat.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,8 @@ image::monitoring/images/metricbeat.png[Example monitoring architecture]
To learn about monitoring in general, see
{stack-ov}/xpack-monitoring.html[Monitoring the {stack}].

//NOTE: The tagged regions are re-used in the Stack Overview.

. Enable the collection of monitoring data. Set
`xpack.monitoring.collection.enabled` to `true` on each node in the production
cluster. By default, it is is disabled (`false`).
Expand Down Expand Up @@ -71,13 +73,13 @@ PUT _cluster/settings
Leave `xpack.monitoring.enabled` set to its default value (`true`).
--

. On each {es} node in the production cluster:

.. {metricbeat-ref}/metricbeat-installation.html[Install {metricbeat}].
. {metricbeat-ref}/metricbeat-installation.html[Install {metricbeat}] on each
{es} node in the production cluster.

.. Enable the {es} module in {metricbeat}. +
. Enable the {es} module in {metricbeat} on each {es} node. +
+
--
// tag::enable-es-module[]
For example, to enable the default configuration in the `modules.d` directory,
run the following command:

Expand All @@ -89,12 +91,15 @@ metricbeat modules enable elasticsearch
For more information, see
{metricbeat-ref}/configuration-metricbeat.html[Specify which modules to run] and
{metricbeat-ref}/metricbeat-module-elasticsearch.html[{es} module].

// end::enable-es-module[]
--

.. Configure the {es} module in {metricbeat}. +
. Configure the {es} module in {metricbeat}. +
+
--
You must specify the following settings in the `modules.d/elasticsearch.yml` file:
For example, specify the following settings in the `modules.d/elasticsearch.yml`
file:

[source,yaml]
----------------------------------
Expand All @@ -109,42 +114,32 @@ You must specify the following settings in the `modules.d/elasticsearch.yml` fil
- node_stats
- shard
period: 10s
hosts: ["http://localhost:9200"] <1>
xpack.enabled: true <2>
hosts: ["http://localhost:9200"]
#username: "user"
#password: "secret"
xpack.enabled: true
----------------------------------
<1> This setting identifies the host and port number that are used to access {es}.
<2> This setting ensures that {kib} can read this monitoring data successfully.
That is to say, it's stored in the same location and format as monitoring data
that is sent by <<es-monitoring-exporters,exporters>>.
--

.. If Elastic {security-features} are enabled, you must also provide a user ID
and password so that {metricbeat} can collect metrics successfully.
By default, the module collects {es} monitoring metrics from
`http://localhost:9200`. If that host and port number are not correct, you must
update the `hosts` setting. If you configured {es} to use encrypted
communications, you must access it via HTTPS. For example, use a `hosts` setting like `https://localhost:9200`.
// end::configure-es-module[]

... Create a user on the production cluster that has the
// tag::remote-monitoring-user[]
If Elastic {security-features} are enabled, you must also provide a user ID
and password so that {metricbeat} can collect metrics successfully:

.. Create a user on the production cluster that has the
{stack-ov}/built-in-roles.html[`remote_monitoring_collector` built-in role].
Alternatively, use the {stack-ov}/built-in-users.html[`remote_monitoring_user` built-in user].

... Add the `username` and `password` settings to the {es} module configuration
.. Add the `username` and `password` settings to the {es} module configuration
file.
+
--
For example, add the following settings in the `modules.d/elasticsearch.yml` file:

[source,yaml]
----------------------------------
- module: elasticsearch
...
username: remote_monitoring_user
password: YOUR_PASSWORD
----------------------------------
// end::remote-monitoring-user[]
--

.. If you configured {es} to use <<configuring-tls,encrypted communications>>,
you must access it via HTTPS. For example, use a `hosts` setting like
`https://localhost:9200` in the `modules.d/elasticsearch.yml` file.

.. Identify where to send the monitoring data. +
. Identify where to send the monitoring data. +
+
--
TIP: In production environments, we strongly recommend using a separate cluster
Expand All @@ -159,48 +154,43 @@ configuration file (`metricbeat.yml`):
[source,yaml]
----------------------------------
output.elasticsearch:
# Array of hosts to connect to.
hosts: ["http://es-mon-1:9200", "http://es-mon2:9200"] <1>

# Optional protocol and basic auth credentials.
#protocol: "https"
#username: "elastic"
#password: "changeme"
----------------------------------
<1> In this example, the data is stored on a monitoring cluster with nodes
`es-mon-1` and `es-mon-2`.
`es-mon-1` and `es-mon-2`.

If you configured the monitoring cluster to use encrypted communications, you
must access it via HTTPS. For example, use a `hosts` setting like
`https://es-mon-1:9200`.

IMPORTANT: The {es} {monitor-features} use ingest pipelines, therefore the
cluster that stores the monitoring data must have at least one
<<ingest,ingest node>>.

For more information about these configuration options, see
{metricbeat-ref}/elasticsearch-output.html[Configure the {es} output].
--

.. If {es} {security-features} are enabled on the monitoring cluster, you
If {es} {security-features} are enabled on the monitoring cluster, you
must provide a valid user ID and password so that {metricbeat} can send metrics
successfully.

... Create a user on the monitoring cluster that has the
.. Create a user on the monitoring cluster that has the
{stack-ov}/built-in-roles.html[`remote_monitoring_agent` built-in role].
Alternatively, use the
{stack-ov}/built-in-users.html[`remote_monitoring_user` built-in user].

... Add the `username` and `password` settings to the {es} output information in
the {metricbeat} configuration file (`metricbeat.yml`):
+
--
[source,yaml]
----------------------------------
output.elasticsearch:
...
username: remote_monitoring_user
password: YOUR_PASSWORD
----------------------------------
--
.. Add the `username` and `password` settings to the {es} output information in
the {metricbeat} configuration file.

.. If you configured the monitoring cluster to use
<<configuring-tls,encrypted communications>>, you must access it via
HTTPS. For example, use a `hosts` setting like `https://es-mon-1:9200` in the
`metricbeat.yml` file.
For more information about these configuration options, see
{metricbeat-ref}/elasticsearch-output.html[Configure the {es} output].
--

. <<starting-elasticsearch,Start {es}>>.
. <<starting-elasticsearch,Start {es}>> on each node.

. {metricbeat-ref}/metricbeat-starting.html[Start {metricbeat}].
. {metricbeat-ref}/metricbeat-starting.html[Start {metricbeat}] on each node.

. {kibana-ref}/monitoring-data.html[View the monitoring data in {kib}].