From 8d9ec6d41180fbd24ff38206ccb4dc2225845d32 Mon Sep 17 00:00:00 2001 From: lcawl Date: Fri, 26 Oct 2018 09:43:57 -0700 Subject: [PATCH 01/13] [DOCS] Adds metricbeat info to monitoring overview --- docs/reference/monitoring/index.asciidoc | 25 +++++++++++++----------- 1 file changed, 14 insertions(+), 11 deletions(-) diff --git a/docs/reference/monitoring/index.asciidoc b/docs/reference/monitoring/index.asciidoc index d6a55f44585dd..d53e2e3858104 100644 --- a/docs/reference/monitoring/index.asciidoc +++ b/docs/reference/monitoring/index.asciidoc @@ -17,26 +17,29 @@ All settings associated with {monitoring} in {es} must be set in either the cluster settings. For more information, see <>. [[es-monitoring-overview]] -{es} is also at the core of {monitoring} across the Elastic Stack. In all cases, +{es} is also at the core of {monitoring} across the {stack}. In all cases, {monitoring} documents are just ordinary JSON documents built by monitoring each -Elastic Stack component at some collection interval, then indexing those -documents into the monitoring cluster. Each component in the stack is -responsible for monitoring itself and then forwarding those documents to {es} -for both routing and indexing (storage). +{stack} component at some collection interval, then indexing those +documents into the monitoring cluster. +Each component in the stack is responsible for monitoring itself and then +forwarding those documents to {es} for both routing and indexing (storage). The routing and indexing processes in {es} are handled by what are called -<> and -<>. In the past, collectors and exporters -were considered to be part of a monitoring "agent", but that term is generally -not used anymore. +<> and <>. +In the past, collectors and exporters were considered to be part of a monitoring +"agent", but that term is generally not used anymore. + +beta[] Alternatively, in 6.4 and later, you can use {metricbeat} to collect and +ship data about {kib}, rather than routing it through {es}. In 6.5 and later, you +can also use {metricbeat} to collect and ship data about {es}. You can view monitoring data from {kib} where it’s easy to spot issues at a glance or delve into the system behavior over time to diagnose operational issues. In addition to the built-in status warnings, you can also set up custom alerts based on the data in the monitoring indices. -For an introduction to monitoring your Elastic stack, including Beats, Logstash, -and {kib}, see {xpack-ref}/xpack-monitoring.html[Monitoring the Elastic Stack]. +For an introduction to monitoring your {stack}, including Beats, {ls}, and {kib}, +see {stack-ov}/xpack-monitoring.html[Monitoring the {stack}]. -- From 1310fa5475899c521c62ab359b0b7a2c9ef7c77c Mon Sep 17 00:00:00 2001 From: lcawl Date: Tue, 30 Oct 2018 18:08:05 -0700 Subject: [PATCH 02/13] [DOCS] Removes duplicated information --- .../configuring-monitoring.asciidoc | 99 ++++++++++++------- 1 file changed, 62 insertions(+), 37 deletions(-) diff --git a/docs/reference/monitoring/configuring-monitoring.asciidoc b/docs/reference/monitoring/configuring-monitoring.asciidoc index 81a9cce4f12ec..89d8c3ee52527 100644 --- a/docs/reference/monitoring/configuring-monitoring.asciidoc +++ b/docs/reference/monitoring/configuring-monitoring.asciidoc @@ -20,7 +20,7 @@ monitoring indices. You can also adjust how monitoring data is displayed. To learn about monitoring in general, see {stack-ov}/xpack-monitoring.html[Monitoring the {stack}]. -. To collect monitoring data about your {es} cluster: +. Configure your cluster to collect monitoring data: .. Verify that the `xpack.monitoring.enabled`, `xpack.monitoring.collection.enabled`, and @@ -53,14 +53,6 @@ patterns. For example, to include all indices that start with `test` except the `xpack.monitoring.collection.interval` setting 10 seconds. See <>. -. Optional: Configure your cluster to route monitoring data from sources such -as {kib}, Beats, and Logstash to a monitoring cluster: - -.. Verify that `xpack.monitoring.collection.enabled` settings are `true` on each -node in the cluster. - -.. {stack-ov}/xpack-monitoring.html[Configure {monitoring} across the Elastic Stack]. - . Identify where to store monitoring data. + -- @@ -74,10 +66,26 @@ For more information about typical monitoring architectures, see {stack-ov}/how-monitoring-works.html[How Monitoring Works]. -- -. If {security} is enabled and you are using an `http` exporter to send data to - a dedicated monitoring cluster: +. If you choose to use an `http` exporter: + +.. Configure each {es} node in the cluster you are monitoring to send metrics +to your monitoring cluster by configuring an HTTP exporter in the +`xpack.monitoring.exporters` settings in `elasticsearch.yml`. For example: ++ +-- +[source,yaml] +-------------------------------------------------- +xpack.monitoring.exporters: + id1: + type: http + host: ["http://es-mon-1:9200", "http://es-mon2:9200"] +-------------------------------------------------- +-- + +.. If {security} is enabled on the monitoring cluster, you must provide +appropriate credentials when data is shipped to the monitoring cluster: -.. Create a user on the monitoring cluster that has the +... Create a user on the monitoring cluster that has the {stack-ov}/built-in-roles.html#built-in-roles-remote-monitoring-agent[`remote_monitoring_agent` built-in role]. For example, the following request creates a `remote_monitor` user that has the `remote_monitoring_agent` role: @@ -96,19 +104,33 @@ POST /_xpack/security/user/remote_monitor // TEST[skip:needs-gold+-license] -- -.. On each node in the cluster that is being monitored, configure the `http` -exporter to use the appropriate credentials when data is shipped to the -monitoring cluster. +... Add the user ID and password settings to the HTTP exporter settings in the +`elasticsearch.yml` file on each node. + + -- -If SSL/TLS is enabled on the monitoring cluster, you must use the HTTPS protocol -in the `host` setting. You must also include the CA certificate in each node's -trusted certificates in order to verify the identities of the nodes in the -monitoring cluster. +For example: -The following example specifies the location of the PEM encoded certificate with -the `certificate_authorities` setting: +[source,yaml] +-------------------------------------------------- +xpack.monitoring.exporters: + id1: + type: http + host: ["http://es-mon-1:9200", "http://es-mon2:9200"] + auth.username: remote_monitor + auth.password: changeme +-------------------------------------------------- +-- + +.. If you configured the monitoring cluster to use +<>, you must use the HTTPS protocol in +the `host` setting. You must also specify the trusted CA certificates that will +be used to verify the identity of the nodes in the monitoring cluster. +*** To add a CA certificate to an {es} node's trusted certificates, you can +specify the location of the PEM encoded certificate with the +`certificate_authorities` setting. For example: ++ +-- [source,yaml] -------------------------------------------------- xpack.monitoring.exporters: @@ -116,18 +138,17 @@ xpack.monitoring.exporters: type: http host: ["https://es-mon1:9200", "https://es-mon2:9200"] auth: - username: remote_monitor <1> + username: remote_monitor password: changeme ssl: certificate_authorities: [ "/path/to/ca.crt" ] - id2: - type: local -------------------------------------------------- -<1> The `username` and `password` parameters provide the user credentials. - -Alternatively, you can configure trusted certificates using a truststore -(a Java Keystore file that contains the certificates): +-- +*** Alternatively, you can configure trusted certificates using a truststore +(a Java Keystore file that contains the certificates). For example: ++ +-- [source,yaml] -------------------------------------------------- xpack.monitoring.exporters: @@ -140,20 +161,24 @@ xpack.monitoring.exporters: ssl: truststore.path: /path/to/file truststore.password: password - id2: - type: local -------------------------------------------------- -- -. If {security} is enabled and you want to visualize monitoring data in {kib}, -you must create users that have access to the {kib} indices and permission to -read from the monitoring indices. +. Optional: Configure your cluster to route monitoring data from sources such +as {kib}, Beats, and Logstash to the monitoring cluster. + +.. Verify that `xpack.monitoring.collection.enabled` settings are `true` on each +node in the cluster. + +.. {stack-ov}/xpack-monitoring.html[Configure {monitoring} across the Elastic Stack]. + +. Restart {es} on the nodes in your production cluster. + -- -You set up {monitoring} UI users on the cluster where the monitoring data is -stored, that is to say the monitoring cluster. To grant all of the necessary permissions, assign users the -`monitoring_user` and `kibana_user` roles. For more information, see -{stack-ov}/mapping-roles.html[Mapping users and groups to roles]. +TIP: You may want to temporarily {ref}/modules-cluster.html[disable shard +allocation] before you restart your nodes to avoid unnecessary shard +reallocation during the install process. + -- . Optional: From 357f6439d218c7fd0e2fc9869bd186f80ff84983 Mon Sep 17 00:00:00 2001 From: lcawl Date: Thu, 1 Nov 2018 09:36:35 -0700 Subject: [PATCH 03/13] [DOCS] Updates feature terminology --- .../configuring-metricbeat.asciidoc | 9 ++--- .../configuring-monitoring.asciidoc | 34 +++++++++---------- docs/reference/monitoring/index.asciidoc | 11 +++--- 3 files changed, 27 insertions(+), 27 deletions(-) diff --git a/docs/reference/monitoring/configuring-metricbeat.asciidoc b/docs/reference/monitoring/configuring-metricbeat.asciidoc index 3d4a37861f1ae..6484399db8aa9 100644 --- a/docs/reference/monitoring/configuring-metricbeat.asciidoc +++ b/docs/reference/monitoring/configuring-metricbeat.asciidoc @@ -99,8 +99,8 @@ You must specify the following settings in the `modules.d/elasticsearch.yml` fil <1> This setting identifies the host and port number that are used to access {es}. -- -.. If {security} is enabled, you must also provide a user ID and password so that -{metricbeat} can collect metrics successfully. +.. If Elastic {security-features} are enabled, you must also provide a user ID +and password so that {metricbeat} can collect metrics successfully. ... Create or identify a user that you want to use to collect the metrics. + @@ -156,8 +156,9 @@ For more information about these configuration options, see {metricbeat-ref}/elasticsearch-output.html[Configure the {es} output]. -- -.. If {security} is enabled on the monitoring cluster, you must provide a valid -user ID and password so that {metricbeat} can send metrics successfully. +.. If Elastic {security-features} are enabled on the monitoring cluster, you +must provide a valid user ID and password so that {metricbeat} can send metrics +successfully. ... Create or identify a user that you want to use to send the metrics. + diff --git a/docs/reference/monitoring/configuring-monitoring.asciidoc b/docs/reference/monitoring/configuring-monitoring.asciidoc index 89d8c3ee52527..f1fc23848d501 100644 --- a/docs/reference/monitoring/configuring-monitoring.asciidoc +++ b/docs/reference/monitoring/configuring-monitoring.asciidoc @@ -6,8 +6,8 @@ Configuring monitoring ++++ -If you enable the collection of monitoring data in your cluster, you can -optionally collect metrics about {es}. By default, {monitoring} is enabled but +If you enable the Elastic {monitoring-features} in your cluster, you can +optionally collect metrics about {es}. By default, monitoring is enabled but data collection is disabled. The following method involves sending the metrics to the monitoring cluster by @@ -56,21 +56,21 @@ the `xpack.monitoring.collection.interval` setting 10 seconds. See . Identify where to store monitoring data. + -- -By default, {monitoring} uses a `local` exporter that indexes monitoring data -on the same cluster. See <> and <>. +By default, `local` exporters index monitoring data on the same cluster. See <> and <>. -Alternatively, you can use an `http` exporter to send data to a separate -monitoring cluster. See <>. +Alternatively, you can use `http` exporters to send data to a separate +_monitoring cluster_. See <>. For more information about typical monitoring architectures, see {stack-ov}/how-monitoring-works.html[How Monitoring Works]. -- -. If you choose to use an `http` exporter: +. If you choose to use `http` exporters: -.. Configure each {es} node in the cluster you are monitoring to send metrics -to your monitoring cluster by configuring an HTTP exporter in the -`xpack.monitoring.exporters` settings in `elasticsearch.yml`. For example: +.. On the cluster that you want to monitor (often called the _production cluster_), +configure each node to send metrics to your monitoring cluster. Configure an +HTTP exporter in the `xpack.monitoring.exporters` settings in the +`elasticsearch.yml` file. For example: + -- [source,yaml] @@ -82,8 +82,8 @@ xpack.monitoring.exporters: -------------------------------------------------- -- -.. If {security} is enabled on the monitoring cluster, you must provide -appropriate credentials when data is shipped to the monitoring cluster: +.. If the Elastic {security-features} are enabled on the monitoring cluster, you +must provide appropriate credentials when data is shipped to the monitoring cluster: ... Create a user on the monitoring cluster that has the {stack-ov}/built-in-roles.html#built-in-roles-remote-monitoring-agent[`remote_monitoring_agent` built-in role]. @@ -165,12 +165,10 @@ xpack.monitoring.exporters: -- . Optional: Configure your cluster to route monitoring data from sources such -as {kib}, Beats, and Logstash to the monitoring cluster. - -.. Verify that `xpack.monitoring.collection.enabled` settings are `true` on each -node in the cluster. - -.. {stack-ov}/xpack-monitoring.html[Configure {monitoring} across the Elastic Stack]. +as {kib}, Beats, and Logstash to the monitoring cluster. The +`xpack.monitoring.collection.enabled` setting must be `true` on each node in the +cluster. For information about configuring each product to collect and send +monitoring data, see {stack-ov}/xpack-monitoring.html[Monitoring the {stack}]. . Restart {es} on the nodes in your production cluster. + diff --git a/docs/reference/monitoring/index.asciidoc b/docs/reference/monitoring/index.asciidoc index d53e2e3858104..ea8995f2aa420 100644 --- a/docs/reference/monitoring/index.asciidoc +++ b/docs/reference/monitoring/index.asciidoc @@ -5,20 +5,21 @@ [partintro] -- -{monitoring} enables you to easily monitor the health of your {es} cluster. The -monitoring metrics are collected from each node and stored in {es} indices. +The Elastic {monitoring-features} enable you to easily monitor the health of +your {es} cluster. The monitoring metrics are collected from each node and +stored in {es} indices. Each {es} node is considered unique based on its persistent UUID, which is written on first start to its <> directory, which defaults to `./data`. -All settings associated with {monitoring} in {es} must be set in either the +All settings associated with monitoring in {es} must be set in either the `elasticsearch.yml` file for each node or, where possible, in the dynamic cluster settings. For more information, see <>. [[es-monitoring-overview]] -{es} is also at the core of {monitoring} across the {stack}. In all cases, -{monitoring} documents are just ordinary JSON documents built by monitoring each +{es} is also at the core of monitoring across the {stack}. In all cases, +monitoring documents are just ordinary JSON documents built by monitoring each {stack} component at some collection interval, then indexing those documents into the monitoring cluster. From 851b1ae60ed105d7ca58f903b5268eb144368a3d Mon Sep 17 00:00:00 2001 From: lcawl Date: Thu, 1 Nov 2018 10:22:32 -0700 Subject: [PATCH 04/13] [DOCS] Adds links in monitoring pages --- .../monitoring/configuring-monitoring.asciidoc | 18 ++++++++++-------- docs/reference/monitoring/index.asciidoc | 2 +- 2 files changed, 11 insertions(+), 9 deletions(-) diff --git a/docs/reference/monitoring/configuring-monitoring.asciidoc b/docs/reference/monitoring/configuring-monitoring.asciidoc index f1fc23848d501..470d4f658f6e5 100644 --- a/docs/reference/monitoring/configuring-monitoring.asciidoc +++ b/docs/reference/monitoring/configuring-monitoring.asciidoc @@ -6,7 +6,7 @@ Configuring monitoring ++++ -If you enable the Elastic {monitoring-features} in your cluster, you can +If you enable the Elastic {monitor-features} in your cluster, you can optionally collect metrics about {es}. By default, monitoring is enabled but data collection is disabled. @@ -25,7 +25,7 @@ To learn about monitoring in general, see .. Verify that the `xpack.monitoring.enabled`, `xpack.monitoring.collection.enabled`, and `xpack.monitoring.elasticsearch.collection.enabled` settings are `true` on each -node in the cluster. By default `xpack.monitoring.collection.enabled` is disabled +node in the cluster. By default `xpack.monitoring.collection.enabled` is disabled (`false`), and that overrides `xpack.monitoring.elasticsearch.collection.enabled`, which defaults to being enabled (`true`). Both settings can be set dynamically at runtime. For more information, see <>. @@ -56,10 +56,11 @@ the `xpack.monitoring.collection.interval` setting 10 seconds. See . Identify where to store monitoring data. + -- -By default, `local` exporters index monitoring data on the same cluster. See <> and <>. +By default, the data is stored on the same cluster by using +<>. -Alternatively, you can use `http` exporters to send data to a separate -_monitoring cluster_. See <>. +Alternatively, you can use <> to send data to a +separate _monitoring cluster_. For more information about typical monitoring architectures, see {stack-ov}/how-monitoring-works.html[How Monitoring Works]. @@ -164,13 +165,14 @@ xpack.monitoring.exporters: -------------------------------------------------- -- -. Optional: Configure your cluster to route monitoring data from sources such -as {kib}, Beats, and Logstash to the monitoring cluster. The +. Configure your cluster to route monitoring data from sources such as {kib}, +Beats, and {ls} to the monitoring cluster. The `xpack.monitoring.collection.enabled` setting must be `true` on each node in the cluster. For information about configuring each product to collect and send monitoring data, see {stack-ov}/xpack-monitoring.html[Monitoring the {stack}]. -. Restart {es} on the nodes in your production cluster. +. If you updated settings in the `elasticsearch.yml` files on your production +cluster, restart {es}. See <> and <>. + -- TIP: You may want to temporarily {ref}/modules-cluster.html[disable shard diff --git a/docs/reference/monitoring/index.asciidoc b/docs/reference/monitoring/index.asciidoc index ea8995f2aa420..32385b7d9c71f 100644 --- a/docs/reference/monitoring/index.asciidoc +++ b/docs/reference/monitoring/index.asciidoc @@ -5,7 +5,7 @@ [partintro] -- -The Elastic {monitoring-features} enable you to easily monitor the health of +The Elastic {monitor-features} enable you to easily monitor the health of your {es} cluster. The monitoring metrics are collected from each node and stored in {es} indices. From 8a9fb8b268268dcdee69e60df5acf425919078ab Mon Sep 17 00:00:00 2001 From: lcawl Date: Fri, 2 Nov 2018 11:16:15 -0700 Subject: [PATCH 05/13] [DOCS] Addressed feedback about exporters --- .../configuring-monitoring.asciidoc | 10 ++++----- docs/reference/monitoring/index.asciidoc | 22 +++++++++++-------- 2 files changed, 18 insertions(+), 14 deletions(-) diff --git a/docs/reference/monitoring/configuring-monitoring.asciidoc b/docs/reference/monitoring/configuring-monitoring.asciidoc index 470d4f658f6e5..b2d2a2b48fbb1 100644 --- a/docs/reference/monitoring/configuring-monitoring.asciidoc +++ b/docs/reference/monitoring/configuring-monitoring.asciidoc @@ -56,17 +56,17 @@ the `xpack.monitoring.collection.interval` setting 10 seconds. See . Identify where to store monitoring data. + -- -By default, the data is stored on the same cluster by using -<>. +By default, the data is stored on the same cluster by using a +<>. -Alternatively, you can use <> to send data to a -separate _monitoring cluster_. +Alternatively, you can use an <> to send data to +a separate _monitoring cluster_. For more information about typical monitoring architectures, see {stack-ov}/how-monitoring-works.html[How Monitoring Works]. -- -. If you choose to use `http` exporters: +. If you choose to use an `http` exporter: .. On the cluster that you want to monitor (often called the _production cluster_), configure each node to send metrics to your monitoring cluster. Configure an diff --git a/docs/reference/monitoring/index.asciidoc b/docs/reference/monitoring/index.asciidoc index 32385b7d9c71f..13e7314f8af5f 100644 --- a/docs/reference/monitoring/index.asciidoc +++ b/docs/reference/monitoring/index.asciidoc @@ -7,7 +7,11 @@ -- The Elastic {monitor-features} enable you to easily monitor the health of your {es} cluster. The monitoring metrics are collected from each node and -stored in {es} indices. +stored in {es} indices. + +TIP: In production environments, it is recommended to store the monitoring data +in a separate _monitoring cluster_. See +{stack-ov}/monitoring-production.html[Monitoring in a production environment]. Each {es} node is considered unique based on its persistent UUID, which is written on first start to its <> directory, which @@ -24,14 +28,14 @@ monitoring documents are just ordinary JSON documents built by monitoring each documents into the monitoring cluster. Each component in the stack is responsible for monitoring itself and then -forwarding those documents to {es} for both routing and indexing (storage). -The routing and indexing processes in {es} are handled by what are called -<> and <>. -In the past, collectors and exporters were considered to be part of a monitoring -"agent", but that term is generally not used anymore. - -beta[] Alternatively, in 6.4 and later, you can use {metricbeat} to collect and -ship data about {kib}, rather than routing it through {es}. In 6.5 and later, you +forwarding those documents to the {es} production cluster for both routing and +indexing (storage). The routing and indexing processes in {es} are handled by +what are called <> and +<>. + +beta[] Alternatively, in 6.4 and later, you can use {metricbeat} to collect +monitoring data about {kib} and ship it directly to the monitoring cluster, +rather than routing it through the production cluster. In 6.5 and later, you can also use {metricbeat} to collect and ship data about {es}. You can view monitoring data from {kib} where it’s easy to spot issues at a From 6cad72c785d3c1d8e319d0bd58afc8625d8d3e2d Mon Sep 17 00:00:00 2001 From: lcawl Date: Fri, 2 Nov 2018 11:40:02 -0700 Subject: [PATCH 06/13] [DOCS] Adds examples for cluster-wide settings --- .../configuring-monitoring.asciidoc | 38 +++++++++++++++---- 1 file changed, 31 insertions(+), 7 deletions(-) diff --git a/docs/reference/monitoring/configuring-monitoring.asciidoc b/docs/reference/monitoring/configuring-monitoring.asciidoc index b2d2a2b48fbb1..1efb1869ddca0 100644 --- a/docs/reference/monitoring/configuring-monitoring.asciidoc +++ b/docs/reference/monitoring/configuring-monitoring.asciidoc @@ -22,13 +22,37 @@ To learn about monitoring in general, see . Configure your cluster to collect monitoring data: -.. Verify that the `xpack.monitoring.enabled`, -`xpack.monitoring.collection.enabled`, and -`xpack.monitoring.elasticsearch.collection.enabled` settings are `true` on each -node in the cluster. By default `xpack.monitoring.collection.enabled` is disabled -(`false`), and that overrides `xpack.monitoring.elasticsearch.collection.enabled`, -which defaults to being enabled (`true`). Both settings can be set dynamically -at runtime. For more information, see <>. +.. Verify that the `xpack.monitoring.enabled` setting is `true`, which is its +default value, on each node in the cluster. For more information, see +<>. + +.. Set the `xpack.monitoring.collection.enabled` and +`xpack.monitoring.elasticsearch.collection.enabled` settings to `true` on each +node in the cluster. ++ +-- +For example, you can use the following APIs to review and change these +cluster-wide settings: + +[source,js] +---------------------------------- +GET _cluster/settings + +PUT _cluster/settings +{ + "persistent": { + "xpack.monitoring.collection.enabled": true, + "xpack.monitoring.elasticsearch.collection.enabled": true + } +} +---------------------------------- +// CONSOLE + +By default `xpack.monitoring.collection.enabled` is disabled (`false`) and that +overrides `xpack.monitoring.elasticsearch.collection.enabled`, which by default +is enabled (`true`). For more information, see <> and +<>. +-- .. Optional: Specify which indices you want to monitor. + From 662bcc80565e97bdc4117fb294b9511bddd70fd4 Mon Sep 17 00:00:00 2001 From: lcawl Date: Mon, 5 Nov 2018 11:55:36 -0800 Subject: [PATCH 07/13] [DOCS] Adds description for xpack.enabled parameter --- docs/reference/monitoring/configuring-metricbeat.asciidoc | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/docs/reference/monitoring/configuring-metricbeat.asciidoc b/docs/reference/monitoring/configuring-metricbeat.asciidoc index 6484399db8aa9..6c297ebb6c36a 100644 --- a/docs/reference/monitoring/configuring-metricbeat.asciidoc +++ b/docs/reference/monitoring/configuring-metricbeat.asciidoc @@ -94,9 +94,12 @@ You must specify the following settings in the `modules.d/elasticsearch.yml` fil - shard period: 10s hosts: ["http://localhost:9200"] <1> - xpack.enabled: true + xpack.enabled: true <2> ---------------------------------- -<1> This setting identifies the host and port number that are used to access {es}. +<1> This setting identifies the host and port number that are used to access {es}. +<2> This setting ensures that {kib} can read this monitoring data successfully. +That is to say, it's stored in the same location and format as monitoring data +that is sent by <>. -- .. If Elastic {security-features} are enabled, you must also provide a user ID From 0c54ace54abeddc23fa7ade9d035771a746af926 Mon Sep 17 00:00:00 2001 From: lcawl Date: Wed, 7 Nov 2018 09:29:55 -0800 Subject: [PATCH 08/13] [DOCS] Clarifies dynamic cluster settings --- .../configuring-metricbeat.asciidoc | 13 ++++++-- .../configuring-monitoring.asciidoc | 30 ++++++++++++------- 2 files changed, 29 insertions(+), 14 deletions(-) diff --git a/docs/reference/monitoring/configuring-metricbeat.asciidoc b/docs/reference/monitoring/configuring-metricbeat.asciidoc index 6c297ebb6c36a..b4d38173b6531 100644 --- a/docs/reference/monitoring/configuring-metricbeat.asciidoc +++ b/docs/reference/monitoring/configuring-metricbeat.asciidoc @@ -13,9 +13,13 @@ To learn about monitoring in general, see {stack-ov}/xpack-monitoring.html[Monitoring the {stack}]. . Enable the collection of monitoring data. Set -`xpack.monitoring.collection.enabled` to `true` on the production cluster. + +`xpack.monitoring.collection.enabled` to `true` on each node in the production +cluster. By default, it is is disabled (`false`). + -- +NOTE: You can specify this setting in either the `elasticsearch.yml` on each +node or across the cluster as a dynamic cluster setting. + For example, you can use the following APIs to review and change this setting: [source,js] @@ -35,10 +39,13 @@ For more information, see <> and < -- . Disable the default collection of {es} monitoring metrics. Set -`xpack.monitoring.elasticsearch.collection.enabled` to `false` on the production -cluster. + +`xpack.monitoring.elasticsearch.collection.enabled` to `false` on each node in +the production cluster. + -- +NOTE: You can specify this setting in either the `elasticsearch.yml` on each +node or across the cluster as a dynamic cluster setting. + For example, you can use the following API to change this setting: [source,js] diff --git a/docs/reference/monitoring/configuring-monitoring.asciidoc b/docs/reference/monitoring/configuring-monitoring.asciidoc index 1efb1869ddca0..67a3e0d7d7243 100644 --- a/docs/reference/monitoring/configuring-monitoring.asciidoc +++ b/docs/reference/monitoring/configuring-monitoring.asciidoc @@ -26,13 +26,24 @@ To learn about monitoring in general, see default value, on each node in the cluster. For more information, see <>. -.. Set the `xpack.monitoring.collection.enabled` and -`xpack.monitoring.elasticsearch.collection.enabled` settings to `true` on each -node in the cluster. +.. Verify that the `xpack.monitoring.elasticsearch.collection.enabled` setting +is `true`, which is its default value, on each node in the cluster. ++ +-- +NOTE: You can specify this setting in either the `elasticsearch.yml` on each +node or across the cluster as a dynamic cluster setting. + +For more information, see <> and <>. +-- + +.. Set the `xpack.monitoring.collection.enabled` setting to `true` on each +node in the cluster. By default, it is is disabled (`false`). + -- -For example, you can use the following APIs to review and change these -cluster-wide settings: +NOTE: You can specify this setting in either the `elasticsearch.yml` on each +node or across the cluster as a dynamic cluster setting. + +For example, use the following APIs to review and change this setting: [source,js] ---------------------------------- @@ -41,17 +52,14 @@ GET _cluster/settings PUT _cluster/settings { "persistent": { - "xpack.monitoring.collection.enabled": true, - "xpack.monitoring.elasticsearch.collection.enabled": true + "xpack.monitoring.collection.enabled": true } } ---------------------------------- // CONSOLE -By default `xpack.monitoring.collection.enabled` is disabled (`false`) and that -overrides `xpack.monitoring.elasticsearch.collection.enabled`, which by default -is enabled (`true`). For more information, see <> and -<>. +For more +information, see <> and <>. -- .. Optional: Specify which indices you want to monitor. From b484fb469701cafd4d56f10aa4a6ef4f6c53b37a Mon Sep 17 00:00:00 2001 From: lcawl Date: Wed, 7 Nov 2018 15:32:39 -0800 Subject: [PATCH 09/13] [DOCS] Clarified security rsteps --- .../configuring-metricbeat.asciidoc | 31 ++++++------------- 1 file changed, 9 insertions(+), 22 deletions(-) diff --git a/docs/reference/monitoring/configuring-metricbeat.asciidoc b/docs/reference/monitoring/configuring-metricbeat.asciidoc index b4d38173b6531..275ca047199c9 100644 --- a/docs/reference/monitoring/configuring-metricbeat.asciidoc +++ b/docs/reference/monitoring/configuring-metricbeat.asciidoc @@ -112,22 +112,15 @@ that is sent by <>. .. If Elastic {security-features} are enabled, you must also provide a user ID and password so that {metricbeat} can collect metrics successfully. -... Create or identify a user that you want to use to collect the metrics. -+ --- -TIP: There is a `remote_monitoring_user` built-in user that grants the privileges -necessary for {metricbeat} to monitor {stack} products. See -{stack-ov}/built-in-users.html[Built-in users]. - -Alternatively, you can choose a different user and give them the -`remote_monitoring_collector` {stack-ov}/built-in-roles.html[built-in role]. --- +... Create a user that has the +{stack-ov}/built-in-roles.html[`remote_monitoring_collector` built-in role]. +Alternatively, use the {stack-ov}/built-in-users.html[`remote_monitoring_user` built-in user]. ... Add the `username` and `password` settings to the {es} module configuration file. + -- -For example, add the following settings in the `modules.d/kibana.yml` file: +For example, add the following settings in the `modules.d/elasticsearch.yml` file: [source,yaml] ---------------------------------- @@ -166,20 +159,14 @@ For more information about these configuration options, see {metricbeat-ref}/elasticsearch-output.html[Configure the {es} output]. -- -.. If Elastic {security-features} are enabled on the monitoring cluster, you +.. If {es} {security-features} are enabled on the monitoring cluster, you must provide a valid user ID and password so that {metricbeat} can send metrics successfully. -... Create or identify a user that you want to use to send the metrics. -+ --- -TIP: There is a `remote_monitoring_user` built-in user that grants the privileges -necessary for {metricbeat} to monitor {stack} products. See -{stack-ov}/built-in-users.html[Built-in users]. - -Alternatively, you can choose a different user and give them the -`remote_monitoring_agent` {stack-ov}/built-in-roles.html[built-in role]. --- +... Create a user that has the +{stack-ov}/built-in-roles.html[`remote_monitoring_agent` built-in role]. +Alternatively, use the +{stack-ov}/built-in-users.html[`remote_monitoring_user` built-in user]. ... Add the `username` and `password` settings to the {es} output information in the {metricbeat} configuration file (`metricbeat.yml`): From d2ba9c0f9a893056aa35ac1e998b305ea87cbfb0 Mon Sep 17 00:00:00 2001 From: lcawl Date: Wed, 7 Nov 2018 15:39:28 -0800 Subject: [PATCH 10/13] [DOCS] Adds remote_monitoring_user to original monitoring method --- .../configuring-monitoring.asciidoc | 32 ++++++------------- 1 file changed, 9 insertions(+), 23 deletions(-) diff --git a/docs/reference/monitoring/configuring-monitoring.asciidoc b/docs/reference/monitoring/configuring-monitoring.asciidoc index 67a3e0d7d7243..9aa68b338287f 100644 --- a/docs/reference/monitoring/configuring-monitoring.asciidoc +++ b/docs/reference/monitoring/configuring-monitoring.asciidoc @@ -119,23 +119,9 @@ xpack.monitoring.exporters: must provide appropriate credentials when data is shipped to the monitoring cluster: ... Create a user on the monitoring cluster that has the -{stack-ov}/built-in-roles.html#built-in-roles-remote-monitoring-agent[`remote_monitoring_agent` built-in role]. -For example, the following request creates a `remote_monitor` user that has the -`remote_monitoring_agent` role: -+ --- -[source, sh] ---------------------------------------------------------------- -POST /_xpack/security/user/remote_monitor -{ - "password" : "changeme", - "roles" : [ "remote_monitoring_agent"], - "full_name" : "Internal Agent For Remote Monitoring" -} ---------------------------------------------------------------- -// CONSOLE -// TEST[skip:needs-gold+-license] --- +{stack-ov}/built-in-roles.html[`remote_monitoring_agent` built-in role]. +Alternatively, use the +{stack-ov}/built-in-users.html[`remote_monitoring_user` built-in user]. ... Add the user ID and password settings to the HTTP exporter settings in the `elasticsearch.yml` file on each node. + @@ -149,8 +135,8 @@ xpack.monitoring.exporters: id1: type: http host: ["http://es-mon-1:9200", "http://es-mon2:9200"] - auth.username: remote_monitor - auth.password: changeme + auth.username: remote_monitoring_user + auth.password: YOUR_PASSWORD -------------------------------------------------- -- @@ -171,8 +157,8 @@ xpack.monitoring.exporters: type: http host: ["https://es-mon1:9200", "https://es-mon2:9200"] auth: - username: remote_monitor - password: changeme + username: remote_monitoring_user + password: YOUR_PASSWORD ssl: certificate_authorities: [ "/path/to/ca.crt" ] -------------------------------------------------- @@ -189,8 +175,8 @@ xpack.monitoring.exporters: type: http host: ["https://es-mon1:9200", "https://es-mon2:9200"] auth: - username: remote_monitor - password: changeme + username: remote_monitoring_user + password: YOUR_PASSWORD ssl: truststore.path: /path/to/file truststore.password: password From 86d1d1c01ab99002166e2eb3b1b6730a94a28952 Mon Sep 17 00:00:00 2001 From: lcawl Date: Wed, 7 Nov 2018 16:07:33 -0800 Subject: [PATCH 11/13] [DOCS] Clarified location of users --- docs/reference/monitoring/configuring-metricbeat.asciidoc | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/reference/monitoring/configuring-metricbeat.asciidoc b/docs/reference/monitoring/configuring-metricbeat.asciidoc index 275ca047199c9..eb34c03bbc085 100644 --- a/docs/reference/monitoring/configuring-metricbeat.asciidoc +++ b/docs/reference/monitoring/configuring-metricbeat.asciidoc @@ -112,7 +112,7 @@ that is sent by <>. .. If Elastic {security-features} are enabled, you must also provide a user ID and password so that {metricbeat} can collect metrics successfully. -... Create a user that has the +... Create a user on the production cluster that has the {stack-ov}/built-in-roles.html[`remote_monitoring_collector` built-in role]. Alternatively, use the {stack-ov}/built-in-users.html[`remote_monitoring_user` built-in user]. @@ -163,7 +163,7 @@ For more information about these configuration options, see must provide a valid user ID and password so that {metricbeat} can send metrics successfully. -... Create a user that has the +... Create a user on the monitoring cluster that has the {stack-ov}/built-in-roles.html[`remote_monitoring_agent` built-in role]. Alternatively, use the {stack-ov}/built-in-users.html[`remote_monitoring_user` built-in user]. From 5b474d26c60ff7bf48f97354aaa02661edc6693d Mon Sep 17 00:00:00 2001 From: lcawl Date: Wed, 7 Nov 2018 17:17:01 -0800 Subject: [PATCH 12/13] [DOCS] Adds info about required privileges --- docs/reference/monitoring/configuring-metricbeat.asciidoc | 3 +++ docs/reference/monitoring/configuring-monitoring.asciidoc | 3 +++ 2 files changed, 6 insertions(+) diff --git a/docs/reference/monitoring/configuring-metricbeat.asciidoc b/docs/reference/monitoring/configuring-metricbeat.asciidoc index eb34c03bbc085..317a79ce65e19 100644 --- a/docs/reference/monitoring/configuring-metricbeat.asciidoc +++ b/docs/reference/monitoring/configuring-metricbeat.asciidoc @@ -35,6 +35,9 @@ PUT _cluster/settings ---------------------------------- // CONSOLE +NOTE: If {es} {security-features} are enabled, you must have `monitor` cluster +privileges to view the settings and `manage` cluster privileges to change them. + For more information, see <> and <>. -- diff --git a/docs/reference/monitoring/configuring-monitoring.asciidoc b/docs/reference/monitoring/configuring-monitoring.asciidoc index 9aa68b338287f..da018196c6721 100644 --- a/docs/reference/monitoring/configuring-monitoring.asciidoc +++ b/docs/reference/monitoring/configuring-monitoring.asciidoc @@ -58,6 +58,9 @@ PUT _cluster/settings ---------------------------------- // CONSOLE +NOTE: If {es} {security-features} are enabled, you must have `monitor` cluster +privileges to view the settings and `manage` cluster privileges to change them. + For more information, see <> and <>. -- From 7d7a6c8101b05b7e84e624af0a3a7375fccdde1b Mon Sep 17 00:00:00 2001 From: lcawl Date: Wed, 7 Nov 2018 18:34:34 -0800 Subject: [PATCH 13/13] [DOCS] Fixes location of privilege information --- .../monitoring/configuring-metricbeat.asciidoc | 13 +++++++------ .../monitoring/configuring-monitoring.asciidoc | 11 ++++++----- 2 files changed, 13 insertions(+), 11 deletions(-) diff --git a/docs/reference/monitoring/configuring-metricbeat.asciidoc b/docs/reference/monitoring/configuring-metricbeat.asciidoc index 317a79ce65e19..ef9a0cb6d771e 100644 --- a/docs/reference/monitoring/configuring-metricbeat.asciidoc +++ b/docs/reference/monitoring/configuring-metricbeat.asciidoc @@ -18,7 +18,9 @@ cluster. By default, it is is disabled (`false`). + -- NOTE: You can specify this setting in either the `elasticsearch.yml` on each -node or across the cluster as a dynamic cluster setting. +node or across the cluster as a dynamic cluster setting. If {es} +{security-features} are enabled, you must have `monitor` cluster privileges to +view the cluster settings and `manage` cluster privileges to change them. For example, you can use the following APIs to review and change this setting: @@ -33,10 +35,7 @@ PUT _cluster/settings } } ---------------------------------- -// CONSOLE - -NOTE: If {es} {security-features} are enabled, you must have `monitor` cluster -privileges to view the settings and `manage` cluster privileges to change them. +// CONSOLE For more information, see <> and <>. -- @@ -47,7 +46,9 @@ the production cluster. + -- NOTE: You can specify this setting in either the `elasticsearch.yml` on each -node or across the cluster as a dynamic cluster setting. +node or across the cluster as a dynamic cluster setting. If {es} +{security-features} are enabled, you must have `monitor` cluster privileges to +view the cluster settings and `manage` cluster privileges to change them. For example, you can use the following API to change this setting: diff --git a/docs/reference/monitoring/configuring-monitoring.asciidoc b/docs/reference/monitoring/configuring-monitoring.asciidoc index da018196c6721..e7ed3a4539739 100644 --- a/docs/reference/monitoring/configuring-monitoring.asciidoc +++ b/docs/reference/monitoring/configuring-monitoring.asciidoc @@ -31,7 +31,9 @@ is `true`, which is its default value, on each node in the cluster. + -- NOTE: You can specify this setting in either the `elasticsearch.yml` on each -node or across the cluster as a dynamic cluster setting. +node or across the cluster as a dynamic cluster setting. If {es} +{security-features} are enabled, you must have `monitor` cluster privileges to +view the cluster settings and `manage` cluster privileges to change them. For more information, see <> and <>. -- @@ -41,7 +43,9 @@ node in the cluster. By default, it is is disabled (`false`). + -- NOTE: You can specify this setting in either the `elasticsearch.yml` on each -node or across the cluster as a dynamic cluster setting. +node or across the cluster as a dynamic cluster setting. If {es} +{security-features} are enabled, you must have `monitor` cluster privileges to +view the cluster settings and `manage` cluster privileges to change them. For example, use the following APIs to review and change this setting: @@ -58,9 +62,6 @@ PUT _cluster/settings ---------------------------------- // CONSOLE -NOTE: If {es} {security-features} are enabled, you must have `monitor` cluster -privileges to view the settings and `manage` cluster privileges to change them. - For more information, see <> and <>. --