Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file modified docs/images/monitoring-dashboard.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
2 changes: 1 addition & 1 deletion docs/index.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ include::introduction.asciidoc[]

include::setup.asciidoc[]

include::monitoring/monitoring-kibana.asciidoc[]
include::monitoring/configuring-monitoring.asciidoc[]

include::security/securing-kibana.asciidoc[]

Expand Down
2 changes: 1 addition & 1 deletion docs/monitoring/beats-details.asciidoc
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[role="xpack"]
[[beats-page]]
=== Beats Monitoring Metrics
== Beats Monitoring Metrics
++++
<titleabbrev>Beats Metrics</titleabbrev>
++++
Expand Down
2 changes: 1 addition & 1 deletion docs/monitoring/cluster-alerts.asciidoc
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[role="xpack"]
[[cluster-alerts]]
=== Cluster Alerts
== Cluster Alerts

The *Monitoring > Clusters* page in {kib} summarizes the status of your Elastic
stack. You can drill down into the metrics to view more information about your
Expand Down
26 changes: 26 additions & 0 deletions docs/monitoring/configuring-monitoring.asciidoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
[role="xpack"]
[[configuring-monitoring]]
== Configuring monitoring in {kib}
++++
<titleabbrev>Configuring monitoring</titleabbrev>
++++

If you enable the Elastic {monitor-features} in your cluster, you can
optionally collect metrics about {kib}:

* <<monitoring-kibana>>
* <<monitoring-metricbeat>>


You can also use {kib} to visualize
monitoring data from across the {stack}:

* <<monitoring-data>>

To learn about monitoring in general, see
{stack-ov}/xpack-monitoring.html[Monitoring the {stack}].

include::monitoring-kibana.asciidoc[]
include::monitoring-metricbeat.asciidoc[]
include::viewing-metrics.asciidoc[]
include::{kib-repo-dir}/settings/monitoring-settings.asciidoc[]
2 changes: 1 addition & 1 deletion docs/monitoring/elasticsearch-details.asciidoc
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[role="xpack"]
[[elasticsearch-metrics]]
=== {es} Monitoring Metrics
== {es} Monitoring Metrics
++++
<titleabbrev>{es} Metrics</titleabbrev>
++++
Expand Down
26 changes: 15 additions & 11 deletions docs/monitoring/index.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -5,24 +5,28 @@
[partintro]
--

{monitoring} in {kib} serves two separate purposes:
The {kib} {monitor-features} serve two separate purposes:

. To visualize monitoring data from across the Elastic Stack. You can view
health and performance data for {es}, Logstash, and Beats in real time, as well
as analyze past performance. For more information, see {xpack-ref}/xpack-monitoring.html[Monitoring the Elastic Stack].
. To visualize monitoring data from across the {stack}. You can view health and
performance data for {es}, {ls}, and Beats in real time, as well as analyze past
performance. For more information, see
{stack-ov}/xpack-monitoring.html[Monitoring the {stack}].

. To monitor {kib} itself and route that data to the monitoring cluster.

If you enable {monitoring} across the Elastic Stack, a monitoring agent runs on
each {es} node, Logstash node, {kib} instance, and Beat to collect and index
metrics. Each node and instance is considered unique based on its persistent
If you enable monitoring across the {stack}, each {es} node, {ls} node, {kib}
instance, and Beat is considered unique based on its persistent
UUID, which is written to the <<settings,`path.data`>> directory when the node
or instance starts.
or instance starts.

* <<monitoring-kibana>>
* <<monitoring-data>>
NOTE: Watcher must be enabled to view cluster alerts. If you have a Basic
license, Top Cluster Alerts are not displayed.

--

include::viewing-metrics.asciidoc[]
include::beats-details.asciidoc[]
include::cluster-alerts.asciidoc[]
include::elasticsearch-details.asciidoc[]
include::kibana-details.asciidoc[]
include::logstash-details.asciidoc[]
include::monitoring-troubleshooting.asciidoc[]
2 changes: 1 addition & 1 deletion docs/monitoring/kibana-details.asciidoc
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[role="xpack"]
[[kibana-page]]
=== {kib} Monitoring Metrics
== {kib} Monitoring Metrics
++++
<titleabbrev>{kib} Metrics</titleabbrev>
++++
Expand Down
2 changes: 1 addition & 1 deletion docs/monitoring/logstash-details.asciidoc
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[role="xpack"]
[[logstash-page]]
=== Logstash Monitoring Metrics
== Logstash Monitoring Metrics
++++
<titleabbrev>Logstash Metrics</titleabbrev>
++++
Expand Down
39 changes: 30 additions & 9 deletions docs/monitoring/monitoring-kibana.asciidoc
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
[role="xpack"]
[[monitoring-kibana]]
== Configuring monitoring in {kib}
=== Collecting {kib} monitoring data
++++
<titleabbrev>Configuring monitoring</titleabbrev>
<titleabbrev>Collecting monitoring data</titleabbrev>
++++

If you enable the collection of monitoring data in your cluster, you can
If you enable the Elastic {monitor-features} in your cluster, you can
optionally collect metrics about {kib}.

The following method involves sending the metrics to the production cluster,
Expand All @@ -15,10 +15,34 @@ method, see <<monitoring-metricbeat>>.
To learn about monitoring in general, see
{stack-ov}/xpack-monitoring.html[Monitoring the {stack}].

. Enable the collection of monitoring data in {stack}. Set
`xpack.monitoring.collection.enabled` to `true` on the production cluster. +
. Set the `xpack.monitoring.collection.enabled` setting to `true` on each
node in the production cluster. By default, it is is disabled (`false`).
+
--
NOTE: You can specify this setting in either the `elasticsearch.yml` on each
node or across the cluster as a dynamic cluster setting. If {es}
{security-features} are enabled, you must have `monitor` cluster privileges to
view the cluster settings and `manage` cluster privileges to change them.

--

** To update the cluster settings in {kib}:

... Open {kib} in your web browser.
+
--
By default, if you are running {kib} locally, go to `http://localhost:5601/`.

If {es} {security-features} are enabled, log in.
--

... In the side navigation, click *Monitoring*. If data collection is disabled,
you are prompted to turn it on.

** From the Console or command line, set `xpack.monitoring.collection.enabled`
to `true` on the production cluster. +
+
--
For example, you can use the following APIs to review and change this setting:

[source,js]
Expand Down Expand Up @@ -62,7 +86,7 @@ as `https://<your_production_cluster>:9200` in this setting.

--

. If {security} is enabled on the production cluster:
. If the Elastic {security-features} are enabled on the production cluster:

.. Verify that there is a
valid user ID and password in the `elasticsearch.username` and
Expand Down Expand Up @@ -95,6 +119,3 @@ For more information, see <<using-kibana-with-security>>.
. <<start-stop,Start {kib}>>.

. <<monitoring-data,View the monitoring data in {kib}>>.

include::monitoring-metricbeat.asciidoc[]
include::{kib-repo-dir}/settings/monitoring-settings.asciidoc[]
116 changes: 68 additions & 48 deletions docs/monitoring/monitoring-metricbeat.asciidoc
Original file line number Diff line number Diff line change
@@ -1,8 +1,13 @@
[role="xpack"]
[[monitoring-metricbeat]]
=== Monitoring {kib} with {metricbeat}
=== Collecting {kib} monitoring data with {metricbeat}
++++
<titleabbrev>Collecting monitoring data with {metricbeat}</titleabbrev>
++++

beta[] In 6.4 and later, you can use {metricbeat} to collect data about {kib}
beta[]

In 6.4 and later, you can use {metricbeat} to collect data about {kib}
and ship it to the monitoring cluster, rather than routing it through the
production cluster as described in <<monitoring-kibana>>.

Expand All @@ -11,9 +16,51 @@ image::monitoring/images/metricbeat.png[Example monitoring architecture]
To learn about monitoring in general, see
{stack-ov}/xpack-monitoring.html[Monitoring the {stack}].

. Enable the collection of monitoring data. Set
`xpack.monitoring.collection.enabled` to `true` on the production cluster. +
+
. Disable the default collection of {kib} monitoring metrics. +
+
--
Add the following setting in the {kib} configuration file (`kibana.yml`):

[source,yaml]
----------------------------------
xpack.monitoring.kibana.collection.enabled: false
----------------------------------

Leave the `xpack.monitoring.enabled` set to its default value (`true`).

For more information, see
<<monitoring-settings-kb,Monitoring settings in {kib}>>.
--

. <<start-stop,Start {kib}>>.

. Set the `xpack.monitoring.collection.enabled` setting to `true` on
each node in the production cluster. By default, it is disabled (`false`).
+
--
NOTE: You can specify this setting in either the `elasticsearch.yml` on each
node or across the cluster as a dynamic cluster setting. If {es}
{security-features} are enabled, you must have `monitor` cluster privileges to
view the cluster settings and `manage` cluster privileges to change them.

--

** In {kib}:

... Open {kib} in your web browser.
+
--
If you are running {kib} locally, go to `http://localhost:5601/`.

If the Elastic {security-features} are enabled, log in.
--

... In the side navigation, click *Monitoring*. If data collection is disabled,
you are prompted to turn it on.

** From the Console or command line, set `xpack.monitoring.collection.enabled`
to `true` on the production cluster. +
+
--
For example, you can use the following APIs to review and change this setting:

Expand All @@ -33,22 +80,6 @@ For more information, see {ref}/monitoring-settings.html[Monitoring settings in
and {ref}/cluster-update-settings.html[Cluster update settings].
--

. Disable the default collection of {kib} monitoring metrics. +
+
--
Add the following setting in the {kib} configuration file (`kibana.yml`):

[source,yaml]
----------------------------------
xpack.monitoring.kibana.collection.enabled: false
----------------------------------

Leave the `xpack.monitoring.enabled` set to its default value (`true`).

For more information, see
<<monitoring-settings-kb,Monitoring settings in {kib}>>.
--

. {metricbeat-ref}/metricbeat-installation.html[Install {metricbeat}] on the
same server as {kib}.

Expand Down Expand Up @@ -80,25 +111,21 @@ You must specify the following settings in the `modules.d/kibana.yml` file:
- stats
period: 10s
hosts: ["http://localhost:5601"] <1>
xpack.enabled: true
xpack.enabled: true <2>
----------------------------------
<1> This setting identifies the host and port number that are used to access {kib}.

<1> This setting identifies the host and port number that are used to access {kib}.
<2> This setting ensures that {kib} can read the monitoring data successfully.
That is to say, it's stored in the same location and format as monitoring data
that is sent by {ref}/es-monitoring-exporters.html[exporters].
--

. If {security} is enabled, you must also provide a user ID and password so that
{metricbeat} can collect metrics successfully.

.. Create or identify a user that you want to use to collect the metrics.
+
--
TIP: There is a `remote_monitoring_user` built-in user that grants the privileges
necessary for {metricbeat} to monitor {stack} products. See
{stack-ov}/built-in-users.html[Built-in users].
. If the Elastic {security-features} are enabled, you must also provide a user
ID and password so that {metricbeat} can collect metrics successfully.

Alternatively, you can choose a different user and give them the
... Create a user on the production cluster that has the
`remote_monitoring_collector` {stack-ov}/built-in-roles.html[built-in role].
--
Alternatively, use the `remote_monitoring_user`
{stack-ov}/built-in-users.html[built-in user].

.. Add the `username` and `password` settings to the {kib} module configuration
file.
Expand Down Expand Up @@ -144,19 +171,14 @@ For more information about these configuration options, see

--

. If {security} is enabled on the monitoring cluster, you must provide a valid
user ID and password so that {metricbeat} can send metrics successfully.

.. Create or identify a user that you want to use to send the metrics.
+
--
TIP: There is a `remote_monitoring_user` built-in user that grants the privileges
necessary for {metricbeat} to monitor {stack} products. See
{stack-ov}/built-in-users.html[Built-in users].
. If the {es} {security-features} are enabled on the monitoring cluster, you
must provide a valid user ID and password so that {metricbeat} can send metrics
successfully.

Alternatively, you can choose a different user and give them the
... Create a user on the monitoring cluster that has the
`remote_monitoring_agent` {stack-ov}/built-in-roles.html[built-in role].
--
Alternatively, use the `remote_monitoring_user`
{stack-ov}/built-in-users.html[built-in user].

.. Add the `username` and `password` settings to the {es} output information in
the {metricbeat} configuration file (`metricbeat.yml`):
Expand All @@ -176,8 +198,6 @@ output.elasticsearch:
HTTPS. For example, use a `hosts` setting like `https://es-mon-1:9200` in the
`metricbeat.yml` file.

. <<start-stop,Start {kib}>>.

. {metricbeat-ref}/metricbeat-starting.html[Start {metricbeat}].

. <<monitoring-data,View the monitoring data in {kib}>>.
Loading