[7.1][DOCS] Adds troubleshooting and production monitoring pages

docs/reference/monitoring/collecting-monitoring-data.asciidoc

@@ -18,9 +18,6 @@ Advanced monitoring settings enable you to control how frequently data is
 collected, configure timeouts, and set the retention period for locally-stored 
 monitoring indices. You can also adjust how monitoring data is displayed. 
 
-To learn about monitoring in general, see 
-{stack-ov}/xpack-monitoring.html[Monitoring the {stack}]. 
-
 . Configure your cluster to collect monitoring data:
 
 .. Verify that the `xpack.monitoring.enabled` setting is `true`, which is its 
@@ -108,7 +105,7 @@ cluster that stores the monitoring data must have at least one
 <<ingest,ingest node>>. 
 
 For more information about typical monitoring architectures, 
-see {stack-ov}/how-monitoring-works.html[How Monitoring Works].
+see <<monitoring-overview>>.
 --
 
 . If you choose to use an `http` exporter: 
@@ -197,9 +194,7 @@ xpack.monitoring.exporters:
 --
 
 . Configure your cluster to route monitoring data from sources such as {kib}, 
-Beats, and {ls} to the monitoring cluster. For information about configuring
-each product to collect and send monitoring data, see
-{stack-ov}/xpack-monitoring.html[Monitoring the {stack}].
+Beats, and {ls} to the monitoring cluster.
 
 . If you updated settings in the `elasticsearch.yml` files on your production 
 cluster, restart {es}. See <<stopping-elasticsearch>> and <<starting-elasticsearch>>. 

docs/reference/monitoring/collectors.asciidoc

@@ -140,4 +140,4 @@ related to monitoring data, which can be very useful when there are a large
 number of Logstash nodes or Beats.
 
 For more information about typical monitoring architectures, see
-{xpack-ref}/how-monitoring-works.html[How Monitoring Works].
+<<monitoring-overview>>.

docs/reference/monitoring/index.asciidoc

@@ -10,15 +10,18 @@ performance of your {es} cluster.
 
 * <<monitoring-overview>>
 * <<how-monitoring-works>>
+* <<monitoring-production>>
 * <<collecting-monitoring-data>>
 * <<configuring-metricbeat>>
 * <<config-monitoring-indices>>
 * <<es-monitoring-collectors>>
 * <<es-monitoring-exporters>>
+* <<monitoring-troubleshooting>>
 --
 
 include::overview.asciidoc[]
 include::how-monitoring-works.asciidoc[]
+include::production.asciidoc[]
 include::collecting-monitoring-data.asciidoc[]
 include::pause-export.asciidoc[]
 include::configuring-metricbeat.asciidoc[]
@@ -27,4 +30,5 @@ include::collectors.asciidoc[]
 include::exporters.asciidoc[]
 include::local-export.asciidoc[]
 include::http-export.asciidoc[]
+include::troubleshooting.asciidoc[]
 

docs/reference/monitoring/production.asciidoc

@@ -0,0 +1,126 @@
+[role="xpack"]
+[testenv="gold"]
+[[monitoring-production]]
+== Monitoring in a production environment
+
+In production, you should send monitoring data to a separate _monitoring cluster_
+so that historical data is available even when the nodes you are monitoring are
+not. For example, you can use {metricbeat} to ship monitoring data about {kib}
+and {es} to the monitoring cluster.
+//If you are sending your data to the ESMS, see <<esms>>.
+
+If you have an appropriate license, using a dedicated monitoring cluster also 
+enables you to monitor multiple clusters from a central location.
+
+To store monitoring data in a separate cluster:
+
+. Set up the {es} cluster you want to use as the monitoring cluster. 
+For example, you might set up a two host cluster with the nodes `es-mon-1` and 
+`es-mon-2`.
++
+--
+[IMPORTANT]
+===============================
+* To monitor an {es} 7.x cluster, you must run {es}
+7.x on the monitoring cluster.
+* There must be at least one <<ingest,ingest node>> in the monitoring
+cluster; it does not need to be a dedicated ingest node.
+===============================
+--
+
+.. (Optional) Verify that the collection of monitoring data is disabled on the 
+monitoring cluster. By default, the `xpack.monitoring.collection.enabled` setting 
+is `false`. 
++ 
+--
+For example, you can use the following APIs to review and change this setting:
+
+[source,js]
+----------------------------------
+GET _cluster/settings
+
+PUT _cluster/settings
+{
+  "persistent": {
+    "xpack.monitoring.collection.enabled": false
+  }
+}
+----------------------------------
+// CONSOLE
+--
+
+.. If the {es} {security-features} are enabled on the monitoring cluster, create 
+users that can send and retrieve monitoring data. 
++
+--
+NOTE: If you plan to use {kib} to view monitoring data, username and password 
+credentials must be valid on both the {kib} server and the monitoring cluster. 
+
+--
+
+*** If you plan to use {metricbeat} to collect data about {es} or {kib}, 
+create a user that has the `remote_monitoring_collector` built-in role and a 
+user that has the `remote_monitoring_agent` 
+{stack-ov}/built-in-roles.html#built-in-roles-remote-monitoring-agent[built-in role].
+Alternatively, use the `remote_monitoring_user`
+{stack-ov}/built-in-users.html[built-in user]. 
+
+*** If you plan to use HTTP exporters to route data through your production 
+cluster, create a user that has the `remote_monitoring_agent` 
+{stack-ov}/built-in-roles.html#built-in-roles-remote-monitoring-agent[built-in role]. 
++
+--
+For example, the 
+following request creates a `remote_monitor` user that has the 
+`remote_monitoring_agent` role:
+
+[source, sh]
+---------------------------------------------------------------
+POST /_security/user/remote_monitor
+{
+  "password" : "changeme",
+  "roles" : [ "remote_monitoring_agent"],
+  "full_name" : "Internal Agent For Remote Monitoring"
+}
+---------------------------------------------------------------
+// CONSOLE
+// TEST[skip:needs-gold+-license]
+
+Alternatively, use the `remote_monitoring_user` 
+{stack-ov}/built-in-users.html[built-in user]. 
+--
+
+. Configure your production cluster to collect data and send it to the 
+monitoring cluster. 
+
+** {ref}/configuring-metricbeat.html[Use {metricbeat}]. This option 
+is available in 6.5 and later versions. 
+
+** {ref}/configuring-monitoring.html[Use HTTP exporters].
+
+. (Optional)
+{logstash-ref}/configuring-logstash.html[Configure {ls} to collect data and send it to the monitoring cluster]. 
++
+--
+NOTE: You must configure HTTP exporters in the production cluster to route this 
+data to the monitoring cluster. It cannot be accomplished by using {metricbeat}. 
+
+--
+
+. (Optional) Configure {kib} to collect data and send it to the monitoring cluster:
+
+** {kibana-ref}/monitoring-metricbeat.html[Use {metricbeat}]. This 
+option is available in 6.4 and later versions. 
+
+** {kibana-ref}/monitoring-kibana.html[Use HTTP exporters].
+
+. (Optional) Create a dedicated {kib} instance for monitoring, rather than using 
+a single {kib} instance to access both your production cluster and monitoring 
+cluster.
+
+.. (Optional) Disable the collection of monitoring data in this {kib} instance. 
+Set the `xpack.monitoring.kibana.collection.enabled` setting to `false` in the 
+`kibana.yml` file. For more information about this setting, see 
+{kibana-ref}/monitoring-settings-kb.html[Monitoring settings in {kib}]. 
+
+. {kibana-ref}/monitoring-data.html[Configure {kib} to retrieve and display the monitoring data]. 

docs/reference/monitoring/troubleshooting.asciidoc

@@ -0,0 +1,28 @@
+[[monitoring-troubleshooting]]
+== Troubleshooting monitoring
+++++
+<titleabbrev>Troubleshooting</titleabbrev>
+++++
+
+Use the information in this section to troubleshoot common problems and find
+answers for frequently asked questions. See also
+{logstash-ref}/monitoring-troubleshooting.html[Troubleshooting monitoring in {ls}].
+
+For issues that you cannot fix yourself … we’re here to help.
+If you are an existing Elastic customer with a support contract, please create
+a ticket in the
+https://support.elastic.co/customers/s/login/[Elastic Support portal].
+Or post in the https://discuss.elastic.co/[Elastic forum].
+
+*Symptoms*:
+There is no information about your cluster on the *Monitoring* page in {kib}.
+
+*Resolution*:
+Check whether the appropriate indices exist on the monitoring cluster. For
+example, use the {ref}/cat-indices.html[cat indices] command to verify that
+there is a `.monitoring-kibana*` index for your {kib} monitoring data and a
+`.monitoring-es*` index for your {es} monitoring data. If you are collecting
+monitoring data by using {metricbeat} the indices have `-mb` in their names. If
+the indices do not exist, review your configuration. For example, see
+ <<monitoring-production>>.
+

docs/reference/settings/monitoring-settings.asciidoc

@@ -22,9 +22,6 @@ Logstash, configure
 {logstash-ref}/configuring-logstash.html#monitoring-settings[`xpack.monitoring` settings]
 in `logstash.yml`.
 
-For more information, see
-{xpack-ref}/xpack-monitoring.html[Monitoring the Elastic Stack].
-
 [float]
 [[general-monitoring-settings]]
 ==== General Monitoring Settings
@@ -117,9 +114,9 @@ the `http` exporter will not be deleted automatically.
 Configures where the agent stores monitoring data. By default, the agent uses a
 local exporter that indexes monitoring data on the cluster where it is installed.
 Use an HTTP exporter to send data to a separate monitoring cluster. For more
-information, see <<local-exporter-settings,Local Exporter Settings>>,
-<<http-exporter-settings,HTTP Exporter Settings>>, and
-{xpack-ref}/how-monitoring-works.html[How Monitoring Works].
+information, see <<local-exporter-settings,local exporter settings>>,
+<<http-exporter-settings,HTTP exporter settings>>, and
+<<how-monitoring-works>>.
 
 [float]
 [[local-exporter-settings]]