[Doc] added description of monitoring setting write_direct

docs/static/docker.asciidoc

@@ -153,7 +153,7 @@ Some example translations are shown here:
 **Environment Variable**:: **Logstash Setting**
 `PIPELINE_WORKERS`:: `pipeline.workers`
 `LOG_LEVEL`:: `log.level`
-`XPACK_MONITORING_ENABLED`:: `xpack.monitoring.enabled`
+`MONITORING_ENABLED`:: `monitoring.enabled`
 
 In general, any setting listed in the <<logstash-settings-file, settings
 documentation>> can be configured with this technique.
@@ -170,9 +170,9 @@ images:
 
 [horizontal]
 `http.host`:: `0.0.0.0`
-`xpack.monitoring.elasticsearch.hosts`:: `http://elasticsearch:9200`
+`monitoring.elasticsearch.hosts`:: `http://elasticsearch:9200`
 
-NOTE: The setting `xpack.monitoring.elasticsearch.hosts` is not
+NOTE: The setting `monitoring.elasticsearch.hosts` is not
 defined in the `-oss` image.
 
 These settings are defined in the default `logstash.yml`. They can be overridden

docs/static/monitoring/collectors.asciidoc

@@ -40,8 +40,8 @@ data collection. For more information about the configuration options for the
 collectors, see <<monitoring-settings>>.
 
 WARNING: Unlike {monitoring} for {es} and {kib}, there is no 
-`xpack.monitoring.collection.enabled` setting on Logstash. You must use the 
-`xpack.monitoring.enabled` setting to enable and disable data collection. 
+`pack.monitoring.collection.enabled` setting on Logstash. You must use the
+`monitoring.enabled` setting to enable and disable data collection.
 
 If gaps exist in the monitoring charts in {kib}, it is typically because either 
 a collector failed or the monitoring cluster did not receive the data (for 

docs/static/monitoring/monitoring-internal-legacy.asciidoc

@@ -0,0 +1,17 @@
+[role="xpack"]
+[[monitoring-internal-collection-legacy]]
+=== Collect {ls} monitoring data using internal legacy collectors
+++++
+<titleabbrev>Legacy internal collection (deprecated)</titleabbrev>
+++++
+
+deprecated[7.7.0]
+
+Starting in {ls} version 7.7, the legacy settings for internal collection are deprecated and
+will be removed in version 8.0.0. Instead of sending monitoring data to {es} exporter,
+it’s recommended that you use the <<monitoring-internal-collection,internal collection>>
+to route monitoring data directly to your monitoring indices.
+
+Or as an alternative to internal collection, use <<monitoring-with-metricbeat, {metricbeat} collection>>.
+
+

docs/static/monitoring/monitoring-internal.asciidoc

@@ -53,22 +53,22 @@ monitoring cluster will show the Logstash metrics under the _monitoring_ cluster
 
 --
 
-. Verify that the `xpack.monitoring.collection.enabled` setting is `true` on the
+. Verify that the `monitoring.collection.enabled` setting is `true` on the
 production cluster. If that setting is `false`, the collection of monitoring data
 is disabled in {es} and data is ignored from all other sources.
 
 . Configure your Logstash nodes to send metrics by setting the
-`xpack.monitoring.elasticsearch.hosts` in `logstash.yml`. If {security} is
+`monitoring.elasticsearch.hosts` in `logstash.yml`. If {security} is
 enabled, you also need to specify the credentials for the
 {ref}/built-in-users.html[built-in `logstash_system` user]. For more
 information about these settings, see <<monitoring-settings>>.
 +
 --
 [source,yaml]
 --------------------------------------------------
-xpack.monitoring.elasticsearch.hosts: ["http://es-prod-node-1:9200", "http://es-prod-node-2:9200"] <1>
-xpack.monitoring.elasticsearch.username: "logstash_system" <2>
-xpack.monitoring.elasticsearch.password: "changeme"
+monitoring.elasticsearch.hosts: ["http://es-prod-node-1:9200", "http://es-prod-node-2:9200"] <1>
+monitoring.elasticsearch.username: "logstash_system" <2>
+monitoring.elasticsearch.password: "changeme"
 --------------------------------------------------
 
 <1> If SSL/TLS is enabled on the production cluster, you must
@@ -91,33 +91,33 @@ can specify the location of the PEM encoded certificate with the
 
 [source,yaml]
 --------------------------------------------------
-xpack.monitoring.elasticsearch.ssl.certificate_authority: /path/to/ca.crt
+monitoring.elasticsearch.ssl.certificate_authority: /path/to/ca.crt
 --------------------------------------------------
 
 Alternatively, you can configure trusted certificates using a truststore
 (a Java Keystore file that contains the certificates):
 
 [source,yaml]
 --------------------------------------------------
-xpack.monitoring.elasticsearch.ssl.truststore.path: /path/to/file
-xpack.monitoring.elasticsearch.ssl.truststore.password: password
+monitoring.elasticsearch.ssl.truststore.path: /path/to/file
+monitoring.elasticsearch.ssl.truststore.password: password
 --------------------------------------------------
 
 Also, optionally, you can set up client certificate using a keystore
 (a Java Keystore file that contains the certificate):
 
 [source,yaml]
 --------------------------------------------------
-xpack.monitoring.elasticsearch.ssl.keystore.path: /path/to/file
-xpack.monitoring.elasticsearch.ssl.keystore.password: password
+monitoring.elasticsearch.ssl.keystore.path: /path/to/file
+monitoring.elasticsearch.ssl.keystore.password: password
 --------------------------------------------------
 
 Set sniffing to `true` to enable discovery of other nodes of the {es} cluster.
 It defaults to `false`.
 
 [source,yaml]
 --------------------------------------------------
-xpack.monitoring.elasticsearch.sniffing: false
+monitoring.elasticsearch.sniffing: false
 --------------------------------------------------
 
 --

docs/static/monitoring/monitoring-mb.asciidoc

@@ -29,7 +29,7 @@ commented out:
 
 [source,yaml]
 ----------------------------------
-xpack.monitoring.enabled: false
+monitoring.enabled: false
 ----------------------------------
 
 Remove the `#` at the beginning of the line to enable the setting.

docs/static/monitoring/monitoring-output.asciidoc

@@ -7,7 +7,7 @@ Like all Logstash pipelines, the purpose of the dedicated monitoring pipeline is
 to send events to outputs. In the case of {monitoring} for Logstash, the output 
 is always an `elasticsearch` output. However, unlike ordinary Logstash pipelines, 
 the output is configured within the `logstash.yml` settings file via the
-`xpack.monitoring.elasticsearch.*` settings.
+`monitoring.elasticsearch.*` settings.
 
 Other than its unique manner of configuration, this `elasticsearch` output
 behaves like all `elasticsearch` outputs, including its ability to pause data 
@@ -25,7 +25,7 @@ the following default configuration is used:
 
 [source,yaml]
 ---------------------------------------------------
-xpack.monitoring.elasticsearch.hosts: [ "http://localhost:9200" ]
+monitoring.elasticsearch.hosts: [ "http://localhost:9200" ]
 ---------------------------------------------------
 
 All data produced by {monitoring} for Logstash is indexed in the monitoring 
@@ -38,7 +38,7 @@ necessary to properly configure Logstash. For more information, see
 
 IMPORTANT: When discussing security relative to the `elasticsearch` output, it
 is critical to remember that all users are managed on the production cluster, 
-which is identified in the `xpack.monitoring.elasticsearch.hosts` setting.
+which is identified in the `monitoring.elasticsearch.hosts` setting.
 This is particularly important to remember when you move from development 
 environments to production environments, where you often have dedicated 
 monitoring clusters.

docs/static/monitoring/monitoring-overview.asciidoc

@@ -17,11 +17,13 @@ Make sure monitoring is enabled on your {es} cluster. Then configure *one* of
 these methods to collect {ls} metrics:
 
 * <<monitoring-with-metricbeat, {metricbeat} collection>>
-* <<monitoring-internal-collection,Internal collection (deprecated)>>
+* <<monitoring-internal-collection,Internal collection>>
+* <<monitoring-internal-collection-legacy,Legacy internal collection>>
 
 
 include::monitoring-mb.asciidoc[]
 include::monitoring-internal.asciidoc[]
+include::monitoring-internal-legacy.asciidoc[]
 include::monitoring-ui.asciidoc[]
 include::pipeline-viewer.asciidoc[]
 include::troubleshooting.asciidoc[]

docs/static/security/logstash.asciidoc

@@ -226,8 +226,8 @@ Then configure the user and password in the `logstash.yml` configuration file:
 
 [source,yaml]
 ----------------------------------------------------------
-xpack.monitoring.elasticsearch.username: logstash_system
-xpack.monitoring.elasticsearch.password: t0p.s3cr3t
+monitoring.elasticsearch.username: logstash_system
+monitoring.elasticsearch.password: t0p.s3cr3t
 ----------------------------------------------------------
 
 If you initially installed an older version of {xpack}, and then upgraded, the

docs/static/settings/monitoring-settings.asciidoc

@@ -5,7 +5,7 @@
 <titleabbrev>Monitoring Settings</titleabbrev>
 ++++
 
-You can set the following `xpack.monitoring` settings in `logstash.yml` to
+You can set the following `monitoring` settings in `logstash.yml` to
 control how monitoring data is collected from your Logstash nodes. However, the
 defaults work best in most circumstances. For more information about configuring
 Logstash, see <<logstash-settings-file>>.
@@ -14,11 +14,14 @@ Logstash, see <<logstash-settings-file>>.
 [[monitoring-general-settings]]
 ===== General monitoring settings
 
-`xpack.monitoring.enabled`::
+NOTE: In {ls} 7.7 the {xpack} is still able to use the deprecated `xpack.monitoring.*` settings, but if it
+find that the settings is mixed with the new ones, it exits with an error.
+
+`monitoring.enabled`::
 
 Monitoring is disabled by default. Set to `true` to enable {xpack} monitoring.
 
-`xpack.monitoring.elasticsearch.hosts`::
+`monitoring.elasticsearch.hosts`::
 
 The {es} instances that you want to ship your Logstash metrics to. This might be
 the same {es} instance specified in the `outputs` section in your Logstash
@@ -28,13 +31,13 @@ Logstash metrics must be routed through your production cluster. You can specify
 a single host as a string, or specify multiple hosts as an array. Defaults to
 `http://localhost:9200`.
 
-NOTE: If your Elasticsearch cluster is configured with dedicated master-eliglble
+NOTE: If your Elasticsearch cluster is configured with dedicated master-eligible
 nodes, Logstash metrics should _not_ be routed to these nodes, as doing so can
 create resource contention and impact the stability of the Elasticsearch
 cluster. Therefore, do not include such nodes in
-`xpack.monitoring.elasticsearch.hosts`.
+`monitoring.elasticsearch.hosts`.
 
-`xpack.monitoring.elasticsearch.username` and `xpack.monitoring.elasticsearch.password`::
+`monitoring.elasticsearch.username` and `monitoring.elasticsearch.password`::
 
 If your {es} is protected with basic authentication, these settings provide the
 username and password that the Logstash instance uses to authenticate for
@@ -44,12 +47,17 @@ shipping monitoring data.
 [[monitoring-collection-settings]]
 ===== Monitoring collection settings
 
-`xpack.monitoring.collection.interval`::
+`monitoring.collection.interval`::
 
 Controls how often data samples are collected and shipped on the Logstash side.
 Defaults to `10s`. If you modify the collection interval, set the 
-`xpack.monitoring.min_interval_seconds` option in `kibana.yml` to the same value.
+`monitoring.min_interval_seconds` option in `kibana.yml` to the same value.
+
+`monitoring.cluster_uuid`::
 
+By default {ls} identify the `cluster uuid` bringing it from the first `ouput-elasticsearch` plugin defined
+in the pipelines.
+If it's not found, then uses this `cluster_uuid` setting to tag the data shipped to the monitoring cluster.
 
 [[monitoring-ssl-settings]]
 ===== {monitoring} TLS/SSL settings
@@ -58,44 +66,44 @@ You can configure the following Transport Layer Security (TLS) or
 Secure Sockets Layer (SSL) settings. For more information, see 
 <<ls-monitoring-user>>.
 
-`xpack.monitoring.elasticsearch.ssl.certificate_authority`::
+`monitoring.elasticsearch.ssl.certificate_authority`::
 
 Optional setting that enables you to specify a path to the `.pem` file for the
 certificate authority for your {es} instance.
 
-`xpack.monitoring.elasticsearch.ssl.truststore.path`::
+`monitoring.elasticsearch.ssl.truststore.path`::
 
 Optional settings that provide the paths to the Java keystore (JKS) to validate
 the server’s certificate.
 
-`xpack.monitoring.elasticsearch.ssl.truststore.password`::
+`monitoring.elasticsearch.ssl.truststore.password`::
 
 Optional settings that provide the password to the truststore.
 
-`xpack.monitoring.elasticsearch.ssl.keystore.path`::
+`monitoring.elasticsearch.ssl.keystore.path`::
 
 Optional settings that provide the paths to the Java keystore (JKS) to validate
 the client’s certificate.
 
-`xpack.monitoring.elasticsearch.ssl.keystore.password`::
+`monitoring.elasticsearch.ssl.keystore.password`::
 
 Optional settings that provide the password to the keystore.
 
 [[monitoring-additional-settings]]
 ===== Additional settings
 
-`xpack.monitoring.elasticsearch.cloud_id`::
+`monitoring.elasticsearch.cloud_id`::
 
 If you're using {es} in {ecloud}, you should specify the identifier here.
-This setting is an alternative to `xpack.monitoring.elasticsearch.hosts`.
-If `cloud_id` is configured, `xpack.monitoring.elasticsearch.hosts` should not be used.
+This setting is an alternative to `monitoring.elasticsearch.hosts`.
+If `cloud_id` is configured, `monitoring.elasticsearch.hosts` should not be used.
 The {es} instances that you want to ship your Logstash metrics to. This might be
 the same {es} instance specified in the `outputs` section in your Logstash
 configuration, or a different one.
 
-`xpack.monitoring.elasticsearch.cloud_auth`::
+`monitoring.elasticsearch.cloud_auth`::
 
 If you're using {es} in {ecloud}, you can set your auth credentials here.
-This setting is an alternative to both `xpack.monitoring.elasticsearch.username`
-and `xpack.monitoring.elasticsearch.password`. If `cloud_auth` is configured,
+This setting is an alternative to both `monitoring.elasticsearch.username`
+and `monitoring.elasticsearch.password`. If `cloud_auth` is configured,
 those settings should not be used.