Update experimental notices for 5.3 (#3525)

Configuration reloading -> beta[]
 Filebeat symlinks -> GA
 Filebeat harvester_limit -> GA
 Filebeat publish_async -> experimental & deprecated
 Using environment variables in the Configuration -> GA
 Metricbeat ceph -> beta[]
 Metricbeat couchbase -> beta[]
 Metricbeat docker -> beta[]
 Metricbeat haproxy -> GA
 Metricbeat kafka -> beta[]
 Metricbeat php_fpm -> beta[]
 Metricbeat prometheus -> beta[]

This also cleans up the CHANGELOG.

Author: tsg

CHANGELOG.asciidoc

@@ -16,123 +16,172 @@ https://github.com/elastic/beats/compare/v5.1.1...master[Check the HEAD diff]
 
 - Change beat generator. Use `$GOPATH/src/github.com/elastic/beats/script/generate.py` to generate a beat. {pull}3452[3452]
 
+*Filebeat*
+
+*Heartbeat*
+
 *Metricbeat*
 - Linux cgroup metrics are now enabled by default for the system process
   metricset. The configuration option for the feature was renamed from
   `cgroups` to `process.cgroups.enabled`. {pull}3519[3519]
 
 *Packetbeat*
 
-*Topbeat*
-
-*Filebeat*
-
 *Winlogbeat*
 
 
 ==== Bugfixes
 
 *Affecting all Beats*
 
-- Add `_id`, `_type`, `_index` and `_score` fields in the generated index pattern. {pull}3282}[3282]
+- Add `_id`, `_type`, `_index` and `_score` fields in the generated index pattern. {pull}3282[3282]
+
+*Filebeat*
+
+*Heartbeat*
 
 *Metricbeat*
 
-- Fix service times-out at startup. {pull}3056[3056]
-- Kafka module case sensitive host name matching. {pull}3193[3193]
-- Fix interface conversion panic in couchbase module {pull}3272[3272]
-- Fix overwriting explicit empty config sections {issue}2918[2918]
 - Fix go routine leak in docker module. {pull}3492[3492]
 
 *Packetbeat*
 
-- Fix issue where some Cassandra visualizations were showing data from all protocols. {issue}3314[3314]
-
-*Topbeat*
-
-*Filebeat*
-- Fix registry cleanup issue when files falling under ignore_older after restart. {issue}2818[2818]
-- Fix registry migration issue from old states were files were only harvested after second restart. {pull}3322[3322]
-- Fix alignment issue were Filebeat compiled with Go 1.7.4 was crashing on 32 bits system. {issue}3273[3273]
 
 *Winlogbeat*
-- Fix for "The array bounds are invalid" error when reading large events. {issue}3076[3076]
+
 
 ==== Added
 
 *Affecting all Beats*
-- Add support for passing list and dictionary settings via -E flag.
-- Support for parsing list and dictionary setting from environment variables.
-- Added new flags to import_dashboards (-cacert, -cert, -key, -insecure). {pull}3139[3139] {pull}3163[3163]
-- The limit for the number of fields is increased via the mapping template. {pull}3275[3275]
-- Updated to Go 1.7.4. {pull}3277[3277]
-- Added a NOTICE file containing the notices and licenses of the dependencies. {pull}3334[3334].
+
 - Files created by Beats (logs, registry, file output) will have 0600 permissions. {pull}3387[3387].
 - RPM/deb packages will now install the config file with 0600 permissions. {pull}3382[3382]
 - Add the option to pass custom HTTP headers to the Elasticsearch output. {pull}3400[3400]
 - Unify `regexp` and `contains` conditionals, for both to support array of strings and convert numbers to strings if required. {pull}3469[3469]
 - Add the option to load the sample dashboards during the Beat startup phase. {pull}3506[3506]
 - Disabled date detection in Elasticsearch index templates. Date fields must be explicitly defined in index templates. {pull}3528[3528]
+- Using environment variables in the configuration file is now GA, instead of experimental. {pull}3525[3525]
+
+*Filebeat*
+
+- Add the `pipeline` config option at the prospector level, for configuring the Ingest Node pipeline ID. {pull}3433[3433]
+- Update regular expressions used for matching file names or lines (multiline, include/exclude functionality) to new matchers improving performance of simple string matches. {pull}3469[3469]
+- The `symlinks` and `harverster_limit` settings are now GA, instead of experimental. {pull}3525[3525]
+
+*Heartbeat*
 
 *Metricbeat*
 
-- Add experimental filebeat metricset in the beats module. {pull}2297[2297]
-- Add experimental libbeat metricset in the beats module. {pull}2339[2339]
-- Add experimental docker module. Provided by Ingensi and @douaejeouit based on dockbeat.
-- Add username and password config options to the MongoDB module. {pull}2889[2889]
-- Add username and password config options to the PostgreSQL module. {pull}2889[2890]
-- Add system core metricset for Windows. {pull}2883[2883]
-- Add a sample Redis Kibana dashboard. {pull}2916[2916]
-- Add support for MongoDB 3.4 and WiredTiger metrics. {pull}2999[2999]
-- Add experimental kafka module with partition metricset. {pull}2969[2969]
-- Add raw config option for mysql/status metricset. {pull}3001[3001]
 - Add experimental dbstats metricset to MongoDB module. {pull}3228[3228]
 - Use persistent, direct connections to the configured nodes for MongoDB module. {pull}3228[3228]
-- Kafka module broker matching enhancements. {pull}3129[3129]
-- Add a couchbase module with metricsets for node, cluster and bucket. {pull}3081[3081]
-- Export number of cores for cpu module. {pull}3192[3192]
-- Experimental Prometheus module. {pull}3202[3202]
-- Add system socket module that reports all TCP sockets. {pull}3246[3246]
-- Kafka consumer groups metricset. {pull}3240[3240]
 - Add dynamic configuration reloading for modules. {pull}3281[3281]
 - Add docker health metricset {pull}3357[3357]
 - Add docker image metricset {pull}3467[3467]
 - System module uses new matchers for white-listing processes. {pull}3469[3469]
-- Add CEPH module with health metricset. {pull}3311[3311]
-- Add php_fpm module with pool metricset. {pull}3415[3415]
-- Add cluster_disk and pool_disk metricsets to CEPH module. {pull}3499[3499]
+- Add Beta CEPH module with health metricset. {pull}3311[3311]
+- Add Beta php_fpm module with pool metricset. {pull}3415[3415]
+- The Docker, Kafka, and Prometheus modules are now Beta, instead of experimental. {pull}3525[3525]
+- The HAProxy module is now GA, instead of experimental. {pull}3525[3525]
 
 *Packetbeat*
 
-*Topbeat*
+*Winlogbeat*
+
+==== Deprecated
+
+*Affecting all Beats*
+
+- Usage of field _type is deprecated. It should not be used in queries or dashboards. {pull}3409[3409]
 
 *Filebeat*
-- Add enabled config option to prospectors. {pull}3157[3157]
-- Add target option for decoded_json_field. {pull}3169[3169]
-- Add the `pipeline` config option at the prospector level, for configuring the Ingest Node pipeline ID. {pull}3433[3433]
-- Update regular expressions used for matching file names or lines (multiline, include/exclude functionality) to new matchers improving performance of simple string matches. {pull}3469[3469]
+
+- The experimental `publish_async` option is now deprecated and is planned to be removed in 6.0. {pull}3525[3525]
+
+*Heartbeat*
+
+*Metricbeat*
+
+*Packetbeat*
 
 *Winlogbeat*
 
-- Reduced amount of memory allocated while reading event log records. {pull}3113[3113] {pull}3118[3113]
+////////////////////////////////////////////////////////////
 
-==== Deprecated
+[[release-notes-5.2.0]]
+=== Beats version 5.2.0
+https://github.com/elastic/beats/compare/v5.1.2...v5.2.0[View commits]
+
+==== Bugfixes
 
 *Affecting all Beats*
-- Usage of field _type is deprecated. It should not be used in queries or dashboards. {pull}3409[3409]
+
+- Fix overwriting explicit empty config sections. {issue}2918[2918]
+
+*Filebeat*
+
+- Fix alignment issue were Filebeat compiled with Go 1.7.4 was crashing on 32 bits system. {issue}3273[3273]
 
 *Metricbeat*
 
+- Fix service times-out at startup. {pull}3056[3056]
+- Kafka module case sensitive host name matching. {pull}3193[3193]
+- Fix interface conversion panic in couchbase module {pull}3272[3272]
+
 *Packetbeat*
 
-*Topbeat*
-- Fix error on importing dashboards due to colons in the Caassandra dashboard. {issue}3140[3140]
+- Fix issue where some Cassandra visualizations were showing data from all protocols. {issue}3314[3314]
+
+==== Added
+
+*Affecting all Beats*
+
+- Add support for passing list and dictionary settings via -E flag.
+- Support for parsing list and dictionary setting from environment variables.
+- Added new flags to import_dashboards (-cacert, -cert, -key, -insecure). {pull}3139[3139] {pull}3163[3163]
+- The limit for the number of fields is increased via the mapping template. {pull}3275[3275]
+- Updated to Go 1.7.4. {pull}3277[3277]
+- Added a NOTICE file containing the notices and licenses of the dependencies. {pull}3334[3334].
+
+*Heartbeat*
+
+- First release, containing monitors for ICMP, TCP, and HTTP.
 
 *Filebeat*
 
+- Add enabled config option to prospectors. {pull}3157[3157]
+- Add target option for decoded_json_field. {pull}3169[3169]
+
+*Metricbeat*
+
+- Kafka module broker matching enhancements. {pull}3129[3129]
+- Add a couchbase module with metricsets for node, cluster and bucket. {pull}3081[3081]
+- Export number of cores for CPU module. {pull}3192[3192]
+- Experimental Prometheus module. {pull}3202[3202]
+- Add system socket module that reports all TCP sockets. {pull}3246[3246]
+- Kafka consumer groups metricset. {pull}3240[3240]
+
 *Winlogbeat*
 
-////////////////////////////////////////////////////////////
+- Reduced amount of memory allocated while reading event log records. {pull}3113[3113] {pull}3118[3118]
+
+[[release-notes-5.1.2]]
+=== Beats version 5.1.2
+https://github.com/elastic/beats/compare/v5.1.1...v5.1.2[View commits]
+
+==== Bugfixes
+
+*Filebeat*
+
+- Fix registry migration issue from old states where files were only harvested after second restart. {pull}3322[3322]
+
+*Packetbeat*
+
+- Fix error on importing dashboards due to colons in the Cassandra dashboard. {issue}3140[3140]
+- Fix error on importing dashboards due to the wrong type for the geo_point fields. {pull}3147[3147]
+
+*Winlogbeat*
+
+- Fix for "The array bounds are invalid" error when reading large events. {issue}3076[3076]
 
 [[release-notes-5.1.1]]
 === Beats version 5.1.1

filebeat/_meta/common.full.p2.yml

@@ -138,7 +138,7 @@ filebeat.prospectors:
   # overwrites the pipeline option from the Elasticsearch output.
   #pipeline:
 
-  # Experimental: If symlinks is enabled, symlinks are opened and harvested. The harvester is openening the
+  # If symlinks is enabled, symlinks are opened and harvested. The harvester is openening the
   # original for harvesting but will report the symlink name as source.
   #symlinks: false
 
@@ -160,7 +160,7 @@ filebeat.prospectors:
   # The backoff value will be multiplied each time with the backoff_factor until max_backoff is reached
   #backoff_factor: 2
 
-  # Experimental: Max number of harvesters that are started in parallel.
+  # Max number of harvesters that are started in parallel.
   # Default is 0 which means unlimited
   #harvester_limit: 0
 

filebeat/crawler/crawler.go

@@ -44,7 +44,7 @@ func (c *Crawler) Start(r *registrar.Registrar, reloaderConfig *common.Config) e
 	}
 
 	if reloaderConfig.Enabled() {
-		logp.Warn("EXPERIMENTAL feature dynamic configuration reloading is enabled.")
+		logp.Warn("BETA feature dynamic configuration reloading is enabled.")
 
 		c.reloader = cfgfile.NewReloader(reloaderConfig)
 		factory := prospector.NewFactory(c.out, r)

filebeat/docs/reference/configuration/filebeat-options.asciidoc

@@ -415,8 +415,6 @@ NOTE: The pipeline ID can also be configured in the Elasticsearch output, but th
 
 ===== symlinks
 
-experimental[]
-
 The `symlinks` option allows Filebeat to harvest symlinks in addition to regular files. When harvesting symlinks, Filebeat opens and reads the original file even though it reports the path of the symlink.
 
 When you configure a symlink for harvesting, make sure the original path is excluded. If a single prospector is configured to harvest both the symlink and the original file, the prospector will detect the problem and only process the first file it finds. However, if two different prospectors are configured (one to read the symlink and the other the original path), both paths will be harvested, causing Filebeat to send duplicate data and the prospectors to overwrite each other's state.
@@ -457,8 +455,6 @@ lines. The `backoff` value will be multiplied each time with the `backoff_factor
 [[harvester-limit]]
 ===== harvester_limit
 
-experimental[]
-
 The `harvester_limit` option limits the number of harvesters that are started in parallel for one prospector. This directly relates
 to the maximum number of file handlers that are opened. The default for `harvester_limit` is 0, which means there is no limit. This
 configuration is useful if the number of files to be harvested exceeds the open file handler limit of the operating system.
@@ -498,6 +494,7 @@ See <<load-balancing>> for more information about how this setting affects load
 ===== publish_async
 
 experimental[]
+deprecated[5.3.0]
 
 If enabled, the publisher pipeline in Filebeat operates in async mode preparing
 a new batch of lines while waiting for ACK. This option can improve load-balancing

filebeat/docs/reference/configuration/reload-configuration.asciidoc

@@ -1,7 +1,7 @@
 [[filebeat-configuration-reloading]]
 === Reload Configuration
 
-experimental[]
+beta[]
 
 Reload configuration allows to dynamically reload prospector configuration files. A glob can be defined which should be watched
  for prospector configuration changes. New prospectors will be started / stopped accordingly. This is especially useful in

filebeat/filebeat.full.yml

@@ -254,7 +254,7 @@ filebeat.prospectors:
   # overwrites the pipeline option from the Elasticsearch output.
   #pipeline:
 
-  # Experimental: If symlinks is enabled, symlinks are opened and harvested. The harvester is openening the
+  # If symlinks is enabled, symlinks are opened and harvested. The harvester is openening the
   # original for harvesting but will report the symlink name as source.
   #symlinks: false
 
@@ -276,7 +276,7 @@ filebeat.prospectors:
   # The backoff value will be multiplied each time with the backoff_factor until max_backoff is reached
   #backoff_factor: 2
 
-  # Experimental: Max number of harvesters that are started in parallel.
+  # Max number of harvesters that are started in parallel.
   # Default is 0 which means unlimited
   #harvester_limit: 0
 

filebeat/publisher/publisher.go

@@ -34,7 +34,7 @@ func New(
 	pub publisher.Publisher,
 ) LogPublisher {
 	if async {
-		logp.Warn("Using publish_async is experimental!")
+		logp.Warn("publish_async is experimental and will be removed in a future version!")
 		return newAsyncLogPublisher(in, out, pub)
 	}
 	return newSyncLogPublisher(in, out, pub)

libbeat/docs/shared-env-vars.asciidoc

@@ -12,8 +12,6 @@
 [[using-environ-vars]]
 == Using Environment Variables in the Configuration
 
-experimental[]
-
 You can use environment variable references in the +{beatname_lc}.yml+ file to
 set values that need to be configurable during deployment. To do this, use: