Implement xpack.monitoring.elasticsearch.collection.enabled setting

[ycombinator]

Backport of #33474.

This PR implements an optional setting named xpack.monitoring.elasticsearch.collection.enabled, defaulting to true.

This setting will determine whether X-Pack Monitoring (when active overall) should collect Elasticsearch metrics or not.

[elasticmachine]

Pinging @elastic/es-core-infra

[tlrx]

Sorry, I missed the master pull request.

There's a subtle thing that I am missing, because in the current state xpack.monitoring.elasticsearch.collection.enabled seems to be a duplicate of xpack.monitoring.collection.enabled to me.

Today xpack.monitoring.collection.enabled controls the collection of the local Elasticsearch cluster and has no impact on how Beats/LS/KB monitoring data are exported. Is the plan to change that?

[ycombinator]

Hi @tlrx, sorry for any confusion. Let me try to clarify how things are working right now (I just tested in master to confirm).

Conceptually, the xpack.monitoring.collection.enabled setting does everything the xpack.monitoring.elasticsearch.collection.enabled setting does and more. The xpack.monitoring.collection.enabled setting controls whether all of Monitoring should be on or off. With it on, the xpack.monitoring.elasticsearch.collection.enabled setting can then be used to turn off just Elasticsearch monitoring. Conceptually this setting is similar to the xpack.monitoring.kibana.collection.enabled setting that can be set in kibana.yml to turn off just Kibana monitoring.

Put differently:

The xpack.monitoring.elasticsearch.collection.enabled setting stops collection of only Elasticsearch monitoring data. Monitoring data from other products (Beats/LS/KB) sent to the bulk monitoring API endpoint will still be exported and indexed into monitoring indices.

On the other hand, the xpack.monitoring.collection.enabled setting stops collection of Elasticsearch monitoring data and ignores any monitoring data sent from other products (Beats/LS/KB) sent to the bulk monitoring API endpoint. Therefore, no monitoring data is exported and indexed into monitoring indices.

You can verify this as follows:

1. Start up Elasticsearch and Kibana with default settings + basic license (default build).
2. At this point no monitoring data should be indexed. Verify this with:

   GET _cat/indices/.m*`
   

3. Turn on all monitoring with:

   PUT _cluster/settings
   {
     "persistent": {
       "xpack.monitoring.collection.enabled": true
     }
   }
   

4. After 30s or so, verify that both ES and Kibana monitoring indices are being created and growing, with:

   GET _cat/indices/.m*`
   

5. Now turn off only Elasticsearch monitoring with:

   PUT _cluster/settings
   {
     "persistent": {
       "xpack.monitoring.elasticsearch.collection.enabled": false
     }
   }
   

6. After 30s or so, verify that only Kibana monitoring indices are growing but the Elasticsearc monitoring indices have stopped growing, with:

   GET _cat/indices/.m*`
   

Does that help?

[tlrx]

Thanks for the reminder! I was looking at where ENABLED was used - and saw nothing in the monitoring bulk code - but looking at where the isMonitoringActive() is used answered my question.

[tlrx]

LGTM