Cherry-pick #22879 to 7.x: libbeat: override /sys/fs/cgroup paths in Docker

[axw]

Cherry-pick of PR #22879 to 7.x branch. Original message:

What does this PR do?

This PR adds an undocumented environment variable, LIBBEAT_MONITORING_CGROUPS_HIERARCHY_OVERRIDE, which can be used to override where cgroup metrics are found. This follows the approach taken by Elasticsearch (elastic/elasticsearch#22757), Kibana, and Logstash. The environment variable is set (to "/") in the Docker images.

Why is it important?

Under Docker, the paths in /proc/<pid>/cgroup do not have corresponding paths under /sys/fs/cgroup/<subsystem>. Instead, we should assume a root cgroup and read directly from the files in the subsystem directories.

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
  - [ ] I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
  - [ ] I have added tests that prove my fix is effective or that my feature works
• I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

How to test this PR locally

• Build the Docker image for x-pack/filebeat: PLATFORMS=linux/amd64 TYPES=docker mage package)
• Load the Docker image: docker image load < build/distributions/filebeat-8.0.0-linux-amd64.docker.tar.gz
• Run a filebeat container under Docker with monitoring enabled, and with a CPU quota defined (I set it to 20000, for example).
• Wait for metrics to be reported, and check that the cgroup metrics are non-zero:

GET /.monitoring-beats-7-2020.12.03/_search
{
  "sort": [
    {
      "timestamp": {
        "order": "desc"
      }
    }
  ],
  "_source": "beats_stats.metrics.beat.cgroup"
}

        "_index" : ".monitoring-beats-7-2020.12.03",
        "_id" : "cz-vJnYBjbWE_R3aj_hz",
        "_score" : null,
        "_source" : {
          "beats_stats" : {
            "metrics" : {
              "beat" : {
                "cgroup" : {
                  "memory" : {
                    "mem" : {
                      "usage" : {
                        "bytes" : 29429760
                      },
                      "limit" : {
                        "bytes" : 9223372036854771712
                      }
                    },
                    "id" : "e62f3fa5f16f05d6edc680e2303ccfb6a323599db6f7dfd2b02640390d1b8319"
                  },
                  "cpu" : {
                    "cfs" : {
                      "period" : {
                        "us" : 100000
                      },
                      "quota" : {
                        "us" : 20000
                      }
                    },
                    "stats" : {
                      "periods" : 254,
                      "throttled" : {
                        "ns" : 9362053609,
                        "periods" : 40
                      }
                    },
                    "id" : "e62f3fa5f16f05d6edc680e2303ccfb6a323599db6f7dfd2b02640390d1b8319"
                  },
                  "cpuacct" : {
                    "total" : {
                      "ns" : 1476308159
                    },
                    "id" : "e62f3fa5f16f05d6edc680e2303ccfb6a323599db6f7dfd2b02640390d1b8319"
                  }
                }
              }
            }
          }
        },

Related issues

Fixes #22844

[elasticmachine]

Pinging @elastic/integrations-services (Team:Services)

[elasticmachine]

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Build Cause: Pull request #23050 updated

• Start Time: 2020-12-16T03:52:26.859+0000

• Duration: 106 min 7 sec

Test stats 🧪

Test	Results
Failed	0
Passed	17458
Skipped	1408
Total	18866

💚 Flaky test report

Tests succeeded.

Expand to view the summary

Test stats 🧪

Test	Results
Failed	0
Passed	17458
Skipped	1408
Total	18866