[Metricbeat] Add README to mocked tests folder

metricbeat/mb/testing/data/README.md

@@ -0,0 +1,33 @@
+## Event generation testing framework
+
+### Introduction
+
+#### What it is
+Many modules do requests to an HTTP endpoint to fetch metrics data. Then they do data manipulation and enriching of this data before sending it to Elasticsearch.
+
+What we wanted to achieve is to mock the HTTP responses from the modules into a generic server that will serve those responses. So, for each tested metricset, an HTTP server is launched at a random port (but be aware that JSON responses written to disk are hardcoded with value "127.0.0.1:5555") with the mocked response from a fixed module version to respond to the HTTP requests, once the test is done, it's shut down. This way we isolate the manipulation of data in some tests and reduce lot of boilerplate we had in many modules.
+
+#### How to use it
+
+The idea is simple, head to `beats/metricbeat/mb/testing/data` and run `go test .` It will run all tests, each metricset of each module.
+An alternative is to just run from metricbeat `mage mockedTests` to achieve the same result but using environment variables instead of flags, for example: `MODULE=apache GENERATE=true mage mockedTests`
+
+##### Worth to mention
+- If the input file in `testdata` folder is prefixed (named) `docs`, whatever its extension is, and the flat `-generate` is passed; the framework will also create a `docs.json` file in `_meta` folder of the metricset as historically has been done in Metricbeat.
+- Config file **must** be called `config.yml` and be located inside `metricbeat/module/{module}/{metricset}/_meta/testdata`
+
+### Available flags / environment variables
+
+- `-generate`: It will regenerate the _expected_ JSON file with the output of an event an place it within `testdata` folder. For example: `go test . -generate`. If using mage, a environment variable `GENERATE` is available to 
+- `-module`: Test only the specified module. For example `go test . -module=apache`. If using mage `MODULE` environment variable must be set with the _module_ name that must be tested.
+
+> You can also combine both flags with `go test . -generate -module=apache` to generate files for Apache module only.
+
+### Available settings in `config.yml`
+
+- `type`: (string) The type of the test to run. At the moment, only `http` is supported.
+- `url`: (string) This is the URL path that the module usually fetches to get metrics. For example, in case of Apache module this url is `/server-status?auto=` 
+- `suffix`: (string) The suffix that the input file has. By default `json` other common suffixes are `plain` (string) for plain text files.
+- `omit_documented_fields_check`: (List of strings) Some fields generated by the modules are completely dynamic so they aren't documented in `fields.yml`. Set a list of fields or paths in your metricset that might not be documented like `apache.status.*` for all fields within `apache.status` object or `apache.status.hostname` just for that specific field. Even you can omit all fields using `*`
+- `remove_fields_from_comparison`: (List of strings) Some fields must be removed for byte-to-byte comparison but they must be printed anyways in the _expected_ JSON files. Write a list of those fields here. For example, `apache.status.hostname` in the Apache module was generating a new port on each run so a comparison wasn't possible. Set one item with `apache.status.hostname` to omit this field when comparing outputs.
+- `module`: (Map) Anything added to this map will be appended in the module config before launching tests. For example, This is useful for some modules that requires the user to specify a `namespace`.

metricbeat/mb/testing/data/config.yml

@@ -5,3 +5,5 @@ omit_documented_fields_check:
   - "apache.status.hostname"
 remove_fields_from_comparison:
   - "apache.status.hostname"
+module:
+  namespace: test