Add Beats module and input info to Telemetry

[tsullivan]

Carry on from #18833

(No release notes needd)

This adds the following info to Beats telemetry data payload:

input: {
  count: <Number>,
  names: [ <String>, ... ], # Example: `stdin`
},
module: {
  count: <Number>,
  names: [ <String>, ... ], # Example: `system`, `elasticsearch`
},

To test:

1. Run some type of Beat with a module configured, or run Filebeat with an input configured
2. Check for the Beat info to appear in Monitoring
3. Run the Telemetry API debug script to ensure Beats data appears in the payload:

   node scripts/api_debug.js telemetry --host=https://virtual-spice:5691 | jq '.[].stack_stats | { beats: .beats }'
   

This may be blocked by elastic/beats#7562.

• This PR does not include any changes to accommodate the schema change.
• The mock data in the test fixtures also uses the 6.3 schema.

[elasticmachine]

💚 Build Succeeded

• continuous-integration/kibana-ci/pull-request

[ruflin]

I assume the schema changes you mention above are the ones needed in Elasticsearch? Or are there also schema changes needed in KB? I will take for of the changes in ES.

[ruflin]

Change looks good to me. Do you plan to add the host info like platform in a follow up PR? See here for an example: elastic/beats#7537

Unfortunately the queue type PR is still open.

[ycombinator]

Minor: I see you have this section of code here for processing beats_stats results and further down a section of code for processing beats_state results. I think it might be a bit more readable if you extracted each section into its own respective function, something like processBeatsStatsResults and processBeatsStateResults, and then call those from here.

[tsullivan]

Do you plan to add the host info like platform in a follow up PR? See here for an example: elastic/beats#7537

Hi, unfortunately I didn't realize those parts were also part of the scope of this work, and the queue name which is still an open PR would put me in a hanging state. If we need that in 6.4 as well, I will try to find someone who can take it on.

[elasticmachine]

💔 Build Failed

• continuous-integration/kibana-ci/pull-request

[ruflin]

@tsullivan With input and module we definitively have the 2 most important ones. host / platform would be great to have. Let's skip queue for now.

[tsullivan]

unrelated selenium failure

jenkins test this

[elasticmachine]

💚 Build Succeeded

• continuous-integration/kibana-ci/pull-request

[tsullivan]

6.x/6.4.0: #20856