[Security Solutions] (Phase 1) Adds an application cache called metrics entities and integrates it within Security Solutions behind a feature flag

[FrankHassanabad]

Summary

Phase 1 of a multi-phase cautious approach for adding an experimental application cache for Kibana solutions called metric_entities and integrates it within Security Solutions.

Phase 1 is putting experimental support into the application without breaking existing features. Lots of TODO's, conversations and a possible RFC from phase 1 to phase 2 approach. Some features are missing, but for phase 1 the general idea and code is all there.

To enable this first phase after checking out the branch add this to your kibana.dev.yml

xpack.metricsEntities.enabled: true
xpack.securitySolution.enableExperimental: ['metricsEntitiesEnabled']

Then go into Stack Management -> Advanced Settings (Under Security Solutions) and set the enabled to true like so:

Next go to the security_solutions page and you will see it being activated and you will have these transforms running if you look under stack management:

On the hosts page, network, page, etc... You can see them being activated when you have no query/filter and you click on request:

You will see in the request the index patterns all starting with estc_xyz*

Checklist

Delete any items that are not applicable to this PR.

• Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support

We have lots of TODO's but no concrete docs with this just yet.

• Documentation was added for features that require explanation or tutorials

Behind a feature flag and this isn't there yet.

• Unit or functional tests were updated or added to match the most common scenarios

[elasticmachine]

Pinging @elastic/security-detections-response (Team:Detections and Resp)

[jbudz]

get_server_watch_paths LGTM

[spong]

Platform discourages --no-base-path. We should adapt our scripts to play nice when without this flag. We've discussed this previously with some options being leveraging a static server.basePath: '/kbn', or wrapping our scripts in a helper that calls curl -L http://localhost:5601/ and looks at the redirect and then uses that.

[FrankHassanabad]

Thanks for the tip

[kibanamachine]

💚 Build Succeeded

• continuous-integration/kibana-ci/pull-request
• Commit: 8c3e322
• Storybooks Preview
• Documentation Changes

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
securitySolution	2013	2029	+16

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id	before	after	diff
metricsEntities	-	5	+5

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
securitySolution	6.5MB	6.6MB	+32.3KB

Public APIs missing exports

Total count of every type that is part of your API that should be exported but is not. This will cause broken links in the API documentation system. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats exports for more detailed information.

id	before	after	diff
metricsEntities	-	1	+1

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
securitySolution	160.0KB	160.8KB	+849.0B

Unknown metric groups

API count

id	before	after	diff
metricsEntities	-	8	+8

History

• 💔 Build #123397 failed 2270709
• 💚 Build #123334 succeeded 06d87df
• 💛 Build #123273 was flaky bd49ce3
• 💚 Build #123118 succeeded 4ad83a3
• 💚 Build #123094 succeeded 71b4f96

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @FrankHassanabad

[spong]

Checked out, tested locally, and pair code-reviewed with @FrankHassanabad. Thank you for all your efforts here, and for taking the time to go over all these changes with me -- this is giant leap in providing fast and snappy UI's via metrics and entities! LGTM! 👍 🙂 🚀

[kibanamachine]

💚 Backport successful

Status	Branch	Result
✅	7.x

This backport PR will be merged automatically after passing CI.