[Console] Implement logic to generate body params

[yuliacech]

Summary

Fixes #160533

This PR is a follow up to work done in #159241 and #160515. The new logic added is to generate body params definitions from the ES specification. The old spec-to-console spec doesn't generate any parameters for request body, because this information is not included into JSON specs. So all autocomplete definitions for Console were created manually: see folders js and overrides.
The new script converts the types included in the specification to create an object representing request body.

Description

The logic of the body params conversion is in the comments of the file body_params_converter.ts.

I noticed that some types cause an endless loop in the script because they use their own type to describe inner fields recursively. For example, the endpoint async_search.submit has a field collapse which is of type FieldCollapse. When we look at the type FieldCollapse, among other fields it has a nested field collapse which is of type FieldCollapse. With such nested recursion it would be impossible to describe this endpoint in a simple json structure. So we have to extract this type into a separate definition and link to it with link_scope. The type then needs to be loaded as global rule similar to definitions in the folder js.

That is why this PR moves the endpoints definitions from the folder generated to the folder generated/endpoints and creates a new folder generated/globals. This folder is empty for now and will contain the global definitions when the new script will be used.

How to test

1. Run the new script with a command like this node scripts/generate_console_definitions.js --source /Users/yulia/elastic/elasticsearch-specification
2. In the file spec_definitions_service.ts comment out line 89 like so //this.loadJSSpec();. This is needed to prevent loading manually defined global rules.
3. Start Kibana and check various endpoints in Console.

Known issues

• Body request params can sometimes be a very large object, that causes definitions files to reach the size of several MBs. This needs to be tested and possibly get limited to not cause any performance issues in the browser.
• Some types are not correct in the ES specification repository. For example, ILM policy phase has a field configurations which is not correct according to the ILM API. This needs to be addressed directly in the ES specification repo.

[kibana-ci]

💚 Build Succeeded

• Buildkite Build
• Commit: cc53d22

Metrics [docs]

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
console	418.5KB	418.5KB	+2.0B

History

• 💔 Build #144785 failed 22f1742
• 💔 Build #144343 failed 6f23062
• 💔 Build #142622 failed 09f6335
• 💔 Build #142529 failed adee5e8
• 💔 Build #142493 failed 7986d80

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

[elasticmachine]

Pinging @elastic/platform-deployment-management (Team:Deployment Management)

[ElenaStoeva]

Great work @yuliacech!

I tested manually around 10-15 randomly picked endpoints with body params and most of them worked as expected. I only noticed some potential issues in two of them:

1. For the msearch_template.json autocomplete definition, no body params are being suggested. This might be due to an error in the Es spec file though as it seems that they specified the type of body to be an array which I'm not sure is valid.
2. For count.json, no params are suggested for the query.filter field where a GLOBAL.query_dsl.QueryContainer type is expected. Same for the watcher.query_watches.json file.

I know these are more complex cases, but for the basic body params the suggestions are great! 🎉

[yuliacech]

Thanks a lot for your review, @ElenaStoeva! I'will close this PR for now following the discussion in the team sync. We will not generate request body parameters with the new script for now, because that brings too many risks to the next release. This work will be continued later as an improvement for Console autocomplete generation process.