[Ingest Manager] Unify install* under installPackage

[jfsiii]

Summary

• Add installPackage with installSource param, to provide a single interface the install* functions.

  -    const res = await installPackageFromRegistry({
  +    const res = await installPackage({
  +      installSource: 'registry',

  and

  -    const res = await installPackageByUpload({
  +    const res = await installPackage({
  +      installSource: 'upload',

• Push some repeated work (install, removable) from install* into _installPackage. Which also simplifies its interface.

installPackage

For now installPackage checks the installSource and calls the same install* functions to prevent any change in behavior but there's still a lot of overlap between installPackageFromRegistry & installPackageByUpload. I think we can bring them together into installPackage using the same branching on installSource.

local checks with curl

curl request/responses for happy path:

## zip:
curl -X POST -u elastic:changeme http://localhost:5601/api/fleet/epm/packages --data-binary @$KIBANA_HOME/x-pack/test/ingest_manager_api_integration/apis/fixtures/direct_upload_packages/apache_0.1.4.zip -H 'kbn-xsrf: xyz' -H 'Content-Type: application/zip'
{"response":[{"id":"apache-Logs-Apache-Dashboard-ecs","type":"dashboard"},{"id":"apache-Metrics-Apache-HTTPD-server-status-ecs","type":"dashboard"},{"id":"Apache-HTTPD-CPU-ecs","type":"visualization"},{"id":"Apache-HTTPD-Hostname-list-ecs","type":"visualization"},{"id":"Apache-HTTPD-Load1-slash-5-slash-15-ecs","type":"visualization"},{"id":"Apache-HTTPD-Scoreboard-ecs","type":"visualization"},{"id":"Apache-HTTPD-Total-accesses-and-kbytes-ecs","type":"visualization"},{"id":"Apache-HTTPD-Uptime-ecs","type":"visualization"},{"id":"Apache-HTTPD-Workers-ecs","type":"visualization"},{"id":"Apache-access-unique-IPs-map-ecs","type":"visualization"},{"id":"Apache-browsers-ecs","type":"visualization"},{"id":"Apache-error-logs-over-time-ecs","type":"visualization"},{"id":"Apache-operating-systems-ecs","type":"visualization"},{"id":"Apache-response-codes-of-top-URLs-ecs","type":"visualization"},{"id":"Apache-response-codes-over-time-ecs","type":"visualization"},{"id":"Apache-HTTPD-ecs","type":"search"},{"id":"Apache-access-logs-ecs","type":"search"},{"id":"Apache-errors-log-ecs","type":"search"}]}

## Uploaded packages can be deleted as expected:
curl -X DELETE -u elastic:changeme http://localhost:5601/api/fleet/epm/packages/apache-0.1.4 -H 'kbn-xsrf: xxx'
{"response":[{"id":"apache-Logs-Apache-Dashboard-ecs","type":"dashboard"},{"id":"apache-Metrics-Apache-HTTPD-server-status-ecs","type":"dashboard"},{"id":"Apache-HTTPD-CPU-ecs","type":"visualization"},{"id":"Apache-HTTPD-Hostname-list-ecs","type":"visualization"},{"id":"Apache-HTTPD-Load1-slash-5-slash-15-ecs","type":"visualization"},{"id":"Apache-HTTPD-Scoreboard-ecs","type":"visualization"},{"id":"Apache-HTTPD-Total-accesses-and-kbytes-ecs","type":"visualization"},{"id":"Apache-HTTPD-Uptime-ecs","type":"visualization"},{"id":"Apache-HTTPD-Workers-ecs","type":"visualization"},{"id":"Apache-access-unique-IPs-map-ecs","type":"visualization"},{"id":"Apache-browsers-ecs","type":"visualization"},{"id":"Apache-error-logs-over-time-ecs","type":"visualization"},{"id":"Apache-operating-systems-ecs","type":"visualization"},{"id":"Apache-response-codes-of-top-URLs-ecs","type":"visualization"},{"id":"Apache-response-codes-over-time-ecs","type":"visualization"},{"id":"Apache-HTTPD-ecs","type":"search"},{"id":"Apache-access-logs-ecs","type":"search"},{"id":"Apache-errors-log-ecs","type":"search"}]}

## Now upload
curl -X POST -u elastic:changeme http://localhost:5601/api/fleet/epm/packages --data-binary @$KIBANA_HOME/x-pack/test/ingest_manager_api_integration/apis/fixtures/direct_upload_packages/apache_0.1.4.tar.gz -H 'kbn-xsrf: xyz' -H 'Content-Type: application/gzip'
{"response":[{"id":"apache-Metrics-Apache-HTTPD-server-status-ecs","type":"dashboard"},{"id":"apache-Logs-Apache-Dashboard-ecs","type":"dashboard"},{"id":"Apache-access-unique-IPs-map-ecs","type":"visualization"},{"id":"Apache-HTTPD-CPU-ecs","type":"visualization"},{"id":"Apache-HTTPD-Load1-slash-5-slash-15-ecs","type":"visualization"},{"id":"Apache-response-codes-over-time-ecs","type":"visualization"},{"id":"Apache-HTTPD-Workers-ecs","type":"visualization"},{"id":"Apache-HTTPD-Hostname-list-ecs","type":"visualization"},{"id":"Apache-error-logs-over-time-ecs","type":"visualization"},{"id":"Apache-HTTPD-Scoreboard-ecs","type":"visualization"},{"id":"Apache-HTTPD-Uptime-ecs","type":"visualization"},{"id":"Apache-operating-systems-ecs","type":"visualization"},{"id":"Apache-HTTPD-Total-accesses-and-kbytes-ecs","type":"visualization"},{"id":"Apache-browsers-ecs","type":"visualization"},{"id":"Apache-response-codes-of-top-URLs-ecs","type":"visualization"},{"id":"Apache-access-logs-ecs","type":"search"},{"id":"Apache-errors-log-ecs","type":"search"},{"id":"Apache-HTTPD-ecs","type":"search"},{"id":"logs-apache.error-0.1.4","type":"ingest_pipeline"},{"id":"logs-apache.access-0.1.4","type":"ingest_pipeline"},{"id":"logs-apache.error","type":"index_template"},{"id":"metrics-apache.status","type":"index_template"},{"id":"logs-apache.access","type":"index_template"}]}

curl request/responses for archive errors:

## Wrong content type:
### tar.gz with application/zip:
curl -X POST -u elastic:changeme http://localhost:5601/api/fleet/epm/packages --data-binary @$KIBANA_HOME/x-pack/test/ingest_manager_api_integration/apis/fixtures/direct_upload_packages/apache_0.1.4.tar.gz -H 'kbn-xsrf: xyz' -H 'Content-Type: application/zip'
{"statusCode":400,"error":"Bad Request","message":"Error during extraction of package: Error: end of central directory record signature not found. Assumed content type was application/zip, check if this matches the archive type."}

### zip with application/gzip:
curl -X POST -u elastic:changeme http://localhost:5601/api/fleet/epm/packages --data-binary @$KIBANA_HOME/x-pack/test/ingest_manager_api_integration/apis/fixtures/direct_upload_packages/apache_0.1.4.zip -H 'kbn-xsrf: xyz' -H 'Content-Type: application/gzip'
{"statusCode":400,"error":"Bad Request","message":"Archive seems empty. Assumed content type was application/gzip, check if this matches the archive type."}

## Invalid packages
### Two top-level directories:
curl -X POST -u elastic:changeme http://localhost:5601/api/fleet/epm/packages --data-binary @$KIBANA_HOME/x-pack/test/ingest_manager_api_integration/apis/fixtures/direct_upload_packages/apache_invalid_two_toplevels_0.1.4.zip -H 'kbn-xsrf: xyz' -H 'Content-Type: application/zip'
{"statusCode":400,"error":"Bad Request","message":"Package contains more than one top-level directory."}

### No manifest:
curl -X POST -u elastic:changeme http://localhost:5601/api/fleet/epm/packages --data-binary @$KIBANA_HOME/x-pack/test/ingest_manager_api_integration/apis/fixtures/direct_upload_packages/apache_invalid_no_manifest_0.1.4.zip -H 'kbn-xsrf: xyz' -H 'Content-Type: application/zip'
{"statusCode":400,"error":"Bad Request","message":"Package must contain a top-level manifest.yml file."}

### Invalid YAML in manifest:
curl -X POST -u elastic:changeme http://localhost:5601/api/fleet/epm/packages --data-binary @$KIBANA_HOME/x-pack/test/ingest_manager_api_integration/apis/fixtures/direct_upload_packages/apache_invalid_manifest_invalid_yaml_0.1.4.zip -H 'kbn-xsrf: xyz' -H 'Content-Type: application/zip'
{"statusCode":400,"error":"Bad Request","message":"Could not parse top-level package manifest: YAMLException: bad indentation of a mapping entry at line 2, column 7:\n      name: apache\n          ^."}

### Mandatory field missing in manifest:
curl -X POST -u elastic:changeme http://localhost:5601/api/fleet/epm/packages --data-binary @$KIBANA_HOME/x-pack/test/ingest_manager_api_integration/apis/fixtures/direct_upload_packages/apache_invalid_manifest_missing_field_0.1.4.zip -H 'kbn-xsrf: xyz' -H 'Content-Type: application/zip'
{"statusCode":400,"error":"Bad Request","message":"Invalid top-level package manifest: one or more fields missing of name, version, description, type, categories, format_version"}

### Top-level directory doesn't match name and version from manifest:
curl -X POST -u elastic:changeme http://localhost:5601/api/fleet/epm/packages --data-binary @$KIBANA_HOME/x-pack/test/ingest_manager_api_integration/apis/fixtures/direct_upload_packages/apache_invalid_toplevel_mismatch_0.1.4.zip -H 'kbn-xsrf: xyz' -H 'Content-Type: application/zip'
{"statusCode":400,"error":"Bad Request","message":"Name thisIsATypo and version 0.1.4 do not match top-level directory apache-0.1.4"}

TS type check examples on installPackage

screenshots

Checklist

• Unit or functional tests were updated or added to match the most common scenarios

[elasticmachine]

Pinging @elastic/ingest-management (Team:Ingest Management)

[jfsiii]

@elasticmachine merge upstream

[kibanamachine]

💚 Build Succeeded

• continuous-integration/kibana-ci/pull-request
• Commit: 3d934a4

Metrics [docs]

✅ unchanged

History

• 💚 Build #86597 succeeded 0085069
• 💚 Build #86596 succeeded 4c0d5c31dfa037fae8b3b9c04a3bd9973c1720bf

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream