remove deps on data plugin

[afharo]

Summary

As detailed in #123720, some tests began to fail when the data plugin adds security as an optional dependency (#121955). All the failures are caused by timeouts.

As we troubleshot the timeouts, we found out that running the server first (node scripts/functional_tests_server) and the tests separately (node scripts/functional_test_runner) wouldn't make it fail. Also, adding 90s wait for the cluster to stabilise after start also turned the CI green. So we focused on any issues with running the tests as soon as Kibana became available (GREEN).

Looking at the logs, we can see that ES is busy installing templates when the tests are running. This makes us believe that the issue is that the tests start too soon after this change.

Here's an example of a successful PR:

Compared to the failure:

Bear in mind the line warn debug logs are being captured, only error logs will be written to the console right before the tests. Looking at the captured .log file, we can see the component templates are installed during the execution of the test.

To ensure that the tests are installed only after all the artifacts are properly set up, this PR uses the status API to explicitly declare the initializing status of the relevant plugins. The relevant commit for this change is 4081b6b (#124682).

👆 The change above fixed the broken tests for the Security HTTP API.

However, the UI tests were still an issue. It looks like those failures are caused by the changes in the custom_integrations. I reverted the changes and, to stop the circular dependencies CI check, I changed presentation_util's kibana.json. However, the circular dependency still exists in the requiredBundles and should be fixed.

Risk Matrix

Risk	Probability	Severity	Mitigation/Notes
Since we are waiting for all artifacts to be installed, it may take longer for Kibana to be GREEN	High	Low	All services are active and working, it just reports YELLOW because some plugins are still setting up.
Taking a longer startup may cause Cloud and other automated deployments to timeout.	Low	High	The automation services might need to increase their timeouts. However, according to the tests, it's just a matter of a few additional seconds.

For maintainers

• This was checked for breaking API changes and was labeled appropriately

[afharo]

@elasticmachine merge upstream

[afharo]

@elasticmachine merge upstream

[afharo]

@elasticmachine merge upstream

[kibana-ci]

💔 Build Failed

• Buildkite Build
• Commit: 428af4c
• Interpreting CI Failures
• Storybooks Preview
• Webpack Bundle Analyzer

Failed CI Steps

• Default CI Group #2
• Default CI Group #5
• Default CI Group #7
• Default CI Group #11
• Default CI Group #13
• Default CI Group #13
• Default CI Group #14
• Default CI Group #14
• Default CI Group #16
• Default CI Group #21
• Default CI Group #21
• Docker CI Group
• OSS CI Group #1
• OSS CI Group #1
• Jest Tests #7
• Jest Integration Tests
• API Integration Tests

Test Failures

• [job] [logs] API Integration Tests / apis home apis sample data apis install in the other space should return 200 if success
• [job] [logs] Default CI Group #11 / apis transform basic license transform /api/transform/delete_transforms with deleteDestIndexPattern setting should delete transform and destination index pattern
• [job] [logs] Default CI Group #14 / apps machine learning basic license permissions for user with full ML access (ft_ml_poweruser) "before all" hook for "should display the ML entry in Kibana app menu"
• [job] [logs] Default CI Group #14 / apps machine learning basic license permissions for user with full ML access (ft_ml_poweruser) "before all" hook for "should display the ML entry in Kibana app menu"
• [job] [logs] Default CI Group #13 / Grok Debugger App Grok Debugger "before all" hook for "Loads the app"
• [job] [logs] Default CI Group #2 / Reporting Functional Tests with Security enabled Security with reporting_user built-in role Dashboard: Download CSV file does not allow user that does not have reporting privileges
• [job] [logs] Default CI Group #5 / rule registry spaces only: trial Rule Registry API with write permissions when creating a rule "after all" hook for "writes alerts data to the alert indices"
• [job] [logs] Default CI Group #5 / rule registry spaces only: trial Rule Registry API with write permissions when creating a rule writes alerts data to the alert indices
• [job] [logs] Jest Tests #7 / SavedObjectSaveModal should render matching snapshot
• [job] [logs] Jest Tests #7 / SavedObjectSaveModal should render matching snapshot when custom isValid is set
• [job] [logs] Jest Tests #7 / SavedObjectSaveModal should render matching snapshot when given options
• [job] [logs] Default CI Group #16 / security APIs - Kerberos Kerberos authentication API access with active session "before each" hook for "should fail and preserve session cookie if unsupported authentication schema is used"
• [job] [logs] Default CI Group #13 / security app - OIDC interactions URL capture can login preserving original URL
• [job] [logs] Default CI Group #7 / security app Security Login Page can login
• [job] [logs] OSS CI Group #1 / status page should display the server metrics
• [job] [logs] OSS CI Group #1 / status page should display the server status
• [job] [logs] Default CI Group #21 / transform permissions for user with full transform access with no data loaded "before all" hook for "should display elements in the Transform list page correctly"
• [job] [logs] Default CI Group #21 / transform permissions for user with full transform access with no data loaded "before all" hook for "should display elements in the Transform list page correctly"
• [job] [logs] Docker CI Group / Uptime app with generated data When on the Synthetics Integration Policy Create Page displays custom UI "before all" hook for "should display policy view"

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
presentationUtil	172	176	+4

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
presentationUtil	135.1KB	140.2KB	+5.1KB

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
presentationUtil	44.9KB	44.8KB	-142.0B
security	50.1KB	50.1KB	-5.0B
total			-147.0B

Unknown metric groups

References to deprecated APIs

id	before	after	diff
presentationUtil	2	0	-2

History

• 💔 Build #22003 failed cd97f30
• 💔 Build #21960 failed e1b3ad8
• 💔 Build #21956 failed 1f572e9
• 💔 Build #21946 failed b2619e8

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream