remove deps on data plugin

[Dosant]

Summary

This pr highlights a blocker for removing data_enhanced plugin in #122075

To remove data_enhanced, we will need data plugin optionally depend on security plugin.
After fixing some circular dependency CI starts flaky failing on commit where security becomes a dependency of data plugin: df7151a

The culprit is not clear yet.

Here you can see how I had 3 successful runs before adding security as dependency and then a failure after adding it in: df7151a

Some observations:

1. One of the ofter failing tests is status page should display the server status. Kibana reports Yellow status when Green is expected. After adding more logging I saw that taskManager is causing Yellow status.
   
   logs
2. Ofter failing tests look like Kibana or a bundle is not loaded properly
   
   logs
3. Some of security / reporting are often failing, for example: Reporting Functional Tests with Security enabled Security with reporting_user built-in role Dashboard: Download CSV file does not allow user that does not have reporting privileges. It fails on page load with timeout. logs. It is also often SAML / OIDC related tests that are likely to fail
4. Sometimes the failure is: error: Setup lifecycle of "infra" plugin wasn't completed in 10sec. Consider disabling the plugin and re-start. I guess this just highlights that a bundle wasn't loaded on time
5. Often flaky test pass by second try. Some of them fail even on second try, but with slightly different error.

What we've tried:

1. There was and idea that plugin topology changed, making plugins resolve in different order highlighting some bugs. But this isn't the case. The list of plugins that core reports on start up is unchanged
2. Tried to reproduce locally. Both with dev and prod build. No realiable reproduction were found.
3. Tried to make security plugin a dependency of a different OSS plugin (discover) - didn't reproduce the issue. So issue is reproduced on data but not on discover

UPDATE 1

In #123220 we've added debug logs for security and taskManager
We've noticed that, for example, in a very simple test:

const { access_token: accessToken } = await es.security.getToken({
        body: {
          grant_type: 'password',
          username: 'elastic',
          password: 'changeme',
        },
      });

      await supertest
        .get('/internal/security/me')
        .set('kbn-xsrf', 'true')
        .set('authorization', `Bearer ${accessToken}`)
        .expect(200);

Call to es takes around 30 seconds!!

The test fails because the token expires before it can continue.
logs

Update 2

We've tried adding a delay between es + kibana start and tests run: 2da369f
🟢 🟢 Tests started passing! 🟢 🟢
This means there is clearly some instablity in the beginning causing kibana / elasticsearch slow to respond and tests to fail

[Dosant]

@elasticmachine merge upstream

[Dosant]

@elasticmachine merge upstream

[Dosant]

@elasticmachine merge upstream

[Dosant]

@elasticmachine merge upstream

[Dosant]

buildkite, test this

[Dosant]

@elasticmachine merge upstream

[afharo]

@elasticmachine merge upstream

[afharo]

FYI: I merged main into this branch and resolved the conflicts in hopes that #121628 made this better. My local tests are passing now, so hopefully CI does as well.

[Dosant]

@elasticmachine merge upstream

[afharo]

Updating from main after #128324 was merged to see if it improves anything. That PR fixed an event-loop delay due to status-dependency processing, which happens a lot during start-up.

[afharo]

@elasticmachine merge upstream

[Dosant]

buildkite, test this

[Dosant]

buildkite, test this

[kibana-ci]

💚 Build Succeeded

• Buildkite Build
• Commit: 0383fef
• Storybooks Preview
• Webpack Bundle Analyzer

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
customIntegrations	17	22	+5

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
customIntegrations	6.1KB	10.0KB	+3.9KB
security	50.2KB	50.2KB	-5.0B
total			+3.9KB

Unknown metric groups

ESLint disabled line counts

id	before	after	diff
customIntegrations	2	3	+1

Total ESLint disabled count

id	before	after	diff
customIntegrations	2	3	+1

History

• 💚 Build #34376 succeeded 0383fef
• 💚 Build #34349 succeeded 0383fef
• 💔 Build #33446 failed 08ba9dc
• 💔 Build #33416 failed 897187f
• 💚 Build #33080 succeeded e13aa88
• 💔 Build #32984 failed a5822e5

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream