[Security Solution] Specific Cypress executions for Rule Management team

[MadameSheema]

Addresses: #153661

Summary

This PR is part of the effort we are making to have our cypres serverless tests ready for the second quality gate (QA environment - real serverless project and also part of the initial effort started by #153664.

With the introduced changes, we are creating specific Rule Management Cypress executions for both ESS and serverless with the aim of:

• To help to identify quickly the ownership of the tests in case of failure.
• To help to raise specific flakiness inside the tests of the team.

In this PR:

• We are creating a specific folder for the rule_management team
• We are creating 2 different executions (just prebuilt rules and all the tests excluding prebuilt rules) for ESS, serverless and the quality gate
• We are integrating everything with buildkite
• We are updating the readme

[elasticmachine]

Pinging @elastic/security-detections-response (Team:Detections and Resp)

[elasticmachine]

Pinging @elastic/security-solution (Team: SecuritySolution)

[delanni]

new test suite additions from the ops perspective 👍

Keep in mind, the current test slice runs on these suites are quite fast (8-12 minutes). Since several other tests are running for ~30 minutes, it could be OK to reduce parallelism, and have fewer, but longer running slices (this would reduce the sum overhead from agent provisioning/bootstrap)

[banderror]

@MadameSheema Looking into it

[banderror]

Everything LGTM, it's just some paths need to be fixed in x-pack/test/security_solution_cypress/package.json.

I also posted a nit comment about folder structure, but it shouldn't block the PR.

[banderror]

Nit: Hey @MadameSheema, was it necessary to move rule management tests outside of the x-pack/test/security_solution_cypress/cypress/e2e/detection_response folder to x-pack/test/security_solution_cypress/cypress/e2e/rule_management? Can we move them back?

I'd prefer having the following structure:

• x-pack/test/security_solution_cypress/cypress/e2e/detection_response/detection_engine
• x-pack/test/security_solution_cypress/cypress/e2e/detection_response/rule_management

cc @yctercero

[yctercero]

++ on suggested change. I'm somewhat hesitant to base folder structure on teams (as we've updated teams several times), but I think it's needed in order to group the configs, right?

[MadameSheema]

@yctercero is around making it easier to maintain the scripts we use as well as checking as knowing by a quick look who is the owner. Another thing that I noticed when we didn't have this structure is that several tests didn't have ownership at all making hard to follow some automated flows on operations side.

[kibana-ci]

💚 Build Succeeded

• Buildkite Build
• Commit: 69e7c79

Metrics [docs]

✅ unchanged

History

• 💛 Build #178738 was flaky bfbac3e
• 💚 Build #178577 succeeded 0fa496c
• 💔 Build #178562 failed ee25e91

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @MadameSheema

[banderror]

LGTM, thanks @MadameSheema 👍

[banderror]

@MadameSheema I added the v8.11.2 label because I think it's always worth backporting changes in tests to minimize merge conflicts in the future. Often when we fix a bug and backport the fix, we backport some changes in tests too. Another example would be fixing flakiness - over the last few months, we've been backporting such PRs to reduce flakiness in release branches as well.

[kibanamachine]

💔 All backports failed

Status	Branch	Result
❌	8.11	Backport failed because of merge conflicts You might need to backport the following PRs to 8.11: - [Security Solution] fix cypress config to run all tests (#169942) - [Security Solution] Adds more cypress tests on serverless for investigations team (#169379) - [Security Solution] Adding serverlessQA tag (#167494)

Manual backport

To create the backport manually run:

node scripts/backport --pr 171868

Questions ?

Please refer to the Backport tool documentation

[jbudz]

This was reverted with 177dbd1 due to a parse error in on_merge.yml. See https://buildkite.com/elastic/kibana-on-merge/builds/38581#018c1627-73c5-4f15-9c17-f45b82626364.