Skip to content

[Cloud Security] Fix serverless quality gate tests and introduce a solution for tests that are only run in MKI Serverless Quality Gates#202038

Merged
seanrathier merged 12 commits intoelastic:mainfrom
seanrathier:11274-fix-the-serverless-quality-gate-tests-for-agentless
Dec 3, 2024
Merged

[Cloud Security] Fix serverless quality gate tests and introduce a solution for tests that are only run in MKI Serverless Quality Gates#202038
seanrathier merged 12 commits intoelastic:mainfrom
seanrathier:11274-fix-the-serverless-quality-gate-tests-for-agentless

Conversation

@seanrathier
Copy link
Copy Markdown
Contributor

@seanrathier seanrathier commented Nov 27, 2024 •
edited by kibanamachine
Loading

Summary

closes https://github.com/elastic/security-team/issues/11274

This change aligns the agentless test in the Serverless Quality Gates so that they are aligned properly to the Kibana boundaries.

More importantly, this introduces a new FTR config for Security teams to be able to have tests that only run in MKI Serverless Quality Gates. Loading test files into config.mki_only.ts configuration ensures that those tests are segregated to only MKI.

Related Issues

Checklist

@seanrathier seanrathier self-assigned this Nov 27, 2024
@seanrathier seanrathier marked this pull request as ready for review November 27, 2024 17:05
@seanrathier seanrathier requested review from a team as code owners November 27, 2024 17:05
@elasticmachine
Copy link
Copy Markdown
Contributor

elasticmachine commented Nov 27, 2024

Pinging @elastic/kibana-cloud-security-posture (Team:Cloud Security)

@seanrathier seanrathier added the release_note:skip Skip the PR/issue when compiling release notes label Nov 27, 2024
@seanrathier seanrathier force-pushed the 11274-fix-the-serverless-quality-gate-tests-for-agentless branch from d2533e0 to e65298b Compare November 27, 2024 18:48
opauloh
opauloh approved these changes Nov 27, 2024
View reviewed changes
Copy link
Copy Markdown
Contributor

@opauloh opauloh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, great work, just some questions and nits

...ional/test_suites/security/ftr/cloud_security_posture/mki_only/agentless/mki_create_agent.ts Outdated
Copy link
Copy Markdown
Contributor

@opauloh opauloh Nov 27, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💯

Copy link
Copy Markdown

@acorretti acorretti Nov 28, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a good compromise 👌

x-pack/test_serverless/functional/test_suites/security/config.mki_only.ts Outdated
Copy link
Copy Markdown
Contributor

@opauloh opauloh Nov 27, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is there a configuration we need to coordinate with the qaf-tests repo to include config files by using the tag?

Copy link
Copy Markdown
Contributor

@opauloh opauloh Nov 27, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I saw now: https://github.com/elastic/qaf-tests/pull/141

@seanrathier seanrathier force-pushed the 11274-fix-the-serverless-quality-gate-tests-for-agentless branch from 5fd43d5 to 30d69f5 Compare November 27, 2024 20:37
@seanrathier seanrathier enabled auto-merge (squash) November 27, 2024 20:37
@seanrathier seanrathier force-pushed the 11274-fix-the-serverless-quality-gate-tests-for-agentless branch from 30d69f5 to 12ee13b Compare November 27, 2024 22:44
@seanrathier seanrathier changed the title [Cloud Security] Fix serverless quality gate tests and introduce a better solution tests that are only run in MKI Serverless Quality Gates [Cloud Security] Fix serverless quality gate tests and introduce a solution for tests that are only run in MKI Serverless Quality Gates Nov 28, 2024
@seanrathier seanrathier mentioned this pull request Nov 28, 2024
Merged
@seanrathier seanrathier force-pushed the 11274-fix-the-serverless-quality-gate-tests-for-agentless branch from 12ee13b to 4f4bbeb Compare November 29, 2024 14:38
@seanrathier seanrathier requested a review from pheyos November 29, 2024 16:31
pheyos
pheyos reviewed Nov 29, 2024
View reviewed changes
@seanrathier seanrathier force-pushed the 11274-fix-the-serverless-quality-gate-tests-for-agentless branch from 8654174 to 6880dae Compare December 2, 2024 16:44
@elasticmachine
Copy link
Copy Markdown
Contributor

elasticmachine commented Dec 2, 2024 •
edited
Loading

💛 Build succeeded, but was flaky

Failed CI Steps

Metrics [docs]

‼️ ERROR: no builds found for mergeBase sha [3e1d62e]

History

cc @seanrathier

YulNaumenko
YulNaumenko approved these changes Dec 3, 2024
View reviewed changes
Copy link
Copy Markdown
Contributor

@YulNaumenko YulNaumenko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@seanrathier seanrathier merged commit acba610 into elastic:main Dec 3, 2024
@kibanamachine
Copy link
Copy Markdown
Contributor

kibanamachine commented Dec 3, 2024

Starting backport for target branches: 8.x

https://github.com/elastic/kibana/actions/runs/12133368406

@kibanamachine
Copy link
Copy Markdown
Contributor

kibanamachine commented Dec 3, 2024

💔 All backports failed

Status Branch Result
❌ 8.x Backport failed because of merge conflicts

You might need to backport the following PRs to 8.x:
- [Security Solution][Endpoint] Add FTR API tests that validates creation of DOT indices (#197899)

Manual backport

To create the backport manually run:

node scripts/backport --pr 202038

Questions ?

Please refer to the Backport tool documentation

@kibanamachine kibanamachine added the backport missing Added to PRs automatically when the are determined to be missing a backport. label Dec 4, 2024
@kibanamachine
Copy link
Copy Markdown
Contributor

kibanamachine commented Dec 5, 2024

Friendly reminder: Looks like this PR hasn’t been backported yet.
To create automatically backports add a backport:* label or prevent reminders by adding the backport:skip label.
You can also create backports manually by running node scripts/backport --pr 202038 locally

@kibanamachine
Copy link
Copy Markdown
Contributor

kibanamachine commented Dec 6, 2024

Friendly reminder: Looks like this PR hasn’t been backported yet.
To create automatically backports add a backport:* label or prevent reminders by adding the backport:skip label.
You can also create backports manually by running node scripts/backport --pr 202038 locally

@seanrathier seanrathier added the backport:skip This PR does not require backporting label Dec 6, 2024
@kibanamachine kibanamachine removed the backport missing Added to PRs automatically when the are determined to be missing a backport. label Dec 6, 2024
@elastic elastic deleted a comment from kibanamachine Dec 6, 2024
CAWilson94 pushed a commit to CAWilson94/kibana that referenced this pull request Dec 12, 2024
@seanrathier @CAWilson94
[Cloud Security] Fix serverless quality gate tests and introduce a be…
cd4880f 
…tter solution tests that are only run in MKI Serverless Quality Gates (elastic#202038)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@pheyos pheyos pheyos approved these changes

@opauloh opauloh opauloh approved these changes

@jbudz jbudz jbudz approved these changes

@YulNaumenko YulNaumenko YulNaumenko approved these changes

+1 more reviewer

@acorretti acorretti acorretti approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

@seanrathier seanrathier

Labels

automation backport:skip This PR does not require backporting release_note:skip Skip the PR/issue when compiling release notes Team:Cloud Security Cloud Security team related v9.0.0

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

8 participants

@seanrathier @elasticmachine @kibanamachine @acorretti @pheyos @jbudz @opauloh @YulNaumenko