Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Security Solution] Extend Prebuilt rules install and update workflow test coverage #161687

Merged

Conversation

jpdjere
Copy link
Contributor

@jpdjere jpdjere commented Jul 11, 2023

Summary

  • Implement test plan as described in x-pack/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/installation_and_upgrade.md

For maintainers

@jpdjere jpdjere force-pushed the extend-install-update-workflow-test-coverage branch 2 times, most recently from 2c7dfa3 to 52e7201 Compare July 14, 2023 16:20
@jpdjere jpdjere self-assigned this Jul 14, 2023
@jpdjere jpdjere added test Team:Detections and Resp Security Detection Response Team Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. Team:Detection Rule Management Security Detection Rule Management Team Feature:Prebuilt Detection Rules Security Solution Prebuilt Detection Rules v8.9.0 release_note:skip Skip the PR/issue when compiling release notes labels Jul 14, 2023
@jpdjere jpdjere marked this pull request as ready for review July 14, 2023 16:22
@jpdjere jpdjere requested review from a team as code owners July 14, 2023 16:22
@jpdjere jpdjere requested a review from spong July 14, 2023 16:22
@elasticmachine
Copy link
Contributor

Pinging @elastic/security-detections-response (Team:Detections and Resp)

@elasticmachine
Copy link
Contributor

Pinging @elastic/security-solution (Team: SecuritySolution)

@jpdjere jpdjere requested review from xcrzx and removed request for spong July 14, 2023 16:23
@jpdjere jpdjere changed the title [Security Solution] Extend install update workflow test coverage [Security Solution] Extend Prebuilt rules install and update workflow test coverage Jul 14, 2023
@jpdjere jpdjere requested a review from a team as a code owner July 18, 2023 11:16
test.txt Outdated
@@ -0,0 +1,36 @@
debg KIBANA_CI_STATS_CONFIG environment variable not found, disabling CiStatsReporter
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this file needed?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, will remove it

@jpdjere
Copy link
Contributor Author

jpdjere commented Jul 19, 2023

@kibana-ci
Copy link
Collaborator

💚 Build Succeeded

Metrics [docs]

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id before after diff
securitySolution 9.8MB 9.8MB +202.0B

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @jpdjere

Copy link
Contributor

@xcrzx xcrzx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've reviewed the Cypress tests to some degree and I've come up with some suggestions for making them more readable and easier to maintain. Given the time pressure to review and merge the PR by the end of the day, I don't believe we can even partially address these comments right now. So, I'm approving the PR, and we can tackle these comments when Juan returns from his PTO.

I also haven't had time to thoroughly review the integrational tests included in this PR or to check how these tests align with the test plan.

login();
resetRulesTableState();
deleteAlertsAndRules();
esArchiverResetKibana();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need esArchiverResetKibana here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't, removing in #165488

describe('Detection rules, Prebuilt Rules Installation and Update - Authorization/RBAC', () => {
beforeEach(() => {
login();
resetRulesTableState();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need resetRulesTableState here?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't, removing in #165488

beforeEach(() => {
login();
resetRulesTableState();
deleteAlertsAndRules();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we delete all existing rules and create new ones only once in before instead of doing that multiple times using beforeEach?

Comment on lines +29 to +48
const OUTDATED_RULE_1 = createRuleAssetSavedObject({
name: 'Outdated rule 1',
rule_id: RULE_1_ID,
version: 1,
});
const UPDATED_RULE_1 = createRuleAssetSavedObject({
name: 'Updated rule 1',
rule_id: RULE_1_ID,
version: 2,
});
const OUTDATED_RULE_2 = createRuleAssetSavedObject({
name: 'Outdated rule 2',
rule_id: RULE_2_ID,
version: 1,
});
const UPDATED_RULE_2 = createRuleAssetSavedObject({
name: 'Updated rule 2',
rule_id: RULE_2_ID,
version: 2,
});
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should only one outdated rule suffice for these tests?

});
beforeEach(() => {
// Now login with read-only user in preparation for test
createAndInstallMockedPrebuiltRules({ rules: [RULE_1, RULE_2], installToKibana: false });
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Understanding the final state of the rules table can be a bit challenging. This is because some rules are set up in the top-level beforeEach, while others are installed in the beforeEach at this level. To grasp the final state, you have to flip back and forth between the two sections and the body of the test. Moreover, it is not clear why outdated rules are required for this particular test case (rule installation). Could we do the precondition setup in one location for clarity?

For this entire suite, we need 1 outdated rule and 1 ready for installation. Let's do this setup once for both test cases in the before section.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, all the setup for this is being simplified in #165488

Comment on lines +189 to +207
const assertUpgradeSuccessOrFailure = (
rules: Array<typeof SAMPLE_PREBUILT_RULE>,
didRequestFail: boolean
) => {
const rulesString = rules.length > 1 ? 'rules' : 'rule';
const toastMessage = didRequestFail
? `${rules.length} ${rulesString} failed to update.`
: `${rules.length} ${rulesString} updated successfully.`;
cy.get(TOASTER).should('be.visible').should('have.text', toastMessage);
if (didRequestFail) {
for (const rule of rules) {
cy.get(RULES_UPDATES_TABLE).contains(rule['security-rule'].name);
}
} else {
for (const rule of rules) {
cy.get(rule['security-rule'].name).should('not.exist');
}
}
};
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be good to avoid using complex logic inside tests. Tests should be as simple as possible to make sure that they are easy to understand and maintain. If there's a lot of logic in our tests, they become like any other code that needs debugging, which can make tracking down test failures more difficult.

I suggest splitting this function into two: assert success and assert failure to use them separately in tests.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, removing all this complex logic in #165488

return `[data-test-subj="upgradeSinglePrebuiltRuleButton-${ruleId}"]`;
};

export const NO_RULES_AVAILABLE_FOR_INSTALL_MESSSAGE =
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typo:

Suggested change
export const NO_RULES_AVAILABLE_FOR_INSTALL_MESSSAGE =
export const NO_RULES_AVAILABLE_FOR_INSTALL_MESSAGE =

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, fixing in #165488


export const NO_RULES_AVAILABLE_FOR_INSTALL_MESSSAGE =
'[data-test-subj="noPrebuiltRulesAvailableForInstall"]';
export const NO_RULES_AVAILABLE_FOR_UPGRADE_MESSSAGE =
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typo:

Suggested change
export const NO_RULES_AVAILABLE_FOR_UPGRADE_MESSSAGE =
export const NO_RULES_AVAILABLE_FOR_UPGRADE_MESSAGE =

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, fixing in #165488

interceptInstallationRequestToFail(rules, didRequestFail);
const rule = rules[0];
cy.get(getInstallSingleRuleButtonByRuleId(rule['security-rule'].rule_id)).click();
cy.wait('@installPrebuiltRules');
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In e2e tests, it might be better to rely on user-visible indicators of request progress, such as spinners. This approach, instead of depending on the completion of HTTP requests, can make our tests closer to real-life scenarios and highlight problems with loading states which we have from time to time.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense, fixing in #165488

});

describe('Rules installation notification when no rules have been installed', () => {
beforeEach(() => {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Similar to previous comments, here we have three levels of beforeEach blocks making it very hard to understand the initial conditions of the test cases. My recommendation would be to restructure these tests for clarity.

@jpdjere jpdjere merged commit 887b3bd into elastic:main Jul 19, 2023
kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Jul 19, 2023
… test coverage (elastic#161687)

## Summary

- Implement test plan as described in
`x-pack/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/installation_and_upgrade.md`

### For maintainers

- [ ] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)

---------

Co-authored-by: kibanamachine <[email protected]>
(cherry picked from commit 887b3bd)
@kibanamachine
Copy link
Contributor

💚 All backports created successfully

Status Branch Result
8.9

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

@jpdjere
Copy link
Contributor Author

jpdjere commented Jul 19, 2023

💚 All backports created successfully

Status Branch Result
8.9

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

jpdjere added a commit to jpdjere/kibana that referenced this pull request Jul 19, 2023
… test coverage (elastic#161687)

## Summary

- Implement test plan as described in
`x-pack/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/installation_and_upgrade.md`

### For maintainers

- [ ] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)

---------

Co-authored-by: kibanamachine <[email protected]>
(cherry picked from commit 887b3bd)
Ruhshan pushed a commit to Ruhshan/kibana that referenced this pull request Jul 19, 2023
… test coverage (elastic#161687)

## Summary

- Implement test plan as described in
`x-pack/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/installation_and_upgrade.md`

### For maintainers

- [ ] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)

---------

Co-authored-by: kibanamachine <[email protected]>
jpdjere added a commit that referenced this pull request Jul 20, 2023
…rkflow test coverage (#161687) (#162268)

# Backport

This will backport the following commits from `main` to `8.9`:
- [[Security Solution] Extend Prebuilt rules install and update workflow
test coverage (#161687)](#161687)

<!--- Backport version: 8.9.7 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Juan Pablo
Djeredjian","email":"[email protected]"},"sourceCommit":{"committedDate":"2023-07-19T15:53:15Z","message":"[Security
Solution] Extend Prebuilt rules install and update workflow test
coverage (#161687)\n\n## Summary\r\n\r\n- Implement test plan as
described
in\r\n`x-pack/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/installation_and_upgrade.md`\r\n\r\n###
For maintainers\r\n\r\n- [ ] This was checked for breaking API changes
and was
[labeled\r\nappropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\r\n\r\n---------\r\n\r\nCo-authored-by:
kibanamachine
<[email protected]>","sha":"887b3bde0598037256ae0bfc0e1eb56a4cd72fbe","branchLabelMapping":{"^v8.10.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["test","release_note:skip","Team:Detections
and Resp","Team: SecuritySolution","Team:Detection Rule
Management","Feature:Prebuilt Detection
Rules","v8.9.0","v8.10.0"],"number":161687,"url":"https://github.com/elastic/kibana/pull/161687","mergeCommit":{"message":"[Security
Solution] Extend Prebuilt rules install and update workflow test
coverage (#161687)\n\n## Summary\r\n\r\n- Implement test plan as
described
in\r\n`x-pack/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/installation_and_upgrade.md`\r\n\r\n###
For maintainers\r\n\r\n- [ ] This was checked for breaking API changes
and was
[labeled\r\nappropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\r\n\r\n---------\r\n\r\nCo-authored-by:
kibanamachine
<[email protected]>","sha":"887b3bde0598037256ae0bfc0e1eb56a4cd72fbe"}},"sourceBranch":"main","suggestedTargetBranches":["8.9"],"targetPullRequestStates":[{"branch":"8.9","label":"v8.9.0","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v8.10.0","labelRegex":"^v8.10.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/161687","number":161687,"mergeCommit":{"message":"[Security
Solution] Extend Prebuilt rules install and update workflow test
coverage (#161687)\n\n## Summary\r\n\r\n- Implement test plan as
described
in\r\n`x-pack/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/installation_and_upgrade.md`\r\n\r\n###
For maintainers\r\n\r\n- [ ] This was checked for breaking API changes
and was
[labeled\r\nappropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)\r\n\r\n---------\r\n\r\nCo-authored-by:
kibanamachine
<[email protected]>","sha":"887b3bde0598037256ae0bfc0e1eb56a4cd72fbe"}}]}]
BACKPORT-->
@mistic
Copy link
Member

mistic commented Jul 25, 2023

This PR didn't make it into the latest BC of 8.9.0. Updating the labels.

@mistic mistic added v8.9.1 and removed v8.9.0 labels Jul 25, 2023
ThomThomson pushed a commit to ThomThomson/kibana that referenced this pull request Aug 1, 2023
… test coverage (elastic#161687)

## Summary

- Implement test plan as described in
`x-pack/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/installation_and_upgrade.md`

### For maintainers

- [ ] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)

---------

Co-authored-by: kibanamachine <[email protected]>
jpdjere added a commit that referenced this pull request Oct 17, 2023
… refactor (#165488)

Fixes: #148192

(Tick the two open checkboxes in that issue when merging this PR)

## Summary

This PR rewrites/refactors Cypress tests for the Installation and
Upgrade of Prebuilt Rules implemented in
#161687. Most of the changes here
address feedback received in that PR - answered those comments there.

- RBAC/Authorization: adds tests scenarios for users with full
privileges (happy path)
- Gets rid of huge util helpers such as
`assertRuleAvailableForInstallAndInstallOne` and rewrites test cases in
a more descriptive way, with step by step actions.
- Gets rid of complex logic in tests and their helpers - removing
if/else logic within them and removing optional flags passed to helpers.
- Fixes `bulkCreateRuleAssets` util and uses it in other helpers to
install multiple `security-rule` assets with a single bulk request to
ES.

Additionally: checked `installation_and_upgrade.md` test plan to make
sure it matches with the test in place. Added
[link](#166215) to a ticket for
a to-do task for the sections:
- Rule installation workflow: filtering, sorting, pagination
- Rule upgrade workflow: filtering, sorting, pagination

## Flaky test runner


~~https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3420~~
~~🟢~~

https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3513

---------

Co-authored-by: kibanamachine <[email protected]>
jpdjere added a commit to jpdjere/kibana that referenced this pull request Oct 17, 2023
… refactor (elastic#165488)

Fixes: elastic#148192

(Tick the two open checkboxes in that issue when merging this PR)

## Summary

This PR rewrites/refactors Cypress tests for the Installation and
Upgrade of Prebuilt Rules implemented in
elastic#161687. Most of the changes here
address feedback received in that PR - answered those comments there.

- RBAC/Authorization: adds tests scenarios for users with full
privileges (happy path)
- Gets rid of huge util helpers such as
`assertRuleAvailableForInstallAndInstallOne` and rewrites test cases in
a more descriptive way, with step by step actions.
- Gets rid of complex logic in tests and their helpers - removing
if/else logic within them and removing optional flags passed to helpers.
- Fixes `bulkCreateRuleAssets` util and uses it in other helpers to
install multiple `security-rule` assets with a single bulk request to
ES.

Additionally: checked `installation_and_upgrade.md` test plan to make
sure it matches with the test in place. Added
[link](elastic#166215) to a ticket for
a to-do task for the sections:
- Rule installation workflow: filtering, sorting, pagination
- Rule upgrade workflow: filtering, sorting, pagination

## Flaky test runner

~~https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3420~~
~~🟢~~

https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3513

---------

Co-authored-by: kibanamachine <[email protected]>
(cherry picked from commit 24c008b)

# Conflicts:
#	x-pack/test/security_solution_cypress/package.json
jpdjere added a commit that referenced this pull request Oct 17, 2023
…ntation refactor (#165488) (#169129)

# Backport

This will backport the following commits from `main` to `8.11`:
- [[Security Solution] Install/Update Prebuilt Rules Test Implementation
refactor (#165488)](#165488)

<!--- Backport version: 8.9.8 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Juan Pablo
Djeredjian","email":"[email protected]"},"sourceCommit":{"committedDate":"2023-10-17T11:05:49Z","message":"[Security
Solution] Install/Update Prebuilt Rules Test Implementation refactor
(#165488)\n\nFixes:
https://github.com/elastic/kibana/issues/148192\r\n\r\n(Tick the two
open checkboxes in that issue when merging this PR)\r\n\r\n##
Summary\r\n\r\nThis PR rewrites/refactors Cypress tests for the
Installation and\r\nUpgrade of Prebuilt Rules implemented
in\r\nhttps://github.com//pull/161687. Most of the changes
here\r\naddress feedback received in that PR - answered those comments
there.\r\n\r\n- RBAC/Authorization: adds tests scenarios for users with
full\r\nprivileges (happy path)\r\n- Gets rid of huge util helpers such
as\r\n`assertRuleAvailableForInstallAndInstallOne` and rewrites test
cases in\r\na more descriptive way, with step by step actions.\r\n- Gets
rid of complex logic in tests and their helpers - removing\r\nif/else
logic within them and removing optional flags passed to helpers.\r\n-
Fixes `bulkCreateRuleAssets` util and uses it in other helpers
to\r\ninstall multiple `security-rule` assets with a single bulk request
to\r\nES.\r\n\r\nAdditionally: checked `installation_and_upgrade.md`
test plan to make\r\nsure it matches with the test in place.
Added\r\n[link](#166215) to a
ticket for\r\na to-do task for the sections:\r\n- Rule installation
workflow: filtering, sorting, pagination\r\n- Rule upgrade workflow:
filtering, sorting, pagination\r\n\r\n## Flaky test
runner\r\n\r\n\r\n~~https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3420~~\r\n~~🟢~~\r\n\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3513\r\n\r\n---------\r\n\r\nCo-authored-by:
kibanamachine
<[email protected]>","sha":"24c008b4c5026dd543f1b4aded94f3787bce5fb0","branchLabelMapping":{"^v8.12.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["refactoring","release_note:skip","test-coverage","Team:Detections
and Resp","Team: SecuritySolution","Team:Detection Rule
Management","Feature:Prebuilt Detection
Rules","v8.11.0","v8.12.0"],"number":165488,"url":"https://github.com/elastic/kibana/pull/165488","mergeCommit":{"message":"[Security
Solution] Install/Update Prebuilt Rules Test Implementation refactor
(#165488)\n\nFixes:
https://github.com/elastic/kibana/issues/148192\r\n\r\n(Tick the two
open checkboxes in that issue when merging this PR)\r\n\r\n##
Summary\r\n\r\nThis PR rewrites/refactors Cypress tests for the
Installation and\r\nUpgrade of Prebuilt Rules implemented
in\r\nhttps://github.com//pull/161687. Most of the changes
here\r\naddress feedback received in that PR - answered those comments
there.\r\n\r\n- RBAC/Authorization: adds tests scenarios for users with
full\r\nprivileges (happy path)\r\n- Gets rid of huge util helpers such
as\r\n`assertRuleAvailableForInstallAndInstallOne` and rewrites test
cases in\r\na more descriptive way, with step by step actions.\r\n- Gets
rid of complex logic in tests and their helpers - removing\r\nif/else
logic within them and removing optional flags passed to helpers.\r\n-
Fixes `bulkCreateRuleAssets` util and uses it in other helpers
to\r\ninstall multiple `security-rule` assets with a single bulk request
to\r\nES.\r\n\r\nAdditionally: checked `installation_and_upgrade.md`
test plan to make\r\nsure it matches with the test in place.
Added\r\n[link](#166215) to a
ticket for\r\na to-do task for the sections:\r\n- Rule installation
workflow: filtering, sorting, pagination\r\n- Rule upgrade workflow:
filtering, sorting, pagination\r\n\r\n## Flaky test
runner\r\n\r\n\r\n~~https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3420~~\r\n~~🟢~~\r\n\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3513\r\n\r\n---------\r\n\r\nCo-authored-by:
kibanamachine
<[email protected]>","sha":"24c008b4c5026dd543f1b4aded94f3787bce5fb0"}},"sourceBranch":"main","suggestedTargetBranches":["8.11"],"targetPullRequestStates":[{"branch":"8.11","label":"v8.11.0","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v8.12.0","labelRegex":"^v8.12.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/165488","number":165488,"mergeCommit":{"message":"[Security
Solution] Install/Update Prebuilt Rules Test Implementation refactor
(#165488)\n\nFixes:
https://github.com/elastic/kibana/issues/148192\r\n\r\n(Tick the two
open checkboxes in that issue when merging this PR)\r\n\r\n##
Summary\r\n\r\nThis PR rewrites/refactors Cypress tests for the
Installation and\r\nUpgrade of Prebuilt Rules implemented
in\r\nhttps://github.com//pull/161687. Most of the changes
here\r\naddress feedback received in that PR - answered those comments
there.\r\n\r\n- RBAC/Authorization: adds tests scenarios for users with
full\r\nprivileges (happy path)\r\n- Gets rid of huge util helpers such
as\r\n`assertRuleAvailableForInstallAndInstallOne` and rewrites test
cases in\r\na more descriptive way, with step by step actions.\r\n- Gets
rid of complex logic in tests and their helpers - removing\r\nif/else
logic within them and removing optional flags passed to helpers.\r\n-
Fixes `bulkCreateRuleAssets` util and uses it in other helpers
to\r\ninstall multiple `security-rule` assets with a single bulk request
to\r\nES.\r\n\r\nAdditionally: checked `installation_and_upgrade.md`
test plan to make\r\nsure it matches with the test in place.
Added\r\n[link](#166215) to a
ticket for\r\na to-do task for the sections:\r\n- Rule installation
workflow: filtering, sorting, pagination\r\n- Rule upgrade workflow:
filtering, sorting, pagination\r\n\r\n## Flaky test
runner\r\n\r\n\r\n~~https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3420~~\r\n~~🟢~~\r\n\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3513\r\n\r\n---------\r\n\r\nCo-authored-by:
kibanamachine
<[email protected]>","sha":"24c008b4c5026dd543f1b4aded94f3787bce5fb0"}}]}]
BACKPORT-->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Feature:Prebuilt Detection Rules Security Solution Prebuilt Detection Rules release_note:skip Skip the PR/issue when compiling release notes Team:Detection Rule Management Security Detection Rule Management Team Team:Detections and Resp Security Detection Response Team Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. test v8.9.1 v8.10.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants