[Security Solution] Implement prebuilt rules upgrade test plan #228620
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
maximpn
added
test
release_note:skip
maximpn
force-pushed
the
implement-upgrade-prebuilt-rules-test-plan
branch
10 times, most recently
from
1cd5a78 to
6d0e860
Compare
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this pull request
Jul 31, 2025
…ic#228620) **Addresses: elastic#202078 ## Summary This PR implements Prebuilt Rules Upgrade [with (via Prebuilt Rule Upgrade Flyout)](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_with_preview.md) and [without (via Prebuilt Rules Upgrade table)](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_without_preview.md) preview test plans. ## Details Since some of the test scenarios had been implemented earlier this PR involves refactoring as well and tests reorganization. The following has been done in the scope - Prebuilt Rules Installation and Prebuilt Rules Upgrade workflows in Cypress have been split - Existing tests were renamed to better match to the test plans - Prebuilt Rules Upgrade Cypress tests were reorganized to - `upgrade_with_preview.cy.ts` - Upgrade via Prebuilt Rule Upgrade Flyout - `upgrade_without_preview.cy.ts` - Upgrade via via Prebuilt Rules Upgrade table - Missing tests have been added to the corresponding files for Jest Integration and Cypress tests ## Not implemented test scenarios The following test scenarios weren't implemented due to time constraints - [Prebuilt rules upgrade with preview test plan](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_with_preview.md#concurrency-control) (Upgrade via Prebuilt Rule Upgrade Flyout) - **Concurrency control** e2e tests The tests verify that a toast message appears after either prebuilt rule has been edited (revision bump) or a new prebuilt rule version is available (version bump, a new prebuilt rules package version has been installed). Since UI refetches data every 5 minutes e2e tests either has to wait for ~5 minutes for toasts to appear or use `cy.clock()` to mock timers. The latter doesn't work in a simple way and requires time for deeper investigation. But there are [unit tests](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/public/detection_engine/rule_management_ui/components/rules_table/upgrade_prebuilt_rules_table/use_prebuilt_rules_upgrade_state.test.ts) for concurrency control. - [Prebuilt rules upgrade without preview test plan](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_without_preview.md) (Prebuilt Rules Bulk upgrade) - **Rule upgrade workflow: filtering, sorting, pagination** There are two existing table filtering tests but besides that no new tests has been added. These tests should be covered in the scope of elastic#166215. - **Rule upgrade workflow: Edge cases**/**Scenario: Rule bound data is preserved after upgrading a rule to a newer version** - **Error handling**/**Scenario: Error is handled when any upgrade operation on prebuilt rules fails** - **Licensing: API endpoints** ## Flaky test runner - ✅ [e2e Cypress tests Rule Management group](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/8812) (100 runs) (cherry picked from commit 67006ad)
Contributor
💔 Some backports could not be created
Note: Successful backport PRs will be merged automatically after passing CI. Manual backportTo create the backport manually run: Questions ?Please refer to the Backport tool documentation |
kibanamachine
added a commit
that referenced
this pull request
Jul 31, 2025
…228620) (#230094) # Backport This will backport the following commits from `main` to `9.1`: - [[Security Solution] Implement prebuilt rules upgrade test plan (#228620)](#228620) <!--- Backport version: 9.6.6 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport) <!--BACKPORT [{"author":{"name":"Maxim Palenov","email":"maxim.palenov@elastic.co"},"sourceCommit":{"committedDate":"2025-07-31T15:22:02Z","message":"[Security Solution] Implement prebuilt rules upgrade test plan (#228620)\n\n**Addresses: https://github.com/elastic/kibana/issues/202078**\n\n## Summary\n\nThis PR implements Prebuilt Rules Upgrade [with (via Prebuilt Rule Upgrade Flyout)](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_with_preview.md) and [without (via Prebuilt Rules Upgrade table)](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_without_preview.md) preview test plans.\n\n## Details\n\nSince some of the test scenarios had been implemented earlier this PR involves refactoring as well and tests reorganization. The following has been done in the scope\n\n- Prebuilt Rules Installation and Prebuilt Rules Upgrade workflows in Cypress have been split\n- Existing tests were renamed to better match to the test plans\n- Prebuilt Rules Upgrade Cypress tests were reorganized to\n - `upgrade_with_preview.cy.ts` - Upgrade via Prebuilt Rule Upgrade Flyout\n - `upgrade_without_preview.cy.ts` - Upgrade via via Prebuilt Rules Upgrade table\n - Missing tests have been added to the corresponding files for Jest Integration and Cypress tests\n\n## Not implemented test scenarios\n\nThe following test scenarios weren't implemented due to time constraints\n\n- [Prebuilt rules upgrade with preview test plan](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_with_preview.md#concurrency-control) (Upgrade via Prebuilt Rule Upgrade Flyout)\n - **Concurrency control** e2e tests\n The tests verify that a toast message appears after either prebuilt rule has been edited (revision bump) or a new prebuilt rule version is available (version bump, a new prebuilt rules package version has been installed). Since UI refetches data every 5 minutes e2e tests either has to wait for ~5 minutes for toasts to appear or use `cy.clock()` to mock timers. The latter doesn't work in a simple way and requires time for deeper investigation.\n\n But there are [unit tests](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/public/detection_engine/rule_management_ui/components/rules_table/upgrade_prebuilt_rules_table/use_prebuilt_rules_upgrade_state.test.ts) for concurrency control.\n\n- [Prebuilt rules upgrade without preview test plan](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_without_preview.md) (Prebuilt Rules Bulk upgrade)\n - **Rule upgrade workflow: filtering, sorting, pagination**\n There are two existing table filtering tests but besides that no new tests has been added. These tests should be covered in the scope of https://github.com/elastic/kibana/issues/166215.\n - **Rule upgrade workflow: Edge cases**/**Scenario: Rule bound data is preserved after upgrading a rule to a newer version**\n - **Error handling**/**Scenario: Error is handled when any upgrade operation on prebuilt rules fails**\n - **Licensing: API endpoints**\n\n## Flaky test runner\n\n- ✅ [e2e Cypress tests Rule Management group](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/8812) (100 runs)","sha":"67006ad7844d956a38eb77ffe2e09979ec59eeca","branchLabelMapping":{"^v9.2.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["test","release_note:skip","impact:high","v9.0.0","Team:Detections and Resp","Team: SecuritySolution","Team:Detection Rule Management","Feature:Prebuilt Detection Rules","backport:version","v8.18.0","v9.1.0","v8.19.0","v9.2.0"],"title":"[Security Solution] Implement prebuilt rules upgrade test plan","number":228620,"url":"https://github.com/elastic/kibana/pull/228620","mergeCommit":{"message":"[Security Solution] Implement prebuilt rules upgrade test plan (#228620)\n\n**Addresses: https://github.com/elastic/kibana/issues/202078**\n\n## Summary\n\nThis PR implements Prebuilt Rules Upgrade [with (via Prebuilt Rule Upgrade Flyout)](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_with_preview.md) and [without (via Prebuilt Rules Upgrade table)](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_without_preview.md) preview test plans.\n\n## Details\n\nSince some of the test scenarios had been implemented earlier this PR involves refactoring as well and tests reorganization. The following has been done in the scope\n\n- Prebuilt Rules Installation and Prebuilt Rules Upgrade workflows in Cypress have been split\n- Existing tests were renamed to better match to the test plans\n- Prebuilt Rules Upgrade Cypress tests were reorganized to\n - `upgrade_with_preview.cy.ts` - Upgrade via Prebuilt Rule Upgrade Flyout\n - `upgrade_without_preview.cy.ts` - Upgrade via via Prebuilt Rules Upgrade table\n - Missing tests have been added to the corresponding files for Jest Integration and Cypress tests\n\n## Not implemented test scenarios\n\nThe following test scenarios weren't implemented due to time constraints\n\n- [Prebuilt rules upgrade with preview test plan](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_with_preview.md#concurrency-control) (Upgrade via Prebuilt Rule Upgrade Flyout)\n - **Concurrency control** e2e tests\n The tests verify that a toast message appears after either prebuilt rule has been edited (revision bump) or a new prebuilt rule version is available (version bump, a new prebuilt rules package version has been installed). Since UI refetches data every 5 minutes e2e tests either has to wait for ~5 minutes for toasts to appear or use `cy.clock()` to mock timers. The latter doesn't work in a simple way and requires time for deeper investigation.\n\n But there are [unit tests](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/public/detection_engine/rule_management_ui/components/rules_table/upgrade_prebuilt_rules_table/use_prebuilt_rules_upgrade_state.test.ts) for concurrency control.\n\n- [Prebuilt rules upgrade without preview test plan](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_without_preview.md) (Prebuilt Rules Bulk upgrade)\n - **Rule upgrade workflow: filtering, sorting, pagination**\n There are two existing table filtering tests but besides that no new tests has been added. These tests should be covered in the scope of https://github.com/elastic/kibana/issues/166215.\n - **Rule upgrade workflow: Edge cases**/**Scenario: Rule bound data is preserved after upgrading a rule to a newer version**\n - **Error handling**/**Scenario: Error is handled when any upgrade operation on prebuilt rules fails**\n - **Licensing: API endpoints**\n\n## Flaky test runner\n\n- ✅ [e2e Cypress tests Rule Management group](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/8812) (100 runs)","sha":"67006ad7844d956a38eb77ffe2e09979ec59eeca"}},"sourceBranch":"main","suggestedTargetBranches":["9.0","8.18","9.1","8.19"],"targetPullRequestStates":[{"branch":"9.0","label":"v9.0.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"8.18","label":"v8.18.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"9.1","label":"v9.1.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"8.19","label":"v8.19.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v9.2.0","branchLabelMappingKey":"^v9.2.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/228620","number":228620,"mergeCommit":{"message":"[Security Solution] Implement prebuilt rules upgrade test plan (#228620)\n\n**Addresses: https://github.com/elastic/kibana/issues/202078**\n\n## Summary\n\nThis PR implements Prebuilt Rules Upgrade [with (via Prebuilt Rule Upgrade Flyout)](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_with_preview.md) and [without (via Prebuilt Rules Upgrade table)](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_without_preview.md) preview test plans.\n\n## Details\n\nSince some of the test scenarios had been implemented earlier this PR involves refactoring as well and tests reorganization. The following has been done in the scope\n\n- Prebuilt Rules Installation and Prebuilt Rules Upgrade workflows in Cypress have been split\n- Existing tests were renamed to better match to the test plans\n- Prebuilt Rules Upgrade Cypress tests were reorganized to\n - `upgrade_with_preview.cy.ts` - Upgrade via Prebuilt Rule Upgrade Flyout\n - `upgrade_without_preview.cy.ts` - Upgrade via via Prebuilt Rules Upgrade table\n - Missing tests have been added to the corresponding files for Jest Integration and Cypress tests\n\n## Not implemented test scenarios\n\nThe following test scenarios weren't implemented due to time constraints\n\n- [Prebuilt rules upgrade with preview test plan](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_with_preview.md#concurrency-control) (Upgrade via Prebuilt Rule Upgrade Flyout)\n - **Concurrency control** e2e tests\n The tests verify that a toast message appears after either prebuilt rule has been edited (revision bump) or a new prebuilt rule version is available (version bump, a new prebuilt rules package version has been installed). Since UI refetches data every 5 minutes e2e tests either has to wait for ~5 minutes for toasts to appear or use `cy.clock()` to mock timers. The latter doesn't work in a simple way and requires time for deeper investigation.\n\n But there are [unit tests](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/public/detection_engine/rule_management_ui/components/rules_table/upgrade_prebuilt_rules_table/use_prebuilt_rules_upgrade_state.test.ts) for concurrency control.\n\n- [Prebuilt rules upgrade without preview test plan](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_without_preview.md) (Prebuilt Rules Bulk upgrade)\n - **Rule upgrade workflow: filtering, sorting, pagination**\n There are two existing table filtering tests but besides that no new tests has been added. These tests should be covered in the scope of https://github.com/elastic/kibana/issues/166215.\n - **Rule upgrade workflow: Edge cases**/**Scenario: Rule bound data is preserved after upgrading a rule to a newer version**\n - **Error handling**/**Scenario: Error is handled when any upgrade operation on prebuilt rules fails**\n - **Licensing: API endpoints**\n\n## Flaky test runner\n\n- ✅ [e2e Cypress tests Rule Management group](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/8812) (100 runs)","sha":"67006ad7844d956a38eb77ffe2e09979ec59eeca"}}]}] BACKPORT--> Co-authored-by: Maxim Palenov <maxim.palenov@elastic.co>
kibanamachine
added a commit
that referenced
this pull request
Jul 31, 2025
…#228620) (#230093) # Backport This will backport the following commits from `main` to `8.19`: - [[Security Solution] Implement prebuilt rules upgrade test plan (#228620)](#228620) <!--- Backport version: 9.6.6 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport) <!--BACKPORT [{"author":{"name":"Maxim Palenov","email":"maxim.palenov@elastic.co"},"sourceCommit":{"committedDate":"2025-07-31T15:22:02Z","message":"[Security Solution] Implement prebuilt rules upgrade test plan (#228620)\n\n**Addresses: https://github.com/elastic/kibana/issues/202078**\n\n## Summary\n\nThis PR implements Prebuilt Rules Upgrade [with (via Prebuilt Rule Upgrade Flyout)](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_with_preview.md) and [without (via Prebuilt Rules Upgrade table)](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_without_preview.md) preview test plans.\n\n## Details\n\nSince some of the test scenarios had been implemented earlier this PR involves refactoring as well and tests reorganization. The following has been done in the scope\n\n- Prebuilt Rules Installation and Prebuilt Rules Upgrade workflows in Cypress have been split\n- Existing tests were renamed to better match to the test plans\n- Prebuilt Rules Upgrade Cypress tests were reorganized to\n - `upgrade_with_preview.cy.ts` - Upgrade via Prebuilt Rule Upgrade Flyout\n - `upgrade_without_preview.cy.ts` - Upgrade via via Prebuilt Rules Upgrade table\n - Missing tests have been added to the corresponding files for Jest Integration and Cypress tests\n\n## Not implemented test scenarios\n\nThe following test scenarios weren't implemented due to time constraints\n\n- [Prebuilt rules upgrade with preview test plan](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_with_preview.md#concurrency-control) (Upgrade via Prebuilt Rule Upgrade Flyout)\n - **Concurrency control** e2e tests\n The tests verify that a toast message appears after either prebuilt rule has been edited (revision bump) or a new prebuilt rule version is available (version bump, a new prebuilt rules package version has been installed). Since UI refetches data every 5 minutes e2e tests either has to wait for ~5 minutes for toasts to appear or use `cy.clock()` to mock timers. The latter doesn't work in a simple way and requires time for deeper investigation.\n\n But there are [unit tests](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/public/detection_engine/rule_management_ui/components/rules_table/upgrade_prebuilt_rules_table/use_prebuilt_rules_upgrade_state.test.ts) for concurrency control.\n\n- [Prebuilt rules upgrade without preview test plan](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_without_preview.md) (Prebuilt Rules Bulk upgrade)\n - **Rule upgrade workflow: filtering, sorting, pagination**\n There are two existing table filtering tests but besides that no new tests has been added. These tests should be covered in the scope of https://github.com/elastic/kibana/issues/166215.\n - **Rule upgrade workflow: Edge cases**/**Scenario: Rule bound data is preserved after upgrading a rule to a newer version**\n - **Error handling**/**Scenario: Error is handled when any upgrade operation on prebuilt rules fails**\n - **Licensing: API endpoints**\n\n## Flaky test runner\n\n- ✅ [e2e Cypress tests Rule Management group](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/8812) (100 runs)","sha":"67006ad7844d956a38eb77ffe2e09979ec59eeca","branchLabelMapping":{"^v9.2.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["test","release_note:skip","impact:high","v9.0.0","Team:Detections and Resp","Team: SecuritySolution","Team:Detection Rule Management","Feature:Prebuilt Detection Rules","backport:version","v8.18.0","v9.1.0","v8.19.0","v9.2.0"],"title":"[Security Solution] Implement prebuilt rules upgrade test plan","number":228620,"url":"https://github.com/elastic/kibana/pull/228620","mergeCommit":{"message":"[Security Solution] Implement prebuilt rules upgrade test plan (#228620)\n\n**Addresses: https://github.com/elastic/kibana/issues/202078**\n\n## Summary\n\nThis PR implements Prebuilt Rules Upgrade [with (via Prebuilt Rule Upgrade Flyout)](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_with_preview.md) and [without (via Prebuilt Rules Upgrade table)](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_without_preview.md) preview test plans.\n\n## Details\n\nSince some of the test scenarios had been implemented earlier this PR involves refactoring as well and tests reorganization. The following has been done in the scope\n\n- Prebuilt Rules Installation and Prebuilt Rules Upgrade workflows in Cypress have been split\n- Existing tests were renamed to better match to the test plans\n- Prebuilt Rules Upgrade Cypress tests were reorganized to\n - `upgrade_with_preview.cy.ts` - Upgrade via Prebuilt Rule Upgrade Flyout\n - `upgrade_without_preview.cy.ts` - Upgrade via via Prebuilt Rules Upgrade table\n - Missing tests have been added to the corresponding files for Jest Integration and Cypress tests\n\n## Not implemented test scenarios\n\nThe following test scenarios weren't implemented due to time constraints\n\n- [Prebuilt rules upgrade with preview test plan](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_with_preview.md#concurrency-control) (Upgrade via Prebuilt Rule Upgrade Flyout)\n - **Concurrency control** e2e tests\n The tests verify that a toast message appears after either prebuilt rule has been edited (revision bump) or a new prebuilt rule version is available (version bump, a new prebuilt rules package version has been installed). Since UI refetches data every 5 minutes e2e tests either has to wait for ~5 minutes for toasts to appear or use `cy.clock()` to mock timers. The latter doesn't work in a simple way and requires time for deeper investigation.\n\n But there are [unit tests](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/public/detection_engine/rule_management_ui/components/rules_table/upgrade_prebuilt_rules_table/use_prebuilt_rules_upgrade_state.test.ts) for concurrency control.\n\n- [Prebuilt rules upgrade without preview test plan](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_without_preview.md) (Prebuilt Rules Bulk upgrade)\n - **Rule upgrade workflow: filtering, sorting, pagination**\n There are two existing table filtering tests but besides that no new tests has been added. These tests should be covered in the scope of https://github.com/elastic/kibana/issues/166215.\n - **Rule upgrade workflow: Edge cases**/**Scenario: Rule bound data is preserved after upgrading a rule to a newer version**\n - **Error handling**/**Scenario: Error is handled when any upgrade operation on prebuilt rules fails**\n - **Licensing: API endpoints**\n\n## Flaky test runner\n\n- ✅ [e2e Cypress tests Rule Management group](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/8812) (100 runs)","sha":"67006ad7844d956a38eb77ffe2e09979ec59eeca"}},"sourceBranch":"main","suggestedTargetBranches":["9.0","8.18","9.1","8.19"],"targetPullRequestStates":[{"branch":"9.0","label":"v9.0.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"8.18","label":"v8.18.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"9.1","label":"v9.1.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"8.19","label":"v8.19.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v9.2.0","branchLabelMappingKey":"^v9.2.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/228620","number":228620,"mergeCommit":{"message":"[Security Solution] Implement prebuilt rules upgrade test plan (#228620)\n\n**Addresses: https://github.com/elastic/kibana/issues/202078**\n\n## Summary\n\nThis PR implements Prebuilt Rules Upgrade [with (via Prebuilt Rule Upgrade Flyout)](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_with_preview.md) and [without (via Prebuilt Rules Upgrade table)](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_without_preview.md) preview test plans.\n\n## Details\n\nSince some of the test scenarios had been implemented earlier this PR involves refactoring as well and tests reorganization. The following has been done in the scope\n\n- Prebuilt Rules Installation and Prebuilt Rules Upgrade workflows in Cypress have been split\n- Existing tests were renamed to better match to the test plans\n- Prebuilt Rules Upgrade Cypress tests were reorganized to\n - `upgrade_with_preview.cy.ts` - Upgrade via Prebuilt Rule Upgrade Flyout\n - `upgrade_without_preview.cy.ts` - Upgrade via via Prebuilt Rules Upgrade table\n - Missing tests have been added to the corresponding files for Jest Integration and Cypress tests\n\n## Not implemented test scenarios\n\nThe following test scenarios weren't implemented due to time constraints\n\n- [Prebuilt rules upgrade with preview test plan](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_with_preview.md#concurrency-control) (Upgrade via Prebuilt Rule Upgrade Flyout)\n - **Concurrency control** e2e tests\n The tests verify that a toast message appears after either prebuilt rule has been edited (revision bump) or a new prebuilt rule version is available (version bump, a new prebuilt rules package version has been installed). Since UI refetches data every 5 minutes e2e tests either has to wait for ~5 minutes for toasts to appear or use `cy.clock()` to mock timers. The latter doesn't work in a simple way and requires time for deeper investigation.\n\n But there are [unit tests](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/public/detection_engine/rule_management_ui/components/rules_table/upgrade_prebuilt_rules_table/use_prebuilt_rules_upgrade_state.test.ts) for concurrency control.\n\n- [Prebuilt rules upgrade without preview test plan](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_without_preview.md) (Prebuilt Rules Bulk upgrade)\n - **Rule upgrade workflow: filtering, sorting, pagination**\n There are two existing table filtering tests but besides that no new tests has been added. These tests should be covered in the scope of https://github.com/elastic/kibana/issues/166215.\n - **Rule upgrade workflow: Edge cases**/**Scenario: Rule bound data is preserved after upgrading a rule to a newer version**\n - **Error handling**/**Scenario: Error is handled when any upgrade operation on prebuilt rules fails**\n - **Licensing: API endpoints**\n\n## Flaky test runner\n\n- ✅ [e2e Cypress tests Rule Management group](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/8812) (100 runs)","sha":"67006ad7844d956a38eb77ffe2e09979ec59eeca"}}]}] BACKPORT--> Co-authored-by: Maxim Palenov <maxim.palenov@elastic.co>
maximpn
added a commit
to maximpn/kibana
that referenced
this pull request
Jul 31, 2025
…ic#228620) **Addresses: elastic#202078 ## Summary This PR implements Prebuilt Rules Upgrade [with (via Prebuilt Rule Upgrade Flyout)](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_with_preview.md) and [without (via Prebuilt Rules Upgrade table)](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_without_preview.md) preview test plans. ## Details Since some of the test scenarios had been implemented earlier this PR involves refactoring as well and tests reorganization. The following has been done in the scope - Prebuilt Rules Installation and Prebuilt Rules Upgrade workflows in Cypress have been split - Existing tests were renamed to better match to the test plans - Prebuilt Rules Upgrade Cypress tests were reorganized to - `upgrade_with_preview.cy.ts` - Upgrade via Prebuilt Rule Upgrade Flyout - `upgrade_without_preview.cy.ts` - Upgrade via via Prebuilt Rules Upgrade table - Missing tests have been added to the corresponding files for Jest Integration and Cypress tests ## Not implemented test scenarios The following test scenarios weren't implemented due to time constraints - [Prebuilt rules upgrade with preview test plan](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_with_preview.md#concurrency-control) (Upgrade via Prebuilt Rule Upgrade Flyout) - **Concurrency control** e2e tests The tests verify that a toast message appears after either prebuilt rule has been edited (revision bump) or a new prebuilt rule version is available (version bump, a new prebuilt rules package version has been installed). Since UI refetches data every 5 minutes e2e tests either has to wait for ~5 minutes for toasts to appear or use `cy.clock()` to mock timers. The latter doesn't work in a simple way and requires time for deeper investigation. But there are [unit tests](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/public/detection_engine/rule_management_ui/components/rules_table/upgrade_prebuilt_rules_table/use_prebuilt_rules_upgrade_state.test.ts) for concurrency control. - [Prebuilt rules upgrade without preview test plan](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_without_preview.md) (Prebuilt Rules Bulk upgrade) - **Rule upgrade workflow: filtering, sorting, pagination** There are two existing table filtering tests but besides that no new tests has been added. These tests should be covered in the scope of elastic#166215. - **Rule upgrade workflow: Edge cases**/**Scenario: Rule bound data is preserved after upgrading a rule to a newer version** - **Error handling**/**Scenario: Error is handled when any upgrade operation on prebuilt rules fails** - **Licensing: API endpoints** ## Flaky test runner - ✅ [e2e Cypress tests Rule Management group](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/8812) (100 runs) (cherry picked from commit 67006ad) # Conflicts: # x-pack/solutions/security/plugins/security_solution/public/common/test/eui/combobox.ts
maximpn
added a commit
to maximpn/kibana
that referenced
this pull request
Jul 31, 2025
…ic#228620) **Addresses: elastic#202078 ## Summary This PR implements Prebuilt Rules Upgrade [with (via Prebuilt Rule Upgrade Flyout)](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_with_preview.md) and [without (via Prebuilt Rules Upgrade table)](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_without_preview.md) preview test plans. ## Details Since some of the test scenarios had been implemented earlier this PR involves refactoring as well and tests reorganization. The following has been done in the scope - Prebuilt Rules Installation and Prebuilt Rules Upgrade workflows in Cypress have been split - Existing tests were renamed to better match to the test plans - Prebuilt Rules Upgrade Cypress tests were reorganized to - `upgrade_with_preview.cy.ts` - Upgrade via Prebuilt Rule Upgrade Flyout - `upgrade_without_preview.cy.ts` - Upgrade via via Prebuilt Rules Upgrade table - Missing tests have been added to the corresponding files for Jest Integration and Cypress tests ## Not implemented test scenarios The following test scenarios weren't implemented due to time constraints - [Prebuilt rules upgrade with preview test plan](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_with_preview.md#concurrency-control) (Upgrade via Prebuilt Rule Upgrade Flyout) - **Concurrency control** e2e tests The tests verify that a toast message appears after either prebuilt rule has been edited (revision bump) or a new prebuilt rule version is available (version bump, a new prebuilt rules package version has been installed). Since UI refetches data every 5 minutes e2e tests either has to wait for ~5 minutes for toasts to appear or use `cy.clock()` to mock timers. The latter doesn't work in a simple way and requires time for deeper investigation. But there are [unit tests](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/public/detection_engine/rule_management_ui/components/rules_table/upgrade_prebuilt_rules_table/use_prebuilt_rules_upgrade_state.test.ts) for concurrency control. - [Prebuilt rules upgrade without preview test plan](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_without_preview.md) (Prebuilt Rules Bulk upgrade) - **Rule upgrade workflow: filtering, sorting, pagination** There are two existing table filtering tests but besides that no new tests has been added. These tests should be covered in the scope of elastic#166215. - **Rule upgrade workflow: Edge cases**/**Scenario: Rule bound data is preserved after upgrading a rule to a newer version** - **Error handling**/**Scenario: Error is handled when any upgrade operation on prebuilt rules fails** - **Licensing: API endpoints** ## Flaky test runner - ✅ [e2e Cypress tests Rule Management group](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/8812) (100 runs) (cherry picked from commit 67006ad) # Conflicts: # x-pack/solutions/security/plugins/security_solution/public/common/test/eui/combobox.ts
Contributor
Author
💚 All backports created successfully
Note: Successful backport PRs will be merged automatically after passing CI. Questions ?Please refer to the Backport tool documentation |
maximpn
added a commit
that referenced
this pull request
Aug 1, 2025
…228620) (#230114) # Backport This will backport the following commits from `main` to `9.0`: - [[Security Solution] Implement prebuilt rules upgrade test plan (#228620)](#228620) <!--- Backport version: 10.0.0 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport) <!--BACKPORT [{"author":{"name":"Maxim Palenov","email":"maxim.palenov@elastic.co"},"sourceCommit":{"committedDate":"2025-07-31T15:22:02Z","message":"[Security Solution] Implement prebuilt rules upgrade test plan (#228620)\n\n**Addresses: https://github.com/elastic/kibana/issues/202078**\n\n## Summary\n\nThis PR implements Prebuilt Rules Upgrade [with (via Prebuilt Rule Upgrade Flyout)](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_with_preview.md) and [without (via Prebuilt Rules Upgrade table)](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_without_preview.md) preview test plans.\n\n## Details\n\nSince some of the test scenarios had been implemented earlier this PR involves refactoring as well and tests reorganization. The following has been done in the scope\n\n- Prebuilt Rules Installation and Prebuilt Rules Upgrade workflows in Cypress have been split\n- Existing tests were renamed to better match to the test plans\n- Prebuilt Rules Upgrade Cypress tests were reorganized to\n - `upgrade_with_preview.cy.ts` - Upgrade via Prebuilt Rule Upgrade Flyout\n - `upgrade_without_preview.cy.ts` - Upgrade via via Prebuilt Rules Upgrade table\n - Missing tests have been added to the corresponding files for Jest Integration and Cypress tests\n\n## Not implemented test scenarios\n\nThe following test scenarios weren't implemented due to time constraints\n\n- [Prebuilt rules upgrade with preview test plan](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_with_preview.md#concurrency-control) (Upgrade via Prebuilt Rule Upgrade Flyout)\n - **Concurrency control** e2e tests\n The tests verify that a toast message appears after either prebuilt rule has been edited (revision bump) or a new prebuilt rule version is available (version bump, a new prebuilt rules package version has been installed). Since UI refetches data every 5 minutes e2e tests either has to wait for ~5 minutes for toasts to appear or use `cy.clock()` to mock timers. The latter doesn't work in a simple way and requires time for deeper investigation.\n\n But there are [unit tests](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/public/detection_engine/rule_management_ui/components/rules_table/upgrade_prebuilt_rules_table/use_prebuilt_rules_upgrade_state.test.ts) for concurrency control.\n\n- [Prebuilt rules upgrade without preview test plan](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_without_preview.md) (Prebuilt Rules Bulk upgrade)\n - **Rule upgrade workflow: filtering, sorting, pagination**\n There are two existing table filtering tests but besides that no new tests has been added. These tests should be covered in the scope of https://github.com/elastic/kibana/issues/166215.\n - **Rule upgrade workflow: Edge cases**/**Scenario: Rule bound data is preserved after upgrading a rule to a newer version**\n - **Error handling**/**Scenario: Error is handled when any upgrade operation on prebuilt rules fails**\n - **Licensing: API endpoints**\n\n## Flaky test runner\n\n- ✅ [e2e Cypress tests Rule Management group](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/8812) (100 runs)","sha":"67006ad7844d956a38eb77ffe2e09979ec59eeca","branchLabelMapping":{"^v9.2.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["test","release_note:skip","impact:high","v9.0.0","Team:Detections and Resp","Team: SecuritySolution","Team:Detection Rule Management","Feature:Prebuilt Detection Rules","backport:version","v8.18.0","v9.1.0","v8.19.0","v9.2.0","v9.1.1"],"title":"[Security Solution] Implement prebuilt rules upgrade test plan","number":228620,"url":"https://github.com/elastic/kibana/pull/228620","mergeCommit":{"message":"[Security Solution] Implement prebuilt rules upgrade test plan (#228620)\n\n**Addresses: https://github.com/elastic/kibana/issues/202078**\n\n## Summary\n\nThis PR implements Prebuilt Rules Upgrade [with (via Prebuilt Rule Upgrade Flyout)](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_with_preview.md) and [without (via Prebuilt Rules Upgrade table)](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_without_preview.md) preview test plans.\n\n## Details\n\nSince some of the test scenarios had been implemented earlier this PR involves refactoring as well and tests reorganization. The following has been done in the scope\n\n- Prebuilt Rules Installation and Prebuilt Rules Upgrade workflows in Cypress have been split\n- Existing tests were renamed to better match to the test plans\n- Prebuilt Rules Upgrade Cypress tests were reorganized to\n - `upgrade_with_preview.cy.ts` - Upgrade via Prebuilt Rule Upgrade Flyout\n - `upgrade_without_preview.cy.ts` - Upgrade via via Prebuilt Rules Upgrade table\n - Missing tests have been added to the corresponding files for Jest Integration and Cypress tests\n\n## Not implemented test scenarios\n\nThe following test scenarios weren't implemented due to time constraints\n\n- [Prebuilt rules upgrade with preview test plan](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_with_preview.md#concurrency-control) (Upgrade via Prebuilt Rule Upgrade Flyout)\n - **Concurrency control** e2e tests\n The tests verify that a toast message appears after either prebuilt rule has been edited (revision bump) or a new prebuilt rule version is available (version bump, a new prebuilt rules package version has been installed). Since UI refetches data every 5 minutes e2e tests either has to wait for ~5 minutes for toasts to appear or use `cy.clock()` to mock timers. The latter doesn't work in a simple way and requires time for deeper investigation.\n\n But there are [unit tests](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/public/detection_engine/rule_management_ui/components/rules_table/upgrade_prebuilt_rules_table/use_prebuilt_rules_upgrade_state.test.ts) for concurrency control.\n\n- [Prebuilt rules upgrade without preview test plan](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_without_preview.md) (Prebuilt Rules Bulk upgrade)\n - **Rule upgrade workflow: filtering, sorting, pagination**\n There are two existing table filtering tests but besides that no new tests has been added. These tests should be covered in the scope of https://github.com/elastic/kibana/issues/166215.\n - **Rule upgrade workflow: Edge cases**/**Scenario: Rule bound data is preserved after upgrading a rule to a newer version**\n - **Error handling**/**Scenario: Error is handled when any upgrade operation on prebuilt rules fails**\n - **Licensing: API endpoints**\n\n## Flaky test runner\n\n- ✅ [e2e Cypress tests Rule Management group](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/8812) (100 runs)","sha":"67006ad7844d956a38eb77ffe2e09979ec59eeca"}},"sourceBranch":"main","suggestedTargetBranches":["9.0","8.18"],"targetPullRequestStates":[{"branch":"9.0","label":"v9.0.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"8.18","label":"v8.18.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"9.1","label":"v9.1.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"url":"https://github.com/elastic/kibana/pull/230094","number":230094,"state":"MERGED","mergeCommit":{"sha":"72e6e89777b84c417487057664612385bb09ddc4","message":"[9.1] [Security Solution] Implement prebuilt rules upgrade test plan (#228620) (#230094)\n\n# Backport\n\nThis will backport the following commits from `main` to `9.1`:\n- [[Security Solution] Implement prebuilt rules upgrade test plan\n(#228620)](https://github.com/elastic/kibana/pull/228620)\n\n\n\n### Questions ?\nPlease refer to the [Backport tool\ndocumentation](https://github.com/sorenlouv/backport)\n\n\n\nCo-authored-by: Maxim Palenov <maxim.palenov@elastic.co>"}},{"branch":"8.19","label":"v8.19.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"url":"https://github.com/elastic/kibana/pull/230093","number":230093,"state":"MERGED","mergeCommit":{"sha":"83abce269477cb1301077081d8a7ae9516bd9797","message":"[8.19] [Security Solution] Implement prebuilt rules upgrade test plan (#228620) (#230093)\n\n# Backport\n\nThis will backport the following commits from `main` to `8.19`:\n- [[Security Solution] Implement prebuilt rules upgrade test plan\n(#228620)](https://github.com/elastic/kibana/pull/228620)\n\n\n\n### Questions ?\nPlease refer to the [Backport tool\ndocumentation](https://github.com/sorenlouv/backport)\n\n\n\nCo-authored-by: Maxim Palenov <maxim.palenov@elastic.co>"}},{"branch":"main","label":"v9.2.0","branchLabelMappingKey":"^v9.2.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/228620","number":228620,"mergeCommit":{"message":"[Security Solution] Implement prebuilt rules upgrade test plan (#228620)\n\n**Addresses: https://github.com/elastic/kibana/issues/202078**\n\n## Summary\n\nThis PR implements Prebuilt Rules Upgrade [with (via Prebuilt Rule Upgrade Flyout)](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_with_preview.md) and [without (via Prebuilt Rules Upgrade table)](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_without_preview.md) preview test plans.\n\n## Details\n\nSince some of the test scenarios had been implemented earlier this PR involves refactoring as well and tests reorganization. The following has been done in the scope\n\n- Prebuilt Rules Installation and Prebuilt Rules Upgrade workflows in Cypress have been split\n- Existing tests were renamed to better match to the test plans\n- Prebuilt Rules Upgrade Cypress tests were reorganized to\n - `upgrade_with_preview.cy.ts` - Upgrade via Prebuilt Rule Upgrade Flyout\n - `upgrade_without_preview.cy.ts` - Upgrade via via Prebuilt Rules Upgrade table\n - Missing tests have been added to the corresponding files for Jest Integration and Cypress tests\n\n## Not implemented test scenarios\n\nThe following test scenarios weren't implemented due to time constraints\n\n- [Prebuilt rules upgrade with preview test plan](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_with_preview.md#concurrency-control) (Upgrade via Prebuilt Rule Upgrade Flyout)\n - **Concurrency control** e2e tests\n The tests verify that a toast message appears after either prebuilt rule has been edited (revision bump) or a new prebuilt rule version is available (version bump, a new prebuilt rules package version has been installed). Since UI refetches data every 5 minutes e2e tests either has to wait for ~5 minutes for toasts to appear or use `cy.clock()` to mock timers. The latter doesn't work in a simple way and requires time for deeper investigation.\n\n But there are [unit tests](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/public/detection_engine/rule_management_ui/components/rules_table/upgrade_prebuilt_rules_table/use_prebuilt_rules_upgrade_state.test.ts) for concurrency control.\n\n- [Prebuilt rules upgrade without preview test plan](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_without_preview.md) (Prebuilt Rules Bulk upgrade)\n - **Rule upgrade workflow: filtering, sorting, pagination**\n There are two existing table filtering tests but besides that no new tests has been added. These tests should be covered in the scope of https://github.com/elastic/kibana/issues/166215.\n - **Rule upgrade workflow: Edge cases**/**Scenario: Rule bound data is preserved after upgrading a rule to a newer version**\n - **Error handling**/**Scenario: Error is handled when any upgrade operation on prebuilt rules fails**\n - **Licensing: API endpoints**\n\n## Flaky test runner\n\n- ✅ [e2e Cypress tests Rule Management group](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/8812) (100 runs)","sha":"67006ad7844d956a38eb77ffe2e09979ec59eeca"}}]}] BACKPORT-->
maximpn
added a commit
that referenced
this pull request
Aug 1, 2025
…#228620) (#230115) # Backport This will backport the following commits from `main` to `8.18`: - [[Security Solution] Implement prebuilt rules upgrade test plan (#228620)](#228620) <!--- Backport version: 10.0.0 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport) <!--BACKPORT [{"author":{"name":"Maxim Palenov","email":"maxim.palenov@elastic.co"},"sourceCommit":{"committedDate":"2025-07-31T15:22:02Z","message":"[Security Solution] Implement prebuilt rules upgrade test plan (#228620)\n\n**Addresses: https://github.com/elastic/kibana/issues/202078**\n\n## Summary\n\nThis PR implements Prebuilt Rules Upgrade [with (via Prebuilt Rule Upgrade Flyout)](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_with_preview.md) and [without (via Prebuilt Rules Upgrade table)](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_without_preview.md) preview test plans.\n\n## Details\n\nSince some of the test scenarios had been implemented earlier this PR involves refactoring as well and tests reorganization. The following has been done in the scope\n\n- Prebuilt Rules Installation and Prebuilt Rules Upgrade workflows in Cypress have been split\n- Existing tests were renamed to better match to the test plans\n- Prebuilt Rules Upgrade Cypress tests were reorganized to\n - `upgrade_with_preview.cy.ts` - Upgrade via Prebuilt Rule Upgrade Flyout\n - `upgrade_without_preview.cy.ts` - Upgrade via via Prebuilt Rules Upgrade table\n - Missing tests have been added to the corresponding files for Jest Integration and Cypress tests\n\n## Not implemented test scenarios\n\nThe following test scenarios weren't implemented due to time constraints\n\n- [Prebuilt rules upgrade with preview test plan](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_with_preview.md#concurrency-control) (Upgrade via Prebuilt Rule Upgrade Flyout)\n - **Concurrency control** e2e tests\n The tests verify that a toast message appears after either prebuilt rule has been edited (revision bump) or a new prebuilt rule version is available (version bump, a new prebuilt rules package version has been installed). Since UI refetches data every 5 minutes e2e tests either has to wait for ~5 minutes for toasts to appear or use `cy.clock()` to mock timers. The latter doesn't work in a simple way and requires time for deeper investigation.\n\n But there are [unit tests](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/public/detection_engine/rule_management_ui/components/rules_table/upgrade_prebuilt_rules_table/use_prebuilt_rules_upgrade_state.test.ts) for concurrency control.\n\n- [Prebuilt rules upgrade without preview test plan](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_without_preview.md) (Prebuilt Rules Bulk upgrade)\n - **Rule upgrade workflow: filtering, sorting, pagination**\n There are two existing table filtering tests but besides that no new tests has been added. These tests should be covered in the scope of https://github.com/elastic/kibana/issues/166215.\n - **Rule upgrade workflow: Edge cases**/**Scenario: Rule bound data is preserved after upgrading a rule to a newer version**\n - **Error handling**/**Scenario: Error is handled when any upgrade operation on prebuilt rules fails**\n - **Licensing: API endpoints**\n\n## Flaky test runner\n\n- ✅ [e2e Cypress tests Rule Management group](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/8812) (100 runs)","sha":"67006ad7844d956a38eb77ffe2e09979ec59eeca","branchLabelMapping":{"^v9.2.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["test","release_note:skip","impact:high","v9.0.0","Team:Detections and Resp","Team: SecuritySolution","Team:Detection Rule Management","Feature:Prebuilt Detection Rules","backport:version","v8.18.0","v9.1.0","v8.19.0","v9.2.0","v9.1.1"],"title":"[Security Solution] Implement prebuilt rules upgrade test plan","number":228620,"url":"https://github.com/elastic/kibana/pull/228620","mergeCommit":{"message":"[Security Solution] Implement prebuilt rules upgrade test plan (#228620)\n\n**Addresses: https://github.com/elastic/kibana/issues/202078**\n\n## Summary\n\nThis PR implements Prebuilt Rules Upgrade [with (via Prebuilt Rule Upgrade Flyout)](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_with_preview.md) and [without (via Prebuilt Rules Upgrade table)](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_without_preview.md) preview test plans.\n\n## Details\n\nSince some of the test scenarios had been implemented earlier this PR involves refactoring as well and tests reorganization. The following has been done in the scope\n\n- Prebuilt Rules Installation and Prebuilt Rules Upgrade workflows in Cypress have been split\n- Existing tests were renamed to better match to the test plans\n- Prebuilt Rules Upgrade Cypress tests were reorganized to\n - `upgrade_with_preview.cy.ts` - Upgrade via Prebuilt Rule Upgrade Flyout\n - `upgrade_without_preview.cy.ts` - Upgrade via via Prebuilt Rules Upgrade table\n - Missing tests have been added to the corresponding files for Jest Integration and Cypress tests\n\n## Not implemented test scenarios\n\nThe following test scenarios weren't implemented due to time constraints\n\n- [Prebuilt rules upgrade with preview test plan](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_with_preview.md#concurrency-control) (Upgrade via Prebuilt Rule Upgrade Flyout)\n - **Concurrency control** e2e tests\n The tests verify that a toast message appears after either prebuilt rule has been edited (revision bump) or a new prebuilt rule version is available (version bump, a new prebuilt rules package version has been installed). Since UI refetches data every 5 minutes e2e tests either has to wait for ~5 minutes for toasts to appear or use `cy.clock()` to mock timers. The latter doesn't work in a simple way and requires time for deeper investigation.\n\n But there are [unit tests](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/public/detection_engine/rule_management_ui/components/rules_table/upgrade_prebuilt_rules_table/use_prebuilt_rules_upgrade_state.test.ts) for concurrency control.\n\n- [Prebuilt rules upgrade without preview test plan](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_without_preview.md) (Prebuilt Rules Bulk upgrade)\n - **Rule upgrade workflow: filtering, sorting, pagination**\n There are two existing table filtering tests but besides that no new tests has been added. These tests should be covered in the scope of https://github.com/elastic/kibana/issues/166215.\n - **Rule upgrade workflow: Edge cases**/**Scenario: Rule bound data is preserved after upgrading a rule to a newer version**\n - **Error handling**/**Scenario: Error is handled when any upgrade operation on prebuilt rules fails**\n - **Licensing: API endpoints**\n\n## Flaky test runner\n\n- ✅ [e2e Cypress tests Rule Management group](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/8812) (100 runs)","sha":"67006ad7844d956a38eb77ffe2e09979ec59eeca"}},"sourceBranch":"main","suggestedTargetBranches":["9.0","8.18"],"targetPullRequestStates":[{"branch":"9.0","label":"v9.0.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"8.18","label":"v8.18.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"9.1","label":"v9.1.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"url":"https://github.com/elastic/kibana/pull/230094","number":230094,"state":"MERGED","mergeCommit":{"sha":"72e6e89777b84c417487057664612385bb09ddc4","message":"[9.1] [Security Solution] Implement prebuilt rules upgrade test plan (#228620) (#230094)\n\n# Backport\n\nThis will backport the following commits from `main` to `9.1`:\n- [[Security Solution] Implement prebuilt rules upgrade test plan\n(#228620)](https://github.com/elastic/kibana/pull/228620)\n\n\n\n### Questions ?\nPlease refer to the [Backport tool\ndocumentation](https://github.com/sorenlouv/backport)\n\n\n\nCo-authored-by: Maxim Palenov <maxim.palenov@elastic.co>"}},{"branch":"8.19","label":"v8.19.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"url":"https://github.com/elastic/kibana/pull/230093","number":230093,"state":"MERGED","mergeCommit":{"sha":"83abce269477cb1301077081d8a7ae9516bd9797","message":"[8.19] [Security Solution] Implement prebuilt rules upgrade test plan (#228620) (#230093)\n\n# Backport\n\nThis will backport the following commits from `main` to `8.19`:\n- [[Security Solution] Implement prebuilt rules upgrade test plan\n(#228620)](https://github.com/elastic/kibana/pull/228620)\n\n\n\n### Questions ?\nPlease refer to the [Backport tool\ndocumentation](https://github.com/sorenlouv/backport)\n\n\n\nCo-authored-by: Maxim Palenov <maxim.palenov@elastic.co>"}},{"branch":"main","label":"v9.2.0","branchLabelMappingKey":"^v9.2.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/228620","number":228620,"mergeCommit":{"message":"[Security Solution] Implement prebuilt rules upgrade test plan (#228620)\n\n**Addresses: https://github.com/elastic/kibana/issues/202078**\n\n## Summary\n\nThis PR implements Prebuilt Rules Upgrade [with (via Prebuilt Rule Upgrade Flyout)](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_with_preview.md) and [without (via Prebuilt Rules Upgrade table)](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_without_preview.md) preview test plans.\n\n## Details\n\nSince some of the test scenarios had been implemented earlier this PR involves refactoring as well and tests reorganization. The following has been done in the scope\n\n- Prebuilt Rules Installation and Prebuilt Rules Upgrade workflows in Cypress have been split\n- Existing tests were renamed to better match to the test plans\n- Prebuilt Rules Upgrade Cypress tests were reorganized to\n - `upgrade_with_preview.cy.ts` - Upgrade via Prebuilt Rule Upgrade Flyout\n - `upgrade_without_preview.cy.ts` - Upgrade via via Prebuilt Rules Upgrade table\n - Missing tests have been added to the corresponding files for Jest Integration and Cypress tests\n\n## Not implemented test scenarios\n\nThe following test scenarios weren't implemented due to time constraints\n\n- [Prebuilt rules upgrade with preview test plan](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_with_preview.md#concurrency-control) (Upgrade via Prebuilt Rule Upgrade Flyout)\n - **Concurrency control** e2e tests\n The tests verify that a toast message appears after either prebuilt rule has been edited (revision bump) or a new prebuilt rule version is available (version bump, a new prebuilt rules package version has been installed). Since UI refetches data every 5 minutes e2e tests either has to wait for ~5 minutes for toasts to appear or use `cy.clock()` to mock timers. The latter doesn't work in a simple way and requires time for deeper investigation.\n\n But there are [unit tests](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/public/detection_engine/rule_management_ui/components/rules_table/upgrade_prebuilt_rules_table/use_prebuilt_rules_upgrade_state.test.ts) for concurrency control.\n\n- [Prebuilt rules upgrade without preview test plan](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_without_preview.md) (Prebuilt Rules Bulk upgrade)\n - **Rule upgrade workflow: filtering, sorting, pagination**\n There are two existing table filtering tests but besides that no new tests has been added. These tests should be covered in the scope of https://github.com/elastic/kibana/issues/166215.\n - **Rule upgrade workflow: Edge cases**/**Scenario: Rule bound data is preserved after upgrading a rule to a newer version**\n - **Error handling**/**Scenario: Error is handled when any upgrade operation on prebuilt rules fails**\n - **Licensing: API endpoints**\n\n## Flaky test runner\n\n- ✅ [e2e Cypress tests Rule Management group](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/8812) (100 runs)","sha":"67006ad7844d956a38eb77ffe2e09979ec59eeca"}}]}] BACKPORT-->
delanni
pushed a commit
to delanni/kibana
that referenced
this pull request
Aug 5, 2025
…ic#228620) **Addresses: elastic#202078 ## Summary This PR implements Prebuilt Rules Upgrade [with (via Prebuilt Rule Upgrade Flyout)](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_with_preview.md) and [without (via Prebuilt Rules Upgrade table)](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_without_preview.md) preview test plans. ## Details Since some of the test scenarios had been implemented earlier this PR involves refactoring as well and tests reorganization. The following has been done in the scope - Prebuilt Rules Installation and Prebuilt Rules Upgrade workflows in Cypress have been split - Existing tests were renamed to better match to the test plans - Prebuilt Rules Upgrade Cypress tests were reorganized to - `upgrade_with_preview.cy.ts` - Upgrade via Prebuilt Rule Upgrade Flyout - `upgrade_without_preview.cy.ts` - Upgrade via via Prebuilt Rules Upgrade table - Missing tests have been added to the corresponding files for Jest Integration and Cypress tests ## Not implemented test scenarios The following test scenarios weren't implemented due to time constraints - [Prebuilt rules upgrade with preview test plan](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_with_preview.md#concurrency-control) (Upgrade via Prebuilt Rule Upgrade Flyout) - **Concurrency control** e2e tests The tests verify that a toast message appears after either prebuilt rule has been edited (revision bump) or a new prebuilt rule version is available (version bump, a new prebuilt rules package version has been installed). Since UI refetches data every 5 minutes e2e tests either has to wait for ~5 minutes for toasts to appear or use `cy.clock()` to mock timers. The latter doesn't work in a simple way and requires time for deeper investigation. But there are [unit tests](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/public/detection_engine/rule_management_ui/components/rules_table/upgrade_prebuilt_rules_table/use_prebuilt_rules_upgrade_state.test.ts) for concurrency control. - [Prebuilt rules upgrade without preview test plan](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_without_preview.md) (Prebuilt Rules Bulk upgrade) - **Rule upgrade workflow: filtering, sorting, pagination** There are two existing table filtering tests but besides that no new tests has been added. These tests should be covered in the scope of elastic#166215. - **Rule upgrade workflow: Edge cases**/**Scenario: Rule bound data is preserved after upgrading a rule to a newer version** - **Error handling**/**Scenario: Error is handled when any upgrade operation on prebuilt rules fails** - **Licensing: API endpoints** ## Flaky test runner - ✅ [e2e Cypress tests Rule Management group](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/8812) (100 runs)
maximpn
added a commit
that referenced
this pull request
Sep 1, 2025
…232758) **Resolves: #220081 ## Summary This PR unskips skipped earlier Prebuilt Rule Upgrade workflow ML jobs field Jest integration test. ## Details The Jest integration test `Upgrade diffable rule "machine_learning_job_id" (machine_learning rule type) after preview in flyout customized field w/ an upgrade resulting in a non-solvable conflict (ABC) upgrades rule to resolved value` was skipped earlier in `9.1` and `9.2` due to flakiness. Since that moment combobox integration flakiness has been addressed in the scope of #228620. Thorough testing in `main` (corresponds to `9.2`) and `9.1` branches hasn't revealed any failures or flakiness. On top of that this test runs in `8.18` and `8.19` smoothly all that time. Taking the above into account it should be safe to unskip the skipped test. In case if the flakiness pops up we'll have more data to analyze.
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this pull request
Sep 1, 2025
…lastic#232758) **Resolves: elastic#220081 ## Summary This PR unskips skipped earlier Prebuilt Rule Upgrade workflow ML jobs field Jest integration test. ## Details The Jest integration test `Upgrade diffable rule "machine_learning_job_id" (machine_learning rule type) after preview in flyout customized field w/ an upgrade resulting in a non-solvable conflict (ABC) upgrades rule to resolved value` was skipped earlier in `9.1` and `9.2` due to flakiness. Since that moment combobox integration flakiness has been addressed in the scope of elastic#228620. Thorough testing in `main` (corresponds to `9.2`) and `9.1` branches hasn't revealed any failures or flakiness. On top of that this test runs in `8.18` and `8.19` smoothly all that time. Taking the above into account it should be safe to unskip the skipped test. In case if the flakiness pops up we'll have more data to analyze. (cherry picked from commit f899266)
ymao1
pushed a commit
to ymao1/kibana
that referenced
this pull request
Sep 2, 2025
…lastic#232758) **Resolves: elastic#220081 ## Summary This PR unskips skipped earlier Prebuilt Rule Upgrade workflow ML jobs field Jest integration test. ## Details The Jest integration test `Upgrade diffable rule "machine_learning_job_id" (machine_learning rule type) after preview in flyout customized field w/ an upgrade resulting in a non-solvable conflict (ABC) upgrades rule to resolved value` was skipped earlier in `9.1` and `9.2` due to flakiness. Since that moment combobox integration flakiness has been addressed in the scope of elastic#228620. Thorough testing in `main` (corresponds to `9.2`) and `9.1` branches hasn't revealed any failures or flakiness. On top of that this test runs in `8.18` and `8.19` smoothly all that time. Taking the above into account it should be safe to unskip the skipped test. In case if the flakiness pops up we'll have more data to analyze.
MichelLosier
pushed a commit
to MichelLosier/kibana
that referenced
this pull request
Sep 2, 2025
…lastic#232758) **Resolves: elastic#220081 ## Summary This PR unskips skipped earlier Prebuilt Rule Upgrade workflow ML jobs field Jest integration test. ## Details The Jest integration test `Upgrade diffable rule "machine_learning_job_id" (machine_learning rule type) after preview in flyout customized field w/ an upgrade resulting in a non-solvable conflict (ABC) upgrades rule to resolved value` was skipped earlier in `9.1` and `9.2` due to flakiness. Since that moment combobox integration flakiness has been addressed in the scope of elastic#228620. Thorough testing in `main` (corresponds to `9.2`) and `9.1` branches hasn't revealed any failures or flakiness. On top of that this test runs in `8.18` and `8.19` smoothly all that time. Taking the above into account it should be safe to unskip the skipped test. In case if the flakiness pops up we'll have more data to analyze.
kowalczyk-krzysztof
pushed a commit
to kowalczyk-krzysztof/kibana
that referenced
this pull request
Sep 3, 2025
…lastic#232758) **Resolves: elastic#220081 ## Summary This PR unskips skipped earlier Prebuilt Rule Upgrade workflow ML jobs field Jest integration test. ## Details The Jest integration test `Upgrade diffable rule "machine_learning_job_id" (machine_learning rule type) after preview in flyout customized field w/ an upgrade resulting in a non-solvable conflict (ABC) upgrades rule to resolved value` was skipped earlier in `9.1` and `9.2` due to flakiness. Since that moment combobox integration flakiness has been addressed in the scope of elastic#228620. Thorough testing in `main` (corresponds to `9.2`) and `9.1` branches hasn't revealed any failures or flakiness. On top of that this test runs in `8.18` and `8.19` smoothly all that time. Taking the above into account it should be safe to unskip the skipped test. In case if the flakiness pops up we'll have more data to analyze.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
8 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Addresses: #202078
Summary
This PR implements Prebuilt Rules Upgrade with (via Prebuilt Rule Upgrade Flyout) and without (via Prebuilt Rules Upgrade table) preview test plans.
Details
Since some of the test scenarios had been implemented earlier this PR involves refactoring as well and tests reorganization. The following has been done in the scope
upgrade_with_preview.cy.ts- Upgrade via Prebuilt Rule Upgrade Flyoutupgrade_without_preview.cy.ts- Upgrade via via Prebuilt Rules Upgrade tableNot implemented test scenarios
The following test scenarios weren't implemented due to time constraints
Prebuilt rules upgrade with preview test plan (Upgrade via Prebuilt Rule Upgrade Flyout)
The tests verify that a toast message appears after either prebuilt rule has been edited (revision bump) or a new prebuilt rule version is available (version bump, a new prebuilt rules package version has been installed). Since UI refetches data every 5 minutes e2e tests either has to wait for ~5 minutes for toasts to appear or use
cy.clock()to mock timers. The latter doesn't work in a simple way and requires time for deeper investigation.But there are unit tests for concurrency control.
Prebuilt rules upgrade without preview test plan (Prebuilt Rules Bulk upgrade)
There are two existing table filtering tests but besides that no new tests has been added. These tests should be covered in the scope of [Security Solution] Write tests for filtering, sorting, pagination in the Rule Installation and Upgrade tables #166215.
Flaky test runner