[8.19] [Security Solution] Implement prebuilt rules upgrade test plan (#228620) #230093
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…ic#228620) **Addresses: elastic#202078 ## Summary This PR implements Prebuilt Rules Upgrade [with (via Prebuilt Rule Upgrade Flyout)](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_with_preview.md) and [without (via Prebuilt Rules Upgrade table)](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_without_preview.md) preview test plans. ## Details Since some of the test scenarios had been implemented earlier this PR involves refactoring as well and tests reorganization. The following has been done in the scope - Prebuilt Rules Installation and Prebuilt Rules Upgrade workflows in Cypress have been split - Existing tests were renamed to better match to the test plans - Prebuilt Rules Upgrade Cypress tests were reorganized to - `upgrade_with_preview.cy.ts` - Upgrade via Prebuilt Rule Upgrade Flyout - `upgrade_without_preview.cy.ts` - Upgrade via via Prebuilt Rules Upgrade table - Missing tests have been added to the corresponding files for Jest Integration and Cypress tests ## Not implemented test scenarios The following test scenarios weren't implemented due to time constraints - [Prebuilt rules upgrade with preview test plan](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_with_preview.md#concurrency-control) (Upgrade via Prebuilt Rule Upgrade Flyout) - **Concurrency control** e2e tests The tests verify that a toast message appears after either prebuilt rule has been edited (revision bump) or a new prebuilt rule version is available (version bump, a new prebuilt rules package version has been installed). Since UI refetches data every 5 minutes e2e tests either has to wait for ~5 minutes for toasts to appear or use `cy.clock()` to mock timers. The latter doesn't work in a simple way and requires time for deeper investigation. But there are [unit tests](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/public/detection_engine/rule_management_ui/components/rules_table/upgrade_prebuilt_rules_table/use_prebuilt_rules_upgrade_state.test.ts) for concurrency control. - [Prebuilt rules upgrade without preview test plan](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_without_preview.md) (Prebuilt Rules Bulk upgrade) - **Rule upgrade workflow: filtering, sorting, pagination** There are two existing table filtering tests but besides that no new tests has been added. These tests should be covered in the scope of elastic#166215. - **Rule upgrade workflow: Edge cases**/**Scenario: Rule bound data is preserved after upgrading a rule to a newer version** - **Error handling**/**Scenario: Error is handled when any upgrade operation on prebuilt rules fails** - **Licensing: API endpoints** ## Flaky test runner - ✅ [e2e Cypress tests Rule Management group](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/8812) (100 runs) (cherry picked from commit 67006ad)
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
Contributor
💚 Build Succeeded
Metrics [docs]Async chunks
Unknown metric groupsESLint disabled line counts
Total ESLint disabled count
cc @maximpn |
maximpn
added a commit
that referenced
this pull request
Aug 1, 2025
…228620) (#230114) # Backport This will backport the following commits from `main` to `9.0`: - [[Security Solution] Implement prebuilt rules upgrade test plan (#228620)](#228620) <!--- Backport version: 10.0.0 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport) <!--BACKPORT [{"author":{"name":"Maxim Palenov","email":"maxim.palenov@elastic.co"},"sourceCommit":{"committedDate":"2025-07-31T15:22:02Z","message":"[Security Solution] Implement prebuilt rules upgrade test plan (#228620)\n\n**Addresses: https://github.com/elastic/kibana/issues/202078**\n\n## Summary\n\nThis PR implements Prebuilt Rules Upgrade [with (via Prebuilt Rule Upgrade Flyout)](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_with_preview.md) and [without (via Prebuilt Rules Upgrade table)](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_without_preview.md) preview test plans.\n\n## Details\n\nSince some of the test scenarios had been implemented earlier this PR involves refactoring as well and tests reorganization. The following has been done in the scope\n\n- Prebuilt Rules Installation and Prebuilt Rules Upgrade workflows in Cypress have been split\n- Existing tests were renamed to better match to the test plans\n- Prebuilt Rules Upgrade Cypress tests were reorganized to\n - `upgrade_with_preview.cy.ts` - Upgrade via Prebuilt Rule Upgrade Flyout\n - `upgrade_without_preview.cy.ts` - Upgrade via via Prebuilt Rules Upgrade table\n - Missing tests have been added to the corresponding files for Jest Integration and Cypress tests\n\n## Not implemented test scenarios\n\nThe following test scenarios weren't implemented due to time constraints\n\n- [Prebuilt rules upgrade with preview test plan](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_with_preview.md#concurrency-control) (Upgrade via Prebuilt Rule Upgrade Flyout)\n - **Concurrency control** e2e tests\n The tests verify that a toast message appears after either prebuilt rule has been edited (revision bump) or a new prebuilt rule version is available (version bump, a new prebuilt rules package version has been installed). Since UI refetches data every 5 minutes e2e tests either has to wait for ~5 minutes for toasts to appear or use `cy.clock()` to mock timers. The latter doesn't work in a simple way and requires time for deeper investigation.\n\n But there are [unit tests](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/public/detection_engine/rule_management_ui/components/rules_table/upgrade_prebuilt_rules_table/use_prebuilt_rules_upgrade_state.test.ts) for concurrency control.\n\n- [Prebuilt rules upgrade without preview test plan](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_without_preview.md) (Prebuilt Rules Bulk upgrade)\n - **Rule upgrade workflow: filtering, sorting, pagination**\n There are two existing table filtering tests but besides that no new tests has been added. These tests should be covered in the scope of https://github.com/elastic/kibana/issues/166215.\n - **Rule upgrade workflow: Edge cases**/**Scenario: Rule bound data is preserved after upgrading a rule to a newer version**\n - **Error handling**/**Scenario: Error is handled when any upgrade operation on prebuilt rules fails**\n - **Licensing: API endpoints**\n\n## Flaky test runner\n\n- ✅ [e2e Cypress tests Rule Management group](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/8812) (100 runs)","sha":"67006ad7844d956a38eb77ffe2e09979ec59eeca","branchLabelMapping":{"^v9.2.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["test","release_note:skip","impact:high","v9.0.0","Team:Detections and Resp","Team: SecuritySolution","Team:Detection Rule Management","Feature:Prebuilt Detection Rules","backport:version","v8.18.0","v9.1.0","v8.19.0","v9.2.0","v9.1.1"],"title":"[Security Solution] Implement prebuilt rules upgrade test plan","number":228620,"url":"https://github.com/elastic/kibana/pull/228620","mergeCommit":{"message":"[Security Solution] Implement prebuilt rules upgrade test plan (#228620)\n\n**Addresses: https://github.com/elastic/kibana/issues/202078**\n\n## Summary\n\nThis PR implements Prebuilt Rules Upgrade [with (via Prebuilt Rule Upgrade Flyout)](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_with_preview.md) and [without (via Prebuilt Rules Upgrade table)](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_without_preview.md) preview test plans.\n\n## Details\n\nSince some of the test scenarios had been implemented earlier this PR involves refactoring as well and tests reorganization. The following has been done in the scope\n\n- Prebuilt Rules Installation and Prebuilt Rules Upgrade workflows in Cypress have been split\n- Existing tests were renamed to better match to the test plans\n- Prebuilt Rules Upgrade Cypress tests were reorganized to\n - `upgrade_with_preview.cy.ts` - Upgrade via Prebuilt Rule Upgrade Flyout\n - `upgrade_without_preview.cy.ts` - Upgrade via via Prebuilt Rules Upgrade table\n - Missing tests have been added to the corresponding files for Jest Integration and Cypress tests\n\n## Not implemented test scenarios\n\nThe following test scenarios weren't implemented due to time constraints\n\n- [Prebuilt rules upgrade with preview test plan](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_with_preview.md#concurrency-control) (Upgrade via Prebuilt Rule Upgrade Flyout)\n - **Concurrency control** e2e tests\n The tests verify that a toast message appears after either prebuilt rule has been edited (revision bump) or a new prebuilt rule version is available (version bump, a new prebuilt rules package version has been installed). Since UI refetches data every 5 minutes e2e tests either has to wait for ~5 minutes for toasts to appear or use `cy.clock()` to mock timers. The latter doesn't work in a simple way and requires time for deeper investigation.\n\n But there are [unit tests](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/public/detection_engine/rule_management_ui/components/rules_table/upgrade_prebuilt_rules_table/use_prebuilt_rules_upgrade_state.test.ts) for concurrency control.\n\n- [Prebuilt rules upgrade without preview test plan](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_without_preview.md) (Prebuilt Rules Bulk upgrade)\n - **Rule upgrade workflow: filtering, sorting, pagination**\n There are two existing table filtering tests but besides that no new tests has been added. These tests should be covered in the scope of https://github.com/elastic/kibana/issues/166215.\n - **Rule upgrade workflow: Edge cases**/**Scenario: Rule bound data is preserved after upgrading a rule to a newer version**\n - **Error handling**/**Scenario: Error is handled when any upgrade operation on prebuilt rules fails**\n - **Licensing: API endpoints**\n\n## Flaky test runner\n\n- ✅ [e2e Cypress tests Rule Management group](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/8812) (100 runs)","sha":"67006ad7844d956a38eb77ffe2e09979ec59eeca"}},"sourceBranch":"main","suggestedTargetBranches":["9.0","8.18"],"targetPullRequestStates":[{"branch":"9.0","label":"v9.0.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"8.18","label":"v8.18.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"9.1","label":"v9.1.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"url":"https://github.com/elastic/kibana/pull/230094","number":230094,"state":"MERGED","mergeCommit":{"sha":"72e6e89777b84c417487057664612385bb09ddc4","message":"[9.1] [Security Solution] Implement prebuilt rules upgrade test plan (#228620) (#230094)\n\n# Backport\n\nThis will backport the following commits from `main` to `9.1`:\n- [[Security Solution] Implement prebuilt rules upgrade test plan\n(#228620)](https://github.com/elastic/kibana/pull/228620)\n\n\n\n### Questions ?\nPlease refer to the [Backport tool\ndocumentation](https://github.com/sorenlouv/backport)\n\n\n\nCo-authored-by: Maxim Palenov <maxim.palenov@elastic.co>"}},{"branch":"8.19","label":"v8.19.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"url":"https://github.com/elastic/kibana/pull/230093","number":230093,"state":"MERGED","mergeCommit":{"sha":"83abce269477cb1301077081d8a7ae9516bd9797","message":"[8.19] [Security Solution] Implement prebuilt rules upgrade test plan (#228620) (#230093)\n\n# Backport\n\nThis will backport the following commits from `main` to `8.19`:\n- [[Security Solution] Implement prebuilt rules upgrade test plan\n(#228620)](https://github.com/elastic/kibana/pull/228620)\n\n\n\n### Questions ?\nPlease refer to the [Backport tool\ndocumentation](https://github.com/sorenlouv/backport)\n\n\n\nCo-authored-by: Maxim Palenov <maxim.palenov@elastic.co>"}},{"branch":"main","label":"v9.2.0","branchLabelMappingKey":"^v9.2.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/228620","number":228620,"mergeCommit":{"message":"[Security Solution] Implement prebuilt rules upgrade test plan (#228620)\n\n**Addresses: https://github.com/elastic/kibana/issues/202078**\n\n## Summary\n\nThis PR implements Prebuilt Rules Upgrade [with (via Prebuilt Rule Upgrade Flyout)](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_with_preview.md) and [without (via Prebuilt Rules Upgrade table)](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_without_preview.md) preview test plans.\n\n## Details\n\nSince some of the test scenarios had been implemented earlier this PR involves refactoring as well and tests reorganization. The following has been done in the scope\n\n- Prebuilt Rules Installation and Prebuilt Rules Upgrade workflows in Cypress have been split\n- Existing tests were renamed to better match to the test plans\n- Prebuilt Rules Upgrade Cypress tests were reorganized to\n - `upgrade_with_preview.cy.ts` - Upgrade via Prebuilt Rule Upgrade Flyout\n - `upgrade_without_preview.cy.ts` - Upgrade via via Prebuilt Rules Upgrade table\n - Missing tests have been added to the corresponding files for Jest Integration and Cypress tests\n\n## Not implemented test scenarios\n\nThe following test scenarios weren't implemented due to time constraints\n\n- [Prebuilt rules upgrade with preview test plan](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_with_preview.md#concurrency-control) (Upgrade via Prebuilt Rule Upgrade Flyout)\n - **Concurrency control** e2e tests\n The tests verify that a toast message appears after either prebuilt rule has been edited (revision bump) or a new prebuilt rule version is available (version bump, a new prebuilt rules package version has been installed). Since UI refetches data every 5 minutes e2e tests either has to wait for ~5 minutes for toasts to appear or use `cy.clock()` to mock timers. The latter doesn't work in a simple way and requires time for deeper investigation.\n\n But there are [unit tests](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/public/detection_engine/rule_management_ui/components/rules_table/upgrade_prebuilt_rules_table/use_prebuilt_rules_upgrade_state.test.ts) for concurrency control.\n\n- [Prebuilt rules upgrade without preview test plan](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_without_preview.md) (Prebuilt Rules Bulk upgrade)\n - **Rule upgrade workflow: filtering, sorting, pagination**\n There are two existing table filtering tests but besides that no new tests has been added. These tests should be covered in the scope of https://github.com/elastic/kibana/issues/166215.\n - **Rule upgrade workflow: Edge cases**/**Scenario: Rule bound data is preserved after upgrading a rule to a newer version**\n - **Error handling**/**Scenario: Error is handled when any upgrade operation on prebuilt rules fails**\n - **Licensing: API endpoints**\n\n## Flaky test runner\n\n- ✅ [e2e Cypress tests Rule Management group](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/8812) (100 runs)","sha":"67006ad7844d956a38eb77ffe2e09979ec59eeca"}}]}] BACKPORT-->
maximpn
added a commit
that referenced
this pull request
Aug 1, 2025
…#228620) (#230115) # Backport This will backport the following commits from `main` to `8.18`: - [[Security Solution] Implement prebuilt rules upgrade test plan (#228620)](#228620) <!--- Backport version: 10.0.0 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport) <!--BACKPORT [{"author":{"name":"Maxim Palenov","email":"maxim.palenov@elastic.co"},"sourceCommit":{"committedDate":"2025-07-31T15:22:02Z","message":"[Security Solution] Implement prebuilt rules upgrade test plan (#228620)\n\n**Addresses: https://github.com/elastic/kibana/issues/202078**\n\n## Summary\n\nThis PR implements Prebuilt Rules Upgrade [with (via Prebuilt Rule Upgrade Flyout)](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_with_preview.md) and [without (via Prebuilt Rules Upgrade table)](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_without_preview.md) preview test plans.\n\n## Details\n\nSince some of the test scenarios had been implemented earlier this PR involves refactoring as well and tests reorganization. The following has been done in the scope\n\n- Prebuilt Rules Installation and Prebuilt Rules Upgrade workflows in Cypress have been split\n- Existing tests were renamed to better match to the test plans\n- Prebuilt Rules Upgrade Cypress tests were reorganized to\n - `upgrade_with_preview.cy.ts` - Upgrade via Prebuilt Rule Upgrade Flyout\n - `upgrade_without_preview.cy.ts` - Upgrade via via Prebuilt Rules Upgrade table\n - Missing tests have been added to the corresponding files for Jest Integration and Cypress tests\n\n## Not implemented test scenarios\n\nThe following test scenarios weren't implemented due to time constraints\n\n- [Prebuilt rules upgrade with preview test plan](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_with_preview.md#concurrency-control) (Upgrade via Prebuilt Rule Upgrade Flyout)\n - **Concurrency control** e2e tests\n The tests verify that a toast message appears after either prebuilt rule has been edited (revision bump) or a new prebuilt rule version is available (version bump, a new prebuilt rules package version has been installed). Since UI refetches data every 5 minutes e2e tests either has to wait for ~5 minutes for toasts to appear or use `cy.clock()` to mock timers. The latter doesn't work in a simple way and requires time for deeper investigation.\n\n But there are [unit tests](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/public/detection_engine/rule_management_ui/components/rules_table/upgrade_prebuilt_rules_table/use_prebuilt_rules_upgrade_state.test.ts) for concurrency control.\n\n- [Prebuilt rules upgrade without preview test plan](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_without_preview.md) (Prebuilt Rules Bulk upgrade)\n - **Rule upgrade workflow: filtering, sorting, pagination**\n There are two existing table filtering tests but besides that no new tests has been added. These tests should be covered in the scope of https://github.com/elastic/kibana/issues/166215.\n - **Rule upgrade workflow: Edge cases**/**Scenario: Rule bound data is preserved after upgrading a rule to a newer version**\n - **Error handling**/**Scenario: Error is handled when any upgrade operation on prebuilt rules fails**\n - **Licensing: API endpoints**\n\n## Flaky test runner\n\n- ✅ [e2e Cypress tests Rule Management group](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/8812) (100 runs)","sha":"67006ad7844d956a38eb77ffe2e09979ec59eeca","branchLabelMapping":{"^v9.2.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["test","release_note:skip","impact:high","v9.0.0","Team:Detections and Resp","Team: SecuritySolution","Team:Detection Rule Management","Feature:Prebuilt Detection Rules","backport:version","v8.18.0","v9.1.0","v8.19.0","v9.2.0","v9.1.1"],"title":"[Security Solution] Implement prebuilt rules upgrade test plan","number":228620,"url":"https://github.com/elastic/kibana/pull/228620","mergeCommit":{"message":"[Security Solution] Implement prebuilt rules upgrade test plan (#228620)\n\n**Addresses: https://github.com/elastic/kibana/issues/202078**\n\n## Summary\n\nThis PR implements Prebuilt Rules Upgrade [with (via Prebuilt Rule Upgrade Flyout)](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_with_preview.md) and [without (via Prebuilt Rules Upgrade table)](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_without_preview.md) preview test plans.\n\n## Details\n\nSince some of the test scenarios had been implemented earlier this PR involves refactoring as well and tests reorganization. The following has been done in the scope\n\n- Prebuilt Rules Installation and Prebuilt Rules Upgrade workflows in Cypress have been split\n- Existing tests were renamed to better match to the test plans\n- Prebuilt Rules Upgrade Cypress tests were reorganized to\n - `upgrade_with_preview.cy.ts` - Upgrade via Prebuilt Rule Upgrade Flyout\n - `upgrade_without_preview.cy.ts` - Upgrade via via Prebuilt Rules Upgrade table\n - Missing tests have been added to the corresponding files for Jest Integration and Cypress tests\n\n## Not implemented test scenarios\n\nThe following test scenarios weren't implemented due to time constraints\n\n- [Prebuilt rules upgrade with preview test plan](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_with_preview.md#concurrency-control) (Upgrade via Prebuilt Rule Upgrade Flyout)\n - **Concurrency control** e2e tests\n The tests verify that a toast message appears after either prebuilt rule has been edited (revision bump) or a new prebuilt rule version is available (version bump, a new prebuilt rules package version has been installed). Since UI refetches data every 5 minutes e2e tests either has to wait for ~5 minutes for toasts to appear or use `cy.clock()` to mock timers. The latter doesn't work in a simple way and requires time for deeper investigation.\n\n But there are [unit tests](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/public/detection_engine/rule_management_ui/components/rules_table/upgrade_prebuilt_rules_table/use_prebuilt_rules_upgrade_state.test.ts) for concurrency control.\n\n- [Prebuilt rules upgrade without preview test plan](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_without_preview.md) (Prebuilt Rules Bulk upgrade)\n - **Rule upgrade workflow: filtering, sorting, pagination**\n There are two existing table filtering tests but besides that no new tests has been added. These tests should be covered in the scope of https://github.com/elastic/kibana/issues/166215.\n - **Rule upgrade workflow: Edge cases**/**Scenario: Rule bound data is preserved after upgrading a rule to a newer version**\n - **Error handling**/**Scenario: Error is handled when any upgrade operation on prebuilt rules fails**\n - **Licensing: API endpoints**\n\n## Flaky test runner\n\n- ✅ [e2e Cypress tests Rule Management group](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/8812) (100 runs)","sha":"67006ad7844d956a38eb77ffe2e09979ec59eeca"}},"sourceBranch":"main","suggestedTargetBranches":["9.0","8.18"],"targetPullRequestStates":[{"branch":"9.0","label":"v9.0.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"8.18","label":"v8.18.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"9.1","label":"v9.1.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"url":"https://github.com/elastic/kibana/pull/230094","number":230094,"state":"MERGED","mergeCommit":{"sha":"72e6e89777b84c417487057664612385bb09ddc4","message":"[9.1] [Security Solution] Implement prebuilt rules upgrade test plan (#228620) (#230094)\n\n# Backport\n\nThis will backport the following commits from `main` to `9.1`:\n- [[Security Solution] Implement prebuilt rules upgrade test plan\n(#228620)](https://github.com/elastic/kibana/pull/228620)\n\n\n\n### Questions ?\nPlease refer to the [Backport tool\ndocumentation](https://github.com/sorenlouv/backport)\n\n\n\nCo-authored-by: Maxim Palenov <maxim.palenov@elastic.co>"}},{"branch":"8.19","label":"v8.19.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"url":"https://github.com/elastic/kibana/pull/230093","number":230093,"state":"MERGED","mergeCommit":{"sha":"83abce269477cb1301077081d8a7ae9516bd9797","message":"[8.19] [Security Solution] Implement prebuilt rules upgrade test plan (#228620) (#230093)\n\n# Backport\n\nThis will backport the following commits from `main` to `8.19`:\n- [[Security Solution] Implement prebuilt rules upgrade test plan\n(#228620)](https://github.com/elastic/kibana/pull/228620)\n\n\n\n### Questions ?\nPlease refer to the [Backport tool\ndocumentation](https://github.com/sorenlouv/backport)\n\n\n\nCo-authored-by: Maxim Palenov <maxim.palenov@elastic.co>"}},{"branch":"main","label":"v9.2.0","branchLabelMappingKey":"^v9.2.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/228620","number":228620,"mergeCommit":{"message":"[Security Solution] Implement prebuilt rules upgrade test plan (#228620)\n\n**Addresses: https://github.com/elastic/kibana/issues/202078**\n\n## Summary\n\nThis PR implements Prebuilt Rules Upgrade [with (via Prebuilt Rule Upgrade Flyout)](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_with_preview.md) and [without (via Prebuilt Rules Upgrade table)](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_without_preview.md) preview test plans.\n\n## Details\n\nSince some of the test scenarios had been implemented earlier this PR involves refactoring as well and tests reorganization. The following has been done in the scope\n\n- Prebuilt Rules Installation and Prebuilt Rules Upgrade workflows in Cypress have been split\n- Existing tests were renamed to better match to the test plans\n- Prebuilt Rules Upgrade Cypress tests were reorganized to\n - `upgrade_with_preview.cy.ts` - Upgrade via Prebuilt Rule Upgrade Flyout\n - `upgrade_without_preview.cy.ts` - Upgrade via via Prebuilt Rules Upgrade table\n - Missing tests have been added to the corresponding files for Jest Integration and Cypress tests\n\n## Not implemented test scenarios\n\nThe following test scenarios weren't implemented due to time constraints\n\n- [Prebuilt rules upgrade with preview test plan](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_with_preview.md#concurrency-control) (Upgrade via Prebuilt Rule Upgrade Flyout)\n - **Concurrency control** e2e tests\n The tests verify that a toast message appears after either prebuilt rule has been edited (revision bump) or a new prebuilt rule version is available (version bump, a new prebuilt rules package version has been installed). Since UI refetches data every 5 minutes e2e tests either has to wait for ~5 minutes for toasts to appear or use `cy.clock()` to mock timers. The latter doesn't work in a simple way and requires time for deeper investigation.\n\n But there are [unit tests](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/public/detection_engine/rule_management_ui/components/rules_table/upgrade_prebuilt_rules_table/use_prebuilt_rules_upgrade_state.test.ts) for concurrency control.\n\n- [Prebuilt rules upgrade without preview test plan](https://github.com/elastic/kibana/blob/main/x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_upgrade_without_preview.md) (Prebuilt Rules Bulk upgrade)\n - **Rule upgrade workflow: filtering, sorting, pagination**\n There are two existing table filtering tests but besides that no new tests has been added. These tests should be covered in the scope of https://github.com/elastic/kibana/issues/166215.\n - **Rule upgrade workflow: Edge cases**/**Scenario: Rule bound data is preserved after upgrading a rule to a newer version**\n - **Error handling**/**Scenario: Error is handled when any upgrade operation on prebuilt rules fails**\n - **Licensing: API endpoints**\n\n## Flaky test runner\n\n- ✅ [e2e Cypress tests Rule Management group](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/8812) (100 runs)","sha":"67006ad7844d956a38eb77ffe2e09979ec59eeca"}}]}] BACKPORT-->
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Backport
This will backport the following commits from
mainto8.19:Questions ?
Please refer to the Backport tool documentation