[Security Solution] Test plan for upgrading prebuilt rules without preview (Rule Updates table) v1#203331
Merged
jpdjere merged 18 commits intoelastic:mainfrom Dec 19, 2024
Merged
Conversation
jpdjere
added
release_note:skip
Contributor
|
Pinging @elastic/security-detections-response (Team:Detections and Resp) |
Contributor
|
Pinging @elastic/security-solution (Team: SecuritySolution) |
Contributor
|
Pinging @elastic/security-detection-rule-management (Team:Detection Rule Management) |
jpdjere
changed the title
Contributor
💚 Build Succeeded
|
jpdjere
changed the title
banderror
added
v9.0.0
backport:version
maximpn
reviewed
Dec 17, 2024
Contributor
maximpn
left a comment
maximpn
left a comment
There was a problem hiding this comment.
@jpdjere I looked through the test plan and left comments with suggestions. I can highlight the following
- Should we add explicit tests for rule type change?
- While not strictly opinionated we could use
N,M,Ketc to denote variable numbers as it's usually used in Math thanX,Y,Z. I see a mix up ofX,Y,ZandN,M,K.
.../security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/installation.md
Outdated
Show resolved
Hide resolved
.../security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/installation.md
Outdated
Show resolved
Hide resolved
.../security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/installation.md
Outdated
Show resolved
Hide resolved
.../security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/installation.md
Outdated
Show resolved
Hide resolved
.../security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/installation.md
Outdated
Show resolved
Hide resolved
Contributor
There was a problem hiding this comment.
Should user be on Rule Upgrade page?
...ugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/upgrade.md
Outdated
Show resolved
Hide resolved
...ugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/upgrade.md
Outdated
Show resolved
Hide resolved
...ugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/upgrade.md
Outdated
Show resolved
Hide resolved
Comment on lines
390
to
392
Contributor
There was a problem hiding this comment.
Suggested change
| And the Z upgraded rules should be removed from the table | |
| And the remaining Y - Z rules should still be present in the table | |
| And user should see the number of rules available to upgrade decreased by Z number of upgraded rules | |
| And the K upgraded rules should be removed from the table | |
| And the remaining M = Y - K rules should still be present in the table | |
| And user should see the number of rules available to upgrade decreased by K number of upgraded rules |
Contributor
Author
|
Thanks for the review @maximpn . Applied most of your suggestions and added test scenarios for rule type changes. Will add those tests in the follow up PR. Also, rebased after the big refactor. So comments might have lost their references. |
Contributor
|
@pborgonovi Have you reviewed this one? |
stratoula
pushed a commit
to stratoula/kibana
that referenced
this pull request
Jan 2, 2025
Partially resolves: elastic#202078 ## Summary This PR refactors the test plans involving the Installation and Upgrade Workflows: - breaks down the gigantic `installation_and_upgrade.md` file into three files: - `installation.md` covering exclusively prebuilt rule install workflows - `upgrade.md` covering exclusively prebuilt rule upgrade workflows - `package_installation_and_upgrade.md` which covers scenarios for the installation and upgrades of the Prebuilt Rules package itself. The *Upgrade of Prebuilt Rules* (`upgrade.md`) was modified as follows: - expanded to cover scenarios detailed in the linked ticket (`Upgrading prebuilt rules one-by-one without preview` and `Upgrading prebuilt rules in bulk` in the Rule Updates table, including cases with and without conflicts, as well as a mixture of both) - internally refactored so that sections that will become legacy with the release of Milestone 3 were pushed to the end and marked as pending deletion once the feature is live. - from point above, the test plan now is in a better position to add the scenarios for the Three-Way-Diff upgrade workflow. cc @maximpn @nikitaindik ### Checklist Check the PR satisfies following conditions. Reviewers should verify this PR satisfies this list as well. - [ ] The PR description includes the appropriate Release Notes section, and the correct `release_note:*` label is applied per the [guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) --------- Co-authored-by: Maxim Palenov <maxim.palenov@elastic.co>
benakansara
pushed a commit
to benakansara/kibana
that referenced
this pull request
Jan 2, 2025
Partially resolves: elastic#202078 ## Summary This PR refactors the test plans involving the Installation and Upgrade Workflows: - breaks down the gigantic `installation_and_upgrade.md` file into three files: - `installation.md` covering exclusively prebuilt rule install workflows - `upgrade.md` covering exclusively prebuilt rule upgrade workflows - `package_installation_and_upgrade.md` which covers scenarios for the installation and upgrades of the Prebuilt Rules package itself. The *Upgrade of Prebuilt Rules* (`upgrade.md`) was modified as follows: - expanded to cover scenarios detailed in the linked ticket (`Upgrading prebuilt rules one-by-one without preview` and `Upgrading prebuilt rules in bulk` in the Rule Updates table, including cases with and without conflicts, as well as a mixture of both) - internally refactored so that sections that will become legacy with the release of Milestone 3 were pushed to the end and marked as pending deletion once the feature is live. - from point above, the test plan now is in a better position to add the scenarios for the Three-Way-Diff upgrade workflow. cc @maximpn @nikitaindik ### Checklist Check the PR satisfies following conditions. Reviewers should verify this PR satisfies this list as well. - [ ] The PR description includes the appropriate Release Notes section, and the correct `release_note:*` label is applied per the [guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) --------- Co-authored-by: Maxim Palenov <maxim.palenov@elastic.co>
maximpn
pushed a commit
that referenced
this pull request
Jan 2, 2025
…workflows (#204677) Addresses: #202078 > NOTE: This PR depends on the test plans created in #203331. If any feedback leads to changes in those test plan, these PR will be updated. ## Summary - Enhances tests on the Rule Upgrade Workflow from the Rule Updates table, as described in test plan created in: #203331 ## Flaky test runner https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7611
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this pull request
Jan 2, 2025
…workflows (elastic#204677) Addresses: elastic#202078 > NOTE: This PR depends on the test plans created in elastic#203331. If any feedback leads to changes in those test plan, these PR will be updated. ## Summary - Enhances tests on the Rule Upgrade Workflow from the Rule Updates table, as described in test plan created in: elastic#203331 ## Flaky test runner https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7611 (cherry picked from commit 529f833)
kibanamachine
added a commit
that referenced
this pull request
Jan 2, 2025
…grade workflows (#204677) (#205381) # Backport This will backport the following commits from `main` to `8.x`: - [[Security Solution] Enhance test coverage for Prebuilt Rules upgrade workflows (#204677)](#204677) <!--- Backport version: 9.4.3 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Juan Pablo Djeredjian","email":"jpdjeredjian@gmail.com"},"sourceCommit":{"committedDate":"2025-01-02T12:18:40Z","message":"[Security Solution] Enhance test coverage for Prebuilt Rules upgrade workflows (#204677)\n\nAddresses: https://github.com/elastic/kibana/issues/202078\r\n\r\n> NOTE: This PR depends on the test plans created in #203331. If any feedback leads to changes in those test plan, these PR will be updated.\r\n\r\n## Summary\r\n\r\n- Enhances tests on the Rule Upgrade Workflow from the Rule Updates table, as described in test plan created in: https://github.com/elastic/kibana/pull/203331\r\n\r\n## Flaky test runner\r\n\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7611","sha":"529f833ac83d2365df2c0ac509c55c436f0736a9","branchLabelMapping":{"^v9.0.0$":"main","^v8.18.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["test","release_note:skip","v9.0.0","Team:Detections and Resp","Team: SecuritySolution","Team:Detection Rule Management","Feature:Prebuilt Detection Rules","backport:version","v8.18.0"],"title":"[Security Solution] Enhance test coverage for Prebuilt Rules upgrade workflows","number":204677,"url":"https://github.com/elastic/kibana/pull/204677","mergeCommit":{"message":"[Security Solution] Enhance test coverage for Prebuilt Rules upgrade workflows (#204677)\n\nAddresses: https://github.com/elastic/kibana/issues/202078\r\n\r\n> NOTE: This PR depends on the test plans created in #203331. If any feedback leads to changes in those test plan, these PR will be updated.\r\n\r\n## Summary\r\n\r\n- Enhances tests on the Rule Upgrade Workflow from the Rule Updates table, as described in test plan created in: https://github.com/elastic/kibana/pull/203331\r\n\r\n## Flaky test runner\r\n\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7611","sha":"529f833ac83d2365df2c0ac509c55c436f0736a9"}},"sourceBranch":"main","suggestedTargetBranches":["8.x"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/204677","number":204677,"mergeCommit":{"message":"[Security Solution] Enhance test coverage for Prebuilt Rules upgrade workflows (#204677)\n\nAddresses: https://github.com/elastic/kibana/issues/202078\r\n\r\n> NOTE: This PR depends on the test plans created in #203331. If any feedback leads to changes in those test plan, these PR will be updated.\r\n\r\n## Summary\r\n\r\n- Enhances tests on the Rule Upgrade Workflow from the Rule Updates table, as described in test plan created in: https://github.com/elastic/kibana/pull/203331\r\n\r\n## Flaky test runner\r\n\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7611","sha":"529f833ac83d2365df2c0ac509c55c436f0736a9"}},{"branch":"8.x","label":"v8.18.0","branchLabelMappingKey":"^v8.18.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> Co-authored-by: Juan Pablo Djeredjian <jpdjeredjian@gmail.com>
Contributor
Hi @banderror |
Contributor
|
@pborgonovi You can catch up with these changes in the follow-up PR #205922 |
viduni94
pushed a commit
to viduni94/kibana
that referenced
this pull request
Jan 23, 2025
Partially resolves: elastic#202078 ## Summary This PR refactors the test plans involving the Installation and Upgrade Workflows: - breaks down the gigantic `installation_and_upgrade.md` file into three files: - `installation.md` covering exclusively prebuilt rule install workflows - `upgrade.md` covering exclusively prebuilt rule upgrade workflows - `package_installation_and_upgrade.md` which covers scenarios for the installation and upgrades of the Prebuilt Rules package itself. The *Upgrade of Prebuilt Rules* (`upgrade.md`) was modified as follows: - expanded to cover scenarios detailed in the linked ticket (`Upgrading prebuilt rules one-by-one without preview` and `Upgrading prebuilt rules in bulk` in the Rule Updates table, including cases with and without conflicts, as well as a mixture of both) - internally refactored so that sections that will become legacy with the release of Milestone 3 were pushed to the end and marked as pending deletion once the feature is live. - from point above, the test plan now is in a better position to add the scenarios for the Three-Way-Diff upgrade workflow. cc @maximpn @nikitaindik ### Checklist Check the PR satisfies following conditions. Reviewers should verify this PR satisfies this list as well. - [ ] The PR description includes the appropriate Release Notes section, and the correct `release_note:*` label is applied per the [guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) --------- Co-authored-by: Maxim Palenov <maxim.palenov@elastic.co>
viduni94
pushed a commit
to viduni94/kibana
that referenced
this pull request
Jan 23, 2025
…workflows (elastic#204677) Addresses: elastic#202078 > NOTE: This PR depends on the test plans created in elastic#203331. If any feedback leads to changes in those test plan, these PR will be updated. ## Summary - Enhances tests on the Rule Upgrade Workflow from the Rule Updates table, as described in test plan created in: elastic#203331 ## Flaky test runner https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7611
banderror
changed the title
banderror
added a commit
that referenced
this pull request
Feb 14, 2025
…eview (Rule Updates table) v2 (#205922) **Addresses:** #202078 ## Summary This PR extends rule upgrade test plan with customizable and non-customizable field examples. Rule upgrade workflow test plan (excluding Rule Upgrade flyout) was initially extended in #203331. #203331 adds the following rule upgrade workflow scenarios - Scenario: User can upgrade conflict-free prebuilt rules one by one - Scenario: User cannot upgrade prebuilt rules one by one from Rules Update table if they have conflicts - Scenario: User can upgrade multiple conflict-free prebuilt rules selected on the page - Scenario: User cannot upgrade selected prebuilt rules with conflicts - Scenario: User can upgrade all available conflict-free prebuilt rules at once* - Scenario: User cannot upgrade all prebuilt rules at once if they have upgrade conflicts - Scenario: User can upgrade only conflict-free rules when a mix of rules with and without conflicts are selected for upgrade - Scenario: User can upgrade only conflict-free rules when attempting to upgrade all rules - Scenario: User can upgrade rule with rule type change individually - Scenario: User can not bulk upgrade selected rules with rule type changes - Scenario: User can not bulk upgrade all rules with rule type changes - Scenario: API does not upgrade prebuilt rules if they are up to date --------- Co-authored-by: Georgii Gorbachev <georgii.gorbachev@elastic.co>
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this pull request
Feb 14, 2025
…eview (Rule Updates table) v2 (elastic#205922) **Addresses:** elastic#202078 ## Summary This PR extends rule upgrade test plan with customizable and non-customizable field examples. Rule upgrade workflow test plan (excluding Rule Upgrade flyout) was initially extended in elastic#203331. elastic#203331 adds the following rule upgrade workflow scenarios - Scenario: User can upgrade conflict-free prebuilt rules one by one - Scenario: User cannot upgrade prebuilt rules one by one from Rules Update table if they have conflicts - Scenario: User can upgrade multiple conflict-free prebuilt rules selected on the page - Scenario: User cannot upgrade selected prebuilt rules with conflicts - Scenario: User can upgrade all available conflict-free prebuilt rules at once* - Scenario: User cannot upgrade all prebuilt rules at once if they have upgrade conflicts - Scenario: User can upgrade only conflict-free rules when a mix of rules with and without conflicts are selected for upgrade - Scenario: User can upgrade only conflict-free rules when attempting to upgrade all rules - Scenario: User can upgrade rule with rule type change individually - Scenario: User can not bulk upgrade selected rules with rule type changes - Scenario: User can not bulk upgrade all rules with rule type changes - Scenario: API does not upgrade prebuilt rules if they are up to date --------- Co-authored-by: Georgii Gorbachev <georgii.gorbachev@elastic.co> (cherry picked from commit e29033a)
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this pull request
Feb 14, 2025
…eview (Rule Updates table) v2 (elastic#205922) **Addresses:** elastic#202078 ## Summary This PR extends rule upgrade test plan with customizable and non-customizable field examples. Rule upgrade workflow test plan (excluding Rule Upgrade flyout) was initially extended in elastic#203331. elastic#203331 adds the following rule upgrade workflow scenarios - Scenario: User can upgrade conflict-free prebuilt rules one by one - Scenario: User cannot upgrade prebuilt rules one by one from Rules Update table if they have conflicts - Scenario: User can upgrade multiple conflict-free prebuilt rules selected on the page - Scenario: User cannot upgrade selected prebuilt rules with conflicts - Scenario: User can upgrade all available conflict-free prebuilt rules at once* - Scenario: User cannot upgrade all prebuilt rules at once if they have upgrade conflicts - Scenario: User can upgrade only conflict-free rules when a mix of rules with and without conflicts are selected for upgrade - Scenario: User can upgrade only conflict-free rules when attempting to upgrade all rules - Scenario: User can upgrade rule with rule type change individually - Scenario: User can not bulk upgrade selected rules with rule type changes - Scenario: User can not bulk upgrade all rules with rule type changes - Scenario: API does not upgrade prebuilt rules if they are up to date --------- Co-authored-by: Georgii Gorbachev <georgii.gorbachev@elastic.co> (cherry picked from commit e29033a)
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this pull request
Feb 14, 2025
…eview (Rule Updates table) v2 (elastic#205922) **Addresses:** elastic#202078 ## Summary This PR extends rule upgrade test plan with customizable and non-customizable field examples. Rule upgrade workflow test plan (excluding Rule Upgrade flyout) was initially extended in elastic#203331. elastic#203331 adds the following rule upgrade workflow scenarios - Scenario: User can upgrade conflict-free prebuilt rules one by one - Scenario: User cannot upgrade prebuilt rules one by one from Rules Update table if they have conflicts - Scenario: User can upgrade multiple conflict-free prebuilt rules selected on the page - Scenario: User cannot upgrade selected prebuilt rules with conflicts - Scenario: User can upgrade all available conflict-free prebuilt rules at once* - Scenario: User cannot upgrade all prebuilt rules at once if they have upgrade conflicts - Scenario: User can upgrade only conflict-free rules when a mix of rules with and without conflicts are selected for upgrade - Scenario: User can upgrade only conflict-free rules when attempting to upgrade all rules - Scenario: User can upgrade rule with rule type change individually - Scenario: User can not bulk upgrade selected rules with rule type changes - Scenario: User can not bulk upgrade all rules with rule type changes - Scenario: API does not upgrade prebuilt rules if they are up to date --------- Co-authored-by: Georgii Gorbachev <georgii.gorbachev@elastic.co> (cherry picked from commit e29033a)
kibanamachine
added a commit
that referenced
this pull request
Feb 14, 2025
…out preview (Rule Updates table) v2 (#205922) (#211246) # Backport This will backport the following commits from `main` to `8.x`: - [[Security Solution] Test plan for upgrading prebuilt rules without preview (Rule Updates table) v2 (#205922)](#205922) <!--- Backport version: 9.4.3 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Maxim Palenov","email":"maxim.palenov@elastic.co"},"sourceCommit":{"committedDate":"2025-02-14T15:18:40Z","message":"[Security Solution] Test plan for upgrading prebuilt rules without preview (Rule Updates table) v2 (#205922)\n\n**Addresses:** https://github.com/elastic/kibana/issues/202078\r\n\r\n## Summary\r\n\r\nThis PR extends rule upgrade test plan with customizable and\r\nnon-customizable field examples. Rule upgrade workflow test plan\r\n(excluding Rule Upgrade flyout) was initially extended in\r\nhttps://github.com//pull/203331.\r\n\r\nhttps://github.com//pull/203331 adds the following rule\r\nupgrade workflow scenarios\r\n\r\n- Scenario: User can upgrade conflict-free prebuilt rules one by one\r\n- Scenario: User cannot upgrade prebuilt rules one by one from Rules\r\nUpdate table if they have conflicts\r\n- Scenario: User can upgrade multiple conflict-free prebuilt rules\r\nselected on the page\r\n- Scenario: User cannot upgrade selected prebuilt rules with conflicts\r\n- Scenario: User can upgrade all available conflict-free prebuilt rules\r\nat once*\r\n- Scenario: User cannot upgrade all prebuilt rules at once if they have\r\nupgrade conflicts\r\n- Scenario: User can upgrade only conflict-free rules when a mix of\r\nrules with and without conflicts are selected for upgrade\r\n- Scenario: User can upgrade only conflict-free rules when attempting to\r\nupgrade all rules\r\n- Scenario: User can upgrade rule with rule type change individually\r\n- Scenario: User can not bulk upgrade selected rules with rule type\r\nchanges\r\n- Scenario: User can not bulk upgrade all rules with rule type changes\r\n- Scenario: API does not upgrade prebuilt rules if they are up to date\r\n\r\n---------\r\n\r\nCo-authored-by: Georgii Gorbachev <georgii.gorbachev@elastic.co>","sha":"e29033a726930c3d891aa77ea4f7ed990c7ee511","branchLabelMapping":{"^v9.1.0$":"main","^v8.19.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","test-plan","v9.0.0","Team:Detections and Resp","Team: SecuritySolution","Team:Detection Rule Management","Feature:Prebuilt Detection Rules","backport:version","v8.18.0","v9.1.0","v8.19.0"],"title":"[Security Solution] Test plan for upgrading prebuilt rules without preview (Rule Updates table) v2","number":205922,"url":"https://github.com/elastic/kibana/pull/205922","mergeCommit":{"message":"[Security Solution] Test plan for upgrading prebuilt rules without preview (Rule Updates table) v2 (#205922)\n\n**Addresses:** https://github.com/elastic/kibana/issues/202078\r\n\r\n## Summary\r\n\r\nThis PR extends rule upgrade test plan with customizable and\r\nnon-customizable field examples. Rule upgrade workflow test plan\r\n(excluding Rule Upgrade flyout) was initially extended in\r\nhttps://github.com//pull/203331.\r\n\r\nhttps://github.com//pull/203331 adds the following rule\r\nupgrade workflow scenarios\r\n\r\n- Scenario: User can upgrade conflict-free prebuilt rules one by one\r\n- Scenario: User cannot upgrade prebuilt rules one by one from Rules\r\nUpdate table if they have conflicts\r\n- Scenario: User can upgrade multiple conflict-free prebuilt rules\r\nselected on the page\r\n- Scenario: User cannot upgrade selected prebuilt rules with conflicts\r\n- Scenario: User can upgrade all available conflict-free prebuilt rules\r\nat once*\r\n- Scenario: User cannot upgrade all prebuilt rules at once if they have\r\nupgrade conflicts\r\n- Scenario: User can upgrade only conflict-free rules when a mix of\r\nrules with and without conflicts are selected for upgrade\r\n- Scenario: User can upgrade only conflict-free rules when attempting to\r\nupgrade all rules\r\n- Scenario: User can upgrade rule with rule type change individually\r\n- Scenario: User can not bulk upgrade selected rules with rule type\r\nchanges\r\n- Scenario: User can not bulk upgrade all rules with rule type changes\r\n- Scenario: API does not upgrade prebuilt rules if they are up to date\r\n\r\n---------\r\n\r\nCo-authored-by: Georgii Gorbachev <georgii.gorbachev@elastic.co>","sha":"e29033a726930c3d891aa77ea4f7ed990c7ee511"}},"sourceBranch":"main","suggestedTargetBranches":["9.0","8.18","8.x"],"targetPullRequestStates":[{"branch":"9.0","label":"v9.0.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"8.18","label":"v8.18.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/205922","number":205922,"mergeCommit":{"message":"[Security Solution] Test plan for upgrading prebuilt rules without preview (Rule Updates table) v2 (#205922)\n\n**Addresses:** https://github.com/elastic/kibana/issues/202078\r\n\r\n## Summary\r\n\r\nThis PR extends rule upgrade test plan with customizable and\r\nnon-customizable field examples. Rule upgrade workflow test plan\r\n(excluding Rule Upgrade flyout) was initially extended in\r\nhttps://github.com//pull/203331.\r\n\r\nhttps://github.com//pull/203331 adds the following rule\r\nupgrade workflow scenarios\r\n\r\n- Scenario: User can upgrade conflict-free prebuilt rules one by one\r\n- Scenario: User cannot upgrade prebuilt rules one by one from Rules\r\nUpdate table if they have conflicts\r\n- Scenario: User can upgrade multiple conflict-free prebuilt rules\r\nselected on the page\r\n- Scenario: User cannot upgrade selected prebuilt rules with conflicts\r\n- Scenario: User can upgrade all available conflict-free prebuilt rules\r\nat once*\r\n- Scenario: User cannot upgrade all prebuilt rules at once if they have\r\nupgrade conflicts\r\n- Scenario: User can upgrade only conflict-free rules when a mix of\r\nrules with and without conflicts are selected for upgrade\r\n- Scenario: User can upgrade only conflict-free rules when attempting to\r\nupgrade all rules\r\n- Scenario: User can upgrade rule with rule type change individually\r\n- Scenario: User can not bulk upgrade selected rules with rule type\r\nchanges\r\n- Scenario: User can not bulk upgrade all rules with rule type changes\r\n- Scenario: API does not upgrade prebuilt rules if they are up to date\r\n\r\n---------\r\n\r\nCo-authored-by: Georgii Gorbachev <georgii.gorbachev@elastic.co>","sha":"e29033a726930c3d891aa77ea4f7ed990c7ee511"}},{"branch":"8.x","label":"v8.19.0","branchLabelMappingKey":"^v8.19.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> Co-authored-by: Maxim Palenov <maxim.palenov@elastic.co>
kibanamachine
added a commit
that referenced
this pull request
Feb 14, 2025
…out preview (Rule Updates table) v2 (#205922) (#211247) # Backport This will backport the following commits from `main` to `9.0`: - [[Security Solution] Test plan for upgrading prebuilt rules without preview (Rule Updates table) v2 (#205922)](#205922) <!--- Backport version: 9.4.3 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Maxim Palenov","email":"maxim.palenov@elastic.co"},"sourceCommit":{"committedDate":"2025-02-14T15:18:40Z","message":"[Security Solution] Test plan for upgrading prebuilt rules without preview (Rule Updates table) v2 (#205922)\n\n**Addresses:** https://github.com/elastic/kibana/issues/202078\r\n\r\n## Summary\r\n\r\nThis PR extends rule upgrade test plan with customizable and\r\nnon-customizable field examples. Rule upgrade workflow test plan\r\n(excluding Rule Upgrade flyout) was initially extended in\r\nhttps://github.com//pull/203331.\r\n\r\nhttps://github.com//pull/203331 adds the following rule\r\nupgrade workflow scenarios\r\n\r\n- Scenario: User can upgrade conflict-free prebuilt rules one by one\r\n- Scenario: User cannot upgrade prebuilt rules one by one from Rules\r\nUpdate table if they have conflicts\r\n- Scenario: User can upgrade multiple conflict-free prebuilt rules\r\nselected on the page\r\n- Scenario: User cannot upgrade selected prebuilt rules with conflicts\r\n- Scenario: User can upgrade all available conflict-free prebuilt rules\r\nat once*\r\n- Scenario: User cannot upgrade all prebuilt rules at once if they have\r\nupgrade conflicts\r\n- Scenario: User can upgrade only conflict-free rules when a mix of\r\nrules with and without conflicts are selected for upgrade\r\n- Scenario: User can upgrade only conflict-free rules when attempting to\r\nupgrade all rules\r\n- Scenario: User can upgrade rule with rule type change individually\r\n- Scenario: User can not bulk upgrade selected rules with rule type\r\nchanges\r\n- Scenario: User can not bulk upgrade all rules with rule type changes\r\n- Scenario: API does not upgrade prebuilt rules if they are up to date\r\n\r\n---------\r\n\r\nCo-authored-by: Georgii Gorbachev <georgii.gorbachev@elastic.co>","sha":"e29033a726930c3d891aa77ea4f7ed990c7ee511","branchLabelMapping":{"^v9.1.0$":"main","^v8.19.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","test-plan","v9.0.0","Team:Detections and Resp","Team: SecuritySolution","Team:Detection Rule Management","Feature:Prebuilt Detection Rules","backport:version","v8.18.0","v9.1.0","v8.19.0"],"title":"[Security Solution] Test plan for upgrading prebuilt rules without preview (Rule Updates table) v2","number":205922,"url":"https://github.com/elastic/kibana/pull/205922","mergeCommit":{"message":"[Security Solution] Test plan for upgrading prebuilt rules without preview (Rule Updates table) v2 (#205922)\n\n**Addresses:** https://github.com/elastic/kibana/issues/202078\r\n\r\n## Summary\r\n\r\nThis PR extends rule upgrade test plan with customizable and\r\nnon-customizable field examples. Rule upgrade workflow test plan\r\n(excluding Rule Upgrade flyout) was initially extended in\r\nhttps://github.com//pull/203331.\r\n\r\nhttps://github.com//pull/203331 adds the following rule\r\nupgrade workflow scenarios\r\n\r\n- Scenario: User can upgrade conflict-free prebuilt rules one by one\r\n- Scenario: User cannot upgrade prebuilt rules one by one from Rules\r\nUpdate table if they have conflicts\r\n- Scenario: User can upgrade multiple conflict-free prebuilt rules\r\nselected on the page\r\n- Scenario: User cannot upgrade selected prebuilt rules with conflicts\r\n- Scenario: User can upgrade all available conflict-free prebuilt rules\r\nat once*\r\n- Scenario: User cannot upgrade all prebuilt rules at once if they have\r\nupgrade conflicts\r\n- Scenario: User can upgrade only conflict-free rules when a mix of\r\nrules with and without conflicts are selected for upgrade\r\n- Scenario: User can upgrade only conflict-free rules when attempting to\r\nupgrade all rules\r\n- Scenario: User can upgrade rule with rule type change individually\r\n- Scenario: User can not bulk upgrade selected rules with rule type\r\nchanges\r\n- Scenario: User can not bulk upgrade all rules with rule type changes\r\n- Scenario: API does not upgrade prebuilt rules if they are up to date\r\n\r\n---------\r\n\r\nCo-authored-by: Georgii Gorbachev <georgii.gorbachev@elastic.co>","sha":"e29033a726930c3d891aa77ea4f7ed990c7ee511"}},"sourceBranch":"main","suggestedTargetBranches":["9.0","8.18","8.x"],"targetPullRequestStates":[{"branch":"9.0","label":"v9.0.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"8.18","label":"v8.18.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/205922","number":205922,"mergeCommit":{"message":"[Security Solution] Test plan for upgrading prebuilt rules without preview (Rule Updates table) v2 (#205922)\n\n**Addresses:** https://github.com/elastic/kibana/issues/202078\r\n\r\n## Summary\r\n\r\nThis PR extends rule upgrade test plan with customizable and\r\nnon-customizable field examples. Rule upgrade workflow test plan\r\n(excluding Rule Upgrade flyout) was initially extended in\r\nhttps://github.com//pull/203331.\r\n\r\nhttps://github.com//pull/203331 adds the following rule\r\nupgrade workflow scenarios\r\n\r\n- Scenario: User can upgrade conflict-free prebuilt rules one by one\r\n- Scenario: User cannot upgrade prebuilt rules one by one from Rules\r\nUpdate table if they have conflicts\r\n- Scenario: User can upgrade multiple conflict-free prebuilt rules\r\nselected on the page\r\n- Scenario: User cannot upgrade selected prebuilt rules with conflicts\r\n- Scenario: User can upgrade all available conflict-free prebuilt rules\r\nat once*\r\n- Scenario: User cannot upgrade all prebuilt rules at once if they have\r\nupgrade conflicts\r\n- Scenario: User can upgrade only conflict-free rules when a mix of\r\nrules with and without conflicts are selected for upgrade\r\n- Scenario: User can upgrade only conflict-free rules when attempting to\r\nupgrade all rules\r\n- Scenario: User can upgrade rule with rule type change individually\r\n- Scenario: User can not bulk upgrade selected rules with rule type\r\nchanges\r\n- Scenario: User can not bulk upgrade all rules with rule type changes\r\n- Scenario: API does not upgrade prebuilt rules if they are up to date\r\n\r\n---------\r\n\r\nCo-authored-by: Georgii Gorbachev <georgii.gorbachev@elastic.co>","sha":"e29033a726930c3d891aa77ea4f7ed990c7ee511"}},{"branch":"8.x","label":"v8.19.0","branchLabelMappingKey":"^v8.19.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> Co-authored-by: Maxim Palenov <maxim.palenov@elastic.co>
kibanamachine
added a commit
that referenced
this pull request
Feb 14, 2025
…hout preview (Rule Updates table) v2 (#205922) (#211245) # Backport This will backport the following commits from `main` to `8.18`: - [[Security Solution] Test plan for upgrading prebuilt rules without preview (Rule Updates table) v2 (#205922)](#205922) <!--- Backport version: 9.4.3 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Maxim Palenov","email":"maxim.palenov@elastic.co"},"sourceCommit":{"committedDate":"2025-02-14T15:18:40Z","message":"[Security Solution] Test plan for upgrading prebuilt rules without preview (Rule Updates table) v2 (#205922)\n\n**Addresses:** https://github.com/elastic/kibana/issues/202078\r\n\r\n## Summary\r\n\r\nThis PR extends rule upgrade test plan with customizable and\r\nnon-customizable field examples. Rule upgrade workflow test plan\r\n(excluding Rule Upgrade flyout) was initially extended in\r\nhttps://github.com//pull/203331.\r\n\r\nhttps://github.com//pull/203331 adds the following rule\r\nupgrade workflow scenarios\r\n\r\n- Scenario: User can upgrade conflict-free prebuilt rules one by one\r\n- Scenario: User cannot upgrade prebuilt rules one by one from Rules\r\nUpdate table if they have conflicts\r\n- Scenario: User can upgrade multiple conflict-free prebuilt rules\r\nselected on the page\r\n- Scenario: User cannot upgrade selected prebuilt rules with conflicts\r\n- Scenario: User can upgrade all available conflict-free prebuilt rules\r\nat once*\r\n- Scenario: User cannot upgrade all prebuilt rules at once if they have\r\nupgrade conflicts\r\n- Scenario: User can upgrade only conflict-free rules when a mix of\r\nrules with and without conflicts are selected for upgrade\r\n- Scenario: User can upgrade only conflict-free rules when attempting to\r\nupgrade all rules\r\n- Scenario: User can upgrade rule with rule type change individually\r\n- Scenario: User can not bulk upgrade selected rules with rule type\r\nchanges\r\n- Scenario: User can not bulk upgrade all rules with rule type changes\r\n- Scenario: API does not upgrade prebuilt rules if they are up to date\r\n\r\n---------\r\n\r\nCo-authored-by: Georgii Gorbachev <georgii.gorbachev@elastic.co>","sha":"e29033a726930c3d891aa77ea4f7ed990c7ee511","branchLabelMapping":{"^v9.1.0$":"main","^v8.19.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","test-plan","v9.0.0","Team:Detections and Resp","Team: SecuritySolution","Team:Detection Rule Management","Feature:Prebuilt Detection Rules","backport:version","v8.18.0","v9.1.0","v8.19.0"],"title":"[Security Solution] Test plan for upgrading prebuilt rules without preview (Rule Updates table) v2","number":205922,"url":"https://github.com/elastic/kibana/pull/205922","mergeCommit":{"message":"[Security Solution] Test plan for upgrading prebuilt rules without preview (Rule Updates table) v2 (#205922)\n\n**Addresses:** https://github.com/elastic/kibana/issues/202078\r\n\r\n## Summary\r\n\r\nThis PR extends rule upgrade test plan with customizable and\r\nnon-customizable field examples. Rule upgrade workflow test plan\r\n(excluding Rule Upgrade flyout) was initially extended in\r\nhttps://github.com//pull/203331.\r\n\r\nhttps://github.com//pull/203331 adds the following rule\r\nupgrade workflow scenarios\r\n\r\n- Scenario: User can upgrade conflict-free prebuilt rules one by one\r\n- Scenario: User cannot upgrade prebuilt rules one by one from Rules\r\nUpdate table if they have conflicts\r\n- Scenario: User can upgrade multiple conflict-free prebuilt rules\r\nselected on the page\r\n- Scenario: User cannot upgrade selected prebuilt rules with conflicts\r\n- Scenario: User can upgrade all available conflict-free prebuilt rules\r\nat once*\r\n- Scenario: User cannot upgrade all prebuilt rules at once if they have\r\nupgrade conflicts\r\n- Scenario: User can upgrade only conflict-free rules when a mix of\r\nrules with and without conflicts are selected for upgrade\r\n- Scenario: User can upgrade only conflict-free rules when attempting to\r\nupgrade all rules\r\n- Scenario: User can upgrade rule with rule type change individually\r\n- Scenario: User can not bulk upgrade selected rules with rule type\r\nchanges\r\n- Scenario: User can not bulk upgrade all rules with rule type changes\r\n- Scenario: API does not upgrade prebuilt rules if they are up to date\r\n\r\n---------\r\n\r\nCo-authored-by: Georgii Gorbachev <georgii.gorbachev@elastic.co>","sha":"e29033a726930c3d891aa77ea4f7ed990c7ee511"}},"sourceBranch":"main","suggestedTargetBranches":["9.0","8.18","8.x"],"targetPullRequestStates":[{"branch":"9.0","label":"v9.0.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"8.18","label":"v8.18.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/205922","number":205922,"mergeCommit":{"message":"[Security Solution] Test plan for upgrading prebuilt rules without preview (Rule Updates table) v2 (#205922)\n\n**Addresses:** https://github.com/elastic/kibana/issues/202078\r\n\r\n## Summary\r\n\r\nThis PR extends rule upgrade test plan with customizable and\r\nnon-customizable field examples. Rule upgrade workflow test plan\r\n(excluding Rule Upgrade flyout) was initially extended in\r\nhttps://github.com//pull/203331.\r\n\r\nhttps://github.com//pull/203331 adds the following rule\r\nupgrade workflow scenarios\r\n\r\n- Scenario: User can upgrade conflict-free prebuilt rules one by one\r\n- Scenario: User cannot upgrade prebuilt rules one by one from Rules\r\nUpdate table if they have conflicts\r\n- Scenario: User can upgrade multiple conflict-free prebuilt rules\r\nselected on the page\r\n- Scenario: User cannot upgrade selected prebuilt rules with conflicts\r\n- Scenario: User can upgrade all available conflict-free prebuilt rules\r\nat once*\r\n- Scenario: User cannot upgrade all prebuilt rules at once if they have\r\nupgrade conflicts\r\n- Scenario: User can upgrade only conflict-free rules when a mix of\r\nrules with and without conflicts are selected for upgrade\r\n- Scenario: User can upgrade only conflict-free rules when attempting to\r\nupgrade all rules\r\n- Scenario: User can upgrade rule with rule type change individually\r\n- Scenario: User can not bulk upgrade selected rules with rule type\r\nchanges\r\n- Scenario: User can not bulk upgrade all rules with rule type changes\r\n- Scenario: API does not upgrade prebuilt rules if they are up to date\r\n\r\n---------\r\n\r\nCo-authored-by: Georgii Gorbachev <georgii.gorbachev@elastic.co>","sha":"e29033a726930c3d891aa77ea4f7ed990c7ee511"}},{"branch":"8.x","label":"v8.19.0","branchLabelMappingKey":"^v8.19.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> Co-authored-by: Maxim Palenov <maxim.palenov@elastic.co>
maximpn
added a commit
that referenced
this pull request
Feb 21, 2025
…t rule customization feature (#209260) **Addresses:** #202078 **Partially implements test plan:** #205922 ## Summary This PR implements the Rule Upgrade test plan added in #205922 and #203331. ## Details Tests cover only enabled customization and organized in a following way - `diffable_rule_fields` folder contains per field tests. It focuses on `DiffableRule`* fields. Each field has rule preview and upgrade scenarios throughly covered. Assertion on stats are included in rule preview tests to make sure the maximum coverage with reasonable execution time. **Diffable rule fields` tests were moved to a separate execution group to avoid exceeding execution limits.** - `preview_prebuilt_rules_upgrade.ts` integration tests on `/internal/prebuilt_rules/upgrade/_review` endpoint. It uses only `name` and `tags` fields to minimize tests complexity. Basically it should be considered as smoke tests. - `upgrade_prebuilt_rules` integrations test on `/internal/prebuilt_rules/upgrade/_perform` endpoint. It uses only `name` and `tags` fields to minimize tests complexity. Basically it should be considered as smoke tests. This PR supersedes #205217 and #205339. * `DiffableRule` was added in the scope of prebuilt rules customization epic to serve rule upgrades preview and performing upgrades. It represents slightly reorganized rule fields to simplify prebuilt rule upgrade workflow handling. There are utility functions transforming between `RuleResponse` and `DiffableRule`. ## Flaky test runner Flaky test runs for 100 iterations were successful (#209260 (comment) and #209260 (comment)).
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this pull request
Feb 21, 2025
…t rule customization feature (elastic#209260) **Addresses:** elastic#202078 **Partially implements test plan:** elastic#205922 ## Summary This PR implements the Rule Upgrade test plan added in elastic#205922 and elastic#203331. ## Details Tests cover only enabled customization and organized in a following way - `diffable_rule_fields` folder contains per field tests. It focuses on `DiffableRule`* fields. Each field has rule preview and upgrade scenarios throughly covered. Assertion on stats are included in rule preview tests to make sure the maximum coverage with reasonable execution time. **Diffable rule fields` tests were moved to a separate execution group to avoid exceeding execution limits.** - `preview_prebuilt_rules_upgrade.ts` integration tests on `/internal/prebuilt_rules/upgrade/_review` endpoint. It uses only `name` and `tags` fields to minimize tests complexity. Basically it should be considered as smoke tests. - `upgrade_prebuilt_rules` integrations test on `/internal/prebuilt_rules/upgrade/_perform` endpoint. It uses only `name` and `tags` fields to minimize tests complexity. Basically it should be considered as smoke tests. This PR supersedes elastic#205217 and elastic#205339. * `DiffableRule` was added in the scope of prebuilt rules customization epic to serve rule upgrades preview and performing upgrades. It represents slightly reorganized rule fields to simplify prebuilt rule upgrade workflow handling. There are utility functions transforming between `RuleResponse` and `DiffableRule`. ## Flaky test runner Flaky test runs for 100 iterations were successful (elastic#209260 (comment) and elastic#209260 (comment)). (cherry picked from commit 4909770)
maximpn
added a commit
to maximpn/kibana
that referenced
this pull request
Feb 24, 2025
…t rule customization feature (elastic#209260) **Addresses:** elastic#202078 **Partially implements test plan:** elastic#205922 ## Summary This PR implements the Rule Upgrade test plan added in elastic#205922 and elastic#203331. ## Details Tests cover only enabled customization and organized in a following way - `diffable_rule_fields` folder contains per field tests. It focuses on `DiffableRule`* fields. Each field has rule preview and upgrade scenarios throughly covered. Assertion on stats are included in rule preview tests to make sure the maximum coverage with reasonable execution time. **Diffable rule fields` tests were moved to a separate execution group to avoid exceeding execution limits.** - `preview_prebuilt_rules_upgrade.ts` integration tests on `/internal/prebuilt_rules/upgrade/_review` endpoint. It uses only `name` and `tags` fields to minimize tests complexity. Basically it should be considered as smoke tests. - `upgrade_prebuilt_rules` integrations test on `/internal/prebuilt_rules/upgrade/_perform` endpoint. It uses only `name` and `tags` fields to minimize tests complexity. Basically it should be considered as smoke tests. This PR supersedes elastic#205217 and elastic#205339. * `DiffableRule` was added in the scope of prebuilt rules customization epic to serve rule upgrades preview and performing upgrades. It represents slightly reorganized rule fields to simplify prebuilt rule upgrade workflow handling. There are utility functions transforming between `RuleResponse` and `DiffableRule`. ## Flaky test runner Flaky test runs for 100 iterations were successful (elastic#209260 (comment) and elastic#209260 (comment)). (cherry picked from commit 4909770) # Conflicts: # x-pack/test/security_solution_api_integration/test_suites/detections_response/rules_management/prebuilt_rules/prebuilt_rule_customization/customization_enabled/upgrade_perform_prebuilt_rules.all_rules_mode.ts # x-pack/test/security_solution_api_integration/test_suites/detections_response/rules_management/prebuilt_rules/prebuilt_rule_customization/customization_enabled/upgrade_perform_prebuilt_rules.specific_rules_mode.ts
JoseLuisGJ
pushed a commit
to JoseLuisGJ/kibana
that referenced
this pull request
Feb 27, 2025
…t rule customization feature (elastic#209260) **Addresses:** elastic#202078 **Partially implements test plan:** elastic#205922 ## Summary This PR implements the Rule Upgrade test plan added in elastic#205922 and elastic#203331. ## Details Tests cover only enabled customization and organized in a following way - `diffable_rule_fields` folder contains per field tests. It focuses on `DiffableRule`* fields. Each field has rule preview and upgrade scenarios throughly covered. Assertion on stats are included in rule preview tests to make sure the maximum coverage with reasonable execution time. **Diffable rule fields` tests were moved to a separate execution group to avoid exceeding execution limits.** - `preview_prebuilt_rules_upgrade.ts` integration tests on `/internal/prebuilt_rules/upgrade/_review` endpoint. It uses only `name` and `tags` fields to minimize tests complexity. Basically it should be considered as smoke tests. - `upgrade_prebuilt_rules` integrations test on `/internal/prebuilt_rules/upgrade/_perform` endpoint. It uses only `name` and `tags` fields to minimize tests complexity. Basically it should be considered as smoke tests. This PR supersedes elastic#205217 and elastic#205339. * `DiffableRule` was added in the scope of prebuilt rules customization epic to serve rule upgrades preview and performing upgrades. It represents slightly reorganized rule fields to simplify prebuilt rule upgrade workflow handling. There are utility functions transforming between `RuleResponse` and `DiffableRule`. ## Flaky test runner Flaky test runs for 100 iterations were successful (elastic#209260 (comment) and elastic#209260 (comment)).
CAWilson94
pushed a commit
to CAWilson94/kibana
that referenced
this pull request
Mar 22, 2025
…eview (Rule Updates table) v2 (elastic#205922) **Addresses:** elastic#202078 ## Summary This PR extends rule upgrade test plan with customizable and non-customizable field examples. Rule upgrade workflow test plan (excluding Rule Upgrade flyout) was initially extended in elastic#203331. elastic#203331 adds the following rule upgrade workflow scenarios - Scenario: User can upgrade conflict-free prebuilt rules one by one - Scenario: User cannot upgrade prebuilt rules one by one from Rules Update table if they have conflicts - Scenario: User can upgrade multiple conflict-free prebuilt rules selected on the page - Scenario: User cannot upgrade selected prebuilt rules with conflicts - Scenario: User can upgrade all available conflict-free prebuilt rules at once* - Scenario: User cannot upgrade all prebuilt rules at once if they have upgrade conflicts - Scenario: User can upgrade only conflict-free rules when a mix of rules with and without conflicts are selected for upgrade - Scenario: User can upgrade only conflict-free rules when attempting to upgrade all rules - Scenario: User can upgrade rule with rule type change individually - Scenario: User can not bulk upgrade selected rules with rule type changes - Scenario: User can not bulk upgrade all rules with rule type changes - Scenario: API does not upgrade prebuilt rules if they are up to date --------- Co-authored-by: Georgii Gorbachev <georgii.gorbachev@elastic.co>
CAWilson94
pushed a commit
to CAWilson94/kibana
that referenced
this pull request
Mar 22, 2025
…t rule customization feature (elastic#209260) **Addresses:** elastic#202078 **Partially implements test plan:** elastic#205922 ## Summary This PR implements the Rule Upgrade test plan added in elastic#205922 and elastic#203331. ## Details Tests cover only enabled customization and organized in a following way - `diffable_rule_fields` folder contains per field tests. It focuses on `DiffableRule`* fields. Each field has rule preview and upgrade scenarios throughly covered. Assertion on stats are included in rule preview tests to make sure the maximum coverage with reasonable execution time. **Diffable rule fields` tests were moved to a separate execution group to avoid exceeding execution limits.** - `preview_prebuilt_rules_upgrade.ts` integration tests on `/internal/prebuilt_rules/upgrade/_review` endpoint. It uses only `name` and `tags` fields to minimize tests complexity. Basically it should be considered as smoke tests. - `upgrade_prebuilt_rules` integrations test on `/internal/prebuilt_rules/upgrade/_perform` endpoint. It uses only `name` and `tags` fields to minimize tests complexity. Basically it should be considered as smoke tests. This PR supersedes elastic#205217 and elastic#205339. * `DiffableRule` was added in the scope of prebuilt rules customization epic to serve rule upgrades preview and performing upgrades. It represents slightly reorganized rule fields to simplify prebuilt rule upgrade workflow handling. There are utility functions transforming between `RuleResponse` and `DiffableRule`. ## Flaky test runner Flaky test runs for 100 iterations were successful (elastic#209260 (comment) and elastic#209260 (comment)).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Partially resolves: #202078
Summary
This PR refactors the test plans involving the Installation and Upgrade Workflows:
installation_and_upgrade.mdfile into three files:installation.mdcovering exclusively prebuilt rule install workflowsupgrade.mdcovering exclusively prebuilt rule upgrade workflowspackage_installation_and_upgrade.mdwhich covers scenarios for the installation and upgrades of the Prebuilt Rules package itself.The Upgrade of Prebuilt Rules (
upgrade.md) was modified as follows:Upgrading prebuilt rules one-by-one without previewandUpgrading prebuilt rules in bulkin the Rule Updates table, including cases with and without conflicts, as well as a mixture of both)Checklist
Check the PR satisfies following conditions.
Reviewers should verify this PR satisfies this list as well.
release_note:*label is applied per the guidelines