[Security Solution] Test plan for upgrading prebuilt rules without preview (Rule Updates table) v2#205922
Merged
banderror merged 3 commits intoelastic:mainfrom Feb 14, 2025
Merged
Conversation
maximpn
added
test-plan
v9.0.0
Team:Detections and Resp
Contributor
|
Pinging @elastic/security-detections-response (Team:Detections and Resp) |
Contributor
|
Pinging @elastic/security-solution (Team: SecuritySolution) |
Contributor
|
Pinging @elastic/security-detection-rule-management (Team:Detection Rule Management) |
19 tasks
Contributor
💚 Build Succeeded
cc @maximpn |
maximpn
added
release_note:skip
1 task
pborgonovi
reviewed
Feb 3, 2025
Contributor
pborgonovi
left a comment
pborgonovi
left a comment
There was a problem hiding this comment.
Hey @maximpn
For Rule Type changes, I missed a scenario with mixed selection of rules with and without rule type changes during an upgrade
Contributor
|
@elasticmachine merge upstream |
banderror
changed the title
banderror
reviewed
Feb 5, 2025
Contributor
banderror
left a comment
banderror
left a comment
There was a problem hiding this comment.
Hey @maximpn, this doesn't look like a finished PR. I posted some comments, but it would be faster if I could either push new commits to your branch, or open a PR on top of this PR with some fixes, or open a new PR based on this one.
...ugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/upgrade.md
Outdated
Show resolved
Hide resolved
...ugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/upgrade.md
Outdated
Show resolved
Hide resolved
...ugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/upgrade.md
Outdated
Show resolved
Hide resolved
...ugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/upgrade.md
Outdated
Show resolved
Hide resolved
...ugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/upgrade.md
Outdated
Show resolved
Hide resolved
Contributor
There was a problem hiding this comment.
Diffable rule field Why is it important to specify technical details of our domain model and API contract in a test plan? Do we elaborate on what these are and how are used?
Contributor
There was a problem hiding this comment.
I'm going to address this in a follow-up PR.
Contributor
There was a problem hiding this comment.
Where the following fields support customizations
- Is it important to list the fields that support customization in the test plan for the rule upgrade workflow?
- We have a test plan
rule_customization.mdwhere we don't list them, somehow. - We have two files in
/test_plans/detection_response/prebuilt_rules/shared_assetsthat seem to duplicate this information
Contributor
There was a problem hiding this comment.
I'm going to address this in a follow-up PR.
Comment on lines
170
to
180
Contributor
There was a problem hiding this comment.
Not sure I get what this table is trying to explain.
- Besides these three fields, there are other non-customizable fields. Some of them are technical, such as
rule_sourceandupdated_at. Some of them are rule parameters, such asauthorandlicense. - If "Rule signature id stays unchanged after rule upgrades", why do we list it under "prebuilt rules with upgrades in the following non-customizable fields"? This field is not upgradable or editable at all.
Contributor
There was a problem hiding this comment.
I'm going to address this in a follow-up PR.
banderror
force-pushed
the
rule-upgrade-test-plan
branch
from
4577e96 to
64bf2c8
Compare
banderror
force-pushed
the
rule-upgrade-test-plan
branch
from
64bf2c8 to
3e45d24
Compare
Contributor
|
Starting backport for target branches: 8.18, 8.x, 9.0 https://github.com/elastic/kibana/actions/runs/13332104808 |
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this pull request
Feb 14, 2025
…eview (Rule Updates table) v2 (elastic#205922) **Addresses:** elastic#202078 ## Summary This PR extends rule upgrade test plan with customizable and non-customizable field examples. Rule upgrade workflow test plan (excluding Rule Upgrade flyout) was initially extended in elastic#203331. elastic#203331 adds the following rule upgrade workflow scenarios - Scenario: User can upgrade conflict-free prebuilt rules one by one - Scenario: User cannot upgrade prebuilt rules one by one from Rules Update table if they have conflicts - Scenario: User can upgrade multiple conflict-free prebuilt rules selected on the page - Scenario: User cannot upgrade selected prebuilt rules with conflicts - Scenario: User can upgrade all available conflict-free prebuilt rules at once* - Scenario: User cannot upgrade all prebuilt rules at once if they have upgrade conflicts - Scenario: User can upgrade only conflict-free rules when a mix of rules with and without conflicts are selected for upgrade - Scenario: User can upgrade only conflict-free rules when attempting to upgrade all rules - Scenario: User can upgrade rule with rule type change individually - Scenario: User can not bulk upgrade selected rules with rule type changes - Scenario: User can not bulk upgrade all rules with rule type changes - Scenario: API does not upgrade prebuilt rules if they are up to date --------- Co-authored-by: Georgii Gorbachev <georgii.gorbachev@elastic.co> (cherry picked from commit e29033a)
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this pull request
Feb 14, 2025
…eview (Rule Updates table) v2 (elastic#205922) **Addresses:** elastic#202078 ## Summary This PR extends rule upgrade test plan with customizable and non-customizable field examples. Rule upgrade workflow test plan (excluding Rule Upgrade flyout) was initially extended in elastic#203331. elastic#203331 adds the following rule upgrade workflow scenarios - Scenario: User can upgrade conflict-free prebuilt rules one by one - Scenario: User cannot upgrade prebuilt rules one by one from Rules Update table if they have conflicts - Scenario: User can upgrade multiple conflict-free prebuilt rules selected on the page - Scenario: User cannot upgrade selected prebuilt rules with conflicts - Scenario: User can upgrade all available conflict-free prebuilt rules at once* - Scenario: User cannot upgrade all prebuilt rules at once if they have upgrade conflicts - Scenario: User can upgrade only conflict-free rules when a mix of rules with and without conflicts are selected for upgrade - Scenario: User can upgrade only conflict-free rules when attempting to upgrade all rules - Scenario: User can upgrade rule with rule type change individually - Scenario: User can not bulk upgrade selected rules with rule type changes - Scenario: User can not bulk upgrade all rules with rule type changes - Scenario: API does not upgrade prebuilt rules if they are up to date --------- Co-authored-by: Georgii Gorbachev <georgii.gorbachev@elastic.co> (cherry picked from commit e29033a)
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this pull request
Feb 14, 2025
…eview (Rule Updates table) v2 (elastic#205922) **Addresses:** elastic#202078 ## Summary This PR extends rule upgrade test plan with customizable and non-customizable field examples. Rule upgrade workflow test plan (excluding Rule Upgrade flyout) was initially extended in elastic#203331. elastic#203331 adds the following rule upgrade workflow scenarios - Scenario: User can upgrade conflict-free prebuilt rules one by one - Scenario: User cannot upgrade prebuilt rules one by one from Rules Update table if they have conflicts - Scenario: User can upgrade multiple conflict-free prebuilt rules selected on the page - Scenario: User cannot upgrade selected prebuilt rules with conflicts - Scenario: User can upgrade all available conflict-free prebuilt rules at once* - Scenario: User cannot upgrade all prebuilt rules at once if they have upgrade conflicts - Scenario: User can upgrade only conflict-free rules when a mix of rules with and without conflicts are selected for upgrade - Scenario: User can upgrade only conflict-free rules when attempting to upgrade all rules - Scenario: User can upgrade rule with rule type change individually - Scenario: User can not bulk upgrade selected rules with rule type changes - Scenario: User can not bulk upgrade all rules with rule type changes - Scenario: API does not upgrade prebuilt rules if they are up to date --------- Co-authored-by: Georgii Gorbachev <georgii.gorbachev@elastic.co> (cherry picked from commit e29033a)
Contributor
💚 All backports created successfully
Note: Successful backport PRs will be merged automatically after passing CI. Questions ?Please refer to the Backport tool documentation |
kibanamachine
added a commit
that referenced
this pull request
Feb 14, 2025
…out preview (Rule Updates table) v2 (#205922) (#211246) # Backport This will backport the following commits from `main` to `8.x`: - [[Security Solution] Test plan for upgrading prebuilt rules without preview (Rule Updates table) v2 (#205922)](#205922) <!--- Backport version: 9.4.3 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Maxim Palenov","email":"maxim.palenov@elastic.co"},"sourceCommit":{"committedDate":"2025-02-14T15:18:40Z","message":"[Security Solution] Test plan for upgrading prebuilt rules without preview (Rule Updates table) v2 (#205922)\n\n**Addresses:** https://github.com/elastic/kibana/issues/202078\r\n\r\n## Summary\r\n\r\nThis PR extends rule upgrade test plan with customizable and\r\nnon-customizable field examples. Rule upgrade workflow test plan\r\n(excluding Rule Upgrade flyout) was initially extended in\r\nhttps://github.com//pull/203331.\r\n\r\nhttps://github.com//pull/203331 adds the following rule\r\nupgrade workflow scenarios\r\n\r\n- Scenario: User can upgrade conflict-free prebuilt rules one by one\r\n- Scenario: User cannot upgrade prebuilt rules one by one from Rules\r\nUpdate table if they have conflicts\r\n- Scenario: User can upgrade multiple conflict-free prebuilt rules\r\nselected on the page\r\n- Scenario: User cannot upgrade selected prebuilt rules with conflicts\r\n- Scenario: User can upgrade all available conflict-free prebuilt rules\r\nat once*\r\n- Scenario: User cannot upgrade all prebuilt rules at once if they have\r\nupgrade conflicts\r\n- Scenario: User can upgrade only conflict-free rules when a mix of\r\nrules with and without conflicts are selected for upgrade\r\n- Scenario: User can upgrade only conflict-free rules when attempting to\r\nupgrade all rules\r\n- Scenario: User can upgrade rule with rule type change individually\r\n- Scenario: User can not bulk upgrade selected rules with rule type\r\nchanges\r\n- Scenario: User can not bulk upgrade all rules with rule type changes\r\n- Scenario: API does not upgrade prebuilt rules if they are up to date\r\n\r\n---------\r\n\r\nCo-authored-by: Georgii Gorbachev <georgii.gorbachev@elastic.co>","sha":"e29033a726930c3d891aa77ea4f7ed990c7ee511","branchLabelMapping":{"^v9.1.0$":"main","^v8.19.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","test-plan","v9.0.0","Team:Detections and Resp","Team: SecuritySolution","Team:Detection Rule Management","Feature:Prebuilt Detection Rules","backport:version","v8.18.0","v9.1.0","v8.19.0"],"title":"[Security Solution] Test plan for upgrading prebuilt rules without preview (Rule Updates table) v2","number":205922,"url":"https://github.com/elastic/kibana/pull/205922","mergeCommit":{"message":"[Security Solution] Test plan for upgrading prebuilt rules without preview (Rule Updates table) v2 (#205922)\n\n**Addresses:** https://github.com/elastic/kibana/issues/202078\r\n\r\n## Summary\r\n\r\nThis PR extends rule upgrade test plan with customizable and\r\nnon-customizable field examples. Rule upgrade workflow test plan\r\n(excluding Rule Upgrade flyout) was initially extended in\r\nhttps://github.com//pull/203331.\r\n\r\nhttps://github.com//pull/203331 adds the following rule\r\nupgrade workflow scenarios\r\n\r\n- Scenario: User can upgrade conflict-free prebuilt rules one by one\r\n- Scenario: User cannot upgrade prebuilt rules one by one from Rules\r\nUpdate table if they have conflicts\r\n- Scenario: User can upgrade multiple conflict-free prebuilt rules\r\nselected on the page\r\n- Scenario: User cannot upgrade selected prebuilt rules with conflicts\r\n- Scenario: User can upgrade all available conflict-free prebuilt rules\r\nat once*\r\n- Scenario: User cannot upgrade all prebuilt rules at once if they have\r\nupgrade conflicts\r\n- Scenario: User can upgrade only conflict-free rules when a mix of\r\nrules with and without conflicts are selected for upgrade\r\n- Scenario: User can upgrade only conflict-free rules when attempting to\r\nupgrade all rules\r\n- Scenario: User can upgrade rule with rule type change individually\r\n- Scenario: User can not bulk upgrade selected rules with rule type\r\nchanges\r\n- Scenario: User can not bulk upgrade all rules with rule type changes\r\n- Scenario: API does not upgrade prebuilt rules if they are up to date\r\n\r\n---------\r\n\r\nCo-authored-by: Georgii Gorbachev <georgii.gorbachev@elastic.co>","sha":"e29033a726930c3d891aa77ea4f7ed990c7ee511"}},"sourceBranch":"main","suggestedTargetBranches":["9.0","8.18","8.x"],"targetPullRequestStates":[{"branch":"9.0","label":"v9.0.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"8.18","label":"v8.18.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/205922","number":205922,"mergeCommit":{"message":"[Security Solution] Test plan for upgrading prebuilt rules without preview (Rule Updates table) v2 (#205922)\n\n**Addresses:** https://github.com/elastic/kibana/issues/202078\r\n\r\n## Summary\r\n\r\nThis PR extends rule upgrade test plan with customizable and\r\nnon-customizable field examples. Rule upgrade workflow test plan\r\n(excluding Rule Upgrade flyout) was initially extended in\r\nhttps://github.com//pull/203331.\r\n\r\nhttps://github.com//pull/203331 adds the following rule\r\nupgrade workflow scenarios\r\n\r\n- Scenario: User can upgrade conflict-free prebuilt rules one by one\r\n- Scenario: User cannot upgrade prebuilt rules one by one from Rules\r\nUpdate table if they have conflicts\r\n- Scenario: User can upgrade multiple conflict-free prebuilt rules\r\nselected on the page\r\n- Scenario: User cannot upgrade selected prebuilt rules with conflicts\r\n- Scenario: User can upgrade all available conflict-free prebuilt rules\r\nat once*\r\n- Scenario: User cannot upgrade all prebuilt rules at once if they have\r\nupgrade conflicts\r\n- Scenario: User can upgrade only conflict-free rules when a mix of\r\nrules with and without conflicts are selected for upgrade\r\n- Scenario: User can upgrade only conflict-free rules when attempting to\r\nupgrade all rules\r\n- Scenario: User can upgrade rule with rule type change individually\r\n- Scenario: User can not bulk upgrade selected rules with rule type\r\nchanges\r\n- Scenario: User can not bulk upgrade all rules with rule type changes\r\n- Scenario: API does not upgrade prebuilt rules if they are up to date\r\n\r\n---------\r\n\r\nCo-authored-by: Georgii Gorbachev <georgii.gorbachev@elastic.co>","sha":"e29033a726930c3d891aa77ea4f7ed990c7ee511"}},{"branch":"8.x","label":"v8.19.0","branchLabelMappingKey":"^v8.19.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> Co-authored-by: Maxim Palenov <maxim.palenov@elastic.co>
kibanamachine
added a commit
that referenced
this pull request
Feb 14, 2025
…out preview (Rule Updates table) v2 (#205922) (#211247) # Backport This will backport the following commits from `main` to `9.0`: - [[Security Solution] Test plan for upgrading prebuilt rules without preview (Rule Updates table) v2 (#205922)](#205922) <!--- Backport version: 9.4.3 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Maxim Palenov","email":"maxim.palenov@elastic.co"},"sourceCommit":{"committedDate":"2025-02-14T15:18:40Z","message":"[Security Solution] Test plan for upgrading prebuilt rules without preview (Rule Updates table) v2 (#205922)\n\n**Addresses:** https://github.com/elastic/kibana/issues/202078\r\n\r\n## Summary\r\n\r\nThis PR extends rule upgrade test plan with customizable and\r\nnon-customizable field examples. Rule upgrade workflow test plan\r\n(excluding Rule Upgrade flyout) was initially extended in\r\nhttps://github.com//pull/203331.\r\n\r\nhttps://github.com//pull/203331 adds the following rule\r\nupgrade workflow scenarios\r\n\r\n- Scenario: User can upgrade conflict-free prebuilt rules one by one\r\n- Scenario: User cannot upgrade prebuilt rules one by one from Rules\r\nUpdate table if they have conflicts\r\n- Scenario: User can upgrade multiple conflict-free prebuilt rules\r\nselected on the page\r\n- Scenario: User cannot upgrade selected prebuilt rules with conflicts\r\n- Scenario: User can upgrade all available conflict-free prebuilt rules\r\nat once*\r\n- Scenario: User cannot upgrade all prebuilt rules at once if they have\r\nupgrade conflicts\r\n- Scenario: User can upgrade only conflict-free rules when a mix of\r\nrules with and without conflicts are selected for upgrade\r\n- Scenario: User can upgrade only conflict-free rules when attempting to\r\nupgrade all rules\r\n- Scenario: User can upgrade rule with rule type change individually\r\n- Scenario: User can not bulk upgrade selected rules with rule type\r\nchanges\r\n- Scenario: User can not bulk upgrade all rules with rule type changes\r\n- Scenario: API does not upgrade prebuilt rules if they are up to date\r\n\r\n---------\r\n\r\nCo-authored-by: Georgii Gorbachev <georgii.gorbachev@elastic.co>","sha":"e29033a726930c3d891aa77ea4f7ed990c7ee511","branchLabelMapping":{"^v9.1.0$":"main","^v8.19.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","test-plan","v9.0.0","Team:Detections and Resp","Team: SecuritySolution","Team:Detection Rule Management","Feature:Prebuilt Detection Rules","backport:version","v8.18.0","v9.1.0","v8.19.0"],"title":"[Security Solution] Test plan for upgrading prebuilt rules without preview (Rule Updates table) v2","number":205922,"url":"https://github.com/elastic/kibana/pull/205922","mergeCommit":{"message":"[Security Solution] Test plan for upgrading prebuilt rules without preview (Rule Updates table) v2 (#205922)\n\n**Addresses:** https://github.com/elastic/kibana/issues/202078\r\n\r\n## Summary\r\n\r\nThis PR extends rule upgrade test plan with customizable and\r\nnon-customizable field examples. Rule upgrade workflow test plan\r\n(excluding Rule Upgrade flyout) was initially extended in\r\nhttps://github.com//pull/203331.\r\n\r\nhttps://github.com//pull/203331 adds the following rule\r\nupgrade workflow scenarios\r\n\r\n- Scenario: User can upgrade conflict-free prebuilt rules one by one\r\n- Scenario: User cannot upgrade prebuilt rules one by one from Rules\r\nUpdate table if they have conflicts\r\n- Scenario: User can upgrade multiple conflict-free prebuilt rules\r\nselected on the page\r\n- Scenario: User cannot upgrade selected prebuilt rules with conflicts\r\n- Scenario: User can upgrade all available conflict-free prebuilt rules\r\nat once*\r\n- Scenario: User cannot upgrade all prebuilt rules at once if they have\r\nupgrade conflicts\r\n- Scenario: User can upgrade only conflict-free rules when a mix of\r\nrules with and without conflicts are selected for upgrade\r\n- Scenario: User can upgrade only conflict-free rules when attempting to\r\nupgrade all rules\r\n- Scenario: User can upgrade rule with rule type change individually\r\n- Scenario: User can not bulk upgrade selected rules with rule type\r\nchanges\r\n- Scenario: User can not bulk upgrade all rules with rule type changes\r\n- Scenario: API does not upgrade prebuilt rules if they are up to date\r\n\r\n---------\r\n\r\nCo-authored-by: Georgii Gorbachev <georgii.gorbachev@elastic.co>","sha":"e29033a726930c3d891aa77ea4f7ed990c7ee511"}},"sourceBranch":"main","suggestedTargetBranches":["9.0","8.18","8.x"],"targetPullRequestStates":[{"branch":"9.0","label":"v9.0.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"8.18","label":"v8.18.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/205922","number":205922,"mergeCommit":{"message":"[Security Solution] Test plan for upgrading prebuilt rules without preview (Rule Updates table) v2 (#205922)\n\n**Addresses:** https://github.com/elastic/kibana/issues/202078\r\n\r\n## Summary\r\n\r\nThis PR extends rule upgrade test plan with customizable and\r\nnon-customizable field examples. Rule upgrade workflow test plan\r\n(excluding Rule Upgrade flyout) was initially extended in\r\nhttps://github.com//pull/203331.\r\n\r\nhttps://github.com//pull/203331 adds the following rule\r\nupgrade workflow scenarios\r\n\r\n- Scenario: User can upgrade conflict-free prebuilt rules one by one\r\n- Scenario: User cannot upgrade prebuilt rules one by one from Rules\r\nUpdate table if they have conflicts\r\n- Scenario: User can upgrade multiple conflict-free prebuilt rules\r\nselected on the page\r\n- Scenario: User cannot upgrade selected prebuilt rules with conflicts\r\n- Scenario: User can upgrade all available conflict-free prebuilt rules\r\nat once*\r\n- Scenario: User cannot upgrade all prebuilt rules at once if they have\r\nupgrade conflicts\r\n- Scenario: User can upgrade only conflict-free rules when a mix of\r\nrules with and without conflicts are selected for upgrade\r\n- Scenario: User can upgrade only conflict-free rules when attempting to\r\nupgrade all rules\r\n- Scenario: User can upgrade rule with rule type change individually\r\n- Scenario: User can not bulk upgrade selected rules with rule type\r\nchanges\r\n- Scenario: User can not bulk upgrade all rules with rule type changes\r\n- Scenario: API does not upgrade prebuilt rules if they are up to date\r\n\r\n---------\r\n\r\nCo-authored-by: Georgii Gorbachev <georgii.gorbachev@elastic.co>","sha":"e29033a726930c3d891aa77ea4f7ed990c7ee511"}},{"branch":"8.x","label":"v8.19.0","branchLabelMappingKey":"^v8.19.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> Co-authored-by: Maxim Palenov <maxim.palenov@elastic.co>
kibanamachine
added a commit
that referenced
this pull request
Feb 14, 2025
…hout preview (Rule Updates table) v2 (#205922) (#211245) # Backport This will backport the following commits from `main` to `8.18`: - [[Security Solution] Test plan for upgrading prebuilt rules without preview (Rule Updates table) v2 (#205922)](#205922) <!--- Backport version: 9.4.3 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Maxim Palenov","email":"maxim.palenov@elastic.co"},"sourceCommit":{"committedDate":"2025-02-14T15:18:40Z","message":"[Security Solution] Test plan for upgrading prebuilt rules without preview (Rule Updates table) v2 (#205922)\n\n**Addresses:** https://github.com/elastic/kibana/issues/202078\r\n\r\n## Summary\r\n\r\nThis PR extends rule upgrade test plan with customizable and\r\nnon-customizable field examples. Rule upgrade workflow test plan\r\n(excluding Rule Upgrade flyout) was initially extended in\r\nhttps://github.com//pull/203331.\r\n\r\nhttps://github.com//pull/203331 adds the following rule\r\nupgrade workflow scenarios\r\n\r\n- Scenario: User can upgrade conflict-free prebuilt rules one by one\r\n- Scenario: User cannot upgrade prebuilt rules one by one from Rules\r\nUpdate table if they have conflicts\r\n- Scenario: User can upgrade multiple conflict-free prebuilt rules\r\nselected on the page\r\n- Scenario: User cannot upgrade selected prebuilt rules with conflicts\r\n- Scenario: User can upgrade all available conflict-free prebuilt rules\r\nat once*\r\n- Scenario: User cannot upgrade all prebuilt rules at once if they have\r\nupgrade conflicts\r\n- Scenario: User can upgrade only conflict-free rules when a mix of\r\nrules with and without conflicts are selected for upgrade\r\n- Scenario: User can upgrade only conflict-free rules when attempting to\r\nupgrade all rules\r\n- Scenario: User can upgrade rule with rule type change individually\r\n- Scenario: User can not bulk upgrade selected rules with rule type\r\nchanges\r\n- Scenario: User can not bulk upgrade all rules with rule type changes\r\n- Scenario: API does not upgrade prebuilt rules if they are up to date\r\n\r\n---------\r\n\r\nCo-authored-by: Georgii Gorbachev <georgii.gorbachev@elastic.co>","sha":"e29033a726930c3d891aa77ea4f7ed990c7ee511","branchLabelMapping":{"^v9.1.0$":"main","^v8.19.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","test-plan","v9.0.0","Team:Detections and Resp","Team: SecuritySolution","Team:Detection Rule Management","Feature:Prebuilt Detection Rules","backport:version","v8.18.0","v9.1.0","v8.19.0"],"title":"[Security Solution] Test plan for upgrading prebuilt rules without preview (Rule Updates table) v2","number":205922,"url":"https://github.com/elastic/kibana/pull/205922","mergeCommit":{"message":"[Security Solution] Test plan for upgrading prebuilt rules without preview (Rule Updates table) v2 (#205922)\n\n**Addresses:** https://github.com/elastic/kibana/issues/202078\r\n\r\n## Summary\r\n\r\nThis PR extends rule upgrade test plan with customizable and\r\nnon-customizable field examples. Rule upgrade workflow test plan\r\n(excluding Rule Upgrade flyout) was initially extended in\r\nhttps://github.com//pull/203331.\r\n\r\nhttps://github.com//pull/203331 adds the following rule\r\nupgrade workflow scenarios\r\n\r\n- Scenario: User can upgrade conflict-free prebuilt rules one by one\r\n- Scenario: User cannot upgrade prebuilt rules one by one from Rules\r\nUpdate table if they have conflicts\r\n- Scenario: User can upgrade multiple conflict-free prebuilt rules\r\nselected on the page\r\n- Scenario: User cannot upgrade selected prebuilt rules with conflicts\r\n- Scenario: User can upgrade all available conflict-free prebuilt rules\r\nat once*\r\n- Scenario: User cannot upgrade all prebuilt rules at once if they have\r\nupgrade conflicts\r\n- Scenario: User can upgrade only conflict-free rules when a mix of\r\nrules with and without conflicts are selected for upgrade\r\n- Scenario: User can upgrade only conflict-free rules when attempting to\r\nupgrade all rules\r\n- Scenario: User can upgrade rule with rule type change individually\r\n- Scenario: User can not bulk upgrade selected rules with rule type\r\nchanges\r\n- Scenario: User can not bulk upgrade all rules with rule type changes\r\n- Scenario: API does not upgrade prebuilt rules if they are up to date\r\n\r\n---------\r\n\r\nCo-authored-by: Georgii Gorbachev <georgii.gorbachev@elastic.co>","sha":"e29033a726930c3d891aa77ea4f7ed990c7ee511"}},"sourceBranch":"main","suggestedTargetBranches":["9.0","8.18","8.x"],"targetPullRequestStates":[{"branch":"9.0","label":"v9.0.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"8.18","label":"v8.18.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/205922","number":205922,"mergeCommit":{"message":"[Security Solution] Test plan for upgrading prebuilt rules without preview (Rule Updates table) v2 (#205922)\n\n**Addresses:** https://github.com/elastic/kibana/issues/202078\r\n\r\n## Summary\r\n\r\nThis PR extends rule upgrade test plan with customizable and\r\nnon-customizable field examples. Rule upgrade workflow test plan\r\n(excluding Rule Upgrade flyout) was initially extended in\r\nhttps://github.com//pull/203331.\r\n\r\nhttps://github.com//pull/203331 adds the following rule\r\nupgrade workflow scenarios\r\n\r\n- Scenario: User can upgrade conflict-free prebuilt rules one by one\r\n- Scenario: User cannot upgrade prebuilt rules one by one from Rules\r\nUpdate table if they have conflicts\r\n- Scenario: User can upgrade multiple conflict-free prebuilt rules\r\nselected on the page\r\n- Scenario: User cannot upgrade selected prebuilt rules with conflicts\r\n- Scenario: User can upgrade all available conflict-free prebuilt rules\r\nat once*\r\n- Scenario: User cannot upgrade all prebuilt rules at once if they have\r\nupgrade conflicts\r\n- Scenario: User can upgrade only conflict-free rules when a mix of\r\nrules with and without conflicts are selected for upgrade\r\n- Scenario: User can upgrade only conflict-free rules when attempting to\r\nupgrade all rules\r\n- Scenario: User can upgrade rule with rule type change individually\r\n- Scenario: User can not bulk upgrade selected rules with rule type\r\nchanges\r\n- Scenario: User can not bulk upgrade all rules with rule type changes\r\n- Scenario: API does not upgrade prebuilt rules if they are up to date\r\n\r\n---------\r\n\r\nCo-authored-by: Georgii Gorbachev <georgii.gorbachev@elastic.co>","sha":"e29033a726930c3d891aa77ea4f7ed990c7ee511"}},{"branch":"8.x","label":"v8.19.0","branchLabelMappingKey":"^v8.19.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> Co-authored-by: Maxim Palenov <maxim.palenov@elastic.co>
maximpn
added a commit
that referenced
this pull request
Feb 21, 2025
…t rule customization feature (#209260) **Addresses:** #202078 **Partially implements test plan:** #205922 ## Summary This PR implements the Rule Upgrade test plan added in #205922 and #203331. ## Details Tests cover only enabled customization and organized in a following way - `diffable_rule_fields` folder contains per field tests. It focuses on `DiffableRule`* fields. Each field has rule preview and upgrade scenarios throughly covered. Assertion on stats are included in rule preview tests to make sure the maximum coverage with reasonable execution time. **Diffable rule fields` tests were moved to a separate execution group to avoid exceeding execution limits.** - `preview_prebuilt_rules_upgrade.ts` integration tests on `/internal/prebuilt_rules/upgrade/_review` endpoint. It uses only `name` and `tags` fields to minimize tests complexity. Basically it should be considered as smoke tests. - `upgrade_prebuilt_rules` integrations test on `/internal/prebuilt_rules/upgrade/_perform` endpoint. It uses only `name` and `tags` fields to minimize tests complexity. Basically it should be considered as smoke tests. This PR supersedes #205217 and #205339. * `DiffableRule` was added in the scope of prebuilt rules customization epic to serve rule upgrades preview and performing upgrades. It represents slightly reorganized rule fields to simplify prebuilt rule upgrade workflow handling. There are utility functions transforming between `RuleResponse` and `DiffableRule`. ## Flaky test runner Flaky test runs for 100 iterations were successful (#209260 (comment) and #209260 (comment)).
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this pull request
Feb 21, 2025
…t rule customization feature (elastic#209260) **Addresses:** elastic#202078 **Partially implements test plan:** elastic#205922 ## Summary This PR implements the Rule Upgrade test plan added in elastic#205922 and elastic#203331. ## Details Tests cover only enabled customization and organized in a following way - `diffable_rule_fields` folder contains per field tests. It focuses on `DiffableRule`* fields. Each field has rule preview and upgrade scenarios throughly covered. Assertion on stats are included in rule preview tests to make sure the maximum coverage with reasonable execution time. **Diffable rule fields` tests were moved to a separate execution group to avoid exceeding execution limits.** - `preview_prebuilt_rules_upgrade.ts` integration tests on `/internal/prebuilt_rules/upgrade/_review` endpoint. It uses only `name` and `tags` fields to minimize tests complexity. Basically it should be considered as smoke tests. - `upgrade_prebuilt_rules` integrations test on `/internal/prebuilt_rules/upgrade/_perform` endpoint. It uses only `name` and `tags` fields to minimize tests complexity. Basically it should be considered as smoke tests. This PR supersedes elastic#205217 and elastic#205339. * `DiffableRule` was added in the scope of prebuilt rules customization epic to serve rule upgrades preview and performing upgrades. It represents slightly reorganized rule fields to simplify prebuilt rule upgrade workflow handling. There are utility functions transforming between `RuleResponse` and `DiffableRule`. ## Flaky test runner Flaky test runs for 100 iterations were successful (elastic#209260 (comment) and elastic#209260 (comment)). (cherry picked from commit 4909770)
kibanamachine
added a commit
that referenced
this pull request
Feb 21, 2025
…rebuilt rule customization feature (#209260) (#212045) # Backport This will backport the following commits from `main` to `9.0`: - [[Security Solution] Extend rule upgrade integration tests for prebuilt rule customization feature (#209260)](#209260) <!--- Backport version: 9.6.6 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport) <!--BACKPORT [{"author":{"name":"Maxim Palenov","email":"maxim.palenov@elastic.co"},"sourceCommit":{"committedDate":"2025-02-21T11:11:44Z","message":"[Security Solution] Extend rule upgrade integration tests for prebuilt rule customization feature (#209260)\n\n**Addresses:** https://github.com/elastic/kibana/issues/202078\n**Partially implements test plan:** https://github.com/elastic/kibana/pull/205922\n\n## Summary\n\nThis PR implements the Rule Upgrade test plan added in #205922 and https://github.com/elastic/kibana/pull/203331.\n\n## Details\n\nTests cover only enabled customization and organized in a following way\n\n- `diffable_rule_fields` folder contains per field tests. It focuses on `DiffableRule`* fields. Each field has rule preview and upgrade scenarios throughly covered. Assertion on stats are included in rule preview tests to make sure the maximum coverage with reasonable execution time. **Diffable rule fields` tests were moved to a separate execution group to avoid exceeding execution limits.**\n- `preview_prebuilt_rules_upgrade.ts` integration tests on `/internal/prebuilt_rules/upgrade/_review` endpoint. It uses only `name` and `tags` fields to minimize tests complexity. Basically it should be considered as smoke tests.\n- `upgrade_prebuilt_rules` integrations test on `/internal/prebuilt_rules/upgrade/_perform` endpoint. It uses only `name` and `tags` fields to minimize tests complexity. Basically it should be considered as smoke tests.\n\nThis PR supersedes #205217 and https://github.com/elastic/kibana/pull/205339.\n\n* `DiffableRule` was added in the scope of prebuilt rules customization epic to serve rule upgrades preview and performing upgrades. It represents slightly reorganized rule fields to simplify prebuilt rule upgrade workflow handling. There are utility functions transforming between `RuleResponse` and `DiffableRule`.\n\n## Flaky test runner\n\nFlaky test runs for 100 iterations were successful (#209260 (comment) and https://github.com/elastic/kibana/pull/209260#issuecomment-2649627389).","sha":"4909770664a97f97bc38adc91dc852b5b1e6abf7","branchLabelMapping":{"^v9.1.0$":"main","^v8.19.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["test","release_note:skip","v9.0.0","Team:Detections and Resp","Team: SecuritySolution","Team:Detection Rule Management","Feature:Prebuilt Detection Rules","backport:version","v8.18.0","v9.1.0","v8.19.0"],"title":"[Security Solution] Extend rule upgrade integration tests for prebuilt rule customization feature","number":209260,"url":"https://github.com/elastic/kibana/pull/209260","mergeCommit":{"message":"[Security Solution] Extend rule upgrade integration tests for prebuilt rule customization feature (#209260)\n\n**Addresses:** https://github.com/elastic/kibana/issues/202078\n**Partially implements test plan:** https://github.com/elastic/kibana/pull/205922\n\n## Summary\n\nThis PR implements the Rule Upgrade test plan added in #205922 and https://github.com/elastic/kibana/pull/203331.\n\n## Details\n\nTests cover only enabled customization and organized in a following way\n\n- `diffable_rule_fields` folder contains per field tests. It focuses on `DiffableRule`* fields. Each field has rule preview and upgrade scenarios throughly covered. Assertion on stats are included in rule preview tests to make sure the maximum coverage with reasonable execution time. **Diffable rule fields` tests were moved to a separate execution group to avoid exceeding execution limits.**\n- `preview_prebuilt_rules_upgrade.ts` integration tests on `/internal/prebuilt_rules/upgrade/_review` endpoint. It uses only `name` and `tags` fields to minimize tests complexity. Basically it should be considered as smoke tests.\n- `upgrade_prebuilt_rules` integrations test on `/internal/prebuilt_rules/upgrade/_perform` endpoint. It uses only `name` and `tags` fields to minimize tests complexity. Basically it should be considered as smoke tests.\n\nThis PR supersedes #205217 and https://github.com/elastic/kibana/pull/205339.\n\n* `DiffableRule` was added in the scope of prebuilt rules customization epic to serve rule upgrades preview and performing upgrades. It represents slightly reorganized rule fields to simplify prebuilt rule upgrade workflow handling. There are utility functions transforming between `RuleResponse` and `DiffableRule`.\n\n## Flaky test runner\n\nFlaky test runs for 100 iterations were successful (#209260 (comment) and https://github.com/elastic/kibana/pull/209260#issuecomment-2649627389).","sha":"4909770664a97f97bc38adc91dc852b5b1e6abf7"}},"sourceBranch":"main","suggestedTargetBranches":["9.0","8.18","8.x"],"targetPullRequestStates":[{"branch":"9.0","label":"v9.0.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"8.18","label":"v8.18.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/209260","number":209260,"mergeCommit":{"message":"[Security Solution] Extend rule upgrade integration tests for prebuilt rule customization feature (#209260)\n\n**Addresses:** https://github.com/elastic/kibana/issues/202078\n**Partially implements test plan:** https://github.com/elastic/kibana/pull/205922\n\n## Summary\n\nThis PR implements the Rule Upgrade test plan added in #205922 and https://github.com/elastic/kibana/pull/203331.\n\n## Details\n\nTests cover only enabled customization and organized in a following way\n\n- `diffable_rule_fields` folder contains per field tests. It focuses on `DiffableRule`* fields. Each field has rule preview and upgrade scenarios throughly covered. Assertion on stats are included in rule preview tests to make sure the maximum coverage with reasonable execution time. **Diffable rule fields` tests were moved to a separate execution group to avoid exceeding execution limits.**\n- `preview_prebuilt_rules_upgrade.ts` integration tests on `/internal/prebuilt_rules/upgrade/_review` endpoint. It uses only `name` and `tags` fields to minimize tests complexity. Basically it should be considered as smoke tests.\n- `upgrade_prebuilt_rules` integrations test on `/internal/prebuilt_rules/upgrade/_perform` endpoint. It uses only `name` and `tags` fields to minimize tests complexity. Basically it should be considered as smoke tests.\n\nThis PR supersedes #205217 and https://github.com/elastic/kibana/pull/205339.\n\n* `DiffableRule` was added in the scope of prebuilt rules customization epic to serve rule upgrades preview and performing upgrades. It represents slightly reorganized rule fields to simplify prebuilt rule upgrade workflow handling. There are utility functions transforming between `RuleResponse` and `DiffableRule`.\n\n## Flaky test runner\n\nFlaky test runs for 100 iterations were successful (#209260 (comment) and https://github.com/elastic/kibana/pull/209260#issuecomment-2649627389).","sha":"4909770664a97f97bc38adc91dc852b5b1e6abf7"}},{"branch":"8.x","label":"v8.19.0","branchLabelMappingKey":"^v8.19.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> Co-authored-by: Maxim Palenov <maxim.palenov@elastic.co>
maximpn
added a commit
to maximpn/kibana
that referenced
this pull request
Feb 24, 2025
…t rule customization feature (elastic#209260) **Addresses:** elastic#202078 **Partially implements test plan:** elastic#205922 ## Summary This PR implements the Rule Upgrade test plan added in elastic#205922 and elastic#203331. ## Details Tests cover only enabled customization and organized in a following way - `diffable_rule_fields` folder contains per field tests. It focuses on `DiffableRule`* fields. Each field has rule preview and upgrade scenarios throughly covered. Assertion on stats are included in rule preview tests to make sure the maximum coverage with reasonable execution time. **Diffable rule fields` tests were moved to a separate execution group to avoid exceeding execution limits.** - `preview_prebuilt_rules_upgrade.ts` integration tests on `/internal/prebuilt_rules/upgrade/_review` endpoint. It uses only `name` and `tags` fields to minimize tests complexity. Basically it should be considered as smoke tests. - `upgrade_prebuilt_rules` integrations test on `/internal/prebuilt_rules/upgrade/_perform` endpoint. It uses only `name` and `tags` fields to minimize tests complexity. Basically it should be considered as smoke tests. This PR supersedes elastic#205217 and elastic#205339. * `DiffableRule` was added in the scope of prebuilt rules customization epic to serve rule upgrades preview and performing upgrades. It represents slightly reorganized rule fields to simplify prebuilt rule upgrade workflow handling. There are utility functions transforming between `RuleResponse` and `DiffableRule`. ## Flaky test runner Flaky test runs for 100 iterations were successful (elastic#209260 (comment) and elastic#209260 (comment)). (cherry picked from commit 4909770) # Conflicts: # x-pack/test/security_solution_api_integration/test_suites/detections_response/rules_management/prebuilt_rules/prebuilt_rule_customization/customization_enabled/upgrade_perform_prebuilt_rules.all_rules_mode.ts # x-pack/test/security_solution_api_integration/test_suites/detections_response/rules_management/prebuilt_rules/prebuilt_rule_customization/customization_enabled/upgrade_perform_prebuilt_rules.specific_rules_mode.ts
maximpn
added a commit
that referenced
this pull request
Feb 24, 2025
…rebuilt rule customization feature (#209260) (#212282) # Backport This will backport the following commits from `main` to `8.x`: - [[Security Solution] Extend rule upgrade integration tests for prebuilt rule customization feature (#209260)](#209260) <!--- Backport version: 9.6.6 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport) <!--BACKPORT [{"author":{"name":"Maxim Palenov","email":"maxim.palenov@elastic.co"},"sourceCommit":{"committedDate":"2025-02-21T11:11:44Z","message":"[Security Solution] Extend rule upgrade integration tests for prebuilt rule customization feature (#209260)\n\n**Addresses:** https://github.com/elastic/kibana/issues/202078\n**Partially implements test plan:** https://github.com/elastic/kibana/pull/205922\n\n## Summary\n\nThis PR implements the Rule Upgrade test plan added in #205922 and https://github.com/elastic/kibana/pull/203331.\n\n## Details\n\nTests cover only enabled customization and organized in a following way\n\n- `diffable_rule_fields` folder contains per field tests. It focuses on `DiffableRule`* fields. Each field has rule preview and upgrade scenarios throughly covered. Assertion on stats are included in rule preview tests to make sure the maximum coverage with reasonable execution time. **Diffable rule fields` tests were moved to a separate execution group to avoid exceeding execution limits.**\n- `preview_prebuilt_rules_upgrade.ts` integration tests on `/internal/prebuilt_rules/upgrade/_review` endpoint. It uses only `name` and `tags` fields to minimize tests complexity. Basically it should be considered as smoke tests.\n- `upgrade_prebuilt_rules` integrations test on `/internal/prebuilt_rules/upgrade/_perform` endpoint. It uses only `name` and `tags` fields to minimize tests complexity. Basically it should be considered as smoke tests.\n\nThis PR supersedes #205217 and https://github.com/elastic/kibana/pull/205339.\n\n* `DiffableRule` was added in the scope of prebuilt rules customization epic to serve rule upgrades preview and performing upgrades. It represents slightly reorganized rule fields to simplify prebuilt rule upgrade workflow handling. There are utility functions transforming between `RuleResponse` and `DiffableRule`.\n\n## Flaky test runner\n\nFlaky test runs for 100 iterations were successful (#209260 (comment) and https://github.com/elastic/kibana/pull/209260#issuecomment-2649627389).","sha":"4909770664a97f97bc38adc91dc852b5b1e6abf7","branchLabelMapping":{"^v9.1.0$":"main","^v8.19.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["test","release_note:skip","v9.0.0","Team:Detections and Resp","Team: SecuritySolution","Team:Detection Rule Management","Feature:Prebuilt Detection Rules","backport:version","v8.18.0","v9.1.0","v8.19.0"],"title":"[Security Solution] Extend rule upgrade integration tests for prebuilt rule customization feature","number":209260,"url":"https://github.com/elastic/kibana/pull/209260","mergeCommit":{"message":"[Security Solution] Extend rule upgrade integration tests for prebuilt rule customization feature (#209260)\n\n**Addresses:** https://github.com/elastic/kibana/issues/202078\n**Partially implements test plan:** https://github.com/elastic/kibana/pull/205922\n\n## Summary\n\nThis PR implements the Rule Upgrade test plan added in #205922 and https://github.com/elastic/kibana/pull/203331.\n\n## Details\n\nTests cover only enabled customization and organized in a following way\n\n- `diffable_rule_fields` folder contains per field tests. It focuses on `DiffableRule`* fields. Each field has rule preview and upgrade scenarios throughly covered. Assertion on stats are included in rule preview tests to make sure the maximum coverage with reasonable execution time. **Diffable rule fields` tests were moved to a separate execution group to avoid exceeding execution limits.**\n- `preview_prebuilt_rules_upgrade.ts` integration tests on `/internal/prebuilt_rules/upgrade/_review` endpoint. It uses only `name` and `tags` fields to minimize tests complexity. Basically it should be considered as smoke tests.\n- `upgrade_prebuilt_rules` integrations test on `/internal/prebuilt_rules/upgrade/_perform` endpoint. It uses only `name` and `tags` fields to minimize tests complexity. Basically it should be considered as smoke tests.\n\nThis PR supersedes #205217 and https://github.com/elastic/kibana/pull/205339.\n\n* `DiffableRule` was added in the scope of prebuilt rules customization epic to serve rule upgrades preview and performing upgrades. It represents slightly reorganized rule fields to simplify prebuilt rule upgrade workflow handling. There are utility functions transforming between `RuleResponse` and `DiffableRule`.\n\n## Flaky test runner\n\nFlaky test runs for 100 iterations were successful (#209260 (comment) and https://github.com/elastic/kibana/pull/209260#issuecomment-2649627389).","sha":"4909770664a97f97bc38adc91dc852b5b1e6abf7"}},"sourceBranch":"main","suggestedTargetBranches":["8.18","8.x"],"targetPullRequestStates":[{"branch":"9.0","label":"v9.0.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"url":"https://github.com/elastic/kibana/pull/212045","number":212045,"state":"MERGED","mergeCommit":{"sha":"5e5989ab91c571225a3bba0fa8affd0321312aee","message":"[9.0] [Security Solution] Extend rule upgrade integration tests for prebuilt rule customization feature (#209260) (#212045)\n\n# Backport\n\nThis will backport the following commits from `main` to `9.0`:\n- [[Security Solution] Extend rule upgrade integration tests for\nprebuilt rule customization feature\n(#209260)](https://github.com/elastic/kibana/pull/209260)\n\n\n\n### Questions ?\nPlease refer to the [Backport tool\ndocumentation](https://github.com/sorenlouv/backport)\n\n\n\nCo-authored-by: Maxim Palenov <maxim.palenov@elastic.co>"}},{"branch":"8.18","label":"v8.18.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/209260","number":209260,"mergeCommit":{"message":"[Security Solution] Extend rule upgrade integration tests for prebuilt rule customization feature (#209260)\n\n**Addresses:** https://github.com/elastic/kibana/issues/202078\n**Partially implements test plan:** https://github.com/elastic/kibana/pull/205922\n\n## Summary\n\nThis PR implements the Rule Upgrade test plan added in #205922 and https://github.com/elastic/kibana/pull/203331.\n\n## Details\n\nTests cover only enabled customization and organized in a following way\n\n- `diffable_rule_fields` folder contains per field tests. It focuses on `DiffableRule`* fields. Each field has rule preview and upgrade scenarios throughly covered. Assertion on stats are included in rule preview tests to make sure the maximum coverage with reasonable execution time. **Diffable rule fields` tests were moved to a separate execution group to avoid exceeding execution limits.**\n- `preview_prebuilt_rules_upgrade.ts` integration tests on `/internal/prebuilt_rules/upgrade/_review` endpoint. It uses only `name` and `tags` fields to minimize tests complexity. Basically it should be considered as smoke tests.\n- `upgrade_prebuilt_rules` integrations test on `/internal/prebuilt_rules/upgrade/_perform` endpoint. It uses only `name` and `tags` fields to minimize tests complexity. Basically it should be considered as smoke tests.\n\nThis PR supersedes #205217 and https://github.com/elastic/kibana/pull/205339.\n\n* `DiffableRule` was added in the scope of prebuilt rules customization epic to serve rule upgrades preview and performing upgrades. It represents slightly reorganized rule fields to simplify prebuilt rule upgrade workflow handling. There are utility functions transforming between `RuleResponse` and `DiffableRule`.\n\n## Flaky test runner\n\nFlaky test runs for 100 iterations were successful (#209260 (comment) and https://github.com/elastic/kibana/pull/209260#issuecomment-2649627389).","sha":"4909770664a97f97bc38adc91dc852b5b1e6abf7"}},{"branch":"8.x","label":"v8.19.0","branchLabelMappingKey":"^v8.19.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT-->
maximpn
added a commit
that referenced
this pull request
Feb 24, 2025
…prebuilt rule customization feature (#209260) (#212283) # Backport This will backport the following commits from `main` to `8.18`: - [[Security Solution] Extend rule upgrade integration tests for prebuilt rule customization feature (#209260)](#209260) <!--- Backport version: 9.6.6 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport) <!--BACKPORT [{"author":{"name":"Maxim Palenov","email":"maxim.palenov@elastic.co"},"sourceCommit":{"committedDate":"2025-02-21T11:11:44Z","message":"[Security Solution] Extend rule upgrade integration tests for prebuilt rule customization feature (#209260)\n\n**Addresses:** https://github.com/elastic/kibana/issues/202078\n**Partially implements test plan:** https://github.com/elastic/kibana/pull/205922\n\n## Summary\n\nThis PR implements the Rule Upgrade test plan added in #205922 and https://github.com/elastic/kibana/pull/203331.\n\n## Details\n\nTests cover only enabled customization and organized in a following way\n\n- `diffable_rule_fields` folder contains per field tests. It focuses on `DiffableRule`* fields. Each field has rule preview and upgrade scenarios throughly covered. Assertion on stats are included in rule preview tests to make sure the maximum coverage with reasonable execution time. **Diffable rule fields` tests were moved to a separate execution group to avoid exceeding execution limits.**\n- `preview_prebuilt_rules_upgrade.ts` integration tests on `/internal/prebuilt_rules/upgrade/_review` endpoint. It uses only `name` and `tags` fields to minimize tests complexity. Basically it should be considered as smoke tests.\n- `upgrade_prebuilt_rules` integrations test on `/internal/prebuilt_rules/upgrade/_perform` endpoint. It uses only `name` and `tags` fields to minimize tests complexity. Basically it should be considered as smoke tests.\n\nThis PR supersedes #205217 and https://github.com/elastic/kibana/pull/205339.\n\n* `DiffableRule` was added in the scope of prebuilt rules customization epic to serve rule upgrades preview and performing upgrades. It represents slightly reorganized rule fields to simplify prebuilt rule upgrade workflow handling. There are utility functions transforming between `RuleResponse` and `DiffableRule`.\n\n## Flaky test runner\n\nFlaky test runs for 100 iterations were successful (#209260 (comment) and https://github.com/elastic/kibana/pull/209260#issuecomment-2649627389).","sha":"4909770664a97f97bc38adc91dc852b5b1e6abf7","branchLabelMapping":{"^v9.1.0$":"main","^v8.19.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["test","release_note:skip","v9.0.0","Team:Detections and Resp","Team: SecuritySolution","Team:Detection Rule Management","Feature:Prebuilt Detection Rules","backport:version","v8.18.0","v9.1.0","v8.19.0"],"title":"[Security Solution] Extend rule upgrade integration tests for prebuilt rule customization feature","number":209260,"url":"https://github.com/elastic/kibana/pull/209260","mergeCommit":{"message":"[Security Solution] Extend rule upgrade integration tests for prebuilt rule customization feature (#209260)\n\n**Addresses:** https://github.com/elastic/kibana/issues/202078\n**Partially implements test plan:** https://github.com/elastic/kibana/pull/205922\n\n## Summary\n\nThis PR implements the Rule Upgrade test plan added in #205922 and https://github.com/elastic/kibana/pull/203331.\n\n## Details\n\nTests cover only enabled customization and organized in a following way\n\n- `diffable_rule_fields` folder contains per field tests. It focuses on `DiffableRule`* fields. Each field has rule preview and upgrade scenarios throughly covered. Assertion on stats are included in rule preview tests to make sure the maximum coverage with reasonable execution time. **Diffable rule fields` tests were moved to a separate execution group to avoid exceeding execution limits.**\n- `preview_prebuilt_rules_upgrade.ts` integration tests on `/internal/prebuilt_rules/upgrade/_review` endpoint. It uses only `name` and `tags` fields to minimize tests complexity. Basically it should be considered as smoke tests.\n- `upgrade_prebuilt_rules` integrations test on `/internal/prebuilt_rules/upgrade/_perform` endpoint. It uses only `name` and `tags` fields to minimize tests complexity. Basically it should be considered as smoke tests.\n\nThis PR supersedes #205217 and https://github.com/elastic/kibana/pull/205339.\n\n* `DiffableRule` was added in the scope of prebuilt rules customization epic to serve rule upgrades preview and performing upgrades. It represents slightly reorganized rule fields to simplify prebuilt rule upgrade workflow handling. There are utility functions transforming between `RuleResponse` and `DiffableRule`.\n\n## Flaky test runner\n\nFlaky test runs for 100 iterations were successful (#209260 (comment) and https://github.com/elastic/kibana/pull/209260#issuecomment-2649627389).","sha":"4909770664a97f97bc38adc91dc852b5b1e6abf7"}},"sourceBranch":"main","suggestedTargetBranches":["8.18","8.x"],"targetPullRequestStates":[{"branch":"9.0","label":"v9.0.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"url":"https://github.com/elastic/kibana/pull/212045","number":212045,"state":"MERGED","mergeCommit":{"sha":"5e5989ab91c571225a3bba0fa8affd0321312aee","message":"[9.0] [Security Solution] Extend rule upgrade integration tests for prebuilt rule customization feature (#209260) (#212045)\n\n# Backport\n\nThis will backport the following commits from `main` to `9.0`:\n- [[Security Solution] Extend rule upgrade integration tests for\nprebuilt rule customization feature\n(#209260)](https://github.com/elastic/kibana/pull/209260)\n\n\n\n### Questions ?\nPlease refer to the [Backport tool\ndocumentation](https://github.com/sorenlouv/backport)\n\n\n\nCo-authored-by: Maxim Palenov <maxim.palenov@elastic.co>"}},{"branch":"8.18","label":"v8.18.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/209260","number":209260,"mergeCommit":{"message":"[Security Solution] Extend rule upgrade integration tests for prebuilt rule customization feature (#209260)\n\n**Addresses:** https://github.com/elastic/kibana/issues/202078\n**Partially implements test plan:** https://github.com/elastic/kibana/pull/205922\n\n## Summary\n\nThis PR implements the Rule Upgrade test plan added in #205922 and https://github.com/elastic/kibana/pull/203331.\n\n## Details\n\nTests cover only enabled customization and organized in a following way\n\n- `diffable_rule_fields` folder contains per field tests. It focuses on `DiffableRule`* fields. Each field has rule preview and upgrade scenarios throughly covered. Assertion on stats are included in rule preview tests to make sure the maximum coverage with reasonable execution time. **Diffable rule fields` tests were moved to a separate execution group to avoid exceeding execution limits.**\n- `preview_prebuilt_rules_upgrade.ts` integration tests on `/internal/prebuilt_rules/upgrade/_review` endpoint. It uses only `name` and `tags` fields to minimize tests complexity. Basically it should be considered as smoke tests.\n- `upgrade_prebuilt_rules` integrations test on `/internal/prebuilt_rules/upgrade/_perform` endpoint. It uses only `name` and `tags` fields to minimize tests complexity. Basically it should be considered as smoke tests.\n\nThis PR supersedes #205217 and https://github.com/elastic/kibana/pull/205339.\n\n* `DiffableRule` was added in the scope of prebuilt rules customization epic to serve rule upgrades preview and performing upgrades. It represents slightly reorganized rule fields to simplify prebuilt rule upgrade workflow handling. There are utility functions transforming between `RuleResponse` and `DiffableRule`.\n\n## Flaky test runner\n\nFlaky test runs for 100 iterations were successful (#209260 (comment) and https://github.com/elastic/kibana/pull/209260#issuecomment-2649627389).","sha":"4909770664a97f97bc38adc91dc852b5b1e6abf7"}},{"branch":"8.x","label":"v8.19.0","branchLabelMappingKey":"^v8.19.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT-->
JoseLuisGJ
pushed a commit
to JoseLuisGJ/kibana
that referenced
this pull request
Feb 27, 2025
…t rule customization feature (elastic#209260) **Addresses:** elastic#202078 **Partially implements test plan:** elastic#205922 ## Summary This PR implements the Rule Upgrade test plan added in elastic#205922 and elastic#203331. ## Details Tests cover only enabled customization and organized in a following way - `diffable_rule_fields` folder contains per field tests. It focuses on `DiffableRule`* fields. Each field has rule preview and upgrade scenarios throughly covered. Assertion on stats are included in rule preview tests to make sure the maximum coverage with reasonable execution time. **Diffable rule fields` tests were moved to a separate execution group to avoid exceeding execution limits.** - `preview_prebuilt_rules_upgrade.ts` integration tests on `/internal/prebuilt_rules/upgrade/_review` endpoint. It uses only `name` and `tags` fields to minimize tests complexity. Basically it should be considered as smoke tests. - `upgrade_prebuilt_rules` integrations test on `/internal/prebuilt_rules/upgrade/_perform` endpoint. It uses only `name` and `tags` fields to minimize tests complexity. Basically it should be considered as smoke tests. This PR supersedes elastic#205217 and elastic#205339. * `DiffableRule` was added in the scope of prebuilt rules customization epic to serve rule upgrades preview and performing upgrades. It represents slightly reorganized rule fields to simplify prebuilt rule upgrade workflow handling. There are utility functions transforming between `RuleResponse` and `DiffableRule`. ## Flaky test runner Flaky test runs for 100 iterations were successful (elastic#209260 (comment) and elastic#209260 (comment)).
SoniaSanzV
pushed a commit
to SoniaSanzV/kibana
that referenced
this pull request
Mar 4, 2025
…rebuilt rule customization feature (elastic#209260) (elastic#212282) # Backport This will backport the following commits from `main` to `8.x`: - [[Security Solution] Extend rule upgrade integration tests for prebuilt rule customization feature (elastic#209260)](elastic#209260) <!--- Backport version: 9.6.6 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sorenlouv/backport) <!--BACKPORT [{"author":{"name":"Maxim Palenov","email":"maxim.palenov@elastic.co"},"sourceCommit":{"committedDate":"2025-02-21T11:11:44Z","message":"[Security Solution] Extend rule upgrade integration tests for prebuilt rule customization feature (elastic#209260)\n\n**Addresses:** https://github.com/elastic/kibana/issues/202078\n**Partially implements test plan:** https://github.com/elastic/kibana/pull/205922\n\n## Summary\n\nThis PR implements the Rule Upgrade test plan added in elastic#205922 and https://github.com/elastic/kibana/pull/203331.\n\n## Details\n\nTests cover only enabled customization and organized in a following way\n\n- `diffable_rule_fields` folder contains per field tests. It focuses on `DiffableRule`* fields. Each field has rule preview and upgrade scenarios throughly covered. Assertion on stats are included in rule preview tests to make sure the maximum coverage with reasonable execution time. **Diffable rule fields` tests were moved to a separate execution group to avoid exceeding execution limits.**\n- `preview_prebuilt_rules_upgrade.ts` integration tests on `/internal/prebuilt_rules/upgrade/_review` endpoint. It uses only `name` and `tags` fields to minimize tests complexity. Basically it should be considered as smoke tests.\n- `upgrade_prebuilt_rules` integrations test on `/internal/prebuilt_rules/upgrade/_perform` endpoint. It uses only `name` and `tags` fields to minimize tests complexity. Basically it should be considered as smoke tests.\n\nThis PR supersedes elastic#205217 and https://github.com/elastic/kibana/pull/205339.\n\n* `DiffableRule` was added in the scope of prebuilt rules customization epic to serve rule upgrades preview and performing upgrades. It represents slightly reorganized rule fields to simplify prebuilt rule upgrade workflow handling. There are utility functions transforming between `RuleResponse` and `DiffableRule`.\n\n## Flaky test runner\n\nFlaky test runs for 100 iterations were successful (elastic#209260 (comment) and https://github.com/elastic/kibana/pull/209260#issuecomment-2649627389).","sha":"4909770664a97f97bc38adc91dc852b5b1e6abf7","branchLabelMapping":{"^v9.1.0$":"main","^v8.19.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["test","release_note:skip","v9.0.0","Team:Detections and Resp","Team: SecuritySolution","Team:Detection Rule Management","Feature:Prebuilt Detection Rules","backport:version","v8.18.0","v9.1.0","v8.19.0"],"title":"[Security Solution] Extend rule upgrade integration tests for prebuilt rule customization feature","number":209260,"url":"https://github.com/elastic/kibana/pull/209260","mergeCommit":{"message":"[Security Solution] Extend rule upgrade integration tests for prebuilt rule customization feature (elastic#209260)\n\n**Addresses:** https://github.com/elastic/kibana/issues/202078\n**Partially implements test plan:** https://github.com/elastic/kibana/pull/205922\n\n## Summary\n\nThis PR implements the Rule Upgrade test plan added in elastic#205922 and https://github.com/elastic/kibana/pull/203331.\n\n## Details\n\nTests cover only enabled customization and organized in a following way\n\n- `diffable_rule_fields` folder contains per field tests. It focuses on `DiffableRule`* fields. Each field has rule preview and upgrade scenarios throughly covered. Assertion on stats are included in rule preview tests to make sure the maximum coverage with reasonable execution time. **Diffable rule fields` tests were moved to a separate execution group to avoid exceeding execution limits.**\n- `preview_prebuilt_rules_upgrade.ts` integration tests on `/internal/prebuilt_rules/upgrade/_review` endpoint. It uses only `name` and `tags` fields to minimize tests complexity. Basically it should be considered as smoke tests.\n- `upgrade_prebuilt_rules` integrations test on `/internal/prebuilt_rules/upgrade/_perform` endpoint. It uses only `name` and `tags` fields to minimize tests complexity. Basically it should be considered as smoke tests.\n\nThis PR supersedes elastic#205217 and https://github.com/elastic/kibana/pull/205339.\n\n* `DiffableRule` was added in the scope of prebuilt rules customization epic to serve rule upgrades preview and performing upgrades. It represents slightly reorganized rule fields to simplify prebuilt rule upgrade workflow handling. There are utility functions transforming between `RuleResponse` and `DiffableRule`.\n\n## Flaky test runner\n\nFlaky test runs for 100 iterations were successful (elastic#209260 (comment) and https://github.com/elastic/kibana/pull/209260#issuecomment-2649627389).","sha":"4909770664a97f97bc38adc91dc852b5b1e6abf7"}},"sourceBranch":"main","suggestedTargetBranches":["8.18","8.x"],"targetPullRequestStates":[{"branch":"9.0","label":"v9.0.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"url":"https://github.com/elastic/kibana/pull/212045","number":212045,"state":"MERGED","mergeCommit":{"sha":"5e5989ab91c571225a3bba0fa8affd0321312aee","message":"[9.0] [Security Solution] Extend rule upgrade integration tests for prebuilt rule customization feature (elastic#209260) (elastic#212045)\n\n# Backport\n\nThis will backport the following commits from `main` to `9.0`:\n- [[Security Solution] Extend rule upgrade integration tests for\nprebuilt rule customization feature\n(elastic#209260)](https://github.com/elastic/kibana/pull/209260)\n\n\n\n### Questions ?\nPlease refer to the [Backport tool\ndocumentation](https://github.com/sorenlouv/backport)\n\n\n\nCo-authored-by: Maxim Palenov <maxim.palenov@elastic.co>"}},{"branch":"8.18","label":"v8.18.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/209260","number":209260,"mergeCommit":{"message":"[Security Solution] Extend rule upgrade integration tests for prebuilt rule customization feature (elastic#209260)\n\n**Addresses:** https://github.com/elastic/kibana/issues/202078\n**Partially implements test plan:** https://github.com/elastic/kibana/pull/205922\n\n## Summary\n\nThis PR implements the Rule Upgrade test plan added in elastic#205922 and https://github.com/elastic/kibana/pull/203331.\n\n## Details\n\nTests cover only enabled customization and organized in a following way\n\n- `diffable_rule_fields` folder contains per field tests. It focuses on `DiffableRule`* fields. Each field has rule preview and upgrade scenarios throughly covered. Assertion on stats are included in rule preview tests to make sure the maximum coverage with reasonable execution time. **Diffable rule fields` tests were moved to a separate execution group to avoid exceeding execution limits.**\n- `preview_prebuilt_rules_upgrade.ts` integration tests on `/internal/prebuilt_rules/upgrade/_review` endpoint. It uses only `name` and `tags` fields to minimize tests complexity. Basically it should be considered as smoke tests.\n- `upgrade_prebuilt_rules` integrations test on `/internal/prebuilt_rules/upgrade/_perform` endpoint. It uses only `name` and `tags` fields to minimize tests complexity. Basically it should be considered as smoke tests.\n\nThis PR supersedes elastic#205217 and https://github.com/elastic/kibana/pull/205339.\n\n* `DiffableRule` was added in the scope of prebuilt rules customization epic to serve rule upgrades preview and performing upgrades. It represents slightly reorganized rule fields to simplify prebuilt rule upgrade workflow handling. There are utility functions transforming between `RuleResponse` and `DiffableRule`.\n\n## Flaky test runner\n\nFlaky test runs for 100 iterations were successful (elastic#209260 (comment) and https://github.com/elastic/kibana/pull/209260#issuecomment-2649627389).","sha":"4909770664a97f97bc38adc91dc852b5b1e6abf7"}},{"branch":"8.x","label":"v8.19.0","branchLabelMappingKey":"^v8.19.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT-->
CAWilson94
pushed a commit
to CAWilson94/kibana
that referenced
this pull request
Mar 22, 2025
…eview (Rule Updates table) v2 (elastic#205922) **Addresses:** elastic#202078 ## Summary This PR extends rule upgrade test plan with customizable and non-customizable field examples. Rule upgrade workflow test plan (excluding Rule Upgrade flyout) was initially extended in elastic#203331. elastic#203331 adds the following rule upgrade workflow scenarios - Scenario: User can upgrade conflict-free prebuilt rules one by one - Scenario: User cannot upgrade prebuilt rules one by one from Rules Update table if they have conflicts - Scenario: User can upgrade multiple conflict-free prebuilt rules selected on the page - Scenario: User cannot upgrade selected prebuilt rules with conflicts - Scenario: User can upgrade all available conflict-free prebuilt rules at once* - Scenario: User cannot upgrade all prebuilt rules at once if they have upgrade conflicts - Scenario: User can upgrade only conflict-free rules when a mix of rules with and without conflicts are selected for upgrade - Scenario: User can upgrade only conflict-free rules when attempting to upgrade all rules - Scenario: User can upgrade rule with rule type change individually - Scenario: User can not bulk upgrade selected rules with rule type changes - Scenario: User can not bulk upgrade all rules with rule type changes - Scenario: API does not upgrade prebuilt rules if they are up to date --------- Co-authored-by: Georgii Gorbachev <georgii.gorbachev@elastic.co>
CAWilson94
pushed a commit
to CAWilson94/kibana
that referenced
this pull request
Mar 22, 2025
…t rule customization feature (elastic#209260) **Addresses:** elastic#202078 **Partially implements test plan:** elastic#205922 ## Summary This PR implements the Rule Upgrade test plan added in elastic#205922 and elastic#203331. ## Details Tests cover only enabled customization and organized in a following way - `diffable_rule_fields` folder contains per field tests. It focuses on `DiffableRule`* fields. Each field has rule preview and upgrade scenarios throughly covered. Assertion on stats are included in rule preview tests to make sure the maximum coverage with reasonable execution time. **Diffable rule fields` tests were moved to a separate execution group to avoid exceeding execution limits.** - `preview_prebuilt_rules_upgrade.ts` integration tests on `/internal/prebuilt_rules/upgrade/_review` endpoint. It uses only `name` and `tags` fields to minimize tests complexity. Basically it should be considered as smoke tests. - `upgrade_prebuilt_rules` integrations test on `/internal/prebuilt_rules/upgrade/_perform` endpoint. It uses only `name` and `tags` fields to minimize tests complexity. Basically it should be considered as smoke tests. This PR supersedes elastic#205217 and elastic#205339. * `DiffableRule` was added in the scope of prebuilt rules customization epic to serve rule upgrades preview and performing upgrades. It represents slightly reorganized rule fields to simplify prebuilt rule upgrade workflow handling. There are utility functions transforming between `RuleResponse` and `DiffableRule`. ## Flaky test runner Flaky test runs for 100 iterations were successful (elastic#209260 (comment) and elastic#209260 (comment)).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Addresses: #202078
Summary
This PR extends rule upgrade test plan with customizable and non-customizable field examples. Rule upgrade workflow test plan (excluding Rule Upgrade flyout) was initially extended in #203331.
#203331 adds the following rule upgrade workflow scenarios