From d0149086f1408d642717cf85b966fb88d2fa3cd8 Mon Sep 17 00:00:00 2001 From: Davis Plumlee <56367316+dplumlee@users.noreply.github.com> Date: Mon, 10 Jun 2024 09:17:51 -0400 Subject: [PATCH] [Security Solution] Simple diff algorithm test plan (#184484) ## Summary Adds test plan in accordance to https://github.com/elastic/kibana/issues/180158 Adds test cases for simple diff algorithm to prebuilt rules' `Installation and upgrade` test plan --------- Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com> --- .../installation_and_upgrade.md | 146 ++++++++++++++++++ 1 file changed, 146 insertions(+) diff --git a/x-pack/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/installation_and_upgrade.md b/x-pack/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/installation_and_upgrade.md index beda8a9517830..e6454eb424141 100644 --- a/x-pack/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/installation_and_upgrade.md +++ b/x-pack/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/installation_and_upgrade.md @@ -55,6 +55,14 @@ Status: `in progress`. The current test plan matches `Milestone 2` of the [Rule - [**Scenario: User can see correct rule information in preview before upgrading**](#scenario-user-can-see-correct-rule-information-in-preview-before-upgrading) - [**Scenario: Tabs and sections without content should be hidden in preview before upgrading**](#scenario-tabs-and-sections-without-content-should-be-hidden-in-preview-before-upgrading) - [Rule upgrade workflow: filtering, sorting, pagination](#rule-upgrade-workflow-filtering-sorting-pagination) + - [Rule upgrade workflow: simple diff algorithm](#rule-upgrade-workflow-simple-diff-algorithm) + - [**Scenario: Rule field doesn't have an update and has no custom value**](#scenario-rule-field-doesnt-have-an-update-and-has-no-custom-value) + - [**Scenario: Rule field doesn't have an update but has a custom value**](#scenario-rule-field-doesnt-have-an-update-but-has-a-custom-value) + - [**Scenario: Rule field has an update and doesn't have a custom value**](#scenario-rule-field-has-an-update-and-doesnt-have-a-custom-value) + - [**Scenario: Rule field has an update and a custom value that are the same**](#scenario-rule-field-has-an-update-and-a-custom-value-that-are-the-same) + - [**Scenario: Rule field has an update and a custom value that are NOT the same**](#scenario-rule-field-has-an-update-and-a-custom-value-that-are-not-the-same) + - [**Scenario: Rule field has an update and a custom value that are NOT the same and the rule base version doesn't exist**](#scenario-rule-field-has-an-update-and-a-custom-value-that-are-not-the-same-and-the-rule-base-version-doesnt-exist) + - [**Scenario: Rule field has an update and a custom value that are the same and the rule base version doesn't exist**](#scenario-rule-field-has-an-update-and-a-custom-value-that-are-the-same-and-the-rule-base-version-doesnt-exist) - [Rule upgrade workflow: viewing rule changes in JSON diff view](#rule-upgrade-workflow-viewing-rule-changes-in-json-diff-view) - [**Scenario: User can see changes in a side-by-side JSON diff view**](#scenario-user-can-see-changes-in-a-side-by-side-json-diff-view) - [**Scenario: User can see precisely how property values would change after upgrade**](#scenario-user-can-see-precisely-how-property-values-would-change-after-upgrade) @@ -815,6 +823,144 @@ And the Investigation Guide tab should NOT be displayed TODO: add scenarios https://github.com/elastic/kibana/issues/166215 +### Rule upgrade workflow: simple diff algorithm + +#### **Scenario: Rule field doesn't have an update and has no custom value** + +**Automation**: 2 integration tests with mock rules + a set of unit tests for the algorithm + +```Gherkin +Given at least 1 prebuilt rule is installed in Kibana +And for this rule there is a new version available +And field is not customized by the user (current version == base version) +And field is not updated by Elastic in this upgrade (target version == base version) +Then for field the diff algorithm should output the current version as the merged one without a conflict +And field should not be returned from the `upgrade/_review` API endpoint +And field should not be shown in the upgrade preview UI + +Examples: +| field_name | base_version | current_version | target_version | +| name | "A" | "A" | "A" | +| risk_score | 1 | 1 | 1 | +``` + +#### **Scenario: Rule field doesn't have an update but has a custom value** + +**Automation**: 2 integration tests with mock rules + a set of unit tests for the algorithm + +```Gherkin +Given at least 1 prebuilt rule is installed in Kibana +And for this rule there is a new version available +And field is customized by the user (current version != base version) +And field is not updated by Elastic in this upgrade (target version == base version) +Then for field the diff algorithm should output the current version as the merged one without a conflict +And field should be returned from the `upgrade/_review` API endpoint +And field should be shown in the upgrade preview UI + +Examples: +| field_name | base_version | current_version | target_version | +| name | "A" | "B" | "A" | +| risk_score | 1 | 2 | 1 | +``` + +#### **Scenario: Rule field has an update and doesn't have a custom value** + +**Automation**: 2 integration tests with mock rules + a set of unit tests for the algorithm + +```Gherkin +Given at least 1 prebuilt rule is installed in Kibana +And for this rule there is a new version available +And field is not customized by the user (current version == base version) +And field is updated by Elastic in this upgrade (target version != base version) +Then for field the diff algorithm should output the target version as the merged one without a conflict +And field should be returned from the `upgrade/_review` API endpoint +And field should be shown in the upgrade preview UI + +Examples: +| field_name | base_version | current_version | target_version | +| name | "A" | "A" | "B" | +| risk_score | 1 | 1 | 2 | +``` + +#### **Scenario: Rule field has an update and a custom value that are the same** + +**Automation**: 2 integration tests with mock rules + a set of unit tests for the algorithm + +```Gherkin +Given at least 1 prebuilt rule is installed in Kibana +And for this rule there is a new version available +And field is customized by the user (current version != base version) +And field is updated by Elastic in this upgrade (target version != base version) +And customized field is the same as the Elastic update in this upgrade (current version == target version) +Then for field the diff algorithm should output the current version as the merged one without a conflict +And field should not be returned from the `upgrade/_review` API endpoint +And field should not be shown in the upgrade preview UI + +Examples: +| field_name | base_version | current_version | target_version | +| name | "A" | "B" | "B" | +| risk_score | 1 | 2 | 2 | +``` + +#### **Scenario: Rule field has an update and a custom value that are NOT the same** + +**Automation**: 2 integration tests with mock rules + a set of unit tests for the algorithm + +```Gherkin +Given at least 1 prebuilt rule is installed in Kibana +And for this rule there is a new version available +And field is customized by the user (current version != base version) +And field is updated by Elastic in this upgrade (target version != base version) +And customized field is different than the Elastic update in this upgrade (current version != target version) +Then for field the diff algorithm should output the current version as the merged one with a conflict +And field should be returned from the `upgrade/_review` API endpoint +And field should be shown in the upgrade preview UI + +Examples: +| field_name | base_version | current_version | target_version | +| name | "A" | "B" | "C" | +| risk_score | 1 | 2 | 3 | +``` + +#### **Scenario: Rule field has an update and a custom value that are NOT the same and the rule base version doesn't exist** + +**Automation**: 2 integration tests with mock rules + a set of unit tests for the algorithm + +```Gherkin +Given at least 1 prebuilt rule is installed in Kibana +And for this rule there is a new version available +And the base version of the rule cannot be determined +And customized field is different than the Elastic update in this upgrade (current version != target version) +Then for field the diff algorithm should output the target version as the merged one with a conflict +And field should be returned from the `upgrade/_review` API endpoint +And field should be shown in the upgrade preview UI + +Examples: +| field_name | base_version | current_version | target_version | +| name | N/A | "B" | "C" | +| risk_score | N/A | 2 | 3 | +``` + +#### **Scenario: Rule field has an update and a custom value that are the same and the rule base version doesn't exist** + +**Automation**: 2 integration tests with mock rules + a set of unit tests for the algorithm + +```Gherkin +Given at least 1 prebuilt rule is installed in Kibana +And for this rule there is a new version available +And the base version of the rule cannot be determined +And customized field is the same as the Elastic update in this upgrade (current version == target version) +Then for field the diff algorithm should output the current version as the merged one without a conflict +And field should not be returned from the `upgrade/_review` API endpoint +And field should not be shown in the upgrade preview UI + + +Examples: +| field_name | base_version | current_version | target_version | +| name | N/A | "A" | "A" | +| risk_score | N/A | 1 | 1 | +``` + ### Rule upgrade workflow: viewing rule changes in JSON diff view #### **Scenario: User can see changes in a side-by-side JSON diff view**