From 3c8952497168de33f838548c31ea886d09f1d1c8 Mon Sep 17 00:00:00 2001 From: Ryland Herrick Date: Mon, 11 Nov 2024 15:01:46 -0600 Subject: [PATCH] [Security Solution] Test plans for prebuilt rule import and export (#191116) ## Summary This PR introduces test plans for both [Prebuilt Rule Import](https://github.com/elastic/kibana/issues/180168) (corresponding [PR](https://github.com/elastic/kibana/pull/190198)) and [Prebuilt Rule Export](https://github.com/elastic/kibana/issues/180167) (corresponding [PR](https://github.com/elastic/kibana/pull/194498)). Import is considerably more complicated as it is calculating new values (for `rule_source`, `immutable`), while the export work is mainly removing existing restrictions (which allowed only custom rules to be exported). --------- Co-authored-by: Elastic Machine (cherry picked from commit e4298492b5e48338396618d51168ea3e8427c103) --- .../prebuilt_rules/exporting.md | 65 +++++++++ .../prebuilt_rules/importing.md | 127 ++++++++++++++++++ 2 files changed, 192 insertions(+) create mode 100644 x-pack/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/exporting.md create mode 100644 x-pack/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/importing.md diff --git a/x-pack/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/exporting.md b/x-pack/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/exporting.md new file mode 100644 index 0000000000000..f4cbc66779a81 --- /dev/null +++ b/x-pack/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/exporting.md @@ -0,0 +1,65 @@ +# Prebuilt Rule Export + +This is a test plan for the exporting of prebuilt rules. This feature is an aspect of `Milestone 2` of the [Rule Immutability/Customization](https://github.com/elastic/security-team/issues/1974) epic. + +Status: `in progress`. + +## Useful information + +### Tickets + +- [Rule Immutability/Customization](https://github.com/elastic/security-team/issues/1974) +- [Rule Exporting Feature](https://github.com/elastic/kibana/issues/180167#issue-2227974379) +- [Rule Export API PR](https://github.com/elastic/kibana/pull/194498) + +### Terminology + +- **prebuilt rule**: A rule contained in our `Prebuilt Security Detection Rules` integration in Fleet. +- **custom rule**: A rule defined by the user, which has no relation to the prebuilt rules +- **rule source, or ruleSource**: A field on the rule that defines the rule's categorization + +## Scenarios + +### Core Functionality + +#### Scenario: Exporting prebuilt rule individually +```Gherkin +Given a space with prebuilt rules installed +When the user selects "Export rule" from the "All actions" dropdown on the rule's page +Then the rule should be exported as an NDJSON file +And it should include an "immutable" field with a value of true +And its "ruleSource" "type" should be "external" +And its "ruleSource" "isCustomized" value should depend on whether the rule was customized +``` + +#### Scenario: Exporting prebuilt rules in bulk +```Gherkin +Given a space with prebuilt rules installed +When the user selects prebuilt rules in the alerts table +And chooses "Export" from bulk actions +Then the selected rules should be exported as an NDJSON file +And they should include an "immutable" field with a value of true +And their "ruleSource" "type" should be "external" +And their "ruleSource" "isCustomized" should depend on whether the rule was customized +``` + +#### Scenario: Exporting both prebuilt and custom rules in bulk +```Gherkin +Given a space with prebuilt and custom rules installed +When the user selects prebuilt rules in the alerts table +And chooses "Export" from bulk actions +Then the selected rules should be exported as an NDJSON file +And the prebuilt rules should include an "immutable" field with a value of true +And the custom rules should include an "immutable" field with a value of false +And the prebuilt rules' "ruleSource" "type" should be "external" +And the custom rules' "ruleSource" "type" should be "internal" +``` + +### Error Handling + +#### Scenario: Exporting beyond the export limit +```Gherkin +Given a space with prebuilt and custom rules installed +And the number of rules is greater than the export limit (defaults to 10_000) +Then the request should be rejected as a bad request +``` diff --git a/x-pack/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/importing.md b/x-pack/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/importing.md new file mode 100644 index 0000000000000..0c947d0a52b95 --- /dev/null +++ b/x-pack/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/importing.md @@ -0,0 +1,127 @@ +# Prebuilt Rule Import + +This is a test plan for the importing of prebuilt rules. This feature is an aspect of `Milestone 2` of the [Rule Immutability/Customization](https://github.com/elastic/security-team/issues/1974) epic. + +Status: `in progress`. + +## Useful information + +### Tickets + +- [Rule Immutability/Customization](https://github.com/elastic/security-team/issues/1974) +- [Rule Importing Feature](https://github.com/elastic/kibana/issues/180168) +- [Rule Import API PR](https://github.com/elastic/kibana/pull/190198) + +### Terminology + +- **prebuilt rule**: A rule contained in our `Prebuilt Security Detection Rules` integration in Fleet. +- **custom rule**: A rule defined by the user, which has no relation to the prebuilt rules +- **rule source, or ruleSource**: A field on the rule that defines the rule's categorization + +## Scenarios + +### Core Functionality + +#### Scenario: Importing an unmodified prebuilt rule with a matching rule_id and version + +```Gherkin +Given the import payload contains a prebuilt rule with a matching rule_id and version, identical to the published rule +When the user imports the rule +Then the rule should be created or updated +And the ruleSource type should be "external" +And isCustomized should be false +``` + +#### Scenario: Importing a customized prebuilt rule with a matching rule_id and version + +```Gherkin +Given the import payload contains a prebuilt rule with a matching rule_id and version, modified from the published version +When the user imports the rule +Then the rule should be created or updated +And the ruleSource type should be "external" +And isCustomized should be true +``` + +#### Scenario: Importing a prebuilt rule with a matching rule_id but no matching version + +```Gherkin +Given the import payload contains a prebuilt rule with a matching rule_id but no matching version +When the user imports the rule +Then the rule should be created or updated +And the ruleSource type should be "external" +And isCustomized should be true +``` + +#### Scenario: Importing a prebuilt rule with a non-existent rule_id + +```Gherkin +Given the import payload contains a prebuilt rule with a non-existent rule_id +When the user imports the rule +Then the rule should be created +And the ruleSource type should be "internal" +``` + +#### Scenario: Importing a prebuilt rule without a rule_id field + +```Gherkin +Given the import payload contains a prebuilt rule without a rule_id field +When the user imports the rule +Then the import should be rejected with a message "rule_id field is required" +``` + +#### Scenario: Importing a prebuilt rule with a matching rule_id but missing a version field + +```Gherkin +Given the import payload contains a prebuilt rule without a version field +When the user imports the rule +Then the import should be rejected with a message "version field is required" +``` + +#### Scenario: Importing an existing custom rule missing a version field + +```Gherkin +Given the import payload contains an existing custom rule without a version field +When the user imports the rule +Then the rule should be updated +And the ruleSource type should be "internal" +And the "version" field should be set to the existing rule's "version" +``` + +#### Scenario: Importing a new custom rule missing a version field + +```Gherkin +Given the import payload contains a new custom rule without a version field +When the user imports the rule +Then the rule should be created +And the ruleSource type should be "internal" +And the "version" field should be set to 1 +``` + +#### Scenario: Importing a rule with overwrite flag set to true + +```Gherkin +Given the import payload contains a rule with an existing rule_id +And the overwrite flag is set to true +When the user imports the rule +Then the rule should be overwritten +And the ruleSource type should be calculated based on the rule_id and version +``` + +#### Scenario: Importing a rule with overwrite flag set to false + +```Gherkin +Given the import payload contains a rule with an existing rule_id +And the overwrite flag is set to false +When the user imports the rule +Then the import should be rejected with a message "rule_id already exists" +``` + +#### Scenario: Importing both custom and prebuilt rules + +```Gherkin +Given the import payload contains modified and unmodified, custom and prebuilt rules +When the user imports the rule +Then custom rules should be created or updated, with versions defaulted to 1 +And prebuilt rules should be created or updated, +And prebuilt rules missing versions should be rejected +```