[Security Solution] Write test plan and add test coverage for the new workflows of installing and upgrading prebuilt rules #148192
Assignees
Labels
8.11 candidate
Feature:Prebuilt Detection Rules
Security Solution Prebuilt Detection Rules
Team:Detection Rule Management
Security Detection Rule Management Team
Team:Detections and Resp
Security Detection Response Team
Team: SecuritySolution
Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc.
v8.9.0
v8.10.0
v8.11.0
Comments
banderror
added
triage_needed
Team:Detections and Resp
|
Pinging @elastic/security-detections-response (Team:Detections and Resp) |
|
Pinging @elastic/security-solution (Team: SecuritySolution) |
banderror
changed the title
banderror
added a commit
that referenced
this issue
Mar 8, 2023
… initial implementation (#148392) **Addresses:** #148181, #148182, #148185 **Partially addresses:** #148183, #148189 ## Summary Based on the [POC](#144060), this PR adds 4 endpoints for the new upgrade and installation workflows for prebuilt rules: - `GET /internal/detection_engine/prebuilt_rules/status` - `POST /internal/detection_engine/prebuilt_rules/upgrade/_review` - `POST /internal/detection_engine/prebuilt_rules/installation/_review` - `POST /internal/detection_engine/prebuilt_rules/_generate_assets` (temporary helper endpoint for development and testing) The new endpoints are hidden behind a feature flag and can be enabled by the following config setting: ```yaml xpack.securitySolution.enableExperimental: ['prebuiltRulesNewUpgradeAndInstallationWorkflowsEnabled'] ``` ## In the next episodes Will be done later in follow-up PRs: - Implementation of some additional response properties for the `upgrade/_review` endpoint: - #148183 - Making base versions optional for diff calculation (we need to support this in order to be able to still show diffs for rule assets coming from packages without historical versions): - #148189 - Further development of the diff algorithm: - #148191 - Test coverage: - #148192
bmorelli25
pushed a commit
to bmorelli25/kibana
that referenced
this issue
Mar 10, 2023
… initial implementation (elastic#148392) **Addresses:** elastic#148181, elastic#148182, elastic#148185 **Partially addresses:** elastic#148183, elastic#148189 ## Summary Based on the [POC](elastic#144060), this PR adds 4 endpoints for the new upgrade and installation workflows for prebuilt rules: - `GET /internal/detection_engine/prebuilt_rules/status` - `POST /internal/detection_engine/prebuilt_rules/upgrade/_review` - `POST /internal/detection_engine/prebuilt_rules/installation/_review` - `POST /internal/detection_engine/prebuilt_rules/_generate_assets` (temporary helper endpoint for development and testing) The new endpoints are hidden behind a feature flag and can be enabled by the following config setting: ```yaml xpack.securitySolution.enableExperimental: ['prebuiltRulesNewUpgradeAndInstallationWorkflowsEnabled'] ``` ## In the next episodes Will be done later in follow-up PRs: - Implementation of some additional response properties for the `upgrade/_review` endpoint: - elastic#148183 - Making base versions optional for diff calculation (we need to support this in order to be able to still show diffs for rule assets coming from packages without historical versions): - elastic#148189 - Further development of the diff algorithm: - elastic#148191 - Test coverage: - elastic#148192
3 tasks
banderror
added a commit
that referenced
this issue
Apr 11, 2023
…appings (#154473) **Related to:** elastic/security-team#6268 (internal) ## Summary For each of our Saved Object types, we must: 1. Remove any SO field mappings with `index: false` (or `enabled: false`, although a first pass was done in #149102) from our SO `mappings` declarations 2. Audit and remove any _unused_ SO fields to minimize our footprint This PR addresses these two requirements for this `security-rule` saved object type (prebuilt rule asset). ## Details Specifically, the PR removes the `name` field from the mappings because: - We don't filter, sort, search, or aggregate by it. - We might need to do it in the future for our prebuilt rule upgrade/installation workflows, but for now we're going to implement filtering, sorting, and pagination on the client side, thus there's no need for this mapping server-side. <img width="1295" alt="Screenshot 2023-04-05 at 15 19 10" src="https://user-images.githubusercontent.com/7359339/230094740-706a9a78-fec3-469e-a4ad-e8b7d7309c78.png"> Also, we may need to add more fields to this mapping in the future to implement further improvements for the prebuilt rule installation, upgrade, or deprecation workflows. ### Checklist - [ ] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [x] The unit test for SO mapping hashes has been updated. - [ ] More tests will be added as part of #148176 and #148192
jpdjere
added a commit
that referenced
this issue
Oct 17, 2023
… refactor (#165488) Fixes: #148192 (Tick the two open checkboxes in that issue when merging this PR) ## Summary This PR rewrites/refactors Cypress tests for the Installation and Upgrade of Prebuilt Rules implemented in #161687. Most of the changes here address feedback received in that PR - answered those comments there. - RBAC/Authorization: adds tests scenarios for users with full privileges (happy path) - Gets rid of huge util helpers such as `assertRuleAvailableForInstallAndInstallOne` and rewrites test cases in a more descriptive way, with step by step actions. - Gets rid of complex logic in tests and their helpers - removing if/else logic within them and removing optional flags passed to helpers. - Fixes `bulkCreateRuleAssets` util and uses it in other helpers to install multiple `security-rule` assets with a single bulk request to ES. Additionally: checked `installation_and_upgrade.md` test plan to make sure it matches with the test in place. Added [link](#166215) to a ticket for a to-do task for the sections: - Rule installation workflow: filtering, sorting, pagination - Rule upgrade workflow: filtering, sorting, pagination ## Flaky test runner ~~https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3420~~ ~~🟢~~ https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3513 --------- Co-authored-by: kibanamachine <[email protected]>
jpdjere
added a commit
to jpdjere/kibana
that referenced
this issue
Oct 17, 2023
… refactor (elastic#165488) Fixes: elastic#148192 (Tick the two open checkboxes in that issue when merging this PR) ## Summary This PR rewrites/refactors Cypress tests for the Installation and Upgrade of Prebuilt Rules implemented in elastic#161687. Most of the changes here address feedback received in that PR - answered those comments there. - RBAC/Authorization: adds tests scenarios for users with full privileges (happy path) - Gets rid of huge util helpers such as `assertRuleAvailableForInstallAndInstallOne` and rewrites test cases in a more descriptive way, with step by step actions. - Gets rid of complex logic in tests and their helpers - removing if/else logic within them and removing optional flags passed to helpers. - Fixes `bulkCreateRuleAssets` util and uses it in other helpers to install multiple `security-rule` assets with a single bulk request to ES. Additionally: checked `installation_and_upgrade.md` test plan to make sure it matches with the test in place. Added [link](elastic#166215) to a ticket for a to-do task for the sections: - Rule installation workflow: filtering, sorting, pagination - Rule upgrade workflow: filtering, sorting, pagination ## Flaky test runner ~~https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3420~~ ~~🟢~~ https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3513 --------- Co-authored-by: kibanamachine <[email protected]> (cherry picked from commit 24c008b) # Conflicts: # x-pack/test/security_solution_cypress/package.json
jpdjere
referenced
this issue
Oct 17, 2023
…ntation refactor (#165488) (#169129) # Backport This will backport the following commits from `main` to `8.11`: - [[Security Solution] Install/Update Prebuilt Rules Test Implementation refactor (#165488)](#165488) <!--- Backport version: 8.9.8 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Juan Pablo Djeredjian","email":"[email protected]"},"sourceCommit":{"committedDate":"2023-10-17T11:05:49Z","message":"[Security Solution] Install/Update Prebuilt Rules Test Implementation refactor (#165488)\n\nFixes: https://github.com/elastic/kibana/issues/148192\r\n\r\n(Tick the two open checkboxes in that issue when merging this PR)\r\n\r\n## Summary\r\n\r\nThis PR rewrites/refactors Cypress tests for the Installation and\r\nUpgrade of Prebuilt Rules implemented in\r\nhttps://github.com//pull/161687. Most of the changes here\r\naddress feedback received in that PR - answered those comments there.\r\n\r\n- RBAC/Authorization: adds tests scenarios for users with full\r\nprivileges (happy path)\r\n- Gets rid of huge util helpers such as\r\n`assertRuleAvailableForInstallAndInstallOne` and rewrites test cases in\r\na more descriptive way, with step by step actions.\r\n- Gets rid of complex logic in tests and their helpers - removing\r\nif/else logic within them and removing optional flags passed to helpers.\r\n- Fixes `bulkCreateRuleAssets` util and uses it in other helpers to\r\ninstall multiple `security-rule` assets with a single bulk request to\r\nES.\r\n\r\nAdditionally: checked `installation_and_upgrade.md` test plan to make\r\nsure it matches with the test in place. Added\r\n[link](#166215) to a ticket for\r\na to-do task for the sections:\r\n- Rule installation workflow: filtering, sorting, pagination\r\n- Rule upgrade workflow: filtering, sorting, pagination\r\n\r\n## Flaky test runner\r\n\r\n\r\n~~https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3420~~\r\n~~🟢~~\r\n\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3513\r\n\r\n---------\r\n\r\nCo-authored-by: kibanamachine <[email protected]>","sha":"24c008b4c5026dd543f1b4aded94f3787bce5fb0","branchLabelMapping":{"^v8.12.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["refactoring","release_note:skip","test-coverage","Team:Detections and Resp","Team: SecuritySolution","Team:Detection Rule Management","Feature:Prebuilt Detection Rules","v8.11.0","v8.12.0"],"number":165488,"url":"https://github.com/elastic/kibana/pull/165488","mergeCommit":{"message":"[Security Solution] Install/Update Prebuilt Rules Test Implementation refactor (#165488)\n\nFixes: https://github.com/elastic/kibana/issues/148192\r\n\r\n(Tick the two open checkboxes in that issue when merging this PR)\r\n\r\n## Summary\r\n\r\nThis PR rewrites/refactors Cypress tests for the Installation and\r\nUpgrade of Prebuilt Rules implemented in\r\nhttps://github.com//pull/161687. Most of the changes here\r\naddress feedback received in that PR - answered those comments there.\r\n\r\n- RBAC/Authorization: adds tests scenarios for users with full\r\nprivileges (happy path)\r\n- Gets rid of huge util helpers such as\r\n`assertRuleAvailableForInstallAndInstallOne` and rewrites test cases in\r\na more descriptive way, with step by step actions.\r\n- Gets rid of complex logic in tests and their helpers - removing\r\nif/else logic within them and removing optional flags passed to helpers.\r\n- Fixes `bulkCreateRuleAssets` util and uses it in other helpers to\r\ninstall multiple `security-rule` assets with a single bulk request to\r\nES.\r\n\r\nAdditionally: checked `installation_and_upgrade.md` test plan to make\r\nsure it matches with the test in place. Added\r\n[link](#166215) to a ticket for\r\na to-do task for the sections:\r\n- Rule installation workflow: filtering, sorting, pagination\r\n- Rule upgrade workflow: filtering, sorting, pagination\r\n\r\n## Flaky test runner\r\n\r\n\r\n~~https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3420~~\r\n~~🟢~~\r\n\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3513\r\n\r\n---------\r\n\r\nCo-authored-by: kibanamachine <[email protected]>","sha":"24c008b4c5026dd543f1b4aded94f3787bce5fb0"}},"sourceBranch":"main","suggestedTargetBranches":["8.11"],"targetPullRequestStates":[{"branch":"8.11","label":"v8.11.0","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v8.12.0","labelRegex":"^v8.12.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/165488","number":165488,"mergeCommit":{"message":"[Security Solution] Install/Update Prebuilt Rules Test Implementation refactor (#165488)\n\nFixes: https://github.com/elastic/kibana/issues/148192\r\n\r\n(Tick the two open checkboxes in that issue when merging this PR)\r\n\r\n## Summary\r\n\r\nThis PR rewrites/refactors Cypress tests for the Installation and\r\nUpgrade of Prebuilt Rules implemented in\r\nhttps://github.com//pull/161687. Most of the changes here\r\naddress feedback received in that PR - answered those comments there.\r\n\r\n- RBAC/Authorization: adds tests scenarios for users with full\r\nprivileges (happy path)\r\n- Gets rid of huge util helpers such as\r\n`assertRuleAvailableForInstallAndInstallOne` and rewrites test cases in\r\na more descriptive way, with step by step actions.\r\n- Gets rid of complex logic in tests and their helpers - removing\r\nif/else logic within them and removing optional flags passed to helpers.\r\n- Fixes `bulkCreateRuleAssets` util and uses it in other helpers to\r\ninstall multiple `security-rule` assets with a single bulk request to\r\nES.\r\n\r\nAdditionally: checked `installation_and_upgrade.md` test plan to make\r\nsure it matches with the test in place. Added\r\n[link](#166215) to a ticket for\r\na to-do task for the sections:\r\n- Rule installation workflow: filtering, sorting, pagination\r\n- Rule upgrade workflow: filtering, sorting, pagination\r\n\r\n## Flaky test runner\r\n\r\n\r\n~~https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3420~~\r\n~~🟢~~\r\n\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3513\r\n\r\n---------\r\n\r\nCo-authored-by: kibanamachine <[email protected]>","sha":"24c008b4c5026dd543f1b4aded94f3787bce5fb0"}}]}] BACKPORT-->
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Epic: https://github.com/elastic/security-team/issues/1974 (internal)
Summary
Add full test coverage for the new workflows of installing and upgrading prebuilt rules. Write a test plan. Add corresponding e2e, integration, and unit tests.
We will need to do it in two iterations because we're going to release the workflows themselves in several iterations (see the milestones in the epic):
Todo
Basic rule upgrade and installation workflows:
The text was updated successfully, but these errors were encountered: