Skip to content

Commit

Permalink
[8.11] [Security Solution] Install/Update Prebuilt Rules Test Impleme…
Browse files Browse the repository at this point in the history
…ntation refactor (#165488) (#169129)

# Backport

This will backport the following commits from `main` to `8.11`:
- [[Security Solution] Install/Update Prebuilt Rules Test Implementation
refactor (#165488)](#165488)

<!--- Backport version: 8.9.8 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Juan Pablo
Djeredjian","email":"[email protected]"},"sourceCommit":{"committedDate":"2023-10-17T11:05:49Z","message":"[Security
Solution] Install/Update Prebuilt Rules Test Implementation refactor
(#165488)\n\nFixes:
https://github.com/elastic/kibana/issues/148192\r\n\r\n(Tick the two
open checkboxes in that issue when merging this PR)\r\n\r\n##
Summary\r\n\r\nThis PR rewrites/refactors Cypress tests for the
Installation and\r\nUpgrade of Prebuilt Rules implemented
in\r\nhttps://github.com//pull/161687. Most of the changes
here\r\naddress feedback received in that PR - answered those comments
there.\r\n\r\n- RBAC/Authorization: adds tests scenarios for users with
full\r\nprivileges (happy path)\r\n- Gets rid of huge util helpers such
as\r\n`assertRuleAvailableForInstallAndInstallOne` and rewrites test
cases in\r\na more descriptive way, with step by step actions.\r\n- Gets
rid of complex logic in tests and their helpers - removing\r\nif/else
logic within them and removing optional flags passed to helpers.\r\n-
Fixes `bulkCreateRuleAssets` util and uses it in other helpers
to\r\ninstall multiple `security-rule` assets with a single bulk request
to\r\nES.\r\n\r\nAdditionally: checked `installation_and_upgrade.md`
test plan to make\r\nsure it matches with the test in place.
Added\r\n[link](#166215) to a
ticket for\r\na to-do task for the sections:\r\n- Rule installation
workflow: filtering, sorting, pagination\r\n- Rule upgrade workflow:
filtering, sorting, pagination\r\n\r\n## Flaky test
runner\r\n\r\n\r\n~~https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3420~~\r\n~~🟢~~\r\n\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3513\r\n\r\n---------\r\n\r\nCo-authored-by:
kibanamachine
<[email protected]>","sha":"24c008b4c5026dd543f1b4aded94f3787bce5fb0","branchLabelMapping":{"^v8.12.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["refactoring","release_note:skip","test-coverage","Team:Detections
and Resp","Team: SecuritySolution","Team:Detection Rule
Management","Feature:Prebuilt Detection
Rules","v8.11.0","v8.12.0"],"number":165488,"url":"https://github.com/elastic/kibana/pull/165488","mergeCommit":{"message":"[Security
Solution] Install/Update Prebuilt Rules Test Implementation refactor
(#165488)\n\nFixes:
https://github.com/elastic/kibana/issues/148192\r\n\r\n(Tick the two
open checkboxes in that issue when merging this PR)\r\n\r\n##
Summary\r\n\r\nThis PR rewrites/refactors Cypress tests for the
Installation and\r\nUpgrade of Prebuilt Rules implemented
in\r\nhttps://github.com//pull/161687. Most of the changes
here\r\naddress feedback received in that PR - answered those comments
there.\r\n\r\n- RBAC/Authorization: adds tests scenarios for users with
full\r\nprivileges (happy path)\r\n- Gets rid of huge util helpers such
as\r\n`assertRuleAvailableForInstallAndInstallOne` and rewrites test
cases in\r\na more descriptive way, with step by step actions.\r\n- Gets
rid of complex logic in tests and their helpers - removing\r\nif/else
logic within them and removing optional flags passed to helpers.\r\n-
Fixes `bulkCreateRuleAssets` util and uses it in other helpers
to\r\ninstall multiple `security-rule` assets with a single bulk request
to\r\nES.\r\n\r\nAdditionally: checked `installation_and_upgrade.md`
test plan to make\r\nsure it matches with the test in place.
Added\r\n[link](#166215) to a
ticket for\r\na to-do task for the sections:\r\n- Rule installation
workflow: filtering, sorting, pagination\r\n- Rule upgrade workflow:
filtering, sorting, pagination\r\n\r\n## Flaky test
runner\r\n\r\n\r\n~~https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3420~~\r\n~~🟢~~\r\n\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3513\r\n\r\n---------\r\n\r\nCo-authored-by:
kibanamachine
<[email protected]>","sha":"24c008b4c5026dd543f1b4aded94f3787bce5fb0"}},"sourceBranch":"main","suggestedTargetBranches":["8.11"],"targetPullRequestStates":[{"branch":"8.11","label":"v8.11.0","labelRegex":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v8.12.0","labelRegex":"^v8.12.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/165488","number":165488,"mergeCommit":{"message":"[Security
Solution] Install/Update Prebuilt Rules Test Implementation refactor
(#165488)\n\nFixes:
https://github.com/elastic/kibana/issues/148192\r\n\r\n(Tick the two
open checkboxes in that issue when merging this PR)\r\n\r\n##
Summary\r\n\r\nThis PR rewrites/refactors Cypress tests for the
Installation and\r\nUpgrade of Prebuilt Rules implemented
in\r\nhttps://github.com//pull/161687. Most of the changes
here\r\naddress feedback received in that PR - answered those comments
there.\r\n\r\n- RBAC/Authorization: adds tests scenarios for users with
full\r\nprivileges (happy path)\r\n- Gets rid of huge util helpers such
as\r\n`assertRuleAvailableForInstallAndInstallOne` and rewrites test
cases in\r\na more descriptive way, with step by step actions.\r\n- Gets
rid of complex logic in tests and their helpers - removing\r\nif/else
logic within them and removing optional flags passed to helpers.\r\n-
Fixes `bulkCreateRuleAssets` util and uses it in other helpers
to\r\ninstall multiple `security-rule` assets with a single bulk request
to\r\nES.\r\n\r\nAdditionally: checked `installation_and_upgrade.md`
test plan to make\r\nsure it matches with the test in place.
Added\r\n[link](#166215) to a
ticket for\r\na to-do task for the sections:\r\n- Rule installation
workflow: filtering, sorting, pagination\r\n- Rule upgrade workflow:
filtering, sorting, pagination\r\n\r\n## Flaky test
runner\r\n\r\n\r\n~~https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3420~~\r\n~~🟢~~\r\n\r\nhttps://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/3513\r\n\r\n---------\r\n\r\nCo-authored-by:
kibanamachine
<[email protected]>","sha":"24c008b4c5026dd543f1b4aded94f3787bce5fb0"}}]}]
BACKPORT-->
  • Loading branch information
jpdjere authored Oct 17, 2023
1 parent 59a7ede commit 2849d3f
Show file tree
Hide file tree
Showing 16 changed files with 438 additions and 372 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -566,7 +566,7 @@ And the Investigation Guide tab should NOT be displayed

### Rule installation workflow: filtering, sorting, pagination

TODO: add scenarios
TODO: add scenarios https://github.com/elastic/kibana/issues/166215

### Rule installation workflow: misc cases

Expand Down Expand Up @@ -709,7 +709,7 @@ And the Investigation Guide tab should NOT be displayed

### Rule upgrade workflow: filtering, sorting, pagination

TODO: add scenarios
TODO: add scenarios https://github.com/elastic/kibana/issues/166215

### Rule upgrade workflow: misc cases

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@ const RuleName = ({ name, ruleId }: RuleNameProps) => {
onClick={() => {
openRulePreview(ruleId);
}}
data-test-subj="ruleName"
>
{name}
</EuiLink>
Expand Down Expand Up @@ -121,7 +122,14 @@ const createInstallButtonColumn = (
onClick={() => installOneRule(ruleId)}
data-test-subj={`installSinglePrebuiltRuleButton-${ruleId}`}
>
{isRuleInstalling ? <EuiLoadingSpinner size="s" /> : i18n.INSTALL_RULE_BUTTON}
{isRuleInstalling ? (
<EuiLoadingSpinner
size="s"
data-test-subj={`installSinglePrebuiltRuleButton-loadingSpinner-${ruleId}`}
/>
) : (
i18n.INSTALL_RULE_BUTTON
)}
</EuiButtonEmpty>
);
},
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@ const RuleName = ({ name, ruleId }: RuleNameProps) => {
onClick={() => {
openRulePreview(ruleId);
}}
data-test-subj="ruleName"
>
{name}
</EuiLink>
Expand Down Expand Up @@ -120,7 +121,14 @@ const createUpgradeButtonColumn = (
onClick={() => upgradeOneRule(ruleId)}
data-test-subj={`upgradeSinglePrebuiltRuleButton-${ruleId}`}
>
{isRuleUpgrading ? <EuiLoadingSpinner size="s" /> : i18n.UPDATE_RULE_BUTTON}
{isRuleUpgrading ? (
<EuiLoadingSpinner
size="s"
data-test-subj={`upgradeSinglePrebuiltRuleButton-loadingSpinner-${ruleId}`}
/>
) : (
i18n.UPDATE_RULE_BUTTON
)}
</EuiButtonEmpty>
);
},
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,23 +13,30 @@ import {
import { ROLES } from '@kbn/security-solution-plugin/common/test';

import { createRuleAssetSavedObject } from '../../../helpers/rules';
import { createAndInstallMockedPrebuiltRules } from '../../../tasks/api_calls/prebuilt_rules';
import { resetRulesTableState, deleteAlertsAndRules } from '../../../tasks/common';
import { login } from '../../../tasks/login';
import {
createAndInstallMockedPrebuiltRules,
installPrebuiltRuleAssets,
preventPrebuiltRulesPackageInstallation,
} from '../../../tasks/api_calls/prebuilt_rules';
import { visit } from '../../../tasks/navigation';
import { RULES_MANAGEMENT_URL } from '../../../urls/rules_management';
import {
ADD_ELASTIC_RULES_BTN,
getInstallSingleRuleButtonByRuleId,
getUpgradeSingleRuleButtonByRuleId,
GO_BACK_TO_RULES_TABLE_BUTTON,
INSTALL_ALL_RULES_BUTTON,
RULES_MANAGEMENT_TAB,
RULES_MANAGEMENT_TABLE,
RULES_UPDATES_TAB,
RULE_CHECKBOX,
UPGRADE_ALL_RULES_BUTTON,
} from '../../../screens/alerts_detection_rules';
import { cleanKibana } from '../../../tasks/common';
import { login } from '../../../tasks/login';

// Rule to test update
const RULE_1_ID = 'rule_1';
const RULE_2_ID = 'rule_2';
const OUTDATED_RULE_1 = createRuleAssetSavedObject({
name: 'Outdated rule 1',
rule_id: RULE_1_ID,
Expand All @@ -40,52 +47,44 @@ const UPDATED_RULE_1 = createRuleAssetSavedObject({
rule_id: RULE_1_ID,
version: 2,
});
const OUTDATED_RULE_2 = createRuleAssetSavedObject({
name: 'Outdated rule 2',

// Rule to test installation
const RULE_2_ID = 'rule_2';
const RULE_2 = createRuleAssetSavedObject({
name: 'Rule 2',
rule_id: RULE_2_ID,
version: 1,
});
const UPDATED_RULE_2 = createRuleAssetSavedObject({
name: 'Updated rule 2',
rule_id: RULE_2_ID,
version: 2,
});

const loadPageAsReadOnlyUser = (url: string) => {
login(ROLES.reader);
visit(url, { role: ROLES.reader });
};

const loginPageAsWriteAuthorizedUser = (url: string) => {
login(ROLES.hunter);
visit(url);
};

// TODO: https://github.com/elastic/kibana/issues/164451 We should find a way to make this spec work in Serverless
// TODO: https://github.com/elastic/kibana/issues/161540
describe(
'Detection rules, Prebuilt Rules Installation and Update - Authorization/RBAC',
{ tags: ['@ess', '@serverless', '@skipInServerless'] },
() => {
beforeEach(() => {
login();
resetRulesTableState();
deleteAlertsAndRules();
cy.task('esArchiverResetKibana');
createAndInstallMockedPrebuiltRules({ rules: [OUTDATED_RULE_1, OUTDATED_RULE_2] });
preventPrebuiltRulesPackageInstallation();
cleanKibana();
});

describe('User with read privileges on Security Solution', () => {
const RULE_1 = createRuleAssetSavedObject({
name: 'Test rule 1',
rule_id: 'rule_1',
});
const RULE_2 = createRuleAssetSavedObject({
name: 'Test rule 2',
rule_id: 'rule_2',
});
beforeEach(() => {
it('should not be able to install prebuilt rules', () => {
// Install one prebuilt rule asset to assert that user can't install it
installPrebuiltRuleAssets([RULE_2]);

// Now login with read-only user in preparation for test
createAndInstallMockedPrebuiltRules({ rules: [RULE_1, RULE_2], installToKibana: false });
loadPageAsReadOnlyUser(RULES_MANAGEMENT_URL);
});

it('should not be able to install prebuilt rules', () => {
// Check that Add Elastic Rules button is disabled
cy.get(ADD_ELASTIC_RULES_BTN).should('be.disabled');

Expand All @@ -94,25 +93,21 @@ describe(
// installation buttons are disabled
cy.visit(`${APP_PATH}${RULES_ADD_PATH}`);
cy.get(INSTALL_ALL_RULES_BUTTON).should('be.disabled');
cy.get(getInstallSingleRuleButtonByRuleId(RULE_1['security-rule'].rule_id)).should(
cy.get(getInstallSingleRuleButtonByRuleId(UPDATED_RULE_1['security-rule'].rule_id)).should(
'not.exist'
);
cy.get(RULE_CHECKBOX).should('not.exist');
});
});

describe('User with read privileges on Security Solution', () => {
beforeEach(() => {
/* Create a second version of the rule, making it available for update */
createAndInstallMockedPrebuiltRules({
rules: [UPDATED_RULE_1, UPDATED_RULE_2],
installToKibana: false,
});
it('should not be able to upgrade prebuilt rules', () => {
// Install one prebuilt rule asset to assert that user can't upgrade it
createAndInstallMockedPrebuiltRules([OUTDATED_RULE_1]);
// Create a new version of the rule to make it available for upgrade
installPrebuiltRuleAssets([UPDATED_RULE_1]);

// Now login with read-only user in preparation for test
loadPageAsReadOnlyUser(RULES_MANAGEMENT_URL);
});

it('should not be able to upgrade prebuilt rules', () => {
// Check that Rule Update tab is not shown
cy.get(RULES_UPDATES_TAB).should('not.exist');

Expand All @@ -121,11 +116,70 @@ describe(
// upgrade buttons are disabled
cy.visit(`${APP_PATH}${RULES_UPDATES}`);
cy.get(UPGRADE_ALL_RULES_BUTTON).should('be.disabled');

// Upgrade button and selection checkbox should not be visible
cy.get(getUpgradeSingleRuleButtonByRuleId(OUTDATED_RULE_1['security-rule'].rule_id)).should(
'not.exist'
);
cy.get(RULE_CHECKBOX).should('not.exist');
});
});

describe('User with write privileges on Security Solution', () => {
it('should be able to install prebuilt rules', () => {
// Install one prebuilt rule asset to assert that user can install it
installPrebuiltRuleAssets([RULE_2]);
loginPageAsWriteAuthorizedUser(RULES_MANAGEMENT_URL);

// Check that Add Elastic Rules button is enabled
cy.get(ADD_ELASTIC_RULES_BTN).should('not.be.disabled');

// Navigate to Add Elastic Rules page and assert that rules can be selected
// and all installation buttons are enabled
cy.get(ADD_ELASTIC_RULES_BTN).click();
cy.get(INSTALL_ALL_RULES_BUTTON).should('not.be.disabled');
cy.get(getInstallSingleRuleButtonByRuleId(RULE_2['security-rule'].rule_id)).should('exist');
cy.get(RULE_CHECKBOX).should('exist');

// Install all available prebuilt rules
cy.get(INSTALL_ALL_RULES_BUTTON).click();

// Rule shouldn't be available for installation anymore
cy.get(getInstallSingleRuleButtonByRuleId(RULE_2['security-rule'].rule_id)).should(
'not.exist'
);

// Navigate back to rules table and assert rule is installed
cy.get(GO_BACK_TO_RULES_TABLE_BUTTON).click();
cy.get(RULES_MANAGEMENT_TABLE).contains(RULE_2['security-rule'].name);
});

it('should be able to upgrade prebuilt rules', () => {
// Install one prebuilt rule asset to assert that user can upgrade it
createAndInstallMockedPrebuiltRules([OUTDATED_RULE_1]);
// Create a new version of the rule to make it available for upgrade
installPrebuiltRuleAssets([UPDATED_RULE_1]);
loginPageAsWriteAuthorizedUser(RULES_MANAGEMENT_URL);

// Check that Rule Update tab is shown
cy.get(RULES_UPDATES_TAB).should('exist');

// Navigate to Rule Update tab and assert that rules can be selected
// and all upgrade buttons are enabled
cy.get(RULES_UPDATES_TAB).click();
cy.get(UPGRADE_ALL_RULES_BUTTON).should('not.be.disabled');
cy.get(RULE_CHECKBOX).should('exist');
cy.get(getUpgradeSingleRuleButtonByRuleId(OUTDATED_RULE_1['security-rule'].rule_id)).should(
'exist'
);

// Upgrade the rule and assert that it's upgraded
cy.get(
getUpgradeSingleRuleButtonByRuleId(OUTDATED_RULE_1['security-rule'].rule_id)
).click();
cy.get(RULES_MANAGEMENT_TAB).click();
cy.get(RULES_MANAGEMENT_TABLE).contains(UPDATED_RULE_1['security-rule'].name);
});
});
}
);
Loading

0 comments on commit 2849d3f

Please sign in to comment.