Skip to content

[Security Solution] Restructuring folders of Detection Engine + refactoring Rule Management #142950

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
banderror merged 29 commits into from
Oct 21, 2022
Merged

[Security Solution] Restructuring folders of Detection Engine + refactoring Rule Management #142950

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
2eef5de
Preliminary refactoring: folder restructuring and creating API hooks
banderror Oct 13, 2022
20a7de5
Further rules table refactoring
Oct 13, 2022
f03f3d1
Fix CI: linting errors, type errors, tests
banderror Oct 13, 2022
5babefe
Restructuring common folder, take 1
banderror Oct 14, 2022
4657f76
Cleanup
Oct 14, 2022
872394b
Update query hooks
Oct 14, 2022
323db29
Fix name collision
Oct 14, 2022
9eb427b
Restructuring common folder, take 2
banderror Oct 14, 2022
87b7299
Restructuring common folder, take 3
banderror Oct 15, 2022
fe818a1
Fix tests
Oct 16, 2022
1a0f5ba
Fix tests
Oct 16, 2022
31c5a82
Restructuring common folder, take 4
banderror Oct 16, 2022
1e79428
Restructuring common folder, take 5
banderror Oct 16, 2022
4a08b8b
Restructuring common folder, take 6
banderror Oct 17, 2022
5abd594
Restructuring common folder, take 7
banderror Oct 17, 2022
ae573dc
Fix errors
banderror Oct 17, 2022
49d6f94
Update CODEOWNERS
banderror Oct 17, 2022
95c0926
Fix tests
banderror Oct 17, 2022
1bb6651
Fix tests
Oct 17, 2022
38c27c5
Test fixes
Oct 17, 2022
c9476c6
Fix tests
Oct 18, 2022
a55d91d
Fix errors after rebase
banderror Oct 18, 2022
cf04cf4
Fix some TODOs
Oct 18, 2022
25cc4f8
Update and unskip more tests
Oct 18, 2022
66dc2ad
Import bulk action types NOT via index.ts
banderror Oct 18, 2022
0b78136
Merge main into onweek-refactoring-rule-management
banderror Oct 20, 2022
8f55566
Fix post-merge errors
banderror Oct 20, 2022
d6363ee
Address comments
banderror Oct 20, 2022
df5dfd7
Merge branch 'main' into onweek-refactoring-rule-management
banderror Oct 21, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
37 changes: 21 additions & 16 deletions .github/CODEOWNERS
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -477,26 +477,31 @@ x-pack/examples/files_example @elastic/kibana-app-services
/x-pack/plugins/security_solution/common/detection_engine/schemas/alerts @elastic/security-detections-response-alerts
/x-pack/plugins/security_solution/common/field_maps @elastic/security-detections-response-alerts

/x-pack/plugins/security_solution/public/detection_engine/rule_creation_ui @elastic/security-detections-response-alerts
/x-pack/plugins/security_solution/public/detections/pages/alerts @elastic/security-detections-response-alerts

/x-pack/plugins/security_solution/server/lib/detection_engine/migrations @elastic/security-detections-response-alerts
/x-pack/plugins/security_solution/server/lib/detection_engine/notifications @elastic/security-detections-response-alerts
/x-pack/plugins/security_solution/server/lib/detection_engine/schemas @elastic/security-detections-response-alerts
/x-pack/plugins/security_solution/server/lib/detection_engine/rule_preview @elastic/security-detections-response-alerts
/x-pack/plugins/security_solution/server/lib/detection_engine/rule_types @elastic/security-detections-response-alerts
/x-pack/plugins/security_solution/server/lib/detection_engine/signals @elastic/security-detections-response-alerts
/x-pack/plugins/security_solution/server/lib/detection_engine/routes/index @elastic/security-detections-response-alerts
/x-pack/plugins/security_solution/server/lib/detection_engine/routes/signals @elastic/security-detections-response-alerts

## Security Solution sub teams - Detections and Response Rules
/x-pack/plugins/security_solution/common/detection_engine/fleet_integrations @elastic/security-detections-response-rules
/x-pack/plugins/security_solution/common/detection_engine/prebuilt_rules @elastic/security-detections-response-rules
/x-pack/plugins/security_solution/common/detection_engine/rule_management @elastic/security-detections-response-rules
/x-pack/plugins/security_solution/common/detection_engine/rule_monitoring @elastic/security-detections-response-rules
/x-pack/plugins/security_solution/common/detection_engine/schemas/common @elastic/security-detections-response-rules
/x-pack/plugins/security_solution/common/detection_engine/schemas/request @elastic/security-detections-response-rules
/x-pack/plugins/security_solution/common/detection_engine/schemas/response @elastic/security-detections-response-rules
/x-pack/plugins/security_solution/common/detection_engine/rule_schema @elastic/security-detections-response-rules @elastic/security-detections-response-alerts

/x-pack/plugins/security_solution/public/common/components/health_truncate_text @elastic/security-detections-response-rules
/x-pack/plugins/security_solution/public/common/components/links_to_docs @elastic/security-detections-response-rules
/x-pack/plugins/security_solution/public/common/components/ml_popover @elastic/security-detections-response-rules
/x-pack/plugins/security_solution/public/common/components/popover_items @elastic/security-detections-response-rules
/x-pack/plugins/security_solution/public/detection_engine/fleet_integrations @elastic/security-detections-response-rules
/x-pack/plugins/security_solution/public/detection_engine/rule_details_ui @elastic/security-detections-response-rules
/x-pack/plugins/security_solution/public/detection_engine/rule_management @elastic/security-detections-response-rules
/x-pack/plugins/security_solution/public/detection_engine/rule_management_ui @elastic/security-detections-response-rules
/x-pack/plugins/security_solution/public/detection_engine/rule_monitoring @elastic/security-detections-response-rules
/x-pack/plugins/security_solution/public/detections/components/callouts @elastic/security-detections-response-rules
/x-pack/plugins/security_solution/public/detections/components/modals/ml_job_upgrade_modal @elastic/security-detections-response-rules
Expand All @@ -507,17 +512,12 @@ x-pack/examples/files_example @elastic/kibana-app-services
/x-pack/plugins/security_solution/public/detections/pages/detection_engine/rules @elastic/security-detections-response-rules
/x-pack/plugins/security_solution/public/rules @elastic/security-detections-response-rules

/x-pack/plugins/security_solution/server/lib/detection_engine/routes/fleet @elastic/security-detections-response-rules
/x-pack/plugins/security_solution/server/lib/detection_engine/routes/rules @elastic/security-detections-response-rules
/x-pack/plugins/security_solution/server/lib/detection_engine/routes/rules/create_rule_exceptions_route* @elastic/security-solution-platform
/x-pack/plugins/security_solution/server/lib/detection_engine/routes/rules/find_rule_exceptions_route* @elastic/security-solution-platform
/x-pack/plugins/security_solution/server/lib/detection_engine/routes/rules/import_rules_route* @elastic/security-solution-platform
/x-pack/plugins/security_solution/server/lib/detection_engine/routes/rules/preview_rules_route* @elastic/security-detections-response-alerts
/x-pack/plugins/security_solution/server/lib/detection_engine/routes/rules/utils @elastic/security-solution-platform
/x-pack/plugins/security_solution/server/lib/detection_engine/routes/tags @elastic/security-detections-response-rules
/x-pack/plugins/security_solution/server/lib/detection_engine/fleet_integrations @elastic/security-detections-response-rules
/x-pack/plugins/security_solution/server/lib/detection_engine/prebuilt_rules @elastic/security-detections-response-rules
/x-pack/plugins/security_solution/server/lib/detection_engine/rule_management @elastic/security-detections-response-rules
/x-pack/plugins/security_solution/server/lib/detection_engine/rule_monitoring @elastic/security-detections-response-rules
/x-pack/plugins/security_solution/server/lib/detection_engine/rules @elastic/security-detections-response-rules
/x-pack/plugins/security_solution/server/lib/detection_engine/tags @elastic/security-detections-response-rules
/x-pack/plugins/security_solution/server/lib/detection_engine/rule_schema @elastic/security-detections-response-rules @elastic/security-detections-response-alerts

/x-pack/plugins/security_solution/server/utils @elastic/security-detections-response-rules

## Security Solution sub teams - Security Platform
Expand All @@ -527,12 +527,17 @@ x-pack/examples/files_example @elastic/kibana-app-services
/x-pack/plugins/security_solution/cypress/e2e/exceptions @elastic/security-solution-platform
/x-pack/plugins/security_solution/cypress/e2e/value_lists @elastic/security-solution-platform

/x-pack/plugins/security_solution/common/detection_engine/rule_exceptions @elastic/security-solution-platform

/x-pack/plugins/security_solution/public/detection_engine/rule_exceptions @elastic/security-solution-platform
/x-pack/plugins/security_solution/public/detection_engine/rule_exceptions_ui @elastic/security-solution-platform
/x-pack/plugins/security_solution/public/common/components/exceptions @elastic/security-solution-platform
/x-pack/plugins/security_solution/public/exceptions @elastic/security-solution-platform
/x-pack/plugins/security_solution/public/detections/containers/detection_engine/lists @elastic/security-solution-platform
/x-pack/plugins/security_solution/public/common/components/sourcerer @elastic/security-solution-platform

/x-pack/plugins/security_solution/server/lib/detection_engine/rule_actions_legacy @elastic/security-solution-platform
/x-pack/plugins/security_solution/server/lib/detection_engine/rule_exceptions @elastic/security-solution-platform
/x-pack/plugins/security_solution/server/lib/sourcerer @elastic/security-solution-platform

## Security Threat Intelligence - Under Security Platform
Expand Down Expand Up @@ -595,7 +600,7 @@ x-pack/test/threat_intelligence_cypress @elastic/protections-experience


# Security Intelligence And Analytics
/x-pack/plugins/security_solution/server/lib/detection_engine/rules/prepackaged_rules @elastic/security-intelligence-analytics
/x-pack/plugins/security_solution/server/lib/detection_engine/prebuilt_rules/content/prepackaged_rules @elastic/security-intelligence-analytics


# Security Asset Management
Expand Down
2 changes: 1 addition & 1 deletion packages/kbn-securitysolution-io-ts-alerting-types/index.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,14 +20,14 @@ export * from './src/default_severity_mapping_array';
export * from './src/default_threat_array';
export * from './src/default_to_string';
export * from './src/default_uuid';
export * from './src/from';
export * from './src/language';
export * from './src/machine_learning_job_id';
export * from './src/max_signals';
export * from './src/normalized_ml_job_id';
export * from './src/references_default_array';
export * from './src/risk_score';
export * from './src/risk_score_mapping';
export * from './src/rule_schedule';
export * from './src/saved_object_attributes';
export * from './src/severity';
export * from './src/severity_mapping';
Expand Down
55 changes: 30 additions & 25 deletions packages/kbn-securitysolution-io-ts-alerting-types/src/actions/index.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,42 +6,47 @@
* Side Public License, v 1.
*/

/* eslint-disable @typescript-eslint/naming-convention */

import * as t from 'io-ts';
import { saved_object_attributes } from '../saved_object_attributes';

export type RuleActionGroup = t.TypeOf<typeof RuleActionGroup>;
export const RuleActionGroup = t.string;
Copy link
Copy Markdown
Contributor

@marshallmain marshallmain Oct 18, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's the reasoning behind using PascalCase for the io-ts runtime values now? In the past we've used camelCase for objects and PascalCase for types, which generally makes it simple to distinguish between types and concrete instances of that type.

Copy link
Copy Markdown
Contributor Author

@banderror banderror Oct 20, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@marshallmain This is a TypeScript feature called aliases. In this case, it's handy because it allows to write only one JSDoc comment which will be applied to both the TS type and the io-ts schema. So when you hover over them in the IDE you will see the same comment. Without that, we'd need to either duplicate the comment or leave either the TS type or the io-ts schema uncommented. Personally, I also like the fact that it's 1 import instead of 2 when you need both (e.g. in routes).

Copy link
Copy Markdown
Contributor Author

@banderror banderror Oct 20, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One of the examples of how JSDoc comments for rule attributes could look like: x-pack/plugins/security_solution/common/detection_engine/rule_schema/model/common_attributes/related_integrations.ts


export type RuleActionId = t.TypeOf<typeof RuleActionId>;
export const RuleActionId = t.string;

export type RuleActionTypeId = t.TypeOf<typeof RuleActionTypeId>;
export const RuleActionTypeId = t.string;

/**
* Params is an "object", since it is a type of RuleActionParams which is action templates.
* @see x-pack/plugins/alerting/common/rule.ts
*/
export const action_group = t.string;
export const action_id = t.string;
export const action_action_type_id = t.string;
export const action_params = saved_object_attributes;
export type RuleActionParams = t.TypeOf<typeof RuleActionParams>;
export const RuleActionParams = saved_object_attributes;

export const action = t.exact(
export type RuleAction = t.TypeOf<typeof RuleAction>;
export const RuleAction = t.exact(
t.type({
group: action_group,
id: action_id,
action_type_id: action_action_type_id,
params: action_params,
group: RuleActionGroup,
id: RuleActionId,
action_type_id: RuleActionTypeId,
params: RuleActionParams,
})
);

export type Action = t.TypeOf<typeof action>;
export type RuleActionArray = t.TypeOf<typeof RuleActionArray>;
export const RuleActionArray = t.array(RuleAction);

export const actions = t.array(action);
export type Actions = t.TypeOf<typeof actions>;

export const actionsCamel = t.array(
t.exact(
t.type({
group: action_group,
id: action_id,
actionTypeId: action_action_type_id,
params: action_params,
})
)
export type RuleActionCamel = t.TypeOf<typeof RuleActionCamel>;
export const RuleActionCamel = t.exact(
t.type({
group: RuleActionGroup,
id: RuleActionId,
actionTypeId: RuleActionTypeId,
params: RuleActionParams,
})
);
export type ActionsCamel = t.TypeOf<typeof actions>;

export type RuleActionArrayCamel = t.TypeOf<typeof RuleActionArrayCamel>;
export const RuleActionArrayCamel = t.array(RuleActionCamel);
14 changes: 9 additions & 5 deletions packages/kbn-securitysolution-io-ts-alerting-types/src/default_actions_array/index.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,12 +8,16 @@

import * as t from 'io-ts';
import { Either } from 'fp-ts/lib/Either';
import { actions, Actions } from '../actions';
import { RuleActionArray } from '../actions';

export const DefaultActionsArray = new t.Type<Actions, Actions | undefined, unknown>(
export const DefaultActionsArray = new t.Type<
RuleActionArray,
RuleActionArray | undefined,
unknown
>(
'DefaultActionsArray',
actions.is,
(input, context): Either<t.Errors, Actions> =>
input == null ? t.success([]) : actions.validate(input, context),
RuleActionArray.is,
(input, context): Either<t.Errors, RuleActionArray> =>
input == null ? t.success([]) : RuleActionArray.validate(input, context),
t.identity
);
4 changes: 2 additions & 2 deletions packages/kbn-securitysolution-io-ts-alerting-types/src/default_from_string/index.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@

import * as t from 'io-ts';
import { Either } from 'fp-ts/lib/Either';
import { from } from '../from';
import { From } from '../from';

/**
* Types the DefaultFromString as:
Expand All @@ -21,7 +21,7 @@ export const DefaultFromString = new t.Type<string, string | undefined, unknown>
if (input == null) {
return t.success('now-6m');
}
return from.validate(input, context);
return From.validate(input, context);
},
t.identity
);
8 changes: 4 additions & 4 deletions ...s/kbn-securitysolution-io-ts-alerting-types/src/default_risk_score_mapping_array/index.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,20 +8,20 @@

import * as t from 'io-ts';
import { Either } from 'fp-ts/lib/Either';
import { RiskScoreMapping, risk_score_mapping } from '../risk_score_mapping';
import { RiskScoreMapping } from '../risk_score_mapping';

/**
* Types the DefaultStringArray as:
* - If null or undefined, then a default risk_score_mapping array will be set
* - If null or undefined, then a default RiskScoreMapping array will be set
*/
export const DefaultRiskScoreMappingArray = new t.Type<
RiskScoreMapping,
RiskScoreMapping | undefined,
unknown
>(
'DefaultRiskScoreMappingArray',
risk_score_mapping.is,
RiskScoreMapping.is,
(input, context): Either<t.Errors, RiskScoreMapping> =>
input == null ? t.success([]) : risk_score_mapping.validate(input, context),
input == null ? t.success([]) : RiskScoreMapping.validate(input, context),
t.identity
);
8 changes: 4 additions & 4 deletions ...ges/kbn-securitysolution-io-ts-alerting-types/src/default_severity_mapping_array/index.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,20 +8,20 @@

import * as t from 'io-ts';
import { Either } from 'fp-ts/lib/Either';
import { SeverityMapping, severity_mapping } from '../severity_mapping';
import { SeverityMapping } from '../severity_mapping';

/**
* Types the DefaultStringArray as:
* - If null or undefined, then a default severity_mapping array will be set
* - If null or undefined, then a default SeverityMapping array will be set
*/
export const DefaultSeverityMappingArray = new t.Type<
SeverityMapping,
SeverityMapping | undefined,
unknown
>(
'DefaultSeverityMappingArray',
severity_mapping.is,
SeverityMapping.is,
(input, context): Either<t.Errors, SeverityMapping> =>
input == null ? t.success([]) : severity_mapping.validate(input, context),
input == null ? t.success([]) : SeverityMapping.validate(input, context),
t.identity
);
7 changes: 2 additions & 5 deletions packages/kbn-securitysolution-io-ts-alerting-types/src/from/index.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,8 @@ import { parseScheduleDates } from '@kbn/securitysolution-io-ts-utils';

const stringValidator = (input: unknown): input is string => typeof input === 'string';

export const from = new t.Type<string, string, unknown>(
export type From = t.TypeOf<typeof From>;
export const From = new t.Type<string, string, unknown>(
'From',
t.string.is,
(input, context): Either<t.Errors, string> => {
Expand All @@ -23,7 +24,3 @@ export const from = new t.Type<string, string, unknown>(
},
t.identity
);
export type From = t.TypeOf<typeof from>;

export const fromOrUndefined = t.union([from, t.undefined]);
export type FromOrUndefined = t.TypeOf<typeof fromOrUndefined>;
11 changes: 1 addition & 10 deletions packages/kbn-securitysolution-io-ts-alerting-types/src/risk_score/index.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,6 @@
* Side Public License, v 1.
*/

/* eslint-disable @typescript-eslint/naming-convention */

import * as t from 'io-ts';
import { Either } from 'fp-ts/lib/Either';

Expand All @@ -16,6 +14,7 @@ import { Either } from 'fp-ts/lib/Either';
* - Natural Number (positive integer and not a float),
* - Between the values [0 and 100] inclusive.
*/
export type RiskScore = t.TypeOf<typeof RiskScore>;
export const RiskScore = new t.Type<number, number, unknown>(
'RiskScore',
t.number.is,
Expand All @@ -26,11 +25,3 @@ export const RiskScore = new t.Type<number, number, unknown>(
},
t.identity
);

export type RiskScoreC = typeof RiskScore;

export const risk_score = RiskScore;
export type RiskScore = t.TypeOf<typeof risk_score>;

export const riskScoreOrUndefined = t.union([risk_score, t.undefined]);
export type RiskScoreOrUndefined = t.TypeOf<typeof riskScoreOrUndefined>;
22 changes: 8 additions & 14 deletions packages/kbn-securitysolution-io-ts-alerting-types/src/risk_score_mapping/index.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,25 +6,19 @@
* Side Public License, v 1.
*/

/* eslint-disable @typescript-eslint/naming-convention */

import * as t from 'io-ts';
import { operator } from '@kbn/securitysolution-io-ts-types';
import { riskScoreOrUndefined } from '../risk_score';
import { RiskScore } from '../risk_score';

export const risk_score_mapping_field = t.string;
export const risk_score_mapping_value = t.string;
export const risk_score_mapping_item = t.exact(
export type RiskScoreMappingItem = t.TypeOf<typeof RiskScoreMappingItem>;
export const RiskScoreMappingItem = t.exact(
t.type({
field: risk_score_mapping_field,
value: risk_score_mapping_value,
field: t.string,
value: t.string,
operator,
risk_score: riskScoreOrUndefined,
risk_score: t.union([RiskScore, t.undefined]),
})
);

export const risk_score_mapping = t.array(risk_score_mapping_item);
export type RiskScoreMapping = t.TypeOf<typeof risk_score_mapping>;

export const riskScoreMappingOrUndefined = t.union([risk_score_mapping, t.undefined]);
export type RiskScoreMappingOrUndefined = t.TypeOf<typeof riskScoreMappingOrUndefined>;
export type RiskScoreMapping = t.TypeOf<typeof RiskScoreMapping>;
export const RiskScoreMapping = t.array(RiskScoreMappingItem);
22 changes: 22 additions & 0 deletions packages/kbn-securitysolution-io-ts-alerting-types/src/rule_schedule/index.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
/*
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
* or more contributor license agreements. Licensed under the Elastic License
* 2.0 and the Server Side Public License, v 1; you may not use this file except
* in compliance with, at your election, the Elastic License 2.0 or the Server
* Side Public License, v 1.
*/

import * as t from 'io-ts';
import { From } from '../from';

export type RuleInterval = t.TypeOf<typeof RuleInterval>;
export const RuleInterval = t.string; // we need a more specific schema

export type RuleIntervalFrom = t.TypeOf<typeof RuleIntervalFrom>;
export const RuleIntervalFrom = From;

/**
* TODO: Create a regular expression type or custom date math part type here
*/
export type RuleIntervalTo = t.TypeOf<typeof RuleIntervalTo>;
export const RuleIntervalTo = t.string; // we need a more specific schema
7 changes: 2 additions & 5 deletions packages/kbn-securitysolution-io-ts-alerting-types/src/severity/index.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,5 @@

import * as t from 'io-ts';

export const severity = t.keyof({ low: null, medium: null, high: null, critical: null });
export type Severity = t.TypeOf<typeof severity>;

export const severityOrUndefined = t.union([severity, t.undefined]);
export type SeverityOrUndefined = t.TypeOf<typeof severityOrUndefined>;
export type Severity = t.TypeOf<typeof Severity>;
export const Severity = t.keyof({ low: null, medium: null, high: null, critical: null });
Loading