[Security Solution] Add rule_source to the API schema

[jpdjere]

Epics: https://github.com/elastic/security-team/issues/1974 (internal), #174168

Summary

As part of the preparatory changes for the work in Milestone 3, we want to add the new rule_source field to the API schema. In particular we should:

• Add rule_source as an optional property to RuleResponse, by introducing it as an optional property in the ResponseFields schema.
  • For now, all endpoints should return undefined for the rule_source field.
• Add rule_source as an optional property to RuleToImport, which defines the schema of required and accepted fields when importing a rule.
  • For now, the new rule_source field should be ignored in the endpoint logic.

The rule_source field will only be optional in RuleResponse temporarily: it should be marked as required when the normalization on read and migration on write on the endpoints is introduced.

Background

kibana/x-pack/plugins/security_solution/docs/rfcs/detection_response/prebuilt_rules_customization.md

Lines 93 to 170 in de25d7c

	## Necessary rule schema changes
	In order to support the customization of Elastic Prebuilt Rules, we need to modify our rule schema. This involves introducing the new top level field: the nested `rule_source` field, as well deprecating the `immutable` field.
	```ts
	// PSEUDOCODE - see complete schema in detail below
	{
	rule_source: {
	type: 'external'
	is_customized: boolean;
	source_updated_at?: Date;
	} | {
	type: 'internal'
	},
	}
	```
	### `rule_source` field
	The `rule_source` field will be a top-level object field in our rule schema that will be a discriminated union of two types: `'internal'` and `'external'`.
	Rules with `rule_source` of type `internal` are rules generated internally in the Kibana application and which don't have an external origin outside of it.
	Rules with `rule_source` of type `external` are rules who have an external origin or source. Elastic prebuilt rules are a case of `external` rule, with the `detection-rules` repository handled by the TRaDE team being their external source origin.
	This also means that a rule with this type of `rule_source` will determine that the rule is an Elastic Prebuilt Rule that can receive upstream updates via Fleet. This field is intended to partly replace the currently existing `immutable` field, which is used today for the same purpose, but -as its name indicates- also currently determines if a rule's fields can be modified/customized.
	Prebuilt rules will have:
	```ts
	{
	rule_source: {
	/**
	* The discriminant of the discriminated union type of the `rule_source` field.
	*/
	type: 'external';
	/**
	* Determines whether the rule (which is prebuilt/external) has been customized by the user,
	* i.e. if any of its fields have been modified and diverged from the base version of the rule,
	* which is the version that is installed from the `security_detection_engine` Fleet package.
	* The value will be initially set to `false` when a brand new prebuilt rule is installed,
	* but will be rewritten to `true` if a rule's field is edited and diverges from the value
	* from the base version of the rule.
	* See section "Updating the `isCustomized` flag"
	*/
	is_customized: boolean;
	/**
	* A date in ISO 8601 format which describes the last time that this prebuilt rule was created
	* and subsequently updated by the TRaDE team, the team responsible for creating prebuilt rules.
	* Its usage is detailed in https://github.com/elastic/detection-rules/issues/2826.
	* This field will be optional in both the API schema and the internal rule schema, since
	* this value will not exist for prebuilt rules until a new version of each rule which includes
	* this field in the prebuilt rule asset is published by the TRaDE team, and the user installs
	* it or updates to it.
	* NOTE: the field will not be included in first iteration of the implementation. There is a
	* dependency with the TRaDE team that has blocked the inclusion of this field in the `security-rule` SO,
	* and the change has therefore been postponed.
	* See ticket https://github.com/elastic/detection-rules/issues/2826 for details.
	*/
	source_updated_at?: Date;
	};
	}
	```
	Custom rules will have:
	```ts
	{
	rule_source: {
	/**
	* The discriminant of the discriminated union type of the `rule_source` field.
	*/
	type: 'internal';
	};
	}
	```

kibana/x-pack/plugins/security_solution/docs/rfcs/detection_response/prebuilt_rules_customization.md

Lines 185 to 243 in de25d7c

	#### API request and response rule schema
	Notice, as well, that `immutable` and `rule_source` will continue to be part **only of the API response schema**, and never form part of the **request parameters**.
	The OpenAPI schema will need to be modified so:
	_Source: [x-pack/plugins/security_solution/common/api/detection_engine/model/rule_schema/common_attributes.schema.yaml](https://github.com/elastic/kibana/blob/main/x-pack/plugins/security_solution/common/api/detection_engine/model/rule_schema/common_attributes.schema.yaml)_
	```yaml
	# [... file continues above...]
	# Add deprecation warning to `immutable` field
	IsRuleImmutable:
	type: boolean
	description: '[DEPRECATION WARNING! This field is deprecated and... <the warning explains the deprecation period and suggests a replacement> ] - Determines whether the rule is a prebuilt Elastic rule.'
	IsExternalRuleCustomized:
	type: boolean
	description: Determines whether an external/prebuilt rule has been customized by the user (i.e. any of its fields have been modified and diverged from the base value).
	ExternalSourceUpdatedAt:
	type: string
	format: date-time
	description: The date and time that the external/prebuilt rule was last updated in its source repository.
	InternalRuleSource:
	description: Type of rule source for internally sourced rules, i.e. created within the Kibana apps.
	type: object
	properties:
	type:
	type: string
	enum:
	- internal
	required:
	- type
	ExternalRuleSource:
	description: Type of rule source for externally sourced rules, i.e. rules that have an external source, such as the Elastic Prebuilt rules repo.
	type: object
	properties:
	type:
	type: string
	enum:
	- external
	is_customized:
	$ref: '#/components/schemas/IsExternalRuleCustomized'
	source_updated_at:
	$ref: '#/components/schemas/ExternalSourceUpdatedAt'
	required:
	- type
	- is_customized
	RuleSource:
	description: Discriminated union that determines whether the rule is internally sourced (created within the Kibana app) or has an external source, such as the Elastic Prebuilt rules repo.
	oneOf:
	- $ref: '#/components/schemas/ExternalRuleSource'
	- $ref: '#/components/schemas/InternalRuleSource'
	# [... file continues below ...]
	```

Rule to Import schema

kibana/x-pack/plugins/security_solution/docs/rfcs/detection_response/prebuilt_rules_customization.md

Lines 275 to 311 in de25d7c

	#### Rule Import request schema
	We also need to modify the `RuleToImport` schema, since now we will be allowing the importing of both custom rules and prebuilt rules.
	Currently, `RuleToImport` optionally accepts the `immutable` param, but rejects validation if its value is set to anything else than `false` - since we don't currently support importing prebuilt rules.
	We will be changing the mechanism for importing rules so that onlt the `rule_id` is required parameter. This parameters will be used to determine if the rule is prebuilt or not, and dynamically calculate `rule_source` during import.
	The rule import endpoint should:
	- for custom rules being imported, if `version` is not specified, set it to `1`.
	- for prebuilt rules being imported, if `version` is not specified, throw an error.
	See the detailed explanation for this mechanism in the [Exporting and importing rules](#exporting-and-importing-rules) sections.
	The `immutable` and `rule_source` fields will be ignored if passed in the request payload.
	_Source: [x-pack/plugins/security_solution/common/api/detection_engine/rule_management/import_rules/rule_to_import.ts](https://github.com/elastic/kibana/blob/main/x-pack/plugins/security_solution/common/api/detection_engine/rule_management/import_rules/rule_to_import.ts)_
	```ts
	// [... file continues above...]
	import {
	// [...]
	RuleSignatureId,
	IsRuleImmutable,
	ExternalSourceAttributes,
	RuleVersion,
	} from '../../model/rule_schema';
	export type RuleToImport = z.infer<typeof RuleToImport>;
	export type RuleToImportInput = z.input<typeof RuleToImport>;
	export const RuleToImport = BaseCreateProps.and(TypeSpecificCreateProps).and(
	ResponseFields.partial().extend({
	rule_id: RuleSignatureId,
	})
	);
	```

[elasticmachine]

Pinging @elastic/security-detections-response (Team:Detections and Resp)

[elasticmachine]

Pinging @elastic/security-detection-rule-management (Team:Detection Rule Management)

[elasticmachine]

Pinging @elastic/security-solution (Team: SecuritySolution)