Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Security Solution] Allow exporting prebuilt rules at the API level #180167

Open
Tracked by #174168
jpdjere opened this issue Apr 5, 2024 · 4 comments
Open
Tracked by #174168

[Security Solution] Allow exporting prebuilt rules at the API level #180167

jpdjere opened this issue Apr 5, 2024 · 4 comments
Assignees
@rylnd
Labels
8.17 candidate enhancement New value added to drive a business result Feature:Prebuilt Detection Rules Security Solution Prebuilt Detection Rules Team:Detection Rule Management Security Detection Rule Management Team Team:Detections and Resp Security Detection Response Team Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc.

Comments

@jpdjere
Copy link
Contributor

jpdjere commented Apr 5, 2024 •
edited
Loading

Epics: https://github.com/elastic/security-team/issues/1974 (internal), #174168

Summary

  • Allow exporting prebuilt rules at the API level.
  • Remove the checks that we currently do server-side to filter out prebuilt rules from the response payload.

Background

From the RFC:

There are two helper functions used for exporting:

The Export Rules - POST /rules/_export endpoint uses either the first or the second helper depending on the request payload, while Bulk Actions - POST /rules/_bulk_action uses only getExportByObjectIds.

In order to allow the endpoint to export both custom and prebuilt rules, we need to update the logic and remove the checks that we currently do server-side in both of these methods, and which filter out prebuilt rules from the response payload.

Acceptance criteria
@jpdjere jpdjere added triage_needed Team:Detections and Resp Security Detection Response Team Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. Team:Detection Rule Management Security Detection Rule Management Team Feature:Prebuilt Detection Rules Security Solution Prebuilt Detection Rules labels Apr 5, 2024
@elasticmachine
Copy link
Contributor

Pinging @elastic/security-detections-response (Team:Detections and Resp)

@elasticmachine
Copy link
Contributor

Pinging @elastic/security-detection-rule-management (Team:Detection Rule Management)

@elasticmachine
Copy link
Contributor

Pinging @elastic/security-solution (Team: SecuritySolution)

@jpdjere jpdjere mentioned this issue Apr 5, 2024
Open
@banderror banderror changed the title [Security Solution] Allow exporting prebuilt rules at the API level [Security Solution] Allow exporting prebuilt rules at the API level (DRAFT) Apr 17, 2024
@banderror
Copy link
Contributor

Hey @jpdjere, tomorrow we'll have Milestone 3 onboarding meetings with Ryland and Zhenia, and I think it would be great if we could finalize descriptions for all the tickets we're going to ask them to help us with. Could you please prioritize writing a description at least for this one before the meetings?

@banderror banderror changed the title [Security Solution] Allow exporting prebuilt rules at the API level (DRAFT) [Security Solution] Allow exporting prebuilt rules at the API level Jul 17, 2024
@banderror banderror added enhancement New value added to drive a business result and removed triage_needed labels Jul 17, 2024
@rylnd rylnd mentioned this issue Sep 30, 2024
Merged
4 tasks
rylnd added a commit that referenced this issue Oct 15, 2024
@rylnd
[Security Solution] Allow exporting of prebuilt rules via the API (#1…
b67bd83 
…94498)

## Summary

This PR introduces the backend functionality necessary to export
prebuilt rules via our existing export APIs:

1. Export Rules - POST /rules/_export 
2. Bulk Actions - POST /rules/_bulk_action 

The [Prebuilt Rule Customization
RFC](https://github.com/elastic/kibana/blob/main/x-pack/plugins/security_solution/docs/rfcs/detection_response/prebuilt_rules_customization.md)
goes into detail, and the export-specific issue is described
[here](#180167 (comment)).


## Steps to Review
1. Enable the Feature Flag: `prebuiltRulesCustomizationEnabled`
1. Install the prebuilt rules package via fleet  
1. Install some prebuilt rules, and obtain a prebuilt rule's `rule_id`,
e.g. `ac8805f6-1e08-406c-962e-3937057fa86f`
1. Export the rule via the export route, e.g. (in Dev Tools):

        POST kbn:api/detection_engine/rules/_export
        
Note that you may need to use the CURL equivalent for these requests, as
the dev console does not seem to handle file responses:

curl --location --request POST
'http://localhost:5601/api/detection_engine/rules/_export?exclude_export_details=true&file_name=exported_rules.ndjson'
\
        --header 'kbn-xsrf: true' \
        --header 'elastic-api-version: 2023-10-31' \
        --header 'Authorization: Basic waefoijawoefiajweo=='

1. Export the rule via bulk actions, e.g. (in Dev Tools):

        POST kbn:api/detection_engine/rules/_bulk_action
        {
          "action": "export"
        }
        
1. Observe that the exported rules' fields are correct, especially
`rule_source` and `immutable` (see tests added here for examples).

### Checklist

- [ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [ ] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed
### For maintainers

- [ ] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
kibanamachine pushed a commit to kibanamachine/kibana that referenced this issue Oct 15, 2024
@rylnd
[Security Solution] Allow exporting of prebuilt rules via the API (el…
2241023 
…astic#194498)

## Summary

This PR introduces the backend functionality necessary to export
prebuilt rules via our existing export APIs:

1. Export Rules - POST /rules/_export
2. Bulk Actions - POST /rules/_bulk_action

The [Prebuilt Rule Customization
RFC](https://github.com/elastic/kibana/blob/main/x-pack/plugins/security_solution/docs/rfcs/detection_response/prebuilt_rules_customization.md)
goes into detail, and the export-specific issue is described
[here](elastic#180167 (comment)).

## Steps to Review
1. Enable the Feature Flag: `prebuiltRulesCustomizationEnabled`
1. Install the prebuilt rules package via fleet
1. Install some prebuilt rules, and obtain a prebuilt rule's `rule_id`,
e.g. `ac8805f6-1e08-406c-962e-3937057fa86f`
1. Export the rule via the export route, e.g. (in Dev Tools):

        POST kbn:api/detection_engine/rules/_export

Note that you may need to use the CURL equivalent for these requests, as
the dev console does not seem to handle file responses:

curl --location --request POST
'http://localhost:5601/api/detection_engine/rules/_export?exclude_export_details=true&file_name=exported_rules.ndjson'
\
        --header 'kbn-xsrf: true' \
        --header 'elastic-api-version: 2023-10-31' \
        --header 'Authorization: Basic waefoijawoefiajweo=='

1. Export the rule via bulk actions, e.g. (in Dev Tools):

        POST kbn:api/detection_engine/rules/_bulk_action
        {
          "action": "export"
        }

1. Observe that the exported rules' fields are correct, especially
`rule_source` and `immutable` (see tests added here for examples).

### Checklist

- [ ]
[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)
was added for features that require explanation or tutorials
- [ ] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [ ] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed
### For maintainers

- [ ] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)

(cherry picked from commit b67bd83)
@rylnd rylnd mentioned this issue Oct 21, 2024
Merged
rylnd added a commit that referenced this issue Nov 11, 2024
@rylnd @elasticmachine
[Security Solution] Test plans for prebuilt rule import and export (#…
e429849 
…191116)

## Summary

This PR introduces test plans for both [Prebuilt Rule
Import](#180168) (corresponding
[PR](#190198)) and [Prebuilt Rule
Export](#180167) (corresponding
[PR](#194498)). Import is
considerably more complicated as it is calculating new values (for
`rule_source`, `immutable`), while the export work is mainly removing
existing restrictions (which allowed only custom rules to be exported).

---------

Co-authored-by: Elastic Machine <[email protected]>
kibanamachine pushed a commit to kibanamachine/kibana that referenced this issue Nov 11, 2024
@rylnd
[Security Solution] Test plans for prebuilt rule import and export (e…
3c89524 
…lastic#191116)

## Summary

This PR introduces test plans for both [Prebuilt Rule
Import](elastic#180168) (corresponding
[PR](elastic#190198)) and [Prebuilt Rule
Export](elastic#180167) (corresponding
[PR](elastic#194498)). Import is
considerably more complicated as it is calculating new values (for
`rule_source`, `immutable`), while the export work is mainly removing
existing restrictions (which allowed only custom rules to be exported).

---------

Co-authored-by: Elastic Machine <[email protected]>
(cherry picked from commit e429849)
kibanamachine added a commit that referenced this issue Nov 11, 2024
@kibanamachine @rylnd
[8.x] [Security Solution] Test plans for prebuilt rule import and exp…
48c81df 
…ort (#191116) (#199716)

# Backport

This will backport the following commits from `main` to `8.x`:
- [[Security Solution] Test plans for prebuilt rule import and export
(#191116)](#191116)

<!--- Backport version: 9.4.3 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Ryland
Herrick","email":"[email protected]"},"sourceCommit":{"committedDate":"2024-11-11T21:01:46Z","message":"[Security
Solution] Test plans for prebuilt rule import and export (#191116)\n\n##
Summary\r\n\r\nThis PR introduces test plans for both [Prebuilt
Rule\r\nImport](#180168)
(corresponding\r\n[PR](#190198))
and [Prebuilt
Rule\r\nExport](#180167)
(corresponding\r\n[PR](#194498)).
Import is\r\nconsiderably more complicated as it is calculating new
values (for\r\n`rule_source`, `immutable`), while the export work is
mainly removing\r\nexisting restrictions (which allowed only custom
rules to be exported).\r\n\r\n---------\r\n\r\nCo-authored-by: Elastic
Machine
<[email protected]>","sha":"e4298492b5e48338396618d51168ea3e8427c103","branchLabelMapping":{"^v9.0.0$":"main","^v8.17.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","test-plan","v9.0.0","Team:Detections
and Resp","Team: SecuritySolution","Team:Detection Rule
Management","Feature:Prebuilt Detection
Rules","backport:version","v8.17.0"],"title":"[Security Solution] Test
plans for prebuilt rule import and
export","number":191116,"url":"https://github.com/elastic/kibana/pull/191116","mergeCommit":{"message":"[Security
Solution] Test plans for prebuilt rule import and export (#191116)\n\n##
Summary\r\n\r\nThis PR introduces test plans for both [Prebuilt
Rule\r\nImport](#180168)
(corresponding\r\n[PR](#190198))
and [Prebuilt
Rule\r\nExport](#180167)
(corresponding\r\n[PR](#194498)).
Import is\r\nconsiderably more complicated as it is calculating new
values (for\r\n`rule_source`, `immutable`), while the export work is
mainly removing\r\nexisting restrictions (which allowed only custom
rules to be exported).\r\n\r\n---------\r\n\r\nCo-authored-by: Elastic
Machine
<[email protected]>","sha":"e4298492b5e48338396618d51168ea3e8427c103"}},"sourceBranch":"main","suggestedTargetBranches":["8.x"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/191116","number":191116,"mergeCommit":{"message":"[Security
Solution] Test plans for prebuilt rule import and export (#191116)\n\n##
Summary\r\n\r\nThis PR introduces test plans for both [Prebuilt
Rule\r\nImport](#180168)
(corresponding\r\n[PR](#190198))
and [Prebuilt
Rule\r\nExport](#180167)
(corresponding\r\n[PR](#194498)).
Import is\r\nconsiderably more complicated as it is calculating new
values (for\r\n`rule_source`, `immutable`), while the export work is
mainly removing\r\nexisting restrictions (which allowed only custom
rules to be exported).\r\n\r\n---------\r\n\r\nCo-authored-by: Elastic
Machine
<[email protected]>","sha":"e4298492b5e48338396618d51168ea3e8427c103"}},{"branch":"8.x","label":"v8.17.0","branchLabelMappingKey":"^v8.17.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->

Co-authored-by: Ryland Herrick <[email protected]>
tkajtoch pushed a commit to tkajtoch/kibana that referenced this issue Nov 12, 2024
@rylnd @elasticmachine @tkajtoch
[Security Solution] Test plans for prebuilt rule import and export (e…
5d5733e 
…lastic#191116)

## Summary

This PR introduces test plans for both [Prebuilt Rule
Import](elastic#180168) (corresponding
[PR](elastic#190198)) and [Prebuilt Rule
Export](elastic#180167) (corresponding
[PR](elastic#194498)). Import is
considerably more complicated as it is calculating new values (for
`rule_source`, `immutable`), while the export work is mainly removing
existing restrictions (which allowed only custom rules to be exported).

---------

Co-authored-by: Elastic Machine <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@rylnd rylnd

Labels
8.17 candidate enhancement New value added to drive a business result Feature:Prebuilt Detection Rules Security Solution Prebuilt Detection Rules Team:Detection Rule Management Security Detection Rule Management Team Team:Detections and Resp Security Detection Response Team Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@rylnd @jpdjere @banderror @elasticmachine