Skip to content

[Security Solution][CTI] Event enrichment search strategy#101553

Merged
rylnd merged 7 commits intoelastic:masterfrom
rylnd:ad_hoc_enrichment
Jun 15, 2021
Merged

[Security Solution][CTI] Event enrichment search strategy#101553
rylnd merged 7 commits intoelastic:masterfrom
rylnd:ad_hoc_enrichment

Conversation

@rylnd
Copy link
Copy Markdown
Contributor

@rylnd rylnd commented Jun 8, 2021 •
edited
Loading

Summary

This adds a new search strategy implementation to perform simplified "indicator match" logic in service of a new CTI feature. As design for the UI component of this is still being finalized, I'm opening this backend-only work as a self-contained PR to get the ball rolling.

At a high level, this strategy:

  1. Converts an allowlist of field parameters into named queries to be sent to elasticsearch
  2. Transforms the response into an array of enrichments, as specified in the enrichments RFC

Checklist

For maintainers

@rylnd rylnd added v8.0.0 Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. v7.14.0 release_note:feature Makes this part of the condensed release notes Team: CTI Security Solution labels Jun 8, 2021
@rylnd rylnd self-assigned this Jun 8, 2021
rylnd added 5 commits June 10, 2021 14:14
@rylnd
Adding boilerplate for new CTI search strategy type
ec2922a 
This is going to be a subtype of the general SecSol search strategy;
the main functionality is going to be:

* transformation of the incoming parameters into named equivalents
* transformation of responses to include enrichment context fields
  (matched.*)
@rylnd
More boilerplate, including tests
1ef6fd6 
A few type errors because our functions don't actually do anything yet,
nor are our request/response types fleshed out.
@rylnd
Starting to flesh out the request parsing
b3410e5 
* Defines a basic request, along with a mock
* Defines helper function to generate should clauses from field values
* Adds placeholder tests throughout
@rylnd
Fleshing out unit tests around our enrichment query
146f36c
@rylnd
Fleshing out response parsing of eventEnrichment strategy
d5f420c
@rylnd rylnd force-pushed the ad_hoc_enrichment branch from ada56e4 to d5f420c Compare June 10, 2021 19:14
@rylnd rylnd changed the title [Security Solution][CTI] Ad hoc event enrichment [Security Solution][CTI] Event enrichment search strategy Jun 10, 2021
@rylnd rylnd requested a review from ecezalp June 10, 2021 19:33
@rylnd rylnd marked this pull request as ready for review June 10, 2021 19:33
@rylnd rylnd requested a review from a team as a code owner June 10, 2021 19:33
@elasticmachine
Copy link
Copy Markdown
Contributor

elasticmachine commented Jun 10, 2021

Pinging @elastic/security-solution (Team: SecuritySolution)

@rylnd
Copy link
Copy Markdown
Contributor Author

rylnd commented Jun 14, 2021

@elasticmachine merge upstream

@kibanamachine
Copy link
Copy Markdown
Contributor

kibanamachine commented Jun 14, 2021

💚 Build Succeeded

Metrics [docs]

✅ unchanged

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @rylnd

ecezalp
ecezalp approved these changes Jun 15, 2021
View reviewed changes
Copy link
Copy Markdown
Contributor

@ecezalp ecezalp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

x-pack/plugins/security_solution/common/cti/constants.ts
INDICATOR_LASTSEEN,
];

export const EVENT_ENRICHMENT_INDICATOR_FIELD_MAP = {
Copy link
Copy Markdown
Contributor

@ecezalp ecezalp Jun 15, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what do you think about

const specialFields = ["foo", "bar"]

const desiredFieldmap = specialFields.reduce((acc, item) => {
  acc[item] = `${DEFAULT_INDICATOR_SOURCE_PATH}.${item}`; 
  return acc;
}, {})

Copy link
Copy Markdown
Contributor Author

@rylnd rylnd Jun 15, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

source.ip and destination.ip break that pattern, unfortunately.

x-pack/plugins/security_solution/common/search_strategy/security_solution/cti/index.mock.ts
import { IEsSearchResponse } from 'src/plugins/data/public';

import {
CtiEventEnrichmentRequestOptions,
Copy link
Copy Markdown
Contributor

@ecezalp ecezalp Jun 15, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we consider treating CTI as an acronym in general?

Copy link
Copy Markdown
Contributor Author

@rylnd rylnd Jun 15, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ahh, the old "acronyms in camel case" debate! I know it well 😄 .

My personal preference is to use pascal case for acronyms longer than two characters: CtiEventEnrichment or HtmlButton, but I agree that consistency is tantamount. I don't think we have any existing constants that break this pattern, but if you've been taking another approach in some parallel work let's discuss.

@rylnd rylnd merged commit 4d921ff into elastic:master Jun 15, 2021
@rylnd rylnd deleted the ad_hoc_enrichment branch June 15, 2021 22:00
@rylnd rylnd mentioned this pull request Jun 15, 2021
Merged
cuff-links pushed a commit to cuff-links/kibana that referenced this pull request Jun 15, 2021
@rylnd @kibanamachine
[Security Solution][CTI] Event enrichment search strategy (elastic#10…
d1426e3 
…1553)

* Adding boilerplate for new CTI search strategy type

This is going to be a subtype of the general SecSol search strategy;
the main functionality is going to be:

* transformation of the incoming parameters into named equivalents
* transformation of responses to include enrichment context fields
  (matched.*)

* More boilerplate, including tests

A few type errors because our functions don't actually do anything yet,
nor are our request/response types fleshed out.

* Starting to flesh out the request parsing

* Defines a basic request, along with a mock
* Defines helper function to generate should clauses from field values
* Adds placeholder tests throughout

* Fleshing out unit tests around our enrichment query

* Fleshing out response parsing of eventEnrichment strategy

* Fix types from elasticsearch

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
rylnd added a commit to rylnd/kibana that referenced this pull request Jun 15, 2021
@rylnd @kibanamachine
[Security Solution][CTI] Event enrichment search strategy (elastic#10…
bd90914 
…1553)

* Adding boilerplate for new CTI search strategy type

This is going to be a subtype of the general SecSol search strategy;
the main functionality is going to be:

* transformation of the incoming parameters into named equivalents
* transformation of responses to include enrichment context fields
  (matched.*)

* More boilerplate, including tests

A few type errors because our functions don't actually do anything yet,
nor are our request/response types fleshed out.

* Starting to flesh out the request parsing

* Defines a basic request, along with a mock
* Defines helper function to generate should clauses from field values
* Adds placeholder tests throughout

* Fleshing out unit tests around our enrichment query

* Fleshing out response parsing of eventEnrichment strategy

* Fix types from elasticsearch

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
rylnd added a commit that referenced this pull request Jun 16, 2021
@rylnd @kibanamachine
[Security Solution][CTI] Event enrichment search strategy (#101553) (#…
1f89ac3 
…102275)

* Adding boilerplate for new CTI search strategy type

This is going to be a subtype of the general SecSol search strategy;
the main functionality is going to be:

* transformation of the incoming parameters into named equivalents
* transformation of responses to include enrichment context fields
  (matched.*)

* More boilerplate, including tests

A few type errors because our functions don't actually do anything yet,
nor are our request/response types fleshed out.

* Starting to flesh out the request parsing

* Defines a basic request, along with a mock
* Defines helper function to generate should clauses from field values
* Adds placeholder tests throughout

* Fleshing out unit tests around our enrichment query

* Fleshing out response parsing of eventEnrichment strategy

* Fix types from elasticsearch

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
@rylnd rylnd mentioned this pull request Jun 28, 2021
Merged
14 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

1 more reviewer

@ecezalp ecezalp ecezalp approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

@rylnd rylnd

Labels

release_note:feature Makes this part of the condensed release notes Team: CTI Security Solution Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. v7.14.0 v8.0.0

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@rylnd @elasticmachine @kibanamachine @ecezalp