[sentinel_one] Enrich events with event.{category,type,outcome}#3787

Merged

andrewkroh merged 8 commits intoelastic:mainfrom

vinit-chauhan:sentinel_one_pipeline_fix

Aug 23, 2022

Contributor

vinit-chauhan commented Jul 21, 2022 •

edited

Loading

Type of change

Bug

What does this PR do?

This PR fixes the bug related to the value of event.category and event.type fields are being statically mapped. In this PR we have mapped the values of event.category, event.type and event.outcome based on the activity performed.

Checklist

I have reviewed tips for building integrations and this pull request is aligned with them.
I have verified that all data streams collect metrics or logs.
I have added an entry to my package's changelog.yml file.
I have verified that Kibana version constraints are current according to guidelines.

How to test this PR locally

Related issues

Closes The sentinel_one activity dataset events are miscategorized as "malware" in field event.category #3786

Screenshots


          Enrich the event.category, event.type and event.outcome field based o…

9eea50d

…n activity

vinit-chauhan requested a review from a team as a code owner

July 21, 2022 12:30


          Update the changelog file

45058d0

vinit-chauhan added bug Team:Security-External Integrations Integration:sentinel_one labels

elasticmachine commented Jul 21, 2022

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

elasticmachine commented Jul 21, 2022 •

edited

Loading

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Start Time: 2022-08-17T13:38:11.157+0000
Duration: 17 min 5 sec

Test stats 🧪

Test	Results
Failed	0
Passed	31
Skipped	0
Total	31

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.

elasticmachine commented Jul 21, 2022 •

edited

Loading

🌐 Coverage report

Name	Metrics % (`covered/total`)	Diff
Packages	100.0% (`5/5`)	💚
Files	100.0% (`5/5`)	💚 2.825
Classes	100.0% (`5/5`)	💚 2.825
Methods	100.0% (`71/71`)	💚 10.664
Lines	97.97% (`2558/2611`)	👍 7.059
Conditionals	100.0% (`0/0`)	💚

efd6 reviewed

View reviewed changes

packages/sentinel_one/data_stream/activity/elasticsearch/ingest_pipeline/default.yml Outdated

+                        def eventCategory = new HashSet();
+                        def eventType = new HashSet();
+                        if(ctx.json?.threatId != null && ctx.json?.threatId != "") {
+                            if(ctx.json?.data?.threatClassification == "Malware") {

Contributor

efd6 Jul 22, 2022

I there a reason "Trojan" is not a malware classification? It appears in some of the documents that are now classed as "threat"/"indicator".

Contributor Author

vinit-chauhan Jul 22, 2022

Oh, yes. Will update it.

packages/sentinel_one/data_stream/activity/elasticsearch/ingest_pipeline/default.yml Outdated

Comment on lines +41 to +42

		else if(ctx.json?.primaryDescription != null){
		if(ctx.json?.primaryDescription?.toLowerCase().contains("logged in")){

Contributor

efd6 Jul 22, 2022

This could be made less verbose and made to do less work by assigning a local. Also, since we know ctx.json?.primaryDescription != null, the conditional null derefs are not needed in the later steps.

Suggested change

      
                      else if(ctx.json?.primaryDescription != null){
          
                          if(ctx.json?.primaryDescription?.toLowerCase().contains("logged in")){
          
                      else if(ctx.json?.primaryDescription != null) {
          
                          def description = ctx.json.primaryDescription.toLowerCase();
          
                          if (description.contains("logged in")) {

with similar uses in all the following conditions.

packages/sentinel_one/data_stream/activity/elasticsearch/ingest_pipeline/default.yml Outdated

+                                  eventCategory.add("authentication");
+                                  ctx.event.outcome = "failure";
+                            }
+                          eventType.add("info");

Contributor

efd6 Jul 22, 2022

Adjust the indent to match the block.


          Changes as per review comments

b9ad5c4

andrewkroh reviewed

View reviewed changes

packages/sentinel_one/data_stream/activity/elasticsearch/ingest_pipeline/default.yml Outdated

+                            eventType.add("info");
+                        }
+                        ctx.event.category = eventCategory.size()==0 ? null : eventCategory;
+                        ctx.event.type = eventType.size()==0 ? null : eventType;

Member

andrewkroh Aug 2, 2022

How about an if (!eventCategory.isEmpty) { ctx.event.category = eventCategory } so that we don't end up with a null in the doc. Same for event.type.

packages/sentinel_one/data_stream/activity/elasticsearch/ingest_pipeline/default.yml Outdated

+                                eventType.add("start");
+                                ctx.event.outcome = "success";
+                            }
+                            if(description.contains("logged out")){

Member

andrewkroh Aug 2, 2022

Should these be else if?

packages/sentinel_one/data_stream/activity/elasticsearch/ingest_pipeline/default.yml Outdated

+                                eventCategory.add("authentication");
+                                ctx.event.outcome = "failure";
+                            }
+                            eventType.add("info");

Member

andrewkroh Aug 2, 2022

Is this meant to be a catch all? Should it be part of a final else block?

vinit-chauhan and others added 2 commits

August 4, 2022 21:35


          Changes as per review comments

8dbd038


          Merge branch 'main' into sentinel_one_pipeline_fix

c95be9b

vinit-chauhan force-pushed the sentinel_one_pipeline_fix branch from f2157dd to 94b571f Compare

August 12, 2022 15:49


          Merge branch 'main' into sentinel_one_pipeline_fix

510311b

vinit-chauhan force-pushed the sentinel_one_pipeline_fix branch from 94b571f to 510311b Compare

August 12, 2022 16:45

Contributor Author

vinit-chauhan commented Aug 16, 2022

/test

andrewkroh added the conflicts label

andrewkroh changed the title ~~[sentinel_one] Enrich the event.category, event.type and event.outcome field based o…~~ [sentinel_one] Enrich events with event.category/type/outcome

andrewkroh changed the title ~~[sentinel_one] Enrich events with event.category/type/outcome~~ [sentinel_one] Enrich events with event.{category,type,outcome}

efd6 reviewed

View reviewed changes

packages/sentinel_one/data_stream/activity/elasticsearch/ingest_pipeline/default.yml Outdated

+                    source: >
+                        def eventCategory = new HashSet();
+                        def eventType = new HashSet();
+                        if (ctx.json?.threatId != null && ctx.json?.threatId != '') {

Contributor

efd6 Aug 17, 2022

Suggested change

      
                      if (ctx.json?.threatId != null && ctx.json?.threatId != '') {
          
                      if (ctx.json?.threatId != null && ctx.json.threatId != '') {

packages/sentinel_one/data_stream/activity/elasticsearch/ingest_pipeline/default.yml Outdated

+                        if (!eventCategory.isEmpty()) {
+                            ctx.event.category = eventCategory;
+                        }
+                        if (ctx.json?.data?.confidenceLevel != null && ['suspicious', 'malicious'].contains(ctx.json?.data?.confidenceLevel)) {

Contributor

efd6 Aug 17, 2022

[...].contains(null) returns false unless the array list holds a null, so this can be rewritten:

Suggested change

      
                      if (ctx.json?.data?.confidenceLevel != null && ['suspicious', 'malicious'].contains(ctx.json?.data?.confidenceLevel)) {
          
                      if (['suspicious', 'malicious'].contains(ctx.json?.data?.confidenceLevel)) {

packages/sentinel_one/data_stream/threat/elasticsearch/ingest_pipeline/default.yml Outdated

+                            if (threat_classification == 'Exploit' || threat_classification == 'PUA') {
+                                eventCategory.add('threat');
+                                eventType.add('indicator');
+                            } else if (threat_classification == 'Malware' || threat_classification == 'Ransomware' || threat_classification == 'Trojan' || threat_classification == 'Downloader') {

Contributor

efd6 Aug 17, 2022

Suggested change

      
                          } else if (threat_classification == 'Malware' || threat_classification == 'Ransomware' || threat_classification == 'Trojan' || threat_classification == 'Downloader') {
          
                          } else if ['Malware', 'Ransomware', 'Trojan', 'Downloader'].contains(threat_classification)) {

Maybe similar for above? I have no strong opinion either way for that though.

vinit-chauhan and others added 2 commits

August 17, 2022 18:56


          Merge branch 'main' into sentinel_one_pipeline_fix

9f66cc3


          Changes as per review comments

5b0b1a2

andrewkroh approved these changes

View reviewed changes

andrewkroh merged commit 5f36a35 into elastic:main

andrewkroh mentioned this pull request

[production] Promote many packages from snapshot elastic/package-storage#5402

Merged

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug conflicts Integration:sentinel_one