diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS index ea772efe4bc..42a10cae9a4 100644 --- a/.github/CODEOWNERS +++ b/.github/CODEOWNERS @@ -54,6 +54,7 @@ /packages/cyberark @elastic/security-external-integrations /packages/cyberarkpas @elastic/security-external-integrations /packages/cylance @elastic/security-external-integrations +/packages/darktrace @elastic/security-external-integrations /packages/dga @elastic/ml-ui /packages/docker @elastic/obs-cloudnative-monitoring /packages/elastic_agent @elastic/elastic-agent-control-plane diff --git a/packages/darktrace/_dev/build/build.yml b/packages/darktrace/_dev/build/build.yml new file mode 100644 index 00000000000..8d9e4bf7ac8 --- /dev/null +++ b/packages/darktrace/_dev/build/build.yml @@ -0,0 +1,3 @@ +dependencies: + ecs: + reference: git@v8.4.0 diff --git a/packages/darktrace/_dev/build/docs/README.md b/packages/darktrace/_dev/build/docs/README.md new file mode 100644 index 00000000000..691837cee99 --- /dev/null +++ b/packages/darktrace/_dev/build/docs/README.md @@ -0,0 +1,130 @@ +# Darktrace + +## Overview + +The [Darktrace](https://darktrace.com/) integration allows you to monitor Alert Logs. Darktrace is a network solution for detecting and investigating emerging cyber-threats that evade traditional security tools. It is powered by Enterprise Immune System technology, which uses machine learning and mathematics to monitor behaviors and detect anomalies in your organization’s network. + +Use the Darktrace integration to collect and parse data from the REST APIs or via Syslog. Then visualise that data in Kibana. + +For example, you could use the data from this integration to know which model is breached and analyse model breaches, and also know about system health, changes in monitored traffic, and any errors experienced by Darktrace Security Modules or probe instances. + +## Data streams + +The Darktrace integration collects logs for three types of events: AI Analyst Alert, Model Breach Alert and System Status Alert. + +**AI Analyst Alert** is generated by investigates, analyzes, and reports upon threats seen within your Darktrace environment; as a starting point, it reviews and investigates all Model Breaches that occur on the system. If behavior which would be of interest to a cyber analyst is detected, an event is created. See Example Schema [here](https://customerportal.darktrace.com/product-guides/main/syslog-aia-json-schema). + +**Model Breach Alert** is generated when a model breach is triggered. A model is used to define a set of conditions which, when met, will alert the system to the occurrence of a particular event or chain of anomalous behavior. Darktrace models are focused on pattern-of-life anomaly detection, potentially malicious behavior, and compliance issues. See Example Schema [here](https://customerportal.darktrace.com/product-guides/main/syslog-json-schema). + +**System Status Alert** keep Darktrace operators informed of system health, changes in monitored traffic, and any errors experienced by Darktrace Security Modules or probe instances. System Status Alerts include details of the originating host, the severity of the event, and links that may be helpful to investigate or resolve the issue. Notifications are sent for active system events and (optionally) on event resolution. See Example Schema [here](https://customerportal.darktrace.com/product-guides/main/syslog-json-system-schema). + +## Requirements + +You need Elasticsearch for storing and searching your data and Kibana for visualizing and managing it. You can use our hosted Elasticsearch Service on Elastic Cloud, which is recommended, or self-manage the Elastic Stack on your own hardware. + +Firewall exceptions to allow communication from the Darktrace master instance to the Syslog server. + +This module has been tested against **Darktrace Threat Visualizer v5.2**. + +## Setup + +### To collect data from Darktrace REST APIs, follow the below steps: + +1. Hostname URL will be your . (Threat Visualizer Console Hostname) +2. Public and Private Token will be generated by following this [Link](https://customerportal.darktrace.com/product-guides/main/api-tokens). + +**Note:** System Status Alert are not supported by REST API. + +### To collect data from Darktrace via Syslog, follow the below steps: + +The user needs to create a different Syslog Forwarder with different ports for each data stream. + +The process for configuring syslog-format alerts is identical for AI Analyst Alerts, Model Breach Alerts and System Status Alerts. Generic configuration guidance is provided below: + +1. Open the Darktrace Threat Visualizer Dashboard and navigate to the **System Config** page. (**Main menu › Admin**). +2. From the left-side menu, select **Modules**, then navigate to the **Workflow Integrations** section and choose **Syslog**. +3. Select **Syslog JSON** tab and click **New** to set up new Syslog Forwarder. +4. Enter the **IP Address**  and **Port** of the Elastic Agent that is running the integration in the **Server** and **Server Port** field respectively. + +For more details, see [Documentation](https://customerportal.darktrace.com/product-guides/main/json-alerts). + +**Note:** + - It is recommended to turn on **Full Timestamps** toggle in **Show Advanced Options** to get the full timestamp instead of the RFC3164-formatted timestamp. + - It is also recommended to turn off **Reduced Message Size** toggle in **Show Advanced Options** to get more information about alerts. + +### After following generic guidance steps, below are the steps for collecting individual logs for all three data streams. + +#### For AI Analyst Alert, below are the suggested configurations to collect all the events of AI Analyst Alert: + +- Configure the following settings in **Show Advanced Options**: + +| Field Name | Value | +| --------------------------------------- | ----------------------------------- | +| Send AI Analyst Alerts | ON | +| Send AI Analyst Alerts Immediately | ON | +| AI Analyst Behavior Filter | Critical, Suspicious and Compliance | +| Minimum AI Analyst Incident Event Score | 0 | +| Minimum AI Analyst Incident Score | 0 | +| Legacy AI Analyst Alerts | OFF | + +#### For Model Breach Alert, below are the suggested configurations to collect all the events of Model Breach Alert: + +- Configure the following settings in **Show Advanced Options**: + +| Field Name | Value | +| ---------------------------- | -------------------------------------------------- | +| Send Model Breach Alerts | ON | +| Model Breach Behavior Filter | Critical, Suspicious, Compliance and Informational | +| Minimum Breach Score | 0 | +| Minimum Breach Priority | 0 | +| Model Expression | N/A | +| Model Tags Expression | N/A | +| Device IP Addresses | N/A | +| Device Tags Addresses | N/A | + +#### For System Status Alert, below are the suggested configurations to collect all the events of System Status Alert: + +- Configure the following settings in **Show Advanced Options**: + +| Field Name | Value | +| ---------------------------------- | ------------- | +| Send System Status Alerts | ON | +| Send Resolved System Status Alerts | ON | +| Minimum System Status Priority | Informational | + +### See more about [Syslog Filters and Optional Settings](https://customerportal.darktrace.com/product-guides/main/syslog-json-alert-settings) + +**Note** : A Fully Qualified Domain Name (FQDN) must be configured for the Darktrace instance in order for links to be included in external alerts. + - An FQDN can be configured from the **System** subsection on the **Settings** tab of the Darktrace **System Config** page. + +## Logs reference + +### ai_analyst_alert + +This is the `ai_analyst_alert` dataset. + +#### Example + +{{event "ai_analyst_alert"}} + +{{fields "ai_analyst_alert"}} + +### model_breach_alert + +This is the `model_breach_alert` dataset. + +#### Example + +{{event "model_breach_alert"}} + +{{fields "model_breach_alert"}} + +### system_status_alert + +This is the `system_status_alert` dataset. + +#### Example + +{{event "system_status_alert"}} + +{{fields "system_status_alert"}} \ No newline at end of file diff --git a/packages/darktrace/_dev/deploy/docker/docker-compose.yml b/packages/darktrace/_dev/deploy/docker/docker-compose.yml new file mode 100644 index 00000000000..1eba57d48f2 --- /dev/null +++ b/packages/darktrace/_dev/deploy/docker/docker-compose.yml @@ -0,0 +1,60 @@ +version: '2.3' +services: + darktrace-ai_analyst_alert-tls: + image: docker.elastic.co/observability/stream:v0.8.0 + volumes: + - ./sample_logs:/sample_logs:ro + command: log --start-signal=SIGHUP --delay=5s --addr elastic-agent:9571 -p=tls --insecure /sample_logs/ai_analyst_alert.log + darktrace-ai_analyst_alert-tcp: + image: docker.elastic.co/observability/stream:v0.8.0 + volumes: + - ./sample_logs:/sample_logs:ro + command: log --start-signal=SIGHUP --delay=5s --addr elastic-agent:9571 -p=tcp /sample_logs/ai_analyst_alert.log + darktrace-ai_analyst_alert-udp: + image: docker.elastic.co/observability/stream:v0.8.0 + volumes: + - ./sample_logs:/sample_logs:ro + command: log --start-signal=SIGHUP --delay=5s --addr elastic-agent:9574 -p=udp /sample_logs/ai_analyst_alert.log + darktrace-model_breach_alert-tls: + image: docker.elastic.co/observability/stream:v0.8.0 + volumes: + - ./sample_logs:/sample_logs:ro + command: log --start-signal=SIGHUP --delay=5s --addr elastic-agent:9572 -p=tls --insecure /sample_logs/model_breach_alert.log + darktrace-model_breach_alert-tcp: + image: docker.elastic.co/observability/stream:v0.8.0 + volumes: + - ./sample_logs:/sample_logs:ro + command: log --start-signal=SIGHUP --delay=5s --addr elastic-agent:9572 -p=tcp /sample_logs/model_breach_alert.log + darktrace-model_breach_alert-udp: + image: docker.elastic.co/observability/stream:v0.8.0 + volumes: + - ./sample_logs:/sample_logs:ro + command: log --start-signal=SIGHUP --delay=5s --addr elastic-agent:9575 -p=udp /sample_logs/model_breach_alert.log + darktrace-system_status_alert-tls: + image: docker.elastic.co/observability/stream:v0.8.0 + volumes: + - ./sample_logs:/sample_logs:ro + command: log --start-signal=SIGHUP --delay=5s --addr elastic-agent:9573 -p=tls --insecure /sample_logs/system_status_alert.log + darktrace-system_status_alert-tcp: + image: docker.elastic.co/observability/stream:v0.8.0 + volumes: + - ./sample_logs:/sample_logs:ro + command: log --start-signal=SIGHUP --delay=5s --addr elastic-agent:9573 -p=tcp /sample_logs/system_status_alert.log + darktrace-system_status_alert-udp: + image: docker.elastic.co/observability/stream:v0.8.0 + volumes: + - ./sample_logs:/sample_logs:ro + command: log --start-signal=SIGHUP --delay=5s --addr elastic-agent:9576 -p=udp /sample_logs/system_status_alert.log + darktrace: + image: docker.elastic.co/observability/stream:v0.8.0 + hostname: darktrace + ports: + - 8080 + volumes: + - ./files:/files:ro + environment: + PORT: '8080' + command: + - http-server + - --addr=:8080 + - --config=/files/config.yml diff --git a/packages/darktrace/_dev/deploy/docker/files/config.yml b/packages/darktrace/_dev/deploy/docker/files/config.yml new file mode 100644 index 00000000000..6f06f666c65 --- /dev/null +++ b/packages/darktrace/_dev/deploy/docker/files/config.yml @@ -0,0 +1,13 @@ +rules: + - path: /modelbreaches + methods: ['GET'] + responses: + - status_code: 200 + body: | + [{"model":{"name":"Device::Attack and Recon Tools","pid":135,"phid":1198,"uuid":"8abcdefg-1234-1234-1234-5abcdefg12","logic":{"data":[{"cid":2311,"weight":1},{"cid":2312,"weight":1},{"cid":2315,"weight":1},{"cid":2316,"weight":1},{"cid":2314,"weight":1},{"cid":2310,"weight":1},{"cid":2313,"weight":1}],"targetScore":1,"type":"weightedComponentList","version":1},"throttle":604800,"sharedEndpoints":false,"actions":{"alert":true,"antigena":{},"breach":true,"model":true,"setPriority":false,"setTag":false,"setType":false},"tags":["AP: Internal Recon","OT Engineer"],"interval":3600,"delay":0,"sequenced":false,"active":true,"modified":"2022-07-11 11:47:51","activeTimes":{"devices":{},"tags":{},"type":"exclusions","version":2},"autoUpdatable":true,"autoUpdate":true,"autoSuppress":true,"description":"A device is using common penetration testing tools.\\n\\nAction: Review the device to see if it a security device, these can be tagged as such to exclude them from future breaches. Activity from non security devices merit further investigation into what else the device is doing and could be a significant risk within the network.","behaviour":"decreasing","created":{"by":"System"},"edited":{"by":"System"},"version":77,"priority":4,"category":"Suspicious","compliance":false},"device":{"did":7,"ip":"81.2.69.192","ips":[{"ip":"175.16.199.2","timems":1657746000000,"time":"2022-07-13 21:00:00","sid":1}],"sid":1,"hostname":"localhost.local","firstSeen":1657544891000,"lastSeen":1657748638000,"typename":"desktop","typelabel":"Desktop","credentials":["dummy"],"tags":[{"tid":66,"expiry":0,"thid":66,"name":"Domain Authenticated","restricted":false,"data":{"auto":false,"color":183,"description":""},"isReferenced":true},{"tid":29,"expiry":0,"thid":29,"name":"Linux","restricted":false,"data":{"auto":false,"color":168,"description":"","visibility":"Public"},"isReferenced":true},{"tid":45,"expiry":0,"thid":45,"name":"New Device","restricted":false,"data":{"auto":false,"color":130,"description":"","visibility":"Public"},"isReferenced":true}]},"triggeredComponents":[{"time":1657748807000,"cbid":6,"cid":2311,"chid":2676,"size":1,"threshold":0,"interval":300,"logic":{"data":{"left":{"left":"A","operator":"AND","right":{"left":"H","operator":"AND","right":{"left":"J","operator":"AND","right":"N"}}},"operator":"OR","right":{"left":{"left":"B","operator":"AND","right":{"left":"H","operator":"AND","right":{"left":"J","operator":"AND","right":{"left":"N","operator":"AND","right":"O"}}}},"operator":"OR","right":{"left":{"left":"C","operator":"AND","right":{"left":"H","operator":"AND","right":{"left":"J","operator":"AND","right":"N"}}},"operator":"OR","right":{"left":{"left":"E","operator":"AND","right":{"left":"H","operator":"AND","right":{"left":"J","operator":"AND","right":"N"}}},"operator":"OR","right":{"left":{"left":"F","operator":"AND","right":{"left":"H","operator":"AND","right":{"left":"J","operator":"AND","right":"N"}}},"operator":"OR","right":{"left":{"left":"G","operator":"AND","right":{"left":"H","operator":"AND","right":{"left":"J","operator":"AND","right":"N"}}},"operator":"OR","right":{"left":{"left":"H","operator":"AND","right":{"left":"I","operator":"AND","right":{"left":"J","operator":"AND","right":"N"}}},"operator":"OR","right":{"left":{"left":"H","operator":"AND","right":{"left":"J","operator":"AND","right":{"left":"K","operator":"AND","right":"N"}}},"operator":"OR","right":{"left":{"left":"H","operator":"AND","right":{"left":"J","operator":"AND","right":{"left":"L","operator":"AND","right":"N"}}},"operator":"OR","right":{"left":{"left":"H","operator":"AND","right":{"left":"J","operator":"AND","right":{"left":"M","operator":"AND","right":"N"}}},"operator":"OR","right":{"left":"H","operator":"AND","right":{"left":"J","operator":"AND","right":{"left":"N","operator":"AND","right":"P"}}}}}}}}}}}}},"version":"v0.1"},"metric":{"mlid":16,"name":"connections","label":"Connections"},"triggeredFilters":[{"cfid":26781,"id":"H","filterType":"Direction","arguments":{"value":"out"},"comparatorType":"is","trigger":{"value":"out"}},{"cfid":26783,"id":"J","filterType":"Tagged internal source","arguments":{"value":12},"comparatorType":"does not have tag","trigger":{"value":"12","tag":{"tid":12,"expiry":0,"thid":12,"name":"Security Device","restricted":false,"data":{"auto":false,"color":55,"description":"","visibility":"Public"},"isReferenced":true}}},{"cfid":26787,"id":"N","filterType":"Tagged internal destination","arguments":{"value":12},"comparatorType":"does not have tag","trigger":{"value":"12","tag":{"tid":12,"expiry":0,"thid":12,"name":"Security Device","restricted":false,"data":{"auto":false,"color":55,"description":"","visibility":"Public"},"isReferenced":true}}},{"cfid":26788,"id":"O","filterType":"User agent","arguments":{"value":"examples"},"comparatorType":"does not match regular expression","trigger":{"value":""}},{"cfid":26789,"id":"P","filterType":"URI","arguments":{"value":"examples"},"comparatorType":"matches regular expression","trigger":{"value":""}},{"cfid":26790,"id":"d1","filterType":"Proxied connection","arguments":{"value":"true"},"comparatorType":"display","trigger":{"value":"false"}},{"cfid":26791,"id":"d10","filterType":"HTTP response code","arguments":{},"comparatorType":"display","trigger":{"value":"0"}},{"cfid":26792,"id":"d2","filterType":"HTTP referrer","arguments":{},"comparatorType":"display","trigger":{"value":""}},{"cfid":26793,"id":"d3","filterType":"HTTP method","arguments":{},"comparatorType":"display","trigger":{"value":"GET"}},{"cfid":26794,"id":"d4","filterType":"HTTP X-Forwarded-For","arguments":{},"comparatorType":"display","trigger":{"value":""}},{"cfid":26795,"id":"d5","filterType":"URI","arguments":{},"comparatorType":"display","trigger":{"value":""}},{"cfid":26796,"id":"d6","filterType":"User agent","arguments":{},"comparatorType":"display","trigger":{"value":""}},{"cfid":26797,"id":"d7","filterType":"Destination IP","arguments":{},"comparatorType":"display","trigger":{"value":"81.2.69.144"}},{"cfid":26798,"id":"d8","filterType":"Internal destination device name","arguments":{},"comparatorType":"display","trigger":{"value":"localhost.local"}},{"cfid":26799,"id":"d9","filterType":"Connection hostname","arguments":{},"comparatorType":"display","trigger":{"value":""}}]}],"breachUrl":"https://example.com/#modelbreach/6","pbid":6,"score":0.871,"commentCount":0,"creationTime":1657748815000,"time":1657748808000,"mitreTechniques":[{"technique":"Hardware Additions Mitigation","techniqueID":"T1200"}]}] + - path: /aianalyst/incidentevents + methods: ['GET'] + responses: + - status_code: 200 + body: | + [{"summariser":"AdminConnSummary","acknowledged":false,"pinned":true,"createdAt":1628002089240,"attackPhases":[5],"title":"Extensive Unusual SSH Connections","id":"eabc0011-1234-1234-1234-cabcdefg0011","children":["eabcdef0-1234-1234-1234-cabcdefghij9"],"category":"critical","currentGroup":"eabc1234-1234-1234-1234-cabcdefg0011","groupCategory":"critical","groupScore":"72.9174234","groupPreviousGroups":null,"activityId":"abcd1234","groupingIds":["abcdef12"],"groupByActivity":false,"userTriggered":false,"externalTriggered":false,"aiaScore":98,"summary":"The device 175.16.199.1 was observed making unusual internal SSH connections to a wide range of devices.\n\nThough this behaviour could be the result of legitimate remote access or administration, it could also be a sign of attempted lateral movement by a compromised machine.\n\nConsequently, if this activity was not expected, the security team may wish to investigate further.","periods":[{"start":1627985298683,"end":1628000141220}],"breachDevices":[{"identifier":null,"hostname":null,"ip":"81.2.69.144","mac":null,"subnet":"VPN","did":10,"sid":12}],"relatedBreaches":[{"modelName":"Unusual Activity / Unusual Activity from Re-Activated Device","pbid":1234,"threatScore":37,"timestamp":1627997157000}],"details":[[{"header":"Breaching Device","contents":[{"key":null,"type":"device","values":[{"identifier":null,"hostname":null,"ip":"175.16.199.1","mac":null,"subnet":"VPN","did":10,"sid":12}]}]}],[{"header":"SSH Activity","contents":[{"key":"Time","type":"timestampRange","values":[{"start":1627985298683,"end":1628000141220}]},{"key":"Number of unique IPs","type":"integer","values":[16]},{"key":"Targeted IP ranges include","type":"device","values":[{"identifier":null,"hostname":null,"ip":"81.2.69.192","mac":null,"subnet":null,"did":null,"sid":null},{"identifier":null,"hostname":null,"ip":"175.16.199.1","mac":null,"subnet":null,"did":null,"sid":null},{"identifier":null,"hostname":null,"ip":"175.16.199.3","mac":null,"subnet":null,"did":null,"sid":null}]},{"key":"Destination port","type":"integer","values":[22]},{"key":"Connection count","type":"integer","values":[40]},{"key":"Percentage successful","type":"percentage","values":[100]}]}]]}] diff --git a/packages/darktrace/_dev/deploy/docker/sample_logs/ai_analyst_alert.log b/packages/darktrace/_dev/deploy/docker/sample_logs/ai_analyst_alert.log new file mode 100644 index 00000000000..68401029818 --- /dev/null +++ b/packages/darktrace/_dev/deploy/docker/sample_logs/ai_analyst_alert.log @@ -0,0 +1,2 @@ +<165>1 2022-01-10T21:09:27+00:00 example.cloud.darktrace.com darktrace - - - {"summariser":"AdminConnSummary","acknowledged":false,"pinned":true,"createdAt":1628002089240,"attackPhases":[5],"title":"Extensive Unusual SSH Connections","id":"eabc0011-1234-1234-1234-cabcdefg0011","children":["eabcdef0-1234-1234-1234-cabcdefghij9"],"category":"critical","currentGroup":"eabc1234-1234-1234-1234-cabcdefg0011","groupCategory":"critical","groupScore":"72.9174234","groupPreviousGroups":null,"activityId":"abcd1234","groupingIds":["abcdef12"],"groupByActivity":false,"userTriggered":false,"externalTriggered":false,"aiaScore":98,"summary":"The device 175.16.199.1 was observed making unusual internal SSH connections to a wide range of devices.\n\nThough this behaviour could be the result of legitimate remote access or administration, it could also be a sign of attempted lateral movement by a compromised machine.\n\nConsequently, if this activity was not expected, the security team may wish to investigate further.","periods":[{"start":1627985298683,"end":1628000141220}],"breachDevices":[{"identifier":null,"hostname":null,"ip":"81.2.69.144","mac":null,"subnet":"VPN","did":10,"sid":12}],"relatedBreaches":[{"modelName":"Unusual Activity / Unusual Activity from Re-Activated Device","pbid":1234,"threatScore":37,"timestamp":1627997157000}],"details":[[{"header":"Breaching Device","contents":[{"key":null,"type":"device","values":[{"identifier":null,"hostname":null,"ip":"175.16.199.1","mac":null,"subnet":"VPN","did":10,"sid":12}]}]}],[{"header":"SSH Activity","contents":[{"key":"Time","type":"timestampRange","values":[{"start":1627985298683,"end":1628000141220}]},{"key":"Number of unique IPs","type":"integer","values":[16]},{"key":"Targeted IP ranges include","type":"device","values":[{"identifier":null,"hostname":null,"ip":"81.2.69.192","mac":null,"subnet":null,"did":null,"sid":null},{"identifier":null,"hostname":null,"ip":"175.16.199.1","mac":null,"subnet":null,"did":null,"sid":null},{"identifier":null,"hostname":null,"ip":"175.16.199.3","mac":null,"subnet":null,"did":null,"sid":null}]},{"key":"Destination port","type":"integer","values":[22]},{"key":"Connection count","type":"integer","values":[40]},{"key":"Percentage successful","type":"percentage","values":[100]}]}]]} +<165>1 2022-02-19T11:50:10+00:00 example.cloud.darktrace.com darktrace - - - {"summariser":"ScanSummary","acknowledged":false,"pinned":false,"createdAt":1657749437781,"attackPhases":[4],"title":"Port Scanning","id":"eabcdef0-1234-1234-1234-cabcdefghij9","incidentEventUrl":"https://www.example.com/#aiaincidentevent/eabcdef0-1234-1234-1234-cabcdefghij9","children":["eabcdef0-1234-1234-1234-cabcdefghij9"],"category":"suspicious","currentGroup":"eab12345-1234-1234-1234-cabcdef12345","groupCategory":"suspicious","groupScore":6.857722547303857,"groupPreviousGroups":[],"activityId":"abcdefee","groupingIds":["ab123456"],"groupByActivity":false,"userTriggered":false,"externalTriggered":false,"aiaScore":64,"summary":"The device linux.local was observed making an unusually large number of internal connection attempts to multiple devices, suggesting scanning activity.\n\nNetwork scanning can be used during reconnaissance to gather information about internal devices, such as their list of open ports, and is thus a possible indicator of preparation for malicious or unauthorised internal activity.\n\nIf the activity from the device was not expected, it is recommended that the security team investigate it further to determine whether it was part of legitimate network activity.","periods":[{"start":1657747020967,"end":1657749405342}],"breachDevices":[{"identifier":"linux.local","hostname":"linux.local","ip":"175.16.199.1/24","mac":null,"subnet":null,"did":7,"sid":1}],"relatedBreaches":[{"modelName":"Device / Attack and Recon Tools","pbid":6,"threatScore":88,"timestamp":1657748808000}],"details":[[{"header":"Overview of Scan","contents":[{"key":"Time","type":"timestampRange","values":[{"start":1657747020967,"end":1657749405342}]},{"key":"Source device","type":"device","values":[{"identifier":"linux.local","hostname":"linux.local","ip":"175.16.199.3/28","mac":null,"subnet":null,"did":7,"sid":1}]},{"key":"Scanned IPs","type":"device","values":[{"identifier":null,"hostname":null,"ip":"81.2.69.144","mac":null,"subnet":null,"did":null,"sid":null},{"identifier":null,"hostname":null,"ip":"81.2.69.192","mac":null,"subnet":null,"did":null,"sid":null}]},{"key":"Username observed prior to activity","type":"string","values":["user1"]},{"key":"Source of username","type":"string","values":["User login"]},{"key":"Time observed","type":"timestamp","values":[1657739323000]},{"key":"Event UID","type":"string","values":["CABCDabcdABCDabcd000"]}]}],[{"header":"TCP Scanning Activity","contents":[{"key":"Total connections","type":"integer","values":[4537]},{"key":"Total ports scanned","type":"integer","values":[996]},{"key":"Port range","type":"integerRange","values":[{"start":"1","end":"65389"}]},{"key":"Key ports","type":"integer","values":[21,22,23,80,389,443,1433,3128,3306,4444,4899,8080]}]}],[{"header":"UDP Scanning Activity","contents":[{"key":"Total connections","type":"integer","values":[116]},{"key":"Port","type":"integer","values":[137]}]}]]} diff --git a/packages/darktrace/_dev/deploy/docker/sample_logs/model_breach_alert.log b/packages/darktrace/_dev/deploy/docker/sample_logs/model_breach_alert.log new file mode 100644 index 00000000000..d5d96f42422 --- /dev/null +++ b/packages/darktrace/_dev/deploy/docker/sample_logs/model_breach_alert.log @@ -0,0 +1,4 @@ +<165>1 2022-02-12T19:10:23+00:00 example.cloud.darktrace.com darktrace - - - {"model":{"name":"Device::Attack and Recon Tools","pid":135,"phid":1199,"uuid":"7abcdefg-1234-1234-1234-5abcdefg12","logic":{"data":[{"cid":2311,"weight":1},{"cid":2312,"weight":1},{"cid":2315,"weight":1},{"cid":2316,"weight":1},{"cid":2314,"weight":1},{"cid":2310,"weight":1},{"cid":2313,"weight":1}],"targetScore":1,"type":"weightedComponentList","version":1},"throttle":604800,"sharedEndpoints":false,"actions":{"alert":true,"antigena":{},"breach":true,"model":true,"setPriority":false,"setTag":false,"setType":false},"tags":["AP: Internal Recon","OT Engineer"],"interval":3600,"delay":0,"sequenced":false,"active":true,"modified":"2022-07-11 11:47:51","activeTimes":{"devices":{},"tags":{},"type":"exclusions","version":2},"autoUpdatable":true,"autoUpdate":true,"autoSuppress":true,"description":"A device is using common penetration testing tools.\\n\\nAction: Review the device to see if it a security device, these can be tagged as such to exclude them from future breaches. Activity from non security devices merit further investigation into what else the device is doing and could be a significant risk within the network.","behaviour":"decreasing","created":{"by":"System"},"edited":{"by":"System"},"version":77,"priority":4,"category":"Suspicious","compliance":false},"device":{"did":7,"ip":"81.2.69.192","ips":[{"ip":"175.16.199.2","timems":1657746000000,"time":"2022-07-13 21:00:00","sid":1}],"sid":1,"hostname":"localhost.local","firstSeen":1657544891000,"lastSeen":1657748638000,"typename":"desktop","typelabel":"Desktop","credentials":["dummy"],"tags":[{"tid":66,"expiry":0,"thid":66,"name":"Domain Authenticated","restricted":false,"data":{"auto":false,"color":183,"description":""},"isReferenced":true},{"tid":29,"expiry":0,"thid":29,"name":"Linux","restricted":false,"data":{"auto":false,"color":168,"description":"","visibility":"Public"},"isReferenced":true},{"tid":45,"expiry":0,"thid":45,"name":"New Device","restricted":false,"data":{"auto":false,"color":130,"description":"","visibility":"Public"},"isReferenced":true}]},"triggeredComponents":[{"time":1657748807000,"cbid":6,"cid":2311,"chid":2676,"size":1,"threshold":0,"interval":300,"logic":{"data":{"left":{"left":"A","operator":"AND","right":{"left":"H","operator":"AND","right":{"left":"J","operator":"AND","right":"N"}}},"operator":"OR","right":{"left":{"left":"B","operator":"AND","right":{"left":"H","operator":"AND","right":{"left":"J","operator":"AND","right":{"left":"N","operator":"AND","right":"O"}}}},"operator":"OR","right":{"left":{"left":"C","operator":"AND","right":{"left":"H","operator":"AND","right":{"left":"J","operator":"AND","right":"N"}}},"operator":"OR","right":{"left":{"left":"E","operator":"AND","right":{"left":"H","operator":"AND","right":{"left":"J","operator":"AND","right":"N"}}},"operator":"OR","right":{"left":{"left":"F","operator":"AND","right":{"left":"H","operator":"AND","right":{"left":"J","operator":"AND","right":"N"}}},"operator":"OR","right":{"left":{"left":"G","operator":"AND","right":{"left":"H","operator":"AND","right":{"left":"J","operator":"AND","right":"N"}}},"operator":"OR","right":{"left":{"left":"H","operator":"AND","right":{"left":"I","operator":"AND","right":{"left":"J","operator":"AND","right":"N"}}},"operator":"OR","right":{"left":{"left":"H","operator":"AND","right":{"left":"J","operator":"AND","right":{"left":"K","operator":"AND","right":"N"}}},"operator":"OR","right":{"left":{"left":"H","operator":"AND","right":{"left":"J","operator":"AND","right":{"left":"L","operator":"AND","right":"N"}}},"operator":"OR","right":{"left":{"left":"H","operator":"AND","right":{"left":"J","operator":"AND","right":{"left":"M","operator":"AND","right":"N"}}},"operator":"OR","right":{"left":"H","operator":"AND","right":{"left":"J","operator":"AND","right":{"left":"N","operator":"AND","right":"P"}}}}}}}}}}}}},"version":"v0.1"},"metric":{"mlid":16,"name":"connections","label":"Connections"},"triggeredFilters":[{"cfid":26781,"id":"H","filterType":"Direction","arguments":{"value":"out"},"comparatorType":"is","trigger":{"value":"out"}},{"cfid":26783,"id":"J","filterType":"Tagged internal source","arguments":{"value":12},"comparatorType":"does not have tag","trigger":{"value":"12","tag":{"tid":12,"expiry":0,"thid":12,"name":"Security Device","restricted":false,"data":{"auto":false,"color":55,"description":"","visibility":"Public"},"isReferenced":true}}},{"cfid":26787,"id":"N","filterType":"Tagged internal destination","arguments":{"value":12},"comparatorType":"does not have tag","trigger":{"value":"12","tag":{"tid":12,"expiry":0,"thid":12,"name":"Security Device","restricted":false,"data":{"auto":false,"color":55,"description":"","visibility":"Public"},"isReferenced":true}}},{"cfid":26788,"id":"O","filterType":"User agent","arguments":{"value":"examples"},"comparatorType":"does not match regular expression","trigger":{"value":""}},{"cfid":26789,"id":"P","filterType":"URI","arguments":{"value":"examples"},"comparatorType":"matches regular expression","trigger":{"value":""}},{"cfid":26790,"id":"d1","filterType":"Proxied connection","arguments":{"value":"true"},"comparatorType":"display","trigger":{"value":"false"}},{"cfid":26791,"id":"d10","filterType":"HTTP response code","arguments":{},"comparatorType":"display","trigger":{"value":"0"}},{"cfid":26792,"id":"d2","filterType":"HTTP referrer","arguments":{},"comparatorType":"display","trigger":{"value":""}},{"cfid":26793,"id":"d3","filterType":"HTTP method","arguments":{},"comparatorType":"display","trigger":{"value":"GET"}},{"cfid":26794,"id":"d4","filterType":"HTTP X-Forwarded-For","arguments":{},"comparatorType":"display","trigger":{"value":""}},{"cfid":26795,"id":"d5","filterType":"URI","arguments":{},"comparatorType":"display","trigger":{"value":""}},{"cfid":26796,"id":"d6","filterType":"User agent","arguments":{},"comparatorType":"display","trigger":{"value":""}},{"cfid":26797,"id":"d7","filterType":"Destination IP","arguments":{},"comparatorType":"display","trigger":{"value":"81.2.69.144"}},{"cfid":26798,"id":"d8","filterType":"Internal destination device name","arguments":{},"comparatorType":"display","trigger":{"value":"localhost.local"}},{"cfid":26799,"id":"d9","filterType":"Connection hostname","arguments":{},"comparatorType":"display","trigger":{"value":""}}]}],"breachUrl":"https://example.com/#modelbreach/6","pbid":6,"score":0.871,"commentCount":0,"creationTime":1657748815000,"time":1657748808000,"mitreTechniques":[{"technique":"Hardware Additions Mitigation","techniqueID":"T1200"}]} +<165>1 2022-03-09T22:11:20+00:00 example.cloud.darktrace.com darktrace - - - {"commentCount":0,"pbid":1,"time":1657544648000,"creationTime":1657544659000,"aianalystData":[{"uuid":"1234abcd-1234-1234-1234-123456abcdef","related":[1],"summariser":"BeaconSummary"}],"model":{"name":"Compromise::Beaconing Activity To External Rare","pid":156,"phid":1072,"uuid":"1234abcd-1234-1234-1234-123456abcdef","logic":{"data":[{"cid":2026,"weight":1},{"cid":2024,"weight":1},{"cid":2025,"weight":-100}],"targetScore":1,"type":"weightedComponentList","version":1},"throttle":10800,"sharedEndpoints":false,"actions":{"alert":true,"antigena":{},"breach":true,"model":true,"setPriority":false,"setTag":false,"setType":false},"tags":["AP: C2 Comms"],"interval":10800,"delay":0,"sequenced":false,"active":true,"modified":"2022-07-11 11:47:37","activeTimes":{"devices":{},"tags":{},"type":"exclusions","version":2},"autoUpdatable":true,"autoUpdate":true,"autoSuppress":true,"description":"A device has been repeatedly connecting to a rare external location with a beacon score. A beacon score is added when Darktrace identifies that a device is regularly communicating with an endpoint, for example, if a device connects to a rare external endpoint every 12 minutes this would get a beacon score. This model is designed to identify beaconing at a lower threshold and be protocol agnostic.\\n\\nAction: Review the external domains and IPs being connected to to see if they are legitimate and would be expected for business purposes.","behaviour":"incdec1","created":{"by":"System"},"edited":{"by":"System"},"version":23,"priority":2,"category":"Informational","compliance":false},"triggeredComponents":[{"time":1657544648000,"cbid":1,"cid":2026,"chid":2113,"size":11,"threshold":10,"interval":3600,"logic":{"data":{"left":{"left":"A","operator":"AND","right":{"left":"AA","operator":"AND","right":{"left":"AC","operator":"AND","right":{"left":"AD","operator":"AND","right":{"left":"AF","operator":"AND","right":{"left":"AG","operator":"AND","right":{"left":"AH","operator":"AND","right":{"left":"B","operator":"AND","right":{"left":"C","operator":"AND","right":{"left":"D","operator":"AND","right":{"left":"E","operator":"AND","right":{"left":"H","operator":"AND","right":{"left":"I","operator":"AND","right":{"left":"J","operator":"AND","right":{"left":"K","operator":"AND","right":{"left":"L","operator":"AND","right":{"left":"M","operator":"AND","right":{"left":"N","operator":"AND","right":{"left":"O","operator":"AND","right":{"left":"P","operator":"AND","right":{"left":"S","operator":"AND","right":{"left":"U","operator":"AND","right":{"left":"V","operator":"AND","right":{"left":"X","operator":"AND","right":{"left":"Y","operator":"AND","right":"Z"}}}}}}}}}}}}}}}}}}}}}}}}},"operator":"OR","right":{"left":"A","operator":"AND","right":{"left":"AA","operator":"AND","right":{"left":"AB","operator":"AND","right":{"left":"AE","operator":"AND","right":{"left":"AF","operator":"AND","right":{"left":"AG","operator":"AND","right":{"left":"AH","operator":"AND","right":{"left":"C","operator":"AND","right":{"left":"D","operator":"AND","right":{"left":"E","operator":"AND","right":{"left":"H","operator":"AND","right":{"left":"I","operator":"AND","right":{"left":"J","operator":"AND","right":{"left":"K","operator":"AND","right":{"left":"L","operator":"AND","right":{"left":"M","operator":"AND","right":{"left":"N","operator":"AND","right":{"left":"O","operator":"AND","right":{"left":"P","operator":"AND","right":{"left":"S","operator":"AND","right":{"left":"U","operator":"AND","right":{"left":"V","operator":"AND","right":{"left":"X","operator":"AND","right":{"left":"Y","operator":"AND","right":"Z"}}}}}}}}}}}}}}}}}}}}}}}}},"version":"v0.1"},"metric":{"mlid":1,"name":"externalconnections","label":"External Connections"},"triggeredFilters":[{"cfid":23426,"id":"A","filterType":"Beaconing score","arguments":{"value":60},"comparatorType":">","trigger":{"value":"100"}},{"cfid":23427,"id":"AA","filterType":"Individual size up","arguments":{"value":0},"comparatorType":">","trigger":{"value":"4382"}},{"cfid":23428,"id":"AB","filterType":"Rare domain","arguments":{"value":95},"comparatorType":">","trigger":{"value":"100"}},{"cfid":23430,"id":"AD","filterType":"Age of destination","arguments":{"value":1209600},"comparatorType":"<","trigger":{"value":"558"}},{"cfid":23431,"id":"AE","filterType":"Age of external hostname","arguments":{"value":1209600},"comparatorType":"<","trigger":{"value":"558"}},{"cfid":23432,"id":"AF","filterType":"Connection hostname","arguments":{"value":"examples"},"comparatorType":"does not match regular expression","trigger":{"value":"example.com"}},{"cfid":23433,"id":"AG","filterType":"ASN","arguments":{"value":"examples"},"comparatorType":"does not match regular expression","trigger":{"value":"AS12345 LOCAL-02"}},{"cfid":23434,"id":"AH","filterType":"JA3 hash","arguments":{"value":"5d41402abc4b2a76b9719d911017c592"},"comparatorType":"does not match","trigger":{"value":"5d41402abc4b2a76b9719d911017c592"}},{"cfid":23435,"id":"B","filterType":"Rare external IP","arguments":{"value":95},"comparatorType":">","trigger":{"value":"100"}},{"cfid":23436,"id":"C","filterType":"Application protocol","arguments":{"value":"1003"},"comparatorType":"is not","trigger":{"value":"1004"}},{"cfid":23437,"id":"D","filterType":"Destination port","arguments":{"value":53},"comparatorType":"!=","trigger":{"value":"443"}},{"cfid":23438,"id":"E","filterType":"Direction","arguments":{"value":"out"},"comparatorType":"is","trigger":{"value":"out"}},{"cfid":23439,"id":"H","filterType":"Destination port","arguments":{"value":137},"comparatorType":"!=","trigger":{"value":"443"}},{"cfid":23440,"id":"I","filterType":"Destination port","arguments":{"value":161},"comparatorType":"!=","trigger":{"value":"443"}},{"cfid":23441,"id":"J","filterType":"Protocol","arguments":{"value":"6"},"comparatorType":"is","trigger":{"value":"6"}},{"cfid":23442,"id":"K","filterType":"ASN","arguments":{"value":"Company"},"comparatorType":"does not contain","trigger":{"value":"AS12345 LOCAL-02"}},{"cfid":23443,"id":"L","filterType":"ASN","arguments":{"value":"Company"},"comparatorType":"does not contain","trigger":{"value":"AS12345 LOCAL-02"}},{"cfid":23444,"id":"M","filterType":"Internal source device type","arguments":{"value":"13"},"comparatorType":"is not","trigger":{"value":"6"}},{"cfid":23445,"id":"N","filterType":"Internal source device type","arguments":{"value":"5"},"comparatorType":"is not","trigger":{"value":"6"}},{"cfid":23446,"id":"O","filterType":"Internal source device type","arguments":{"value":"9"},"comparatorType":"is not","trigger":{"value":"6"}},{"cfid":23447,"id":"P","filterType":"Internal source device type","arguments":{"value":"12"},"comparatorType":"is not","trigger":{"value":"6"}},{"cfid":23448,"id":"S","filterType":"Internal source device type","arguments":{"value":"30"},"comparatorType":"is not","trigger":{"value":"6"}},{"cfid":23449,"id":"U","filterType":"Internal source device type","arguments":{"value":"4"},"comparatorType":"is not","trigger":{"value":"6"}},{"cfid":23450,"id":"V","filterType":"Internal source device type","arguments":{"value":"3"},"comparatorType":"is not","trigger":{"value":"6"}},{"cfid":23451,"id":"X","filterType":"Trusted hostname","arguments":{"value":"false"},"comparatorType":"is","trigger":{"value":"false"}},{"cfid":23452,"id":"Y","filterType":"Tagged internal source","arguments":{"value":26},"comparatorType":"does not have tag","trigger":{"value":"26","tag":{"tid":26,"expiry":0,"thid":26,"name":"No Device Tracking","restricted":false,"data":{"auto":false,"color":5,"description":"","visibility":"Public"},"isReferenced":true}}},{"cfid":23453,"id":"Z","filterType":"Individual size down","arguments":{"value":0},"comparatorType":">","trigger":{"value":"5862"}},{"cfid":23454,"id":"d1","filterType":"JA3 hash","arguments":{},"comparatorType":"display","trigger":{"value":"5d41402abc4b2a76b9719d911017c592"}},{"cfid":23455,"id":"d2","filterType":"ASN","arguments":{},"comparatorType":"display","trigger":{"value":"AS12345 LOCAL-02"}},{"cfid":23456,"id":"d3","filterType":"Destination IP","arguments":{},"comparatorType":"display","trigger":{"value":"81.2.69.192"}},{"cfid":23457,"id":"d4","filterType":"Connection hostname","arguments":{},"comparatorType":"display","trigger":{"value":"example.com"}}]}],"score":0.674,"device":{"did":3,"ip":"81.2.69.142","sid":1,"firstSeen":1657544089000,"lastSeen":1657544418000,"typename":"desktop","typelabel":"Desktop"}} +<165>1 2022-03-06T10:04:57+00:00 example.cloud.darktrace.com darktrace - - - {"model":{"name":"System::System","pid":802,"phid":803,"uuid":"7abcdefh-1234-1234-1234-5abababab","logic":{"data":[1594],"type":"componentList","version":1},"throttle":10,"sharedEndpoints":false,"actions":{"alert":true,"antigena":{},"breach":true,"model":true,"setPriority":false,"setTag":false,"setType":false},"tags":[],"interval":0,"delay":0,"sequenced":true,"active":true,"modified":"2022-07-11 11:41:08","activeTimes":{"devices":{},"tags":{},"type":"exclusions","version":2},"autoUpdatable":true,"autoUpdate":true,"autoSuppress":true,"description":"An issue with the system has been detected. This system alert is generated for system information that may merit further investigation. This may be due to things like probes failing to connect.\\n\\nAction: Review the system message. Use the status page to see additional system information that may help with diagnostics.","behaviour":"decreasing","defeats":[],"created":{"by":"System"},"edited":{"by":"Nobody"},"version":16,"priority":3,"category":"Informational","compliance":false},"device":{"did":-1},"triggeredComponents":[{"time":1657678365000,"cbid":5,"cid":1594,"chid":1594,"size":1,"threshold":0,"interval":3600,"logic":{"data":{"left":{"left":"A","operator":"AND","right":"B"},"operator":"OR","right":{"left":{"left":"A","operator":"AND","right":"C"},"operator":"OR","right":{"left":{"left":"A","operator":"AND","right":"D"},"operator":"OR","right":{"left":{"left":"A","operator":"AND","right":"E"},"operator":"OR","right":{"left":"A","operator":"AND","right":"F"}}}}},"version":"v0.1"},"metric":{"mlid":207,"name":"dtsystem","label":"System"},"triggeredFilters":[{"cfid":18136,"id":"A","filterType":"Event details","arguments":{"value":"analyze credential ignore list"},"comparatorType":"does not contain","trigger":{"value":"Probe 175.16.199.1 last contact was 18 minutes ago"}},{"cfid":18137,"id":"B","filterType":"System message","arguments":{"value":"Probe error"},"comparatorType":"is","trigger":{"value":"Probe error"}},{"cfid":18142,"id":"d1","filterType":"Event details","arguments":{},"comparatorType":"display","trigger":{"value":"Probe 175.16.199.1 last contact was 18 minutes ago"}},{"cfid":18143,"id":"d2","filterType":"System message","arguments":{},"comparatorType":"display","trigger":{"value":"Probe error"}}]}],"breachUrl":"https://example.com/#modelbreach/5","pbid":5,"score":0.674,"commentCount":0,"creationTime":1657678365000,"time":1657678366000,"mitreTechniques":[]} +<165>Aug 5 03:34:46 example.cloud.darktrace.com darktrace {"model":{"description":"Test model used for testing alerting configuration.","created":{"by":"System"},"edited":{"by":"Nobody"},"name":"Unrestricted Test Model","priority":5},"device":{"ip":"175.16.199.1","hostname":"test-device.example.com","macaddress":"00:00:5e:00:53:00","vendor":"Test Vendor","label":"Test Device"},"triggeredComponents":[{"metric":{"label":"Test Metric"},"triggeredFilters":[{"comparatorType":"display","filterType":"Test Metric Filter","trigger":{"value":"Test filter value"}}]}],"breachUrl":"","pbid":123,"score":1,"creationTime":1659692086317,"time":1659692086317,"mitreTechniques":[]} diff --git a/packages/darktrace/_dev/deploy/docker/sample_logs/system_status_alert.log b/packages/darktrace/_dev/deploy/docker/sample_logs/system_status_alert.log new file mode 100644 index 00000000000..9cbdb646d8b --- /dev/null +++ b/packages/darktrace/_dev/deploy/docker/sample_logs/system_status_alert.log @@ -0,0 +1,2 @@ +<165>1 2022-03-10T10:11:10+00:00 example.cloud.darktrace.com darktrace - - - {"last_updated":1618760651,"uuid":"abcdabcd-1234-1234-1234-3abababcdcd3","priority":43,"priority_level":"medium","hostname":"example-vsensor","ip_address":"175.16.199.1","message":"There have been no Advanced Search hits for this instance seen since Sun 18 April 2021 13:20:23 (UTC). If this is not expected behaviour, please open a ticket using the following link or get in touch with your Cyber Technology Specialist. https://example.com/test","name":"advanced_search","acknowledge_timeout":null,"alert_name":"Advanced Search","child_id":1,"last_updated_status":1618760651,"status":"active"} +<165>1 2022-02-19T04:02:50+00:00 example.cloud.darktrace.com darktrace - - - {"hostname":"local-abc","ip_address":"175.16.199.1","child_id":null,"name":"probe_down-0","priority":98,"priority_level":"critical","alert_name":"Probe Down","status":"Active","message":"The probe 1/175.16.199.1 has lost connection to the Master instance. Please ensure HTTPS bidirectional connectivity exists between the Master and the Probe.\n\nIf you have any issues, please open a ticket using the following link. https://example.com/test","last_updated":1658110810.556194,"last_updated_status":1658110810.556194,"acknowledge_timeout":null,"uuid":"abcd1234-1234-1234-1234-3abababcdcd3","url":"https://example.com/test?value=abcd1234-1234-1234-1234-3abababcdcd3"} diff --git a/packages/darktrace/changelog.yml b/packages/darktrace/changelog.yml new file mode 100644 index 00000000000..f66c5d70aad --- /dev/null +++ b/packages/darktrace/changelog.yml @@ -0,0 +1,6 @@ +# newer versions go on top +- version: '0.1.0' + changes: + - description: Initial Release. + type: enhancement + link: https://github.com/elastic/integrations/pull/4001 diff --git a/packages/darktrace/data_stream/ai_analyst_alert/_dev/test/pipeline/test-ai-analyst-alert.log b/packages/darktrace/data_stream/ai_analyst_alert/_dev/test/pipeline/test-ai-analyst-alert.log new file mode 100644 index 00000000000..72c8836b507 --- /dev/null +++ b/packages/darktrace/data_stream/ai_analyst_alert/_dev/test/pipeline/test-ai-analyst-alert.log @@ -0,0 +1,2 @@ +{"summariser":"AdminConnSummary","acknowledged":false,"pinned":true,"createdAt":1628002089240,"attackPhases":[5],"title":"Extensive Unusual SSH Connections","id":"eabc0011-1234-1234-1234-cabcdefg0011","children":["eabcdef0-1234-1234-1234-cabcdefghij9"],"category":"critical","currentGroup":"eabc1234-1234-1234-1234-cabcdefg0011","groupCategory":"critical","groupScore":"72.9174234","groupPreviousGroups":null,"activityId":"abcd1234","groupingIds":["abcdef12"],"groupByActivity":false,"userTriggered":false,"externalTriggered":false,"aiaScore":98,"summary":"The device 175.16.199.1 was observed making unusual internal SSH connections to a wide range of devices.\n\nThough this behaviour could be the result of legitimate remote access or administration, it could also be a sign of attempted lateral movement by a compromised machine.\n\nConsequently, if this activity was not expected, the security team may wish to investigate further.","periods":[{"start":1627985298683,"end":1628000141220}],"breachDevices":[{"identifier":null,"hostname":"linux.local","ip":"81.2.69.144","mac":null,"subnet":"VPN","did":10,"sid":12}],"relatedBreaches":[{"modelName":"Unusual Activity / Unusual Activity from Re-Activated Device","pbid":1234,"threatScore":37,"timestamp":1627997157000}],"details":[[{"header":"Breaching Device","contents":[{"key":null,"type":"device","values":[{"identifier":null,"hostname":null,"ip":"175.16.199.1","mac":null,"subnet":"VPN","did":10,"sid":12}]}]}],[{"header":"SSH Activity","contents":[{"key":"Time","type":"timestampRange","values":[{"start":1627985298683,"end":1628000141220}]},{"key":"Number of unique IPs","type":"integer","values":[16]},{"key":"Targeted IP ranges include","type":"device","values":[{"identifier":null,"hostname":null,"ip":"81.2.69.192","mac":null,"subnet":null,"did":null,"sid":null},{"identifier":null,"hostname":null,"ip":"175.16.199.1","mac":null,"subnet":null,"did":null,"sid":null},{"identifier":null,"hostname":null,"ip":"175.16.199.3","mac":null,"subnet":null,"did":null,"sid":null}]},{"key":"Destination port","type":"integer","values":[22]},{"key":"Connection count","type":"integer","values":[40]},{"key":"Percentage successful","type":"percentage","values":[100]}]}]]} +{"summariser":"ScanSummary","acknowledged":false,"pinned":false,"createdAt":1657749437781,"attackPhases":[4],"title":"Port Scanning","id":"eabcdef0-1234-1234-1234-cabcdefghij9","incidentEventUrl":"https://www.example.com/#aiaincidentevent/eabcdef0-1234-1234-1234-cabcdefghij9","children":["eabcdef0-1234-1234-1234-cabcdefghij9"],"category":"suspicious","currentGroup":"eab12345-1234-1234-1234-cabcdef12345","groupCategory":"suspicious","groupScore":6.857722547303857,"groupPreviousGroups":[],"activityId":"abcdefee","groupingIds":["ab123456"],"groupByActivity":false,"userTriggered":false,"externalTriggered":false,"aiaScore":64,"summary":"The device linux.local was observed making an unusually large number of internal connection attempts to multiple devices, suggesting scanning activity.\n\nNetwork scanning can be used during reconnaissance to gather information about internal devices, such as their list of open ports, and is thus a possible indicator of preparation for malicious or unauthorised internal activity.\n\nIf the activity from the device was not expected, it is recommended that the security team investigate it further to determine whether it was part of legitimate network activity.","periods":[{"start":1657747020967,"end":1657749405342}],"breachDevices":[{"identifier":"linux.local","hostname":"linux.local","ip":"175.16.199.1/24","mac":null,"subnet":null,"did":7,"sid":1}],"relatedBreaches":[{"modelName":"Device / Attack and Recon Tools","pbid":6,"threatScore":88,"timestamp":1657748808000}],"details":[[{"header":"Overview of Scan","contents":[{"key":"Time","type":"timestampRange","values":[{"start":1657747020967,"end":1657749405342}]},{"key":"Source device","type":"device","values":[{"identifier":"linux.local","hostname":"linux.local","ip":"175.16.199.3/28","mac":null,"subnet":null,"did":7,"sid":1}]},{"key":"Scanned IPs","type":"device","values":[{"identifier":null,"hostname":null,"ip":"81.2.69.144","mac":null,"subnet":null,"did":null,"sid":null},{"identifier":null,"hostname":null,"ip":"81.2.69.192","mac":null,"subnet":null,"did":null,"sid":null}]},{"key":"Username observed prior to activity","type":"string","values":["user1"]},{"key":"Source of username","type":"string","values":["User login"]},{"key":"Time observed","type":"timestamp","values":[1657739323000]},{"key":"Event UID","type":"string","values":["CABCDabcdABCDabcd000"]}]}],[{"header":"TCP Scanning Activity","contents":[{"key":"Total connections","type":"integer","values":[4537]},{"key":"Total ports scanned","type":"integer","values":[996]},{"key":"Port range","type":"integerRange","values":[{"start":"1","end":"65389"}]},{"key":"Key ports","type":"integer","values":[21,22,23,80,389,443,1433,3128,3306,4444,4899,8080]}]}],[{"header":"UDP Scanning Activity","contents":[{"key":"Total connections","type":"integer","values":[116]},{"key":"Port","type":"integer","values":[137]}]}]]} diff --git a/packages/darktrace/data_stream/ai_analyst_alert/_dev/test/pipeline/test-ai-analyst-alert.log-expected.json b/packages/darktrace/data_stream/ai_analyst_alert/_dev/test/pipeline/test-ai-analyst-alert.log-expected.json new file mode 100644 index 00000000000..c2a4fad58b2 --- /dev/null +++ b/packages/darktrace/data_stream/ai_analyst_alert/_dev/test/pipeline/test-ai-analyst-alert.log-expected.json @@ -0,0 +1,482 @@ +{ + "expected": [ + { + "@timestamp": "2021-08-03T14:48:09.240Z", + "darktrace": { + "ai_analyst_alert": { + "activity_id": "abcd1234", + "aia_score": 98.0, + "attack_phases": [ + 5 + ], + "breach_devices": [ + { + "did": 10, + "hostname": "linux.local", + "ip": "81.2.69.144", + "sid": 12, + "subnet": "VPN" + } + ], + "category": "critical", + "children": [ + "eabcdef0-1234-1234-1234-cabcdefghij9" + ], + "created_at": "2021-08-03T14:48:09.240Z", + "current_group": "eabc1234-1234-1234-1234-cabcdefg0011", + "details": [ + [ + { + "contents": [ + { + "type": "device", + "values": [ + { + "did": 10, + "ip": "175.16.199.1", + "sid": 12, + "subnet": "VPN" + } + ] + } + ], + "header": "Breaching Device" + } + ], + [ + { + "contents": [ + { + "key": "Time", + "type": "timestampRange", + "values": [ + { + "end": 1628000141220, + "start": 1627985298683 + } + ] + }, + { + "key": "Number of unique IPs", + "type": "integer", + "values": [ + 16 + ] + }, + { + "key": "Targeted IP ranges include", + "type": "device", + "values": [ + { + "ip": "81.2.69.192" + }, + { + "ip": "175.16.199.1" + }, + { + "ip": "175.16.199.3" + } + ] + }, + { + "key": "Destination port", + "type": "integer", + "values": [ + 22 + ] + }, + { + "key": "Connection count", + "type": "integer", + "values": [ + 40 + ] + }, + { + "key": "Percentage successful", + "type": "percentage", + "values": [ + 100 + ] + } + ], + "header": "SSH Activity" + } + ] + ], + "group_by_activity": false, + "group_category": "critical", + "group_score": 72.9174234, + "grouping_ids": [ + "abcdef12" + ], + "id": "eabc0011-1234-1234-1234-cabcdefg0011", + "is_acknowledged": false, + "is_external_triggered": false, + "is_pinned": true, + "is_user_triggered": false, + "periods": [ + { + "end": "2021-08-03T14:15:41.220Z", + "start": "2021-08-03T10:08:18.683Z" + } + ], + "related_breaches": [ + { + "model_name": "Unusual Activity / Unusual Activity from Re-Activated Device", + "pbid": 1234, + "threat_score": 37, + "timestamp": "2021-08-03T13:25:57.000Z" + } + ], + "summariser": "AdminConnSummary", + "summary": "The device 175.16.199.1 was observed making unusual internal SSH connections to a wide range of devices.\n\nThough this behaviour could be the result of legitimate remote access or administration, it could also be a sign of attempted lateral movement by a compromised machine.\n\nConsequently, if this activity was not expected, the security team may wish to investigate further.", + "title": "Extensive Unusual SSH Connections" + } + }, + "ecs": { + "version": "8.4.0" + }, + "event": { + "category": [ + "threat" + ], + "duration": [ + 14842537000000 + ], + "end": [ + "2021-08-03T14:15:41.220Z" + ], + "id": "eabc0011-1234-1234-1234-cabcdefg0011", + "kind": "alert", + "original": "{\"summariser\":\"AdminConnSummary\",\"acknowledged\":false,\"pinned\":true,\"createdAt\":1628002089240,\"attackPhases\":[5],\"title\":\"Extensive Unusual SSH Connections\",\"id\":\"eabc0011-1234-1234-1234-cabcdefg0011\",\"children\":[\"eabcdef0-1234-1234-1234-cabcdefghij9\"],\"category\":\"critical\",\"currentGroup\":\"eabc1234-1234-1234-1234-cabcdefg0011\",\"groupCategory\":\"critical\",\"groupScore\":\"72.9174234\",\"groupPreviousGroups\":null,\"activityId\":\"abcd1234\",\"groupingIds\":[\"abcdef12\"],\"groupByActivity\":false,\"userTriggered\":false,\"externalTriggered\":false,\"aiaScore\":98,\"summary\":\"The device 175.16.199.1 was observed making unusual internal SSH connections to a wide range of devices.\\n\\nThough this behaviour could be the result of legitimate remote access or administration, it could also be a sign of attempted lateral movement by a compromised machine.\\n\\nConsequently, if this activity was not expected, the security team may wish to investigate further.\",\"periods\":[{\"start\":1627985298683,\"end\":1628000141220}],\"breachDevices\":[{\"identifier\":null,\"hostname\":\"linux.local\",\"ip\":\"81.2.69.144\",\"mac\":null,\"subnet\":\"VPN\",\"did\":10,\"sid\":12}],\"relatedBreaches\":[{\"modelName\":\"Unusual Activity / Unusual Activity from Re-Activated Device\",\"pbid\":1234,\"threatScore\":37,\"timestamp\":1627997157000}],\"details\":[[{\"header\":\"Breaching Device\",\"contents\":[{\"key\":null,\"type\":\"device\",\"values\":[{\"identifier\":null,\"hostname\":null,\"ip\":\"175.16.199.1\",\"mac\":null,\"subnet\":\"VPN\",\"did\":10,\"sid\":12}]}]}],[{\"header\":\"SSH Activity\",\"contents\":[{\"key\":\"Time\",\"type\":\"timestampRange\",\"values\":[{\"start\":1627985298683,\"end\":1628000141220}]},{\"key\":\"Number of unique IPs\",\"type\":\"integer\",\"values\":[16]},{\"key\":\"Targeted IP ranges include\",\"type\":\"device\",\"values\":[{\"identifier\":null,\"hostname\":null,\"ip\":\"81.2.69.192\",\"mac\":null,\"subnet\":null,\"did\":null,\"sid\":null},{\"identifier\":null,\"hostname\":null,\"ip\":\"175.16.199.1\",\"mac\":null,\"subnet\":null,\"did\":null,\"sid\":null},{\"identifier\":null,\"hostname\":null,\"ip\":\"175.16.199.3\",\"mac\":null,\"subnet\":null,\"did\":null,\"sid\":null}]},{\"key\":\"Destination port\",\"type\":\"integer\",\"values\":[22]},{\"key\":\"Connection count\",\"type\":\"integer\",\"values\":[40]},{\"key\":\"Percentage successful\",\"type\":\"percentage\",\"values\":[100]}]}]]}", + "reason": "Extensive Unusual SSH Connections", + "risk_score": 98.0, + "risk_score_norm": 98.0, + "start": [ + "2021-08-03T10:08:18.683Z" + ], + "type": [ + "info" + ] + }, + "host": { + "hostname": [ + "linux.local" + ], + "id": [ + "10" + ], + "ip": [ + "81.2.69.144" + ] + }, + "message": "The device 175.16.199.1 was observed making unusual internal SSH connections to a wide range of devices.\n\nThough this behaviour could be the result of legitimate remote access or administration, it could also be a sign of attempted lateral movement by a compromised machine.\n\nConsequently, if this activity was not expected, the security team may wish to investigate further.", + "related": { + "hosts": [ + "linux.local" + ], + "ip": [ + "81.2.69.144", + "175.16.199.1", + "81.2.69.192", + "175.16.199.3" + ] + }, + "rule": { + "name": [ + "Unusual Activity / Unusual Activity from Re-Activated Device" + ] + }, + "tags": [ + "preserve_original_event", + "preserve_duplicate_custom_fields" + ], + "threat": { + "enrichments": { + "matched": { + "id": [ + "eabcdef0-1234-1234-1234-cabcdefghij9" + ] + } + }, + "group": { + "id": "eabc1234-1234-1234-1234-cabcdefg0011" + } + } + }, + { + "@timestamp": "2022-07-13T21:57:17.781Z", + "darktrace": { + "ai_analyst_alert": { + "activity_id": "abcdefee", + "aia_score": 64.0, + "attack_phases": [ + 4 + ], + "breach_devices": [ + { + "did": 7, + "hostname": "linux.local", + "identifier": "linux.local", + "ip": "175.16.199.1/24", + "sid": 1 + } + ], + "category": "suspicious", + "children": [ + "eabcdef0-1234-1234-1234-cabcdefghij9" + ], + "created_at": "2022-07-13T21:57:17.781Z", + "current_group": "eab12345-1234-1234-1234-cabcdef12345", + "details": [ + [ + { + "contents": [ + { + "key": "Time", + "type": "timestampRange", + "values": [ + { + "end": 1657749405342, + "start": 1657747020967 + } + ] + }, + { + "key": "Source device", + "type": "device", + "values": [ + { + "did": 7, + "hostname": "linux.local", + "identifier": "linux.local", + "ip": "175.16.199.3/28", + "sid": 1 + } + ] + }, + { + "key": "Scanned IPs", + "type": "device", + "values": [ + { + "ip": "81.2.69.144" + }, + { + "ip": "81.2.69.192" + } + ] + }, + { + "key": "Username observed prior to activity", + "type": "string", + "values": [ + "user1" + ] + }, + { + "key": "Source of username", + "type": "string", + "values": [ + "User login" + ] + }, + { + "key": "Time observed", + "type": "timestamp", + "values": [ + 1657739323000 + ] + }, + { + "key": "Event UID", + "type": "string", + "values": [ + "CABCDabcdABCDabcd000" + ] + } + ], + "header": "Overview of Scan" + } + ], + [ + { + "contents": [ + { + "key": "Total connections", + "type": "integer", + "values": [ + 4537 + ] + }, + { + "key": "Total ports scanned", + "type": "integer", + "values": [ + 996 + ] + }, + { + "key": "Port range", + "type": "integerRange", + "values": [ + { + "end": "65389", + "start": "1" + } + ] + }, + { + "key": "Key ports", + "type": "integer", + "values": [ + 21, + 22, + 23, + 80, + 389, + 443, + 1433, + 3128, + 3306, + 4444, + 4899, + 8080 + ] + } + ], + "header": "TCP Scanning Activity" + } + ], + [ + { + "contents": [ + { + "key": "Total connections", + "type": "integer", + "values": [ + 116 + ] + }, + { + "key": "Port", + "type": "integer", + "values": [ + 137 + ] + } + ], + "header": "UDP Scanning Activity" + } + ] + ], + "group_by_activity": false, + "group_category": "suspicious", + "group_score": 6.857722547303857, + "grouping_ids": [ + "ab123456" + ], + "id": "eabcdef0-1234-1234-1234-cabcdefghij9", + "incident_event_url": { + "domain": "www.example.com", + "fragment": "aiaincidentevent/eabcdef0-1234-1234-1234-cabcdefghij9", + "original": "https://www.example.com/#aiaincidentevent/eabcdef0-1234-1234-1234-cabcdefghij9", + "path": "/", + "scheme": "https" + }, + "is_acknowledged": false, + "is_external_triggered": false, + "is_pinned": false, + "is_user_triggered": false, + "periods": [ + { + "end": "2022-07-13T21:56:45.342Z", + "start": "2022-07-13T21:17:00.967Z" + } + ], + "related_breaches": [ + { + "model_name": "Device / Attack and Recon Tools", + "pbid": 6, + "threat_score": 88, + "timestamp": "2022-07-13T21:46:48.000Z" + } + ], + "summariser": "ScanSummary", + "summary": "The device linux.local was observed making an unusually large number of internal connection attempts to multiple devices, suggesting scanning activity.\n\nNetwork scanning can be used during reconnaissance to gather information about internal devices, such as their list of open ports, and is thus a possible indicator of preparation for malicious or unauthorised internal activity.\n\nIf the activity from the device was not expected, it is recommended that the security team investigate it further to determine whether it was part of legitimate network activity.", + "title": "Port Scanning" + } + }, + "ecs": { + "version": "8.4.0" + }, + "event": { + "category": [ + "threat" + ], + "duration": [ + 2384375000000 + ], + "end": [ + "2022-07-13T21:56:45.342Z" + ], + "id": "eabcdef0-1234-1234-1234-cabcdefghij9", + "kind": "alert", + "original": "{\"summariser\":\"ScanSummary\",\"acknowledged\":false,\"pinned\":false,\"createdAt\":1657749437781,\"attackPhases\":[4],\"title\":\"Port Scanning\",\"id\":\"eabcdef0-1234-1234-1234-cabcdefghij9\",\"incidentEventUrl\":\"https://www.example.com/#aiaincidentevent/eabcdef0-1234-1234-1234-cabcdefghij9\",\"children\":[\"eabcdef0-1234-1234-1234-cabcdefghij9\"],\"category\":\"suspicious\",\"currentGroup\":\"eab12345-1234-1234-1234-cabcdef12345\",\"groupCategory\":\"suspicious\",\"groupScore\":6.857722547303857,\"groupPreviousGroups\":[],\"activityId\":\"abcdefee\",\"groupingIds\":[\"ab123456\"],\"groupByActivity\":false,\"userTriggered\":false,\"externalTriggered\":false,\"aiaScore\":64,\"summary\":\"The device linux.local was observed making an unusually large number of internal connection attempts to multiple devices, suggesting scanning activity.\\n\\nNetwork scanning can be used during reconnaissance to gather information about internal devices, such as their list of open ports, and is thus a possible indicator of preparation for malicious or unauthorised internal activity.\\n\\nIf the activity from the device was not expected, it is recommended that the security team investigate it further to determine whether it was part of legitimate network activity.\",\"periods\":[{\"start\":1657747020967,\"end\":1657749405342}],\"breachDevices\":[{\"identifier\":\"linux.local\",\"hostname\":\"linux.local\",\"ip\":\"175.16.199.1/24\",\"mac\":null,\"subnet\":null,\"did\":7,\"sid\":1}],\"relatedBreaches\":[{\"modelName\":\"Device / Attack and Recon Tools\",\"pbid\":6,\"threatScore\":88,\"timestamp\":1657748808000}],\"details\":[[{\"header\":\"Overview of Scan\",\"contents\":[{\"key\":\"Time\",\"type\":\"timestampRange\",\"values\":[{\"start\":1657747020967,\"end\":1657749405342}]},{\"key\":\"Source device\",\"type\":\"device\",\"values\":[{\"identifier\":\"linux.local\",\"hostname\":\"linux.local\",\"ip\":\"175.16.199.3/28\",\"mac\":null,\"subnet\":null,\"did\":7,\"sid\":1}]},{\"key\":\"Scanned IPs\",\"type\":\"device\",\"values\":[{\"identifier\":null,\"hostname\":null,\"ip\":\"81.2.69.144\",\"mac\":null,\"subnet\":null,\"did\":null,\"sid\":null},{\"identifier\":null,\"hostname\":null,\"ip\":\"81.2.69.192\",\"mac\":null,\"subnet\":null,\"did\":null,\"sid\":null}]},{\"key\":\"Username observed prior to activity\",\"type\":\"string\",\"values\":[\"user1\"]},{\"key\":\"Source of username\",\"type\":\"string\",\"values\":[\"User login\"]},{\"key\":\"Time observed\",\"type\":\"timestamp\",\"values\":[1657739323000]},{\"key\":\"Event UID\",\"type\":\"string\",\"values\":[\"CABCDabcdABCDabcd000\"]}]}],[{\"header\":\"TCP Scanning Activity\",\"contents\":[{\"key\":\"Total connections\",\"type\":\"integer\",\"values\":[4537]},{\"key\":\"Total ports scanned\",\"type\":\"integer\",\"values\":[996]},{\"key\":\"Port range\",\"type\":\"integerRange\",\"values\":[{\"start\":\"1\",\"end\":\"65389\"}]},{\"key\":\"Key ports\",\"type\":\"integer\",\"values\":[21,22,23,80,389,443,1433,3128,3306,4444,4899,8080]}]}],[{\"header\":\"UDP Scanning Activity\",\"contents\":[{\"key\":\"Total connections\",\"type\":\"integer\",\"values\":[116]},{\"key\":\"Port\",\"type\":\"integer\",\"values\":[137]}]}]]}", + "reason": "Port Scanning", + "risk_score": 64.0, + "risk_score_norm": 64.0, + "start": [ + "2022-07-13T21:17:00.967Z" + ], + "type": [ + "info" + ], + "url": "https://www.example.com/#aiaincidentevent/eabcdef0-1234-1234-1234-cabcdefghij9" + }, + "host": { + "hostname": [ + "linux.local" + ], + "id": [ + "7" + ], + "name": [ + "linux.local" + ] + }, + "message": "The device linux.local was observed making an unusually large number of internal connection attempts to multiple devices, suggesting scanning activity.\n\nNetwork scanning can be used during reconnaissance to gather information about internal devices, such as their list of open ports, and is thus a possible indicator of preparation for malicious or unauthorised internal activity.\n\nIf the activity from the device was not expected, it is recommended that the security team investigate it further to determine whether it was part of legitimate network activity.", + "related": { + "hosts": [ + "linux.local" + ], + "ip": [ + "81.2.69.144", + "81.2.69.192" + ] + }, + "rule": { + "name": [ + "Device / Attack and Recon Tools" + ] + }, + "tags": [ + "preserve_original_event", + "preserve_duplicate_custom_fields" + ], + "threat": { + "enrichments": { + "matched": { + "id": [ + "eabcdef0-1234-1234-1234-cabcdefghij9" + ] + } + }, + "group": { + "id": "eab12345-1234-1234-1234-cabcdef12345" + } + } + } + ] +} \ No newline at end of file diff --git a/packages/darktrace/data_stream/ai_analyst_alert/_dev/test/pipeline/test-common-config.yml b/packages/darktrace/data_stream/ai_analyst_alert/_dev/test/pipeline/test-common-config.yml new file mode 100644 index 00000000000..be41bb0d476 --- /dev/null +++ b/packages/darktrace/data_stream/ai_analyst_alert/_dev/test/pipeline/test-common-config.yml @@ -0,0 +1,4 @@ +fields: + tags: + - preserve_original_event + - preserve_duplicate_custom_fields diff --git a/packages/darktrace/data_stream/ai_analyst_alert/_dev/test/system/test-httpjson-config.yml b/packages/darktrace/data_stream/ai_analyst_alert/_dev/test/system/test-httpjson-config.yml new file mode 100644 index 00000000000..938018884cf --- /dev/null +++ b/packages/darktrace/data_stream/ai_analyst_alert/_dev/test/system/test-httpjson-config.yml @@ -0,0 +1,10 @@ +input: httpjson +service: darktrace +vars: + url: http://{{Hostname}}:{{Port}} + public_token: xxxx + private_token: xxxx +data_stream: + vars: + preserve_original_event: true + preserve_duplicate_custom_fields: true diff --git a/packages/darktrace/data_stream/ai_analyst_alert/_dev/test/system/test-tcp-config.yml b/packages/darktrace/data_stream/ai_analyst_alert/_dev/test/system/test-tcp-config.yml new file mode 100644 index 00000000000..53314b710b8 --- /dev/null +++ b/packages/darktrace/data_stream/ai_analyst_alert/_dev/test/system/test-tcp-config.yml @@ -0,0 +1,10 @@ +service: darktrace-ai_analyst_alert-tcp +service_notify_signal: SIGHUP +input: tcp +vars: + listen_address: 0.0.0.0 +data_stream: + vars: + listen_port: 9571 + preserve_original_event: true + preserve_duplicate_custom_fields: true diff --git a/packages/darktrace/data_stream/ai_analyst_alert/_dev/test/system/test-tls-config.yml b/packages/darktrace/data_stream/ai_analyst_alert/_dev/test/system/test-tls-config.yml new file mode 100644 index 00000000000..d3819b60c78 --- /dev/null +++ b/packages/darktrace/data_stream/ai_analyst_alert/_dev/test/system/test-tls-config.yml @@ -0,0 +1,62 @@ +service: darktrace-ai_analyst_alert-tls +service_notify_signal: SIGHUP +input: tcp +vars: + listen_address: 0.0.0.0 + ssl: | + key: | + -----BEGIN PRIVATE KEY----- + MIIEvwIBADANBgkqhkiG9w0BAQEFAASCBKkwggSlAgEAAoIBAQDhCLvLsQAHufsN + U+u1x/CequAUphfXZqLhDo2Eo/holfBS0+ey4bnzPL6lS9NFL5JkLQA2gYESqsXU + /Ru8E76Az1egzMwT3TVAPLVU8NbrxBqeNiQa2m9wC37HQy4qC9OxL28LUoKtFjxS + cD1sa0oikXCJN1a3BSoAf9iiZ/dxz4WVfrNhrzq2JFXjravY84n5ujkZOg45Pg70 + 4vHOeg0rBbIoSNfjDUVZWjwC95K1BMN3msOTL9juv/EDa6BujqCxl+G1nY7JPFDL + SHWis65p+1AAa5xieYDb47vyJ0SSR7lEURTXZOkkM6k5JWfgkATEmGzRxPkOloIT + Xg9ag1OlAgMBAAECggEAEHfPJmzhj68wjB0kFr13AmWG2Hv/Kqg8KzQhbx+AwkaW + u7j+L70NGpvLZ9VQtLNyhxoz9cksZO1SZO/Q48aeHlcOFppmJN3/U6AdtQWa9M35 + FLLpmX16wjxVHsfvzOvopgLOoYl8PqZt66qDFDgVyMnT7na6RdJ+7GJuvBPXq+Bc + vgThvAZitHSAOhnBFYmTMlBi6AzOMMsaFlgE3Xf9v3M0pAKItPRKMhXlC3MyvA/v + jgbra4Ib+0ryohggHheHB3bn3Jgv7iFKoW9OQSePVxacJ+kfr9H+No5g495URzqR + mx/96WCiv3rAh3ct8Sk/C4/3zMC8fUueDJIVjhgw0QKBgQD8NufLINNkIpBrLoCS + 972oFEjZB2u6EusQ7X9raROqpaw26ZSu+zSHeIKCGQ93M3aRb3FpdGeOxgZ095MV + 8a+nlh4stOvHj2Mm5YhTBDUavTC7o9aVR3Od5eTXUpHnaJpNI/uyIcKupeK1UJnV + UlBLeIwo/vJ1gsVrKMMAJkuKbwKBgQDkaWRRd0w2gUIbCTGf203BqXft0VdIiOW7 + +gnkeaNHAf09XljzxMcQzrB8kG63aKVGbJffphEfzxtiJ+HRQVH+7QpKRhU/GHmu + +6OKkxTcxJm5zhoRFxcSi2wG4PWmUGJvc7ss1OJGcaOUxwocCepO7N/jfdDz9Uke + KnA+YWOdKwKBgQDteZkYlojT0QOgF8HyH5gQyUCqMKWLJ0LzxltiPCbLV4Dml1pq + w5Z7M8nWS1hXiTpLx93GSFc1hFkSCwYP9GfK6Lryp0sVtHnMZvTMDbseuSJImwRx + vDwtYQfugg1lEQWwOoBEAiu3m/PxernNtNprpU57T0nlwUK3GkM5QdWAuwKBgQCZ + ZF3GiANapzupxGbbH//8Cr9LqsafI7CEqMpz8WxBh4h16iJ6sq+tDeFgBe8UpOY5 + gTwNKg1d+0w8guQYD3HtbWr3rlEeamVtqfiOW3ArQqyqJ0tCJuuLvK3zgKf35Qv2 + JRaSaPT8sdxVUcXsRoxgLJu+vwPQke1koMN4YRbwuQKBgQDJiZ/WSeqa5oIqkXbn + hjm7RXKaf2oE1U/bNjdSFtdEP7T4vUvvr7Hq2f/jiBLtCE7w16PJjKx9iIq2+jMl + qIY43Sk9bdi5FxtYTHda0hwrbH274P+QVcVs5PXCT0TGktOleHGBlXaaPrxl9iCh + 8tmmxZZYa5aQxEO/lxB9xQKaiQ== + -----END PRIVATE KEY----- + certificate: | + -----BEGIN CERTIFICATE----- + MIIDazCCAlOgAwIBAgIUW5TDu1tJMY2Oa7PsL+BQSmeWqz0wDQYJKoZIhvcNAQEL + BQAwRTELMAkGA1UEBhMCVVMxEzARBgNVBAgMClNvbWUtU3RhdGUxITAfBgNVBAoM + GEludGVybmV0IFdpZGdpdHMgUHR5IEx0ZDAeFw0yMTEwMDEwNTAwMjNaFw0yMTEw + MDIwNTAwMjNaMEUxCzAJBgNVBAYTAlVTMRMwEQYDVQQIDApTb21lLVN0YXRlMSEw + HwYDVQQKDBhJbnRlcm5ldCBXaWRnaXRzIFB0eSBMdGQwggEiMA0GCSqGSIb3DQEB + AQUAA4IBDwAwggEKAoIBAQDhCLvLsQAHufsNU+u1x/CequAUphfXZqLhDo2Eo/ho + lfBS0+ey4bnzPL6lS9NFL5JkLQA2gYESqsXU/Ru8E76Az1egzMwT3TVAPLVU8Nbr + xBqeNiQa2m9wC37HQy4qC9OxL28LUoKtFjxScD1sa0oikXCJN1a3BSoAf9iiZ/dx + z4WVfrNhrzq2JFXjravY84n5ujkZOg45Pg704vHOeg0rBbIoSNfjDUVZWjwC95K1 + BMN3msOTL9juv/EDa6BujqCxl+G1nY7JPFDLSHWis65p+1AAa5xieYDb47vyJ0SS + R7lEURTXZOkkM6k5JWfgkATEmGzRxPkOloITXg9ag1OlAgMBAAGjUzBRMB0GA1Ud + DgQWBBRYUSKDHBBE9Q6fTeTqogicCxcXwDAfBgNVHSMEGDAWgBRYUSKDHBBE9Q6f + TeTqogicCxcXwDAPBgNVHRMBAf8EBTADAQH/MA0GCSqGSIb3DQEBCwUAA4IBAQBc + T8B+GpvPy9NQ700LsywRPY0L9IJCKiu6j3TP1tqqSPjAC/cg9ac+bFXuWOu7V+KJ + s09Q/pItq9SLX6UvnfRzTxu5lCBwwGX9cL131mTIu5SmFo7Eks+sorbiIarWDMoC + e+9An3GFpagW+YhOt4BdIM5lTqoeodzganDBsOUZI9aDAj2Yo5h2O7r6Wd12cb6T + mz8vMfB2eG8BxU20ZMfkdERWjiyXHOSBQqeqfkV8d9370gMu5RcJNcIgnbmTRdho + X3HJFiimZVaNjXATqmC/y2A1KXvJdamPLy3mGXkW2cFLoPCdK2OZFUHqiuc1bigA + qEf55SihFqErRMeURPPF + -----END CERTIFICATE----- +data_stream: + vars: + listen_port: 9571 + preserve_original_event: true + preserve_duplicate_custom_fields: true diff --git a/packages/darktrace/data_stream/ai_analyst_alert/_dev/test/system/test-udp-config.yml b/packages/darktrace/data_stream/ai_analyst_alert/_dev/test/system/test-udp-config.yml new file mode 100644 index 00000000000..0d9b6eb6f0e --- /dev/null +++ b/packages/darktrace/data_stream/ai_analyst_alert/_dev/test/system/test-udp-config.yml @@ -0,0 +1,10 @@ +service: darktrace-ai_analyst_alert-udp +service_notify_signal: SIGHUP +input: udp +vars: + listen_address: 0.0.0.0 +data_stream: + vars: + listen_port: 9574 + preserve_original_event: true + preserve_duplicate_custom_fields: true diff --git a/packages/darktrace/data_stream/ai_analyst_alert/agent/stream/httpjson.yml.hbs b/packages/darktrace/data_stream/ai_analyst_alert/agent/stream/httpjson.yml.hbs new file mode 100644 index 00000000000..7f34d9e5ec4 --- /dev/null +++ b/packages/darktrace/data_stream/ai_analyst_alert/agent/stream/httpjson.yml.hbs @@ -0,0 +1,50 @@ +config_version: 2 +interval: {{interval}} +request.method: GET +{{#if proxy_url}} +request.proxy_url: {{proxy_url}} +{{/if}} +{{#if ssl}} +request.ssl: {{ssl}} +{{/if}} +request.url: {{url}}/aianalyst/incidentevents?includeacknowledged=true&includeincidenteventurl=true +{{#if http_client_timeout}} +request.timeout: {{http_client_timeout}} +{{/if}} +request.transforms: + - set: + target: header.DTAPI-Token + value: {{public_token}} + - set: + target: header.DTAPI-Date + value: '[[formatDate (now) "20060102T150405"]]' + - set: + target: url.params.starttime + value: '[[.cursor.last_execution_datetime]]' + default: '[[(now (parseDuration "-{{initial_interval}}")).UnixMilli]]' + - set: + target: url.params.endtime + value: '[[(now).UnixMilli]]' + - set: + target: header.DTAPI-Signature + value: '[[hmac "sha1" "{{private_token}}" (sprintf "%s?%s\n%s\n%s" .url.Path .url.RawQuery "{{public_token}}" (formatDate (now) "20060102T150405"))]]' +cursor: + last_execution_datetime: + value: '[[.last_response.url.params.Get "endtime"]]' +tags: +{{#if preserve_original_event}} + - preserve_original_event +{{/if}} +{{#if preserve_duplicate_custom_fields}} + - preserve_duplicate_custom_fields +{{/if}} +{{#each tags as |tag|}} + - {{tag}} +{{/each}} +{{#contains "forwarded" tags}} +publisher_pipeline.disable_host: true +{{/contains}} +{{#if processors}} +processors: +{{processors}} +{{/if}} diff --git a/packages/darktrace/data_stream/ai_analyst_alert/agent/stream/tcp.yml.hbs b/packages/darktrace/data_stream/ai_analyst_alert/agent/stream/tcp.yml.hbs new file mode 100644 index 00000000000..b1d260f0f9c --- /dev/null +++ b/packages/darktrace/data_stream/ai_analyst_alert/agent/stream/tcp.yml.hbs @@ -0,0 +1,26 @@ +host: "{{listen_address}}:{{listen_port}}" +{{#if tcp_options}} +{{tcp_options}} +{{/if}} +{{#if ssl}} +ssl: {{ssl}} +{{/if}} +tags: +{{#if preserve_original_event}} + - preserve_original_event +{{/if}} +{{#if preserve_duplicate_custom_fields}} + - preserve_duplicate_custom_fields +{{/if}} +{{#each tags as |tag|}} + - {{tag}} +{{/each}} +{{#contains "forwarded" tags}} +publisher_pipeline.disable_host: true +{{/contains}} +processors: +- syslog: + field: message +{{#if processors}} +{{processors}} +{{/if}} diff --git a/packages/darktrace/data_stream/ai_analyst_alert/agent/stream/udp.yml.hbs b/packages/darktrace/data_stream/ai_analyst_alert/agent/stream/udp.yml.hbs new file mode 100644 index 00000000000..f342c4fa75c --- /dev/null +++ b/packages/darktrace/data_stream/ai_analyst_alert/agent/stream/udp.yml.hbs @@ -0,0 +1,23 @@ +host: "{{listen_address}}:{{listen_port}}" +{{#if udp_options}} +{{udp_options}} +{{/if}} +tags: +{{#if preserve_original_event}} + - preserve_original_event +{{/if}} +{{#if preserve_duplicate_custom_fields}} + - preserve_duplicate_custom_fields +{{/if}} +{{#each tags as |tag|}} + - {{tag}} +{{/each}} +{{#contains "forwarded" tags}} +publisher_pipeline.disable_host: true +{{/contains}} +processors: +- syslog: + field: message +{{#if processors}} +{{processors}} +{{/if}} diff --git a/packages/darktrace/data_stream/ai_analyst_alert/elasticsearch/ingest_pipeline/default.yml b/packages/darktrace/data_stream/ai_analyst_alert/elasticsearch/ingest_pipeline/default.yml new file mode 100644 index 00000000000..3b22b43d8bb --- /dev/null +++ b/packages/darktrace/data_stream/ai_analyst_alert/elasticsearch/ingest_pipeline/default.yml @@ -0,0 +1,856 @@ +--- +description: Pipeline for processing AI Analyst Alert logs. +processors: + - set: + field: ecs.version + value: '8.4.0' + - grok: + field: message + patterns: + - "^%{FIELD:log.syslog.appname}\\s*%{GREEDYDATA:message}$" + pattern_definitions: + FIELD: "[a-zA-Z]*" + - rename: + field: message + target_field: event.original + ignore_missing: true + - json: + field: event.original + target_field: json + - fingerprint: + fields: + - json.id + - json.createdAt + - json.activityId + - json.currentGroup + target_field: _id + ignore_missing: true + - set: + field: event.kind + value: alert + if: (['critical','suspicious'].contains(ctx.json?.category?.toLowerCase())) + - set: + field: event.kind + value: event + if: (['compliance','informational'].contains(ctx.json?.category?.toLowerCase())) + - set: + field: event.category + value: [threat] + if: ctx.event?.kind == 'alert' + - set: + field: event.type + value: [info] + - rename: + field: json.activityId + target_field: darktrace.ai_analyst_alert.activity_id + ignore_missing: true + - convert: + field: json.aiaScore + target_field: darktrace.ai_analyst_alert.aia_score + type: double + ignore_missing: true + on_failure: + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - set: + field: event.risk_score + copy_from: darktrace.ai_analyst_alert.aia_score + ignore_failure: true + - set: + field: event.risk_score_norm + copy_from: darktrace.ai_analyst_alert.aia_score + ignore_failure: true + - foreach: + field: json.attackPhases + if: ctx.json?.attackPhases instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value + type: long + on_failure: + - remove: + field: _ingest._value + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - rename: + field: json.attackPhases + target_field: darktrace.ai_analyst_alert.attack_phases + ignore_missing: true + - foreach: + field: json.breachDevices + if: ctx.json?.breachDevices instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value.did + type: long + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.did + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - foreach: + field: json.breachDevices + if: ctx.json?.breachDevices instanceof List + ignore_failure: true + processor: + append: + field: host.id + value: '{{{_ingest._value.did}}}' + allow_duplicates: false + ignore_failure: true + - convert: + field: host.id + type: string + ignore_missing: true + on_failure: + - remove: + field: host.id + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - foreach: + field: json.breachDevices + if: ctx.json?.breachDevices instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value.hostname + type: ip + target_field: _ingest._value._temp_.hostname_ip + ignore_missing: true + on_failure: + - append: + field: host.hostname + value: '{{{_ingest._value.hostname}}}' + allow_duplicates: false + ignore_failure: true + - foreach: + field: json.breachDevices + if: ctx.json?.breachDevices instanceof List + ignore_failure: true + processor: + append: + field: related.ip + value: '{{{_ingest._value._temp_.hostname_ip}}}' + allow_duplicates: false + ignore_failure: true + - set: + field: related.hosts + copy_from: host.hostname + ignore_failure: true + - foreach: + field: json.breachDevices + if: ctx.json?.breachDevices instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value.identifier + target_field: _ingest._value._temp_.identifier_ip + type: ip + ignore_missing: true + on_failure: + - append: + field: host.name + value: '{{{_ingest._value.identifier}}}' + allow_duplicates: false + ignore_failure: true + - foreach: + field: json.breachDevices + if: ctx.json?.breachDevices instanceof List + ignore_failure: true + processor: + append: + field: related.ip + value: '{{{_ingest._value._temp_.identifier_ip}}}' + allow_duplicates: false + ignore_failure: true + - foreach: + field: host.name + if: ctx.host?.name instanceof List + ignore_failure: true + processor: + append: + field: related.hosts + value: '{{{_ingest._value}}}' + allow_duplicates: false + ignore_failure: true + - foreach: + field: json.breachDevices + if: ctx.json?.breachDevices instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value.ip + target_field: _ingest._value._temp_.ip + type: ip + ignore_failure: true + - foreach: + field: json.breachDevices + if: ctx.json?.breachDevices instanceof List + ignore_failure: true + processor: + append: + field: host.ip + value: '{{{_ingest._value._temp_.ip}}}' + allow_duplicates: false + ignore_failure: true + - foreach: + field: host.ip + if: ctx.host?.ip instanceof List + ignore_failure: true + processor: + append: + field: related.ip + value: '{{{_ingest._value}}}' + allow_duplicates: false + ignore_failure: true + - foreach: + field: json.breachDevices + if: ctx.json?.breachDevices instanceof List + ignore_failure: true + processor: + gsub: + field: _ingest._value.mac + target_field: _ingest._value.mac_address + pattern: '[:.]' + replacement: '-' + ignore_missing: true + - foreach: + field: json.breachDevices + if: ctx.json?.breachDevices instanceof List + ignore_failure: true + processor: + uppercase: + field: _ingest._value.mac_address + ignore_missing: true + - foreach: + field: json.breachDevices + if: ctx.json?.breachDevices instanceof List + ignore_failure: true + processor: + append: + field: host.mac + value: '{{{_ingest._value.mac_address}}}' + allow_duplicates: false + ignore_failure: true + - foreach: + field: json.breachDevices + if: ctx.json?.breachDevices instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value.sid + type: long + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.sid + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - foreach: + field: json.breachDevices + if: ctx.json?.breachDevices instanceof List + ignore_failure: true + processor: + remove: + field: + - _ingest._value._temp_ + - _ingest._value.mac + ignore_missing: true + - rename: + field: json.breachDevices + target_field: darktrace.ai_analyst_alert.breach_devices + ignore_missing: true + - rename: + field: json.category + target_field: darktrace.ai_analyst_alert.category + ignore_missing: true + - rename: + field: json.children + target_field: darktrace.ai_analyst_alert.children + ignore_missing: true + - set: + field: threat.enrichments.matched.id + copy_from: darktrace.ai_analyst_alert.children + ignore_failure: true + - date: + field: json.createdAt + target_field: darktrace.ai_analyst_alert.created_at + formats: + - ISO8601 + - UNIX_MS + - 'MMM dd HH:mm:ss' + if: ctx.json?.createdAt != null + on_failure: + - remove: + field: json.createdAt + ignore_missing: true + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - set: + field: '@timestamp' + copy_from: darktrace.ai_analyst_alert.created_at + ignore_failure: true + - rename: + field: json.currentGroup + target_field: darktrace.ai_analyst_alert.current_group + ignore_missing: true + - set: + field: threat.group.id + copy_from: darktrace.ai_analyst_alert.current_group + ignore_failure: true + if: ctx.darktrace?.ai_analyst_alert?.current_group != null + - foreach: + field: json.details + if: ctx.json?.details instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value + ignore_failure: true + processor: + foreach: + field: _ingest._value.contents + ignore_failure: true + processor: + foreach: + field: _ingest._value.values + ignore_failure: true + processor: + convert: + field: _ingest._value.ip + target_field: _ingest._value._temp_.ip + type: ip + ignore_failure: true + - foreach: + field: json.details + if: ctx.json?.details instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value + ignore_failure: true + processor: + foreach: + field: _ingest._value.contents + ignore_failure: true + processor: + foreach: + field: _ingest._value.values + ignore_failure: true + processor: + append: + field: related.ip + value: '{{{_ingest._value._temp_.ip}}}' + allow_duplicates: false + ignore_failure: true + - foreach: + field: json.details + if: ctx.json?.details instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value + ignore_failure: true + processor: + foreach: + field: _ingest._value.contents + ignore_failure: true + processor: + foreach: + field: _ingest._value.values + ignore_failure: true + processor: + convert: + field: _ingest._value.hostname + target_field: _ingest._value._temp_.hostname_ip + type: ip + ignore_missing: true + on_failure: + - append: + field: related.hosts + value: '{{{_ingest._value.hostname}}}' + allow_duplicates: false + ignore_failure: true + - foreach: + field: json.details + if: ctx.json?.details instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value + ignore_failure: true + processor: + foreach: + field: _ingest._value.contents + ignore_failure: true + processor: + foreach: + field: _ingest._value.values + ignore_failure: true + processor: + append: + field: related.ip + value: '{{{_ingest._value._temp_.hostname_ip}}}' + allow_duplicates: false + ignore_failure: true + - foreach: + field: json.details + if: ctx.json?.details instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value + ignore_failure: true + processor: + foreach: + field: _ingest._value.contents + ignore_failure: true + processor: + foreach: + field: _ingest._value.values + ignore_failure: true + processor: + convert: + field: _ingest._value.identifier + target_field: _ingest._value._temp_.identifier_ip + type: ip + ignore_missing: true + on_failure: + - append: + field: related.hosts + value: '{{{_ingest._value.identifier}}}' + allow_duplicates: false + ignore_failure: true + - foreach: + field: json.details + if: ctx.json?.details instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value + ignore_failure: true + processor: + foreach: + field: _ingest._value.contents + ignore_failure: true + processor: + foreach: + field: _ingest._value.values + ignore_failure: true + processor: + append: + field: related.ip + value: '{{{_ingest._value._temp_.identifier_ip}}}' + allow_duplicates: false + ignore_failure: true + - foreach: + field: json.details + if: ctx.json?.details instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value + ignore_failure: true + processor: + foreach: + field: _ingest._value.contents + ignore_failure: true + processor: + foreach: + field: _ingest._value.values + ignore_failure: true + processor: + gsub: + field: _ingest._value.mac + target_field: _ingest._value.mac_address + pattern: '[:.]' + replacement: '-' + ignore_missing: true + - foreach: + field: json.details + if: ctx.json?.details instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value + ignore_failure: true + processor: + foreach: + field: _ingest._value.contents + ignore_failure: true + processor: + foreach: + field: _ingest._value.values + ignore_failure: true + processor: + uppercase: + field: _ingest._value.mac_address + ignore_missing: true + - foreach: + field: json.details + if: ctx.json?.details instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value + ignore_failure: true + processor: + foreach: + field: _ingest._value.contents + ignore_failure: true + processor: + foreach: + field: _ingest._value.values + ignore_failure: true + processor: + remove: + field: + - _ingest._value._temp_ + - _ingest._value.mac + ignore_missing: true + - rename: + field: json.details + target_field: darktrace.ai_analyst_alert.details + ignore_missing: true + - convert: + field: json.groupByActivity + target_field: darktrace.ai_analyst_alert.group_by_activity + type: boolean + ignore_missing: true + on_failure: + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - set: + field: threat.group.id + copy_from: darktrace.ai_analyst_alert.activity_id + ignore_failure: true + if: ctx.threat?.group?.id == null && ctx.darktrace?.ai_analyst_alert?.group_by_activity == true + - rename: + field: json.groupCategory + target_field: darktrace.ai_analyst_alert.group_category + ignore_missing: true + - rename: + field: json.groupPreviousGroups + target_field: darktrace.ai_analyst_alert.group_previous_groups + ignore_missing: true + - convert: + field: json.groupScore + target_field: darktrace.ai_analyst_alert.group_score + type: double + ignore_missing: true + on_failure: + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - rename: + field: json.groupingIds + target_field: darktrace.ai_analyst_alert.grouping_ids + ignore_missing: true + - rename: + field: json.id + target_field: darktrace.ai_analyst_alert.id + ignore_missing: true + - set: + field: event.id + copy_from: darktrace.ai_analyst_alert.id + ignore_failure: true + - uri_parts: + field: json.incidentEventUrl + target_field: darktrace.ai_analyst_alert.incident_event_url + if: ctx.json?.incidentEventUrl != null + keep_original: true + on_failure: + - remove: + field: json.incidentEventUrl + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - set: + field: event.url + copy_from: darktrace.ai_analyst_alert.incident_event_url.original + ignore_failure: true + - convert: + field: json.acknowledged + target_field: darktrace.ai_analyst_alert.is_acknowledged + type: boolean + ignore_missing: true + on_failure: + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - convert: + field: json.externalTriggered + target_field: darktrace.ai_analyst_alert.is_external_triggered + type: boolean + ignore_missing: true + on_failure: + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - convert: + field: json.pinned + target_field: darktrace.ai_analyst_alert.is_pinned + type: boolean + ignore_missing: true + on_failure: + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - convert: + field: json.userTriggered + target_field: darktrace.ai_analyst_alert.is_user_triggered + type: boolean + ignore_missing: true + on_failure: + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - script: + description: Determine event.duration from starting and ending activity timestamp. + if: ctx.json?.periods instanceof List + lang: painless + ignore_failure: true + params: + NANOS_IN_A_MILLI_SECOND: 1000000 + source: + def duration = new ArrayList(); + for (event in ctx.json.periods) { + duration.add((event?.end - event?.start) * params.NANOS_IN_A_MILLI_SECOND); + } + ctx.event.duration = duration; + - foreach: + field: json.periods + if: ctx.json?.periods instanceof List + ignore_failure: true + processor: + date: + field: _ingest._value.end + target_field: _ingest._value.end + formats: + - ISO8601 + - UNIX_MS + - 'MMM dd HH:mm:ss' + on_failure: + - remove: + field: _ingest._value.end + ignore_missing: true + - foreach: + field: json.periods + if: ctx.json?.periods instanceof List + ignore_failure: true + processor: + date: + field: _ingest._value.start + target_field: _ingest._value.start + formats: + - ISO8601 + - UNIX_MS + - 'MMM dd HH:mm:ss' + on_failure: + - remove: + field: _ingest._value.start + ignore_missing: true + - rename: + field: json.periods + target_field: darktrace.ai_analyst_alert.periods + ignore_missing: true + - foreach: + field: darktrace.ai_analyst_alert.periods + if: ctx.darktrace?.ai_analyst_alert?.periods instanceof List + ignore_failure: true + processor: + append: + field: event.end + value: '{{{_ingest._value.end}}}' + allow_duplicates: false + ignore_failure: true + - foreach: + field: darktrace.ai_analyst_alert.periods + if: ctx.darktrace?.ai_analyst_alert?.periods instanceof List + ignore_failure: true + processor: + append: + field: event.start + value: '{{{_ingest._value.start}}}' + allow_duplicates: false + ignore_failure: true + - foreach: + field: json.relatedBreaches + if: ctx.json?.relatedBreaches instanceof List + ignore_failure: true + processor: + date: + field: _ingest._value.timestamp + target_field: _ingest._value.timestamp + formats: + - ISO8601 + - UNIX_MS + - 'MMM dd HH:mm:ss' + on_failure: + - remove: + field: _ingest._value.timestamp + ignore_missing: true + - foreach: + field: json.relatedBreaches + if: ctx.json?.relatedBreaches instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value.pbid + type: long + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.pbid + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - foreach: + field: json.relatedBreaches + if: ctx.json?.relatedBreaches instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value.threatScore + target_field: _ingest._value.threat_score + type: long + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.threatScore + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - foreach: + field: json.relatedBreaches + if: ctx.json?.relatedBreaches instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.modelName + target_field: _ingest._value.model_name + ignore_missing: true + - foreach: + field: json.relatedBreaches + if: ctx.json?.relatedBreaches instanceof List + ignore_failure: true + processor: + append: + field: rule.name + value: '{{{_ingest._value.model_name}}}' + allow_duplicates: false + ignore_failure: true + - foreach: + field: json.relatedBreaches + if: ctx.json?.relatedBreaches instanceof List + ignore_failure: true + processor: + remove: + field: _ingest._value.threatScore + ignore_missing: true + - rename: + field: json.relatedBreaches + target_field: darktrace.ai_analyst_alert.related_breaches + ignore_missing: true + - rename: + field: json.summariser + target_field: darktrace.ai_analyst_alert.summariser + ignore_missing: true + - rename: + field: json.summary + target_field: darktrace.ai_analyst_alert.summary + ignore_missing: true + - set: + field: message + copy_from: darktrace.ai_analyst_alert.summary + ignore_failure: true + - rename: + field: json.title + target_field: darktrace.ai_analyst_alert.title + ignore_missing: true + - set: + field: event.reason + copy_from: darktrace.ai_analyst_alert.title + ignore_failure: true + - remove: + field: json + ignore_missing: true + - remove: + if: ctx.tags == null || !(ctx.tags.contains('preserve_duplicate_custom_fields')) + field: + - darktrace.ai_analyst_alert.created_at + - darktrace.ai_analyst_alert.summary + - darktrace.ai_analyst_alert.id + - darktrace.ai_analyst_alert.title + - darktrace.ai_analyst_alert.aia_score + - darktrace.ai_analyst_alert.children + ignore_failure: true + ignore_missing: true + - foreach: + field: darktrace.ai_analyst_alert.related_breaches + if: ctx.darktrace?.ai_analyst_alert?.related_breaches instanceof List && (ctx.tags == null || !(ctx.tags.contains('preserve_duplicate_custom_fields'))) + ignore_failure: true + processor: + remove: + field: + - _ingest._value.model_name + ignore_missing: true + ignore_failure: true + - foreach: + field: darktrace.ai_analyst_alert.periods + if: ctx.darktrace?.ai_analyst_alert?.periods instanceof List && (ctx.tags == null || !(ctx.tags.contains('preserve_duplicate_custom_fields'))) + ignore_failure: true + processor: + remove: + field: + - _ingest._value.start + - _ingest._value.end + ignore_missing: true + ignore_failure: true + - foreach: + field: darktrace.ai_analyst_alert.breach_devices + if: ctx.darktrace?.ai_analyst_alert?.breach_devices instanceof List && (ctx.tags == null || !(ctx.tags.contains('preserve_duplicate_custom_fields'))) + ignore_failure: true + processor: + remove: + field: + - _ingest._value.did + - _ingest._value.mac_address + ignore_missing: true + ignore_failure: true + - remove: + field: event.original + if: ctx.tags == null || !(ctx.tags.contains('preserve_original_event')) + ignore_failure: true + ignore_missing: true + - script: + description: Drops null/empty values recursively. + lang: painless + source: + boolean dropEmptyFields(Object object) { + if (object == null || object == "") { + return true; + } else if (object instanceof Map) { + ((Map) object).values().removeIf(value -> dropEmptyFields(value)); + return (((Map) object).size() == 0); + } else if (object instanceof List) { + ((List) object).removeIf(value -> dropEmptyFields(value)); + return (((List) object).length == 0); + } + return false; + } + dropEmptyFields(ctx); +on_failure: + - append: + field: error.message + value: '{{{ _ingest.on_failure_message }}}' diff --git a/packages/darktrace/data_stream/ai_analyst_alert/fields/agent.yml b/packages/darktrace/data_stream/ai_analyst_alert/fields/agent.yml new file mode 100644 index 00000000000..10023a11743 --- /dev/null +++ b/packages/darktrace/data_stream/ai_analyst_alert/fields/agent.yml @@ -0,0 +1,183 @@ +- name: cloud + title: Cloud + group: 2 + description: Fields related to the cloud or infrastructure the events are coming from. + footnote: 'Examples: If Metricbeat is running on an EC2 host and fetches data from its host, the cloud info contains the data about this machine. If Metricbeat runs on a remote machine outside the cloud and fetches data from a service running in the cloud, the field contains cloud data from the machine the service is running on.' + type: group + fields: + - name: account.id + level: extended + type: keyword + ignore_above: 1024 + description: 'The cloud account or organization id used to identify different entities in a multi-tenant environment. Examples: AWS account id, Google Cloud ORG Id, or other unique identifier.' + example: 666777888999 + - name: availability_zone + level: extended + type: keyword + ignore_above: 1024 + description: Availability zone in which this host is running. + example: us-east-1c + - name: instance.id + level: extended + type: keyword + ignore_above: 1024 + description: Instance ID of the host machine. + example: i-1234567890abcdef0 + - name: instance.name + level: extended + type: keyword + ignore_above: 1024 + description: Instance name of the host machine. + - name: machine.type + level: extended + type: keyword + ignore_above: 1024 + description: Machine type of the host machine. + example: t2.medium + - name: provider + level: extended + type: keyword + ignore_above: 1024 + description: Name of the cloud provider. Example values are aws, azure, gcp, or digitalocean. + example: aws + - name: region + level: extended + type: keyword + ignore_above: 1024 + description: Region in which this host is running. + example: us-east-1 + - name: project.id + type: keyword + description: Name of the project in Google Cloud. + - name: image.id + type: keyword + description: Image ID for the cloud instance. +- name: container + title: Container + group: 2 + description: 'Container fields are used for meta information about the specific container that is the source of information. These fields help correlate data based containers from any runtime.' + type: group + fields: + - name: id + level: core + type: keyword + ignore_above: 1024 + description: Unique container id. + - name: image.name + level: extended + type: keyword + ignore_above: 1024 + description: Name of the image the container was built on. + - name: labels + level: extended + type: object + object_type: keyword + description: Image labels. + - name: name + level: extended + type: keyword + ignore_above: 1024 + description: Container name. +- name: host + title: Host + group: 2 + description: 'A host is defined as a general computing instance. ECS host.* fields should be populated with details about the host on which the event happened, or from which the measurement was taken. Host types include hardware, virtual machines, Docker containers, and Kubernetes nodes.' + type: group + fields: + - name: architecture + level: core + type: keyword + ignore_above: 1024 + description: Operating system architecture. + example: x86_64 + - name: domain + level: extended + type: keyword + ignore_above: 1024 + description: 'Name of the domain of which the host is a member. For example, on Windows this could be the host''s Active Directory domain or NetBIOS domain name. For Linux this could be the domain of the host''s LDAP provider.' + example: CONTOSO + default_field: false + - name: hostname + level: core + type: keyword + ignore_above: 1024 + description: 'Hostname of the host. It normally contains what the `hostname` command returns on the host machine.' + - name: id + level: core + type: keyword + ignore_above: 1024 + description: 'Unique host id. As hostname is not always unique, use values that are meaningful in your environment. Example: The current usage of `beat.name`.' + - name: ip + level: core + type: ip + description: Host ip addresses. + - name: mac + level: core + type: keyword + ignore_above: 1024 + description: Host mac addresses. + - name: name + level: core + type: keyword + ignore_above: 1024 + description: 'Name of the host. It can contain what `hostname` returns on Unix systems, the fully qualified domain name, or a name specified by the user. The sender decides which value to use.' + - name: os.family + level: extended + type: keyword + ignore_above: 1024 + description: OS family (such as redhat, debian, freebsd, windows). + example: debian + - name: os.kernel + level: extended + type: keyword + ignore_above: 1024 + description: Operating system kernel version as a raw string. + example: 4.4.0-112-generic + - name: os.name + level: extended + type: keyword + ignore_above: 1024 + multi_fields: + - name: text + type: text + norms: false + default_field: false + description: Operating system name, without the version. + example: Mac OS X + - name: os.platform + level: extended + type: keyword + ignore_above: 1024 + description: Operating system platform (such centos, ubuntu, windows). + example: darwin + - name: os.version + level: extended + type: keyword + ignore_above: 1024 + description: Operating system version as a raw string. + example: 10.14.1 + - name: type + level: core + type: keyword + ignore_above: 1024 + description: 'Type of host. For Cloud providers this can be the machine type like `t2.medium`. If vm, this could be the container, for example, or other information meaningful in your environment.' + - name: containerized + type: boolean + description: >- + If the host is a container. + - name: os.build + type: keyword + example: '18D109' + description: >- + OS build information. + - name: os.codename + type: keyword + example: 'stretch' + description: >- + OS codename, if any. +- name: input.type + type: keyword + description: Input type +- name: log.offset + type: long + description: Log offset diff --git a/packages/darktrace/data_stream/ai_analyst_alert/fields/base-fields.yml b/packages/darktrace/data_stream/ai_analyst_alert/fields/base-fields.yml new file mode 100644 index 00000000000..f5f5a863f1d --- /dev/null +++ b/packages/darktrace/data_stream/ai_analyst_alert/fields/base-fields.yml @@ -0,0 +1,20 @@ +- name: data_stream.type + type: constant_keyword + description: Data stream type. +- name: data_stream.dataset + type: constant_keyword + description: Data stream dataset. +- name: data_stream.namespace + type: constant_keyword + description: Data stream namespace. +- name: event.module + type: constant_keyword + description: Event module. + value: darktrace +- name: event.dataset + type: constant_keyword + description: Event dataset. + value: darktrace.ai_analyst_alert +- name: '@timestamp' + type: date + description: Event timestamp. diff --git a/packages/darktrace/data_stream/ai_analyst_alert/fields/ecs.yml b/packages/darktrace/data_stream/ai_analyst_alert/fields/ecs.yml new file mode 100644 index 00000000000..4fcab038289 --- /dev/null +++ b/packages/darktrace/data_stream/ai_analyst_alert/fields/ecs.yml @@ -0,0 +1,64 @@ +- external: ecs + name: ecs.version +- external: ecs + name: event.category +- external: ecs + name: event.created +- external: ecs + name: event.end +- external: ecs + name: event.id +- external: ecs + name: event.kind +- external: ecs + name: event.original +- external: ecs + name: event.reason +- external: ecs + name: event.risk_score +- external: ecs + name: event.risk_score_norm +- external: ecs + name: event.start +- external: ecs + name: event.type +- external: ecs + name: event.url +- external: ecs + name: host.hostname +- external: ecs + name: host.id +- external: ecs + name: host.ip +- external: ecs + name: host.name +- external: ecs + name: log.syslog.appname +- external: ecs + name: log.syslog.facility.code +- external: ecs + name: log.syslog.facility.name +- external: ecs + name: log.syslog.hostname +- external: ecs + name: log.syslog.priority +- external: ecs + name: log.syslog.severity.code +- external: ecs + name: log.syslog.severity.name +- external: ecs + name: log.syslog.version +- external: ecs + name: message +- external: ecs + name: related.hosts +- external: ecs + name: related.ip +- external: ecs + name: rule.name +- external: ecs + name: tags +- external: ecs + name: threat.enrichments.matched.id +- external: ecs + name: threat.group.id diff --git a/packages/darktrace/data_stream/ai_analyst_alert/fields/fields.yml b/packages/darktrace/data_stream/ai_analyst_alert/fields/fields.yml new file mode 100644 index 00000000000..2a6a32bea67 --- /dev/null +++ b/packages/darktrace/data_stream/ai_analyst_alert/fields/fields.yml @@ -0,0 +1,143 @@ +- name: darktrace.ai_analyst_alert + type: group + fields: + - name: activity_id + type: keyword + description: An identifier for the specific activity detected by AI Analyst. If groupByActivity=true , this field should be used to group events together into an incident. + - name: aia_score + type: double + description: The score of the event as classified by AI Analyst - out of 100. + - name: attack_phases + type: long + description: Of the six attack phases, which phases are applicable to the activity. + - name: breach_devices + type: group + fields: + - name: did + type: long + description: The unique device id identifier for the device that triggered the breach. This field is used to group events into device-based incidents within the Threat Visualizer. + - name: hostname + type: keyword + description: The hostname associated with the device, if available. + - name: identifier + type: keyword + description: An identifier for the device used when constructing summaries or reports. May be the device label, hostname or IP, depending on availability. + - name: ip + type: keyword + description: The IP associated with the device. + - name: mac_address + type: keyword + description: The MAC address associated with the device. + - name: sid + type: long + description: The subnet id for the subnet the device is currently located in. + - name: subnet + type: keyword + description: The subnet label for the corresponding subnet, if available. + - name: category + type: keyword + description: The behavior category associated with the incident event. + - name: children + type: keyword + description: One or more unique identifiers that can be used to request this AI Analyst event via the UI or API. Where there is more than one uuid, requests can be made with comma-separated values. + - name: created_at + type: date + description: Timestamp for event creation in epoch time. + - name: current_group + type: keyword + description: The UUID of the current incident this event belongs to. + - name: details + type: flattened + description: An array of multiple sections (sub-arrays) of event information. + - name: group_by_activity + type: boolean + description: Used by pre-v5.2 legacy incident construction. Indicates whether the event should be aggregated by activity or by device to create an incident. When true, the event should be aggregated by activityID, and when false, aggregated by groupingID(s). + - name: group_category + type: keyword + description: The behavior category associated with the incident overall. Relevant for v5.2+ incident construction only. + - name: group_previous_groups + type: keyword + description: If the incident event was part of an incident which was later merged with another, the UUIDs of the incidents before they were merged. + - name: group_score + type: double + description: The current overall score of the incident this event is part of. + - name: grouping_ids + type: keyword + description: Used by pre-v5.2 legacy incident construction. Each entry in the groupingIDs array refers to a device that triggered the activity detection. In single events, should only contain one ID. If groupByActivity=false , this field should be used to group events together into an incident. + - name: id + type: keyword + description: A system field. + - name: incident_event_url + type: group + description: A URL to access the AI Analyst alert in the Threat Visualizer. + fields: + - name: domain + type: keyword + - name: extension + type: keyword + - name: fragment + type: keyword + - name: full + type: keyword + - name: original + type: keyword + - name: password + type: keyword + - name: path + type: keyword + - name: port + type: long + - name: query + type: keyword + - name: scheme + type: keyword + - name: username + type: keyword + - name: is_acknowledged + type: boolean + description: Whether the event has been acknowledged. + - name: is_external_triggered + type: boolean + description: Whether the event was created as a result of an externally triggered AI Analyst investigation. + - name: is_pinned + type: boolean + description: Whether the event, or an incident that the event is associated with, is pinned within the Threat Visualizer user interface. Pinned events will always return regardless of the timeframe specified. + - name: is_user_triggered + type: boolean + description: Whether the event was created as a result of a user-triggered AI Analyst investigation. + - name: periods + type: group + fields: + - name: end + type: date + description: A timestamp for the end of the activity period in epoch time. + - name: start + type: date + description: A timestamp for the start of the activity period in epoch time. + - name: related_breaches + type: group + fields: + - name: model_name + type: keyword + description: The name of the model that breached. + - name: pbid + type: long + description: The policy breach ID unique identifier of the model breach. + - name: threat_score + type: long + description: The breach score of the associated model breach - out of 100. + - name: timestamp + type: date + description: The timestamp at which the model breach occurred in epoch time. + - name: summariser + type: keyword + description: A system field. + - name: summary + type: keyword + description: A textual summary of the suspicious activity. This example is abbreviated. + - name: title + type: keyword + description: A title describing the activity that occurred. +- name: log.source.address + type: keyword + description: Source address from which the log event was read / sent from. diff --git a/packages/darktrace/data_stream/ai_analyst_alert/manifest.yml b/packages/darktrace/data_stream/ai_analyst_alert/manifest.yml new file mode 100644 index 00000000000..6e056b2c96d --- /dev/null +++ b/packages/darktrace/data_stream/ai_analyst_alert/manifest.yml @@ -0,0 +1,179 @@ +title: Collect AI Analyst Alert logs from Darktrace +type: logs +streams: + - input: httpjson + title: AI Analyst Alert logs + description: Collect AI Analyst Alert logs via API. + template_path: httpjson.yml.hbs + vars: + - name: initial_interval + type: text + title: Initial Interval + description: How far back to pull the AI Analyst Alert logs from Darktrace. NOTE:- Supported units for this parameter are h/m/s. + multi: false + required: true + show_user: true + default: 24h + - name: interval + type: text + title: Interval + description: Duration between requests to the Darktrace API. NOTE:- Supported units for this parameter are h/m/s. + default: 1m + multi: false + required: true + show_user: true + - name: http_client_timeout + type: text + title: HTTP Client Timeout + description: Duration before declaring that the HTTP client connection has timed out. NOTE:- Valid time units are ns, us, ms, s, m, h. + multi: false + required: true + show_user: false + default: 30s + - name: tags + type: text + title: Tags + multi: true + required: true + show_user: false + default: + - forwarded + - darktrace-ai_analyst_alert + - name: preserve_original_event + required: true + show_user: true + title: Preserve original event + description: Preserves a raw copy of the original event, added to the field `event.original`. + type: bool + multi: false + default: false + - name: preserve_duplicate_custom_fields + required: true + show_user: false + title: Preserve duplicate custom fields + description: Preserve darktrace.ai_analyst_alert fields that were copied to Elastic Common Schema (ECS) fields. + type: bool + multi: false + default: false + - name: processors + type: yaml + title: Processors + multi: false + required: false + show_user: false + description: >- + Processors are used to reduce the number of fields in the exported event or to enhance the event with metadata. This executes in the agent before the logs are parsed. See [Processors](https://www.elastic.co/guide/en/beats/filebeat/current/filtering-and-enhancing-data.html) for details. + - input: tcp + title: AI Analyst Alert logs + description: Collect AI Analyst Alert logs via TCP input. + template_path: tcp.yml.hbs + vars: + - name: listen_port + type: integer + title: Listen Port + description: The TCP port number to listen on. + multi: false + required: true + show_user: true + default: 9571 + - name: tcp_options + type: yaml + title: Custom TCP Options + multi: false + required: false + show_user: false + default: | + max_message_size: 50KiB + #max_connections: 1 + #framing: delimiter + #line_delimiter: "\n" + description: Specify custom configuration options for the TCP input. + - name: tags + type: text + title: Tags + multi: true + required: true + show_user: false + default: + - forwarded + - darktrace-ai_analyst_alert + - name: preserve_original_event + required: true + show_user: true + title: Preserve original event + description: Preserves a raw copy of the original event, added to the field `event.original`. + type: bool + multi: false + default: false + - name: preserve_duplicate_custom_fields + required: true + show_user: false + title: Preserve duplicate custom fields + description: Preserve darktrace.ai_analyst_alert fields that were copied to Elastic Common Schema (ECS) fields. + type: bool + multi: false + default: false + - name: processors + type: yaml + title: Processors + multi: false + required: false + show_user: false + description: >- + Processors are used to reduce the number of fields in the exported event or to enhance the event with metadata. This executes in the agent before the logs are parsed. See [Processors](https://www.elastic.co/guide/en/beats/filebeat/current/filtering-and-enhancing-data.html) for details. + - input: udp + title: AI Analyst Alert logs + description: Collect AI Analyst Alert logs via UDP input. + template_path: udp.yml.hbs + vars: + - name: listen_port + type: integer + title: Listen Port + description: The UDP port number to listen on. + multi: false + required: true + show_user: true + default: 9574 + - name: udp_options + type: yaml + title: Custom UDP Options + multi: false + required: false + show_user: false + default: | + max_message_size: 50KiB + #timeout: 300s + description: Specify custom configuration options for the UDP input. + - name: tags + type: text + title: Tags + multi: true + required: true + show_user: false + default: + - forwarded + - darktrace-ai_analyst_alert + - name: preserve_original_event + required: true + show_user: true + title: Preserve original event + description: Preserves a raw copy of the original event, added to the field `event.original`. + type: bool + multi: false + default: false + - name: preserve_duplicate_custom_fields + required: true + show_user: false + title: Preserve duplicate custom fields + description: Preserve darktrace.ai_analyst_alert fields that were copied to Elastic Common Schema (ECS) fields. + type: bool + multi: false + default: false + - name: processors + type: yaml + title: Processors + multi: false + required: false + show_user: false + description: >- + Processors are used to reduce the number of fields in the exported event or to enhance the event with metadata. This executes in the agent before the logs are parsed. See [Processors](https://www.elastic.co/guide/en/beats/filebeat/current/filtering-and-enhancing-data.html) for details. diff --git a/packages/darktrace/data_stream/ai_analyst_alert/sample_event.json b/packages/darktrace/data_stream/ai_analyst_alert/sample_event.json new file mode 100644 index 00000000000..fa6272b4acc --- /dev/null +++ b/packages/darktrace/data_stream/ai_analyst_alert/sample_event.json @@ -0,0 +1,241 @@ +{ + "@timestamp": "2021-08-03T14:48:09.240Z", + "agent": { + "ephemeral_id": "82482032-e103-4c45-a00e-103ac604f4ae", + "id": "95d2bc73-8bc8-47d9-b36e-a21b58255eec", + "name": "docker-fleet-agent", + "type": "filebeat", + "version": "8.2.1" + }, + "darktrace": { + "ai_analyst_alert": { + "activity_id": "abcd1234", + "aia_score": 98, + "attack_phases": [ + 5 + ], + "breach_devices": [ + { + "did": 10, + "ip": "81.2.69.144", + "sid": 12, + "subnet": "VPN" + } + ], + "category": "critical", + "children": [ + "eabcdef0-1234-1234-1234-cabcdefghij9" + ], + "created_at": "2021-08-03T14:48:09.240Z", + "current_group": "eabc1234-1234-1234-1234-cabcdefg0011", + "details": [ + [ + { + "contents": [ + { + "type": "device", + "values": [ + { + "did": 10, + "ip": "175.16.199.1", + "sid": 12, + "subnet": "VPN" + } + ] + } + ], + "header": "Breaching Device" + } + ], + [ + { + "contents": [ + { + "key": "Time", + "type": "timestampRange", + "values": [ + { + "end": 1628000141220, + "start": 1627985298683 + } + ] + }, + { + "key": "Number of unique IPs", + "type": "integer", + "values": [ + 16 + ] + }, + { + "key": "Targeted IP ranges include", + "type": "device", + "values": [ + { + "ip": "81.2.69.192" + }, + { + "ip": "175.16.199.1" + }, + { + "ip": "175.16.199.3" + } + ] + }, + { + "key": "Destination port", + "type": "integer", + "values": [ + 22 + ] + }, + { + "key": "Connection count", + "type": "integer", + "values": [ + 40 + ] + }, + { + "key": "Percentage successful", + "type": "percentage", + "values": [ + 100 + ] + } + ], + "header": "SSH Activity" + } + ] + ], + "group_by_activity": false, + "group_category": "critical", + "group_score": 72.9174234, + "grouping_ids": [ + "abcdef12" + ], + "id": "eabc0011-1234-1234-1234-cabcdefg0011", + "is_acknowledged": false, + "is_external_triggered": false, + "is_pinned": true, + "is_user_triggered": false, + "periods": [ + { + "end": "2021-08-03T14:15:41.220Z", + "start": "2021-08-03T10:08:18.683Z" + } + ], + "related_breaches": [ + { + "model_name": "Unusual Activity / Unusual Activity from Re-Activated Device", + "pbid": 1234, + "threat_score": 37, + "timestamp": "2021-08-03T13:25:57.000Z" + } + ], + "summariser": "AdminConnSummary", + "summary": "The device 175.16.199.1 was observed making unusual internal SSH connections to a wide range of devices.\n\nThough this behaviour could be the result of legitimate remote access or administration, it could also be a sign of attempted lateral movement by a compromised machine.\n\nConsequently, if this activity was not expected, the security team may wish to investigate further.", + "title": "Extensive Unusual SSH Connections" + } + }, + "data_stream": { + "dataset": "darktrace.ai_analyst_alert", + "namespace": "ep", + "type": "logs" + }, + "ecs": { + "version": "8.4.0" + }, + "elastic_agent": { + "id": "95d2bc73-8bc8-47d9-b36e-a21b58255eec", + "snapshot": false, + "version": "8.2.1" + }, + "event": { + "agent_id_status": "verified", + "category": [ + "threat" + ], + "dataset": "darktrace.ai_analyst_alert", + "duration": [ + 14842537000000 + ], + "end": [ + "2021-08-03T14:15:41.220Z" + ], + "id": "eabc0011-1234-1234-1234-cabcdefg0011", + "ingested": "2022-09-30T11:36:06Z", + "kind": "alert", + "original": "{\"summariser\":\"AdminConnSummary\",\"acknowledged\":false,\"pinned\":true,\"createdAt\":1628002089240,\"attackPhases\":[5],\"title\":\"Extensive Unusual SSH Connections\",\"id\":\"eabc0011-1234-1234-1234-cabcdefg0011\",\"children\":[\"eabcdef0-1234-1234-1234-cabcdefghij9\"],\"category\":\"critical\",\"currentGroup\":\"eabc1234-1234-1234-1234-cabcdefg0011\",\"groupCategory\":\"critical\",\"groupScore\":\"72.9174234\",\"groupPreviousGroups\":null,\"activityId\":\"abcd1234\",\"groupingIds\":[\"abcdef12\"],\"groupByActivity\":false,\"userTriggered\":false,\"externalTriggered\":false,\"aiaScore\":98,\"summary\":\"The device 175.16.199.1 was observed making unusual internal SSH connections to a wide range of devices.\\n\\nThough this behaviour could be the result of legitimate remote access or administration, it could also be a sign of attempted lateral movement by a compromised machine.\\n\\nConsequently, if this activity was not expected, the security team may wish to investigate further.\",\"periods\":[{\"start\":1627985298683,\"end\":1628000141220}],\"breachDevices\":[{\"identifier\":null,\"hostname\":null,\"ip\":\"81.2.69.144\",\"mac\":null,\"subnet\":\"VPN\",\"did\":10,\"sid\":12}],\"relatedBreaches\":[{\"modelName\":\"Unusual Activity / Unusual Activity from Re-Activated Device\",\"pbid\":1234,\"threatScore\":37,\"timestamp\":1627997157000}],\"details\":[[{\"header\":\"Breaching Device\",\"contents\":[{\"key\":null,\"type\":\"device\",\"values\":[{\"identifier\":null,\"hostname\":null,\"ip\":\"175.16.199.1\",\"mac\":null,\"subnet\":\"VPN\",\"did\":10,\"sid\":12}]}]}],[{\"header\":\"SSH Activity\",\"contents\":[{\"key\":\"Time\",\"type\":\"timestampRange\",\"values\":[{\"start\":1627985298683,\"end\":1628000141220}]},{\"key\":\"Number of unique IPs\",\"type\":\"integer\",\"values\":[16]},{\"key\":\"Targeted IP ranges include\",\"type\":\"device\",\"values\":[{\"identifier\":null,\"hostname\":null,\"ip\":\"81.2.69.192\",\"mac\":null,\"subnet\":null,\"did\":null,\"sid\":null},{\"identifier\":null,\"hostname\":null,\"ip\":\"175.16.199.1\",\"mac\":null,\"subnet\":null,\"did\":null,\"sid\":null},{\"identifier\":null,\"hostname\":null,\"ip\":\"175.16.199.3\",\"mac\":null,\"subnet\":null,\"did\":null,\"sid\":null}]},{\"key\":\"Destination port\",\"type\":\"integer\",\"values\":[22]},{\"key\":\"Connection count\",\"type\":\"integer\",\"values\":[40]},{\"key\":\"Percentage successful\",\"type\":\"percentage\",\"values\":[100]}]}]]}", + "reason": "Extensive Unusual SSH Connections", + "risk_score": 98, + "risk_score_norm": 98, + "start": [ + "2021-08-03T10:08:18.683Z" + ], + "type": [ + "info" + ] + }, + "host": { + "id": [ + "10" + ], + "ip": [ + "81.2.69.144" + ] + }, + "input": { + "type": "udp" + }, + "log": { + "source": { + "address": "192.168.128.5:49066" + }, + "syslog": { + "facility": { + "code": 20, + "name": "local4" + }, + "hostname": "example.cloud.darktrace.com", + "priority": 165, + "severity": { + "code": 5, + "name": "Notice" + }, + "version": "1" + } + }, + "message": "The device 175.16.199.1 was observed making unusual internal SSH connections to a wide range of devices.\n\nThough this behaviour could be the result of legitimate remote access or administration, it could also be a sign of attempted lateral movement by a compromised machine.\n\nConsequently, if this activity was not expected, the security team may wish to investigate further.", + "related": { + "ip": [ + "81.2.69.144", + "175.16.199.1", + "81.2.69.192", + "175.16.199.3" + ] + }, + "rule": { + "name": [ + "Unusual Activity / Unusual Activity from Re-Activated Device" + ] + }, + "tags": [ + "preserve_original_event", + "preserve_duplicate_custom_fields", + "forwarded", + "darktrace-ai_analyst_alert" + ], + "threat": { + "enrichments": { + "matched": { + "id": [ + "eabcdef0-1234-1234-1234-cabcdefghij9" + ] + } + }, + "group": { + "id": "eabc1234-1234-1234-1234-cabcdefg0011" + } + } +} \ No newline at end of file diff --git a/packages/darktrace/data_stream/model_breach_alert/_dev/test/pipeline/test-common-config.yml b/packages/darktrace/data_stream/model_breach_alert/_dev/test/pipeline/test-common-config.yml new file mode 100644 index 00000000000..be41bb0d476 --- /dev/null +++ b/packages/darktrace/data_stream/model_breach_alert/_dev/test/pipeline/test-common-config.yml @@ -0,0 +1,4 @@ +fields: + tags: + - preserve_original_event + - preserve_duplicate_custom_fields diff --git a/packages/darktrace/data_stream/model_breach_alert/_dev/test/pipeline/test-model-breach-alert.log b/packages/darktrace/data_stream/model_breach_alert/_dev/test/pipeline/test-model-breach-alert.log new file mode 100644 index 00000000000..25b22d1a876 --- /dev/null +++ b/packages/darktrace/data_stream/model_breach_alert/_dev/test/pipeline/test-model-breach-alert.log @@ -0,0 +1,5 @@ +{"model":{"name":"System::System","pid":802,"phid":802,"uuid":"8abcdefh-1234-1234-1234-5abababab","logic":{"data":[1594],"type":"componentList","version":1},"throttle":10,"sharedEndpoints":false,"actions":{"alert":true,"antigena":{},"breach":true,"model":true,"setPriority":false,"setTag":false,"setType":false},"tags":[],"interval":0,"delay":0,"sequenced":true,"active":true,"modified":"2022-07-11 11:41:08","activeTimes":{"devices":{},"tags":{},"type":"exclusions","version":2},"autoUpdatable":true,"autoUpdate":true,"autoSuppress":true,"description":"An issue with the system has been detected. This system alert is generated for system information that may merit further investigation. This may be due to things like probes failing to connect.\\n\\nAction: Review the system message. Use the status page to see additional system information that may help with diagnostics.","behaviour":"decreasing","defeats":[],"created":{"by":"System"},"edited":{"by":"Nobody"},"version":16,"priority":3,"category":"Informational","compliance":false},"device":{"did":-1},"triggeredComponents":[{"time":1657678365000,"cbid":5,"cid":1594,"chid":1594,"size":1,"threshold":0,"interval":3600,"logic":{"data":{"left":{"left":"A","operator":"AND","right":"B"},"operator":"OR","right":{"left":{"left":"A","operator":"AND","right":"C"},"operator":"OR","right":{"left":{"left":"A","operator":"AND","right":"D"},"operator":"OR","right":{"left":{"left":"A","operator":"AND","right":"E"},"operator":"OR","right":{"left":"A","operator":"AND","right":"F"}}}}},"version":"v0.1"},"metric":{"mlid":207,"name":"dtsystem","label":"System"},"triggeredFilters":[{"cfid":18136,"id":"A","filterType":"Event details","arguments":{"value":"analyze credential ignore list"},"comparatorType":"does not contain","trigger":{"value":"Probe 175.16.199.1 last contact was 18 minutes ago"}},{"cfid":18137,"id":"B","filterType":"System message","arguments":{"value":"Probe error"},"comparatorType":"is","trigger":{"value":"Probe error"}},{"cfid":18142,"id":"d1","filterType":"Event details","arguments":{},"comparatorType":"display","trigger":{"value":"Probe 175.16.199.1 last contact was 18 minutes ago"}},{"cfid":18143,"id":"d2","filterType":"System message","arguments":{},"comparatorType":"display","trigger":{"value":"Probe error"}}]}],"breachUrl":"https://example.com/#modelbreach/5","pbid":5,"score":0.674,"commentCount":0,"creationTime":1657678365000,"time":1657678366000,"mitreTechniques":[]} +{"model":{"name":"Device::Attack and Recon Tools","pid":135,"phid":1198,"uuid":"8abcdefg-1234-1234-1234-5abcdefg12","logic":{"data":[{"cid":2311,"weight":1},{"cid":2312,"weight":1},{"cid":2315,"weight":1},{"cid":2316,"weight":1},{"cid":2314,"weight":1},{"cid":2310,"weight":1},{"cid":2313,"weight":1}],"targetScore":1,"type":"weightedComponentList","version":1},"throttle":604800,"sharedEndpoints":false,"actions":{"alert":true,"antigena":{},"breach":true,"model":true,"setPriority":false,"setTag":false,"setType":false},"tags":["AP: Internal Recon","OT Engineer"],"interval":3600,"delay":0,"sequenced":false,"active":true,"modified":"2022-07-11 11:47:51","activeTimes":{"devices":{},"tags":{},"type":"exclusions","version":2},"autoUpdatable":true,"autoUpdate":true,"autoSuppress":true,"description":"A device is using common penetration testing tools.\\n\\nAction: Review the device to see if it a security device, these can be tagged as such to exclude them from future breaches. Activity from non security devices merit further investigation into what else the device is doing and could be a significant risk within the network.","behaviour":"decreasing","created":{"by":"System"},"edited":{"by":"System"},"version":77,"priority":4,"category":"Suspicious","compliance":false},"device":{"did":7,"ip":"81.2.69.192","ips":[{"ip":"175.16.199.2","timems":1657746000000,"time":"2022-07-13 21:00:00","sid":1}],"sid":1,"hostname":"175.16.199.1","firstSeen":1657544891000,"lastSeen":1657748638000,"typename":"desktop","typelabel":"Desktop","credentials":["dummy"],"tags":[{"tid":66,"expiry":0,"thid":66,"name":"Domain Authenticated","restricted":false,"data":{"auto":false,"color":183,"description":""},"isReferenced":true},{"tid":29,"expiry":0,"thid":29,"name":"Linux","restricted":false,"data":{"auto":false,"color":168,"description":"","visibility":"Public"},"isReferenced":true},{"tid":45,"expiry":0,"thid":45,"name":"New Device","restricted":false,"data":{"auto":false,"color":130,"description":"","visibility":"Public"},"isReferenced":true}]},"triggeredComponents":[{"time":1657748807000,"cbid":6,"cid":2311,"chid":2676,"size":1,"threshold":0,"interval":300,"logic":{"data":{"left":{"left":"A","operator":"AND","right":{"left":"H","operator":"AND","right":{"left":"J","operator":"AND","right":"N"}}},"operator":"OR","right":{"left":{"left":"B","operator":"AND","right":{"left":"H","operator":"AND","right":{"left":"J","operator":"AND","right":{"left":"N","operator":"AND","right":"O"}}}},"operator":"OR","right":{"left":{"left":"C","operator":"AND","right":{"left":"H","operator":"AND","right":{"left":"J","operator":"AND","right":"N"}}},"operator":"OR","right":{"left":{"left":"E","operator":"AND","right":{"left":"H","operator":"AND","right":{"left":"J","operator":"AND","right":"N"}}},"operator":"OR","right":{"left":{"left":"F","operator":"AND","right":{"left":"H","operator":"AND","right":{"left":"J","operator":"AND","right":"N"}}},"operator":"OR","right":{"left":{"left":"G","operator":"AND","right":{"left":"H","operator":"AND","right":{"left":"J","operator":"AND","right":"N"}}},"operator":"OR","right":{"left":{"left":"H","operator":"AND","right":{"left":"I","operator":"AND","right":{"left":"J","operator":"AND","right":"N"}}},"operator":"OR","right":{"left":{"left":"H","operator":"AND","right":{"left":"J","operator":"AND","right":{"left":"K","operator":"AND","right":"N"}}},"operator":"OR","right":{"left":{"left":"H","operator":"AND","right":{"left":"J","operator":"AND","right":{"left":"L","operator":"AND","right":"N"}}},"operator":"OR","right":{"left":{"left":"H","operator":"AND","right":{"left":"J","operator":"AND","right":{"left":"M","operator":"AND","right":"N"}}},"operator":"OR","right":{"left":"H","operator":"AND","right":{"left":"J","operator":"AND","right":{"left":"N","operator":"AND","right":"P"}}}}}}}}}}}}},"version":"v0.1"},"metric":{"mlid":16,"name":"connections","label":"Connections"},"triggeredFilters":[{"cfid":26781,"id":"H","filterType":"Direction","arguments":{"value":"out"},"comparatorType":"is","trigger":{"value":"out"}},{"cfid":26783,"id":"J","filterType":"Tagged internal source","arguments":{"value":12},"comparatorType":"does not have tag","trigger":{"value":"12","tag":{"tid":12,"expiry":0,"thid":12,"name":"Security Device","restricted":false,"data":{"auto":false,"color":55,"description":"","visibility":"Public"},"isReferenced":true}}},{"cfid":26787,"id":"N","filterType":"Tagged internal destination","arguments":{"value":12},"comparatorType":"does not have tag","trigger":{"value":"12","tag":{"tid":12,"expiry":0,"thid":12,"name":"Security Device","restricted":false,"data":{"auto":false,"color":55,"description":"","visibility":"Public"},"isReferenced":true}}},{"cfid":26788,"id":"O","filterType":"User agent","arguments":{"value":"examples"},"comparatorType":"does not match regular expression","trigger":{"value":""}},{"cfid":26789,"id":"P","filterType":"URI","arguments":{"value":"examples"},"comparatorType":"matches regular expression","trigger":{"value":""}},{"cfid":26790,"id":"d1","filterType":"Proxied connection","arguments":{"value":"true"},"comparatorType":"display","trigger":{"value":"false"}},{"cfid":26791,"id":"d10","filterType":"HTTP response code","arguments":{},"comparatorType":"display","trigger":{"value":"0"}},{"cfid":26792,"id":"d2","filterType":"HTTP referrer","arguments":{},"comparatorType":"display","trigger":{"value":""}},{"cfid":26793,"id":"d3","filterType":"HTTP method","arguments":{},"comparatorType":"display","trigger":{"value":"GET"}},{"cfid":26794,"id":"d4","filterType":"HTTP X-Forwarded-For","arguments":{},"comparatorType":"display","trigger":{"value":""}},{"cfid":26795,"id":"d5","filterType":"URI","arguments":{},"comparatorType":"display","trigger":{"value":""}},{"cfid":26796,"id":"d6","filterType":"User agent","arguments":{},"comparatorType":"display","trigger":{"value":""}},{"cfid":26797,"id":"d7","filterType":"Destination IP","arguments":{},"comparatorType":"display","trigger":{"value":"81.2.69.144"}},{"cfid":26798,"id":"d8","filterType":"Internal destination device name","arguments":{},"comparatorType":"display","trigger":{"value":"localhost.local"}},{"cfid":26799,"id":"d9","filterType":"Connection hostname","arguments":{},"comparatorType":"display","trigger":{"value":""}}]}],"breachUrl":"https://example.com/#modelbreach/6","pbid":6,"score":0.871,"commentCount":0,"creationTime":1657748815000,"time":1657748808000,"mitreTechniques":[{"technique":"Hardware Additions Mitigation","techniqueID":"T1200"}]} +{"commentCount":0,"pbid":1,"time":1657544649000,"creationTime":1657544659000,"aianalystData":[{"uuid":"1234abcd-1234-1234-1234-123456abcdef","related":[1],"summariser":"BeaconSummary"}],"model":{"name":"Compromise::Beaconing Activity To External Rare","pid":156,"phid":1072,"uuid":"1234abcd-1234-1234-1234-123456abcdef","logic":{"data":[{"cid":2026,"weight":1},{"cid":2024,"weight":1},{"cid":2025,"weight":-100}],"targetScore":1,"type":"weightedComponentList","version":1},"throttle":10800,"sharedEndpoints":false,"actions":{"alert":true,"antigena":{},"breach":true,"model":true,"setPriority":false,"setTag":false,"setType":false},"tags":["AP: C2 Comms"],"interval":10800,"delay":0,"sequenced":false,"active":true,"modified":"2022-07-11 11:47:37","activeTimes":{"devices":{},"tags":{},"type":"exclusions","version":2},"autoUpdatable":true,"autoUpdate":true,"autoSuppress":true,"description":"A device has been repeatedly connecting to a rare external location with a beacon score. A beacon score is added when Darktrace identifies that a device is regularly communicating with an endpoint, for example, if a device connects to a rare external endpoint every 12 minutes this would get a beacon score. This model is designed to identify beaconing at a lower threshold and be protocol agnostic.\\n\\nAction: Review the external domains and IPs being connected to to see if they are legitimate and would be expected for business purposes.","behaviour":"incdec1","created":{"by":"System"},"edited":{"by":"System"},"version":23,"priority":2,"category":"Informational","compliance":false},"triggeredComponents":[{"time":1657544648000,"cbid":1,"cid":2026,"chid":2113,"size":11,"threshold":10,"interval":3600,"logic":{"data":{"left":{"left":"A","operator":"AND","right":{"left":"AA","operator":"AND","right":{"left":"AC","operator":"AND","right":{"left":"AD","operator":"AND","right":{"left":"AF","operator":"AND","right":{"left":"AG","operator":"AND","right":{"left":"AH","operator":"AND","right":{"left":"B","operator":"AND","right":{"left":"C","operator":"AND","right":{"left":"D","operator":"AND","right":{"left":"E","operator":"AND","right":{"left":"H","operator":"AND","right":{"left":"I","operator":"AND","right":{"left":"J","operator":"AND","right":{"left":"K","operator":"AND","right":{"left":"L","operator":"AND","right":{"left":"M","operator":"AND","right":{"left":"N","operator":"AND","right":{"left":"O","operator":"AND","right":{"left":"P","operator":"AND","right":{"left":"S","operator":"AND","right":{"left":"U","operator":"AND","right":{"left":"V","operator":"AND","right":{"left":"X","operator":"AND","right":{"left":"Y","operator":"AND","right":"Z"}}}}}}}}}}}}}}}}}}}}}}}}},"operator":"OR","right":{"left":"A","operator":"AND","right":{"left":"AA","operator":"AND","right":{"left":"AB","operator":"AND","right":{"left":"AE","operator":"AND","right":{"left":"AF","operator":"AND","right":{"left":"AG","operator":"AND","right":{"left":"AH","operator":"AND","right":{"left":"C","operator":"AND","right":{"left":"D","operator":"AND","right":{"left":"E","operator":"AND","right":{"left":"H","operator":"AND","right":{"left":"I","operator":"AND","right":{"left":"J","operator":"AND","right":{"left":"K","operator":"AND","right":{"left":"L","operator":"AND","right":{"left":"M","operator":"AND","right":{"left":"N","operator":"AND","right":{"left":"O","operator":"AND","right":{"left":"P","operator":"AND","right":{"left":"S","operator":"AND","right":{"left":"U","operator":"AND","right":{"left":"V","operator":"AND","right":{"left":"X","operator":"AND","right":{"left":"Y","operator":"AND","right":"Z"}}}}}}}}}}}}}}}}}}}}}}}}},"version":"v0.1"},"metric":{"mlid":1,"name":"externalconnections","label":"External Connections"},"triggeredFilters":[{"cfid":23426,"id":"A","filterType":"Beaconing score","arguments":{"value":60},"comparatorType":">","trigger":{"value":"100"}},{"cfid":23427,"id":"AA","filterType":"Individual size up","arguments":{"value":0},"comparatorType":">","trigger":{"value":"4382"}},{"cfid":23428,"id":"AB","filterType":"Rare domain","arguments":{"value":95},"comparatorType":">","trigger":{"value":"100"}},{"cfid":23430,"id":"AD","filterType":"Age of destination","arguments":{"value":1209600},"comparatorType":"<","trigger":{"value":"558"}},{"cfid":23431,"id":"AE","filterType":"Age of external hostname","arguments":{"value":1209600},"comparatorType":"<","trigger":{"value":"558"}},{"cfid":23432,"id":"AF","filterType":"Connection hostname","arguments":{"value":"examples"},"comparatorType":"does not match regular expression","trigger":{"value":"example.com"}},{"cfid":23433,"id":"AG","filterType":"ASN","arguments":{"value":"examples"},"comparatorType":"does not match regular expression","trigger":{"value":"AS12345 LOCAL-02"}},{"cfid":23434,"id":"AH","filterType":"JA3 hash","arguments":{"value":"5d41402abc4b2a76b9719d911017c592"},"comparatorType":"does not match","trigger":{"value":"5d41402abc4b2a76b9719d911017c592"}},{"cfid":23435,"id":"B","filterType":"Rare external IP","arguments":{"value":95},"comparatorType":">","trigger":{"value":"100"}},{"cfid":23436,"id":"C","filterType":"Application protocol","arguments":{"value":"1003"},"comparatorType":"is not","trigger":{"value":"1004"}},{"cfid":23437,"id":"D","filterType":"Destination port","arguments":{"value":53},"comparatorType":"!=","trigger":{"value":"443"}},{"cfid":23438,"id":"E","filterType":"Direction","arguments":{"value":"out"},"comparatorType":"is","trigger":{"value":"out"}},{"cfid":23439,"id":"H","filterType":"Destination port","arguments":{"value":137},"comparatorType":"!=","trigger":{"value":"443"}},{"cfid":23440,"id":"I","filterType":"Destination port","arguments":{"value":161},"comparatorType":"!=","trigger":{"value":"443"}},{"cfid":23441,"id":"J","filterType":"Protocol","arguments":{"value":"6"},"comparatorType":"is","trigger":{"value":"6"}},{"cfid":23442,"id":"K","filterType":"ASN","arguments":{"value":"Company"},"comparatorType":"does not contain","trigger":{"value":"AS12345 LOCAL-02"}},{"cfid":23443,"id":"L","filterType":"ASN","arguments":{"value":"Company"},"comparatorType":"does not contain","trigger":{"value":"AS12345 LOCAL-02"}},{"cfid":23444,"id":"M","filterType":"Internal source device type","arguments":{"value":"13"},"comparatorType":"is not","trigger":{"value":"6"}},{"cfid":23445,"id":"N","filterType":"Internal source device type","arguments":{"value":"5"},"comparatorType":"is not","trigger":{"value":"6"}},{"cfid":23446,"id":"O","filterType":"Internal source device type","arguments":{"value":"9"},"comparatorType":"is not","trigger":{"value":"6"}},{"cfid":23447,"id":"P","filterType":"Internal source device type","arguments":{"value":"12"},"comparatorType":"is not","trigger":{"value":"6"}},{"cfid":23448,"id":"S","filterType":"Internal source device type","arguments":{"value":"30"},"comparatorType":"is not","trigger":{"value":"6"}},{"cfid":23449,"id":"U","filterType":"Internal source device type","arguments":{"value":"4"},"comparatorType":"is not","trigger":{"value":"6"}},{"cfid":23450,"id":"V","filterType":"Internal source device type","arguments":{"value":"3"},"comparatorType":"is not","trigger":{"value":"6"}},{"cfid":23451,"id":"X","filterType":"Trusted hostname","arguments":{"value":"false"},"comparatorType":"is","trigger":{"value":"false"}},{"cfid":23452,"id":"Y","filterType":"Tagged internal source","arguments":{"value":26},"comparatorType":"does not have tag","trigger":{"value":"26","tag":{"tid":26,"expiry":0,"thid":26,"name":"No Device Tracking","restricted":false,"data":{"auto":false,"color":5,"description":"","visibility":"Public"},"isReferenced":true}}},{"cfid":23453,"id":"Z","filterType":"Individual size down","arguments":{"value":0},"comparatorType":">","trigger":{"value":"5862"}},{"cfid":23454,"id":"d1","filterType":"JA3 hash","arguments":{},"comparatorType":"display","trigger":{"value":"5d41402abc4b2a76b9719d911017c592"}},{"cfid":23455,"id":"d2","filterType":"ASN","arguments":{},"comparatorType":"display","trigger":{"value":"AS12345 LOCAL-02"}},{"cfid":23456,"id":"d3","filterType":"Destination IP","arguments":{},"comparatorType":"display","trigger":{"value":"81.2.69.192"}},{"cfid":23457,"id":"d4","filterType":"Connection hostname","arguments":{},"comparatorType":"display","trigger":{"value":"example.com"}}]}],"score":0.674,"device":{"did":3,"ip":"81.2.69.142","sid":1,"firstSeen":1657544089000,"lastSeen":1657544418000,"typename":"desktop","typelabel":"Desktop"}} +{"model":{"name":"Unrestricted Test Model"},"device":{"ip":"175.16.199.1","hostname":"test-device.example.com","macaddress":"00:00:5e:00:53:00","vendor":"Test Vendor","label":"Test Device"},"triggeredComponents":[{"metric":{"label":"Test Metric"},"triggeredFilters":[{"comparatorType":"display","filterType":"Test Metric Filter","trigger":{"value":"Test filter value"}}]}],"breachUrl":"","pbid":123,"score":1,"creationTime":1659692145460,"time":1659692145460,"mitreTechniques":[]} +darktrace {"model":{"description":"Test model used for testing alerting configuration.","created":{"by":"System"},"edited":{"by":"Nobody"},"name":"Unrestricted Test Model","priority":5},"device":{"ip":"175.16.199.1","hostname":"test-device.example.com","macaddress":"00:00:5e:00:53:00","vendor":"Test Vendor","label":"Test Device"},"triggeredComponents":[{"metric":{"label":"Test Metric"},"triggeredFilters":[{"comparatorType":"display","filterType":"Test Metric Filter","trigger":{"value":"Test filter value"}}]}],"breachUrl":"","pbid":123,"score":1,"creationTime":1659692086317,"time":1659692086317,"mitreTechniques":[]} diff --git a/packages/darktrace/data_stream/model_breach_alert/_dev/test/pipeline/test-model-breach-alert.log-expected.json b/packages/darktrace/data_stream/model_breach_alert/_dev/test/pipeline/test-model-breach-alert.log-expected.json new file mode 100644 index 00000000000..25d25301556 --- /dev/null +++ b/packages/darktrace/data_stream/model_breach_alert/_dev/test/pipeline/test-model-breach-alert.log-expected.json @@ -0,0 +1,1285 @@ +{ + "expected": [ + { + "@timestamp": "2022-07-13T02:12:46.000Z", + "darktrace": { + "model_breach_alert": { + "breach_url": { + "domain": "example.com", + "fragment": "modelbreach/5", + "original": "https://example.com/#modelbreach/5", + "path": "/", + "scheme": "https" + }, + "comment": { + "count": 0 + }, + "creation_time": "2022-07-13T02:12:45.000Z", + "model": { + "actions": { + "is_alerting": true, + "is_breach": true, + "is_priority_set": false, + "is_tag_set": false, + "is_type_set": false, + "model": true + }, + "active_times": { + "type": "exclusions", + "version": 2 + }, + "behaviour": "decreasing", + "category": "Informational", + "created": { + "by": "System" + }, + "delay": 0, + "description": "An issue with the system has been detected. This system alert is generated for system information that may merit further investigation. This may be due to things like probes failing to connect.\\n\\nAction: Review the system message. Use the status page to see additional system information that may help with diagnostics.", + "edited": { + "by": "Nobody" + }, + "in_compliance_behavior_category": false, + "interval": 0, + "is_active": true, + "is_auto_suppress": true, + "is_auto_updatable": true, + "is_auto_update": true, + "is_sequenced": true, + "is_shared_endpoints": false, + "logic": { + "data_component_list": [ + 1594 + ], + "type": "componentList", + "version": 1 + }, + "modified": "2022-07-11T11:41:08.000Z", + "name": "System::System", + "phid": 802, + "pid": 802, + "priority": 3, + "throttle": 10, + "uuid": "8abcdefh-1234-1234-1234-5abababab", + "version": 16 + }, + "pbid": 5, + "score": 0.674, + "time": "2022-07-13T02:12:46.000Z", + "triggered_components": [ + { + "cbid": 5, + "chid": 1594, + "cid": 1594, + "interval": 3600, + "logic": { + "data": "{left={left=A, right=B, operator=AND}, right={left={left=A, right=C, operator=AND}, right={left={left=A, right=D, operator=AND}, right={left={left=A, right=E, operator=AND}, right={left=A, right=F, operator=AND}, operator=OR}, operator=OR}, operator=OR}, operator=OR}", + "version": "v0.1" + }, + "metric": { + "label": "System", + "mlid": 207, + "name": "dtsystem" + }, + "size": 1, + "threshold": 0, + "time": "2022-07-13T02:12:45.000Z", + "triggered_filters": [ + { + "arguments": { + "value": "analyze credential ignore list" + }, + "cfid": 18136, + "comparator_type": "does not contain", + "filter_type": "Event details", + "id": "A", + "trigger": { + "value": "Probe 175.16.199.1 last contact was 18 minutes ago" + } + }, + { + "arguments": { + "value": "Probe error" + }, + "cfid": 18137, + "comparator_type": "is", + "filter_type": "System message", + "id": "B", + "trigger": { + "value": "Probe error" + } + }, + { + "cfid": 18142, + "comparator_type": "display", + "filter_type": "Event details", + "id": "d1", + "trigger": { + "value": "Probe 175.16.199.1 last contact was 18 minutes ago" + } + }, + { + "cfid": 18143, + "comparator_type": "display", + "filter_type": "System message", + "id": "d2", + "trigger": { + "value": "Probe error" + } + } + ] + } + ] + } + }, + "ecs": { + "version": "8.4.0" + }, + "event": { + "created": "2022-07-13T02:12:45.000Z", + "kind": "event", + "original": "{\"model\":{\"name\":\"System::System\",\"pid\":802,\"phid\":802,\"uuid\":\"8abcdefh-1234-1234-1234-5abababab\",\"logic\":{\"data\":[1594],\"type\":\"componentList\",\"version\":1},\"throttle\":10,\"sharedEndpoints\":false,\"actions\":{\"alert\":true,\"antigena\":{},\"breach\":true,\"model\":true,\"setPriority\":false,\"setTag\":false,\"setType\":false},\"tags\":[],\"interval\":0,\"delay\":0,\"sequenced\":true,\"active\":true,\"modified\":\"2022-07-11 11:41:08\",\"activeTimes\":{\"devices\":{},\"tags\":{},\"type\":\"exclusions\",\"version\":2},\"autoUpdatable\":true,\"autoUpdate\":true,\"autoSuppress\":true,\"description\":\"An issue with the system has been detected. This system alert is generated for system information that may merit further investigation. This may be due to things like probes failing to connect.\\\\n\\\\nAction: Review the system message. Use the status page to see additional system information that may help with diagnostics.\",\"behaviour\":\"decreasing\",\"defeats\":[],\"created\":{\"by\":\"System\"},\"edited\":{\"by\":\"Nobody\"},\"version\":16,\"priority\":3,\"category\":\"Informational\",\"compliance\":false},\"device\":{\"did\":-1},\"triggeredComponents\":[{\"time\":1657678365000,\"cbid\":5,\"cid\":1594,\"chid\":1594,\"size\":1,\"threshold\":0,\"interval\":3600,\"logic\":{\"data\":{\"left\":{\"left\":\"A\",\"operator\":\"AND\",\"right\":\"B\"},\"operator\":\"OR\",\"right\":{\"left\":{\"left\":\"A\",\"operator\":\"AND\",\"right\":\"C\"},\"operator\":\"OR\",\"right\":{\"left\":{\"left\":\"A\",\"operator\":\"AND\",\"right\":\"D\"},\"operator\":\"OR\",\"right\":{\"left\":{\"left\":\"A\",\"operator\":\"AND\",\"right\":\"E\"},\"operator\":\"OR\",\"right\":{\"left\":\"A\",\"operator\":\"AND\",\"right\":\"F\"}}}}},\"version\":\"v0.1\"},\"metric\":{\"mlid\":207,\"name\":\"dtsystem\",\"label\":\"System\"},\"triggeredFilters\":[{\"cfid\":18136,\"id\":\"A\",\"filterType\":\"Event details\",\"arguments\":{\"value\":\"analyze credential ignore list\"},\"comparatorType\":\"does not contain\",\"trigger\":{\"value\":\"Probe 175.16.199.1 last contact was 18 minutes ago\"}},{\"cfid\":18137,\"id\":\"B\",\"filterType\":\"System message\",\"arguments\":{\"value\":\"Probe error\"},\"comparatorType\":\"is\",\"trigger\":{\"value\":\"Probe error\"}},{\"cfid\":18142,\"id\":\"d1\",\"filterType\":\"Event details\",\"arguments\":{},\"comparatorType\":\"display\",\"trigger\":{\"value\":\"Probe 175.16.199.1 last contact was 18 minutes ago\"}},{\"cfid\":18143,\"id\":\"d2\",\"filterType\":\"System message\",\"arguments\":{},\"comparatorType\":\"display\",\"trigger\":{\"value\":\"Probe error\"}}]}],\"breachUrl\":\"https://example.com/#modelbreach/5\",\"pbid\":5,\"score\":0.674,\"commentCount\":0,\"creationTime\":1657678365000,\"time\":1657678366000,\"mitreTechniques\":[]}", + "risk_score": 0.674, + "risk_score_norm": 67.4, + "severity": 3, + "start": [ + "2022-07-13T02:12:45.000Z" + ], + "type": [ + "info" + ], + "url": "https://example.com/#modelbreach/5" + }, + "related": { + "user": [ + "System", + "Nobody" + ] + }, + "rule": { + "author": "System", + "category": "Informational", + "description": "An issue with the system has been detected. This system alert is generated for system information that may merit further investigation. This may be due to things like probes failing to connect.\\n\\nAction: Review the system message. Use the status page to see additional system information that may help with diagnostics.", + "name": "System::System", + "uuid": "8abcdefh-1234-1234-1234-5abababab", + "version": "16" + }, + "tags": [ + "preserve_original_event", + "preserve_duplicate_custom_fields" + ] + }, + { + "@timestamp": "2022-07-13T21:46:48.000Z", + "darktrace": { + "model_breach_alert": { + "breach_url": { + "domain": "example.com", + "fragment": "modelbreach/6", + "original": "https://example.com/#modelbreach/6", + "path": "/", + "scheme": "https" + }, + "comment": { + "count": 0 + }, + "creation_time": "2022-07-13T21:46:55.000Z", + "device": { + "credentials": [ + "dummy" + ], + "did": 7, + "first_seen": "2022-07-11T13:08:11.000Z", + "hostname": "175.16.199.1", + "ip": "81.2.69.192", + "ips": [ + { + "ip": "175.16.199.2", + "sid": 1, + "time": "2022-07-13T21:00:00.000Z", + "timems": "2022-07-13T21:00:00.000Z" + } + ], + "last_seen": "2022-07-13T21:43:58.000Z", + "sid": 1, + "tags": [ + { + "data": { + "auto": false, + "color": 183 + }, + "expiry": 0, + "is_referenced": true, + "name": "Domain Authenticated", + "restricted": false, + "thid": 66, + "tid": 66 + }, + { + "data": { + "auto": false, + "color": 168, + "visibility": "Public" + }, + "expiry": 0, + "is_referenced": true, + "name": "Linux", + "restricted": false, + "thid": 29, + "tid": 29 + }, + { + "data": { + "auto": false, + "color": 130, + "visibility": "Public" + }, + "expiry": 0, + "is_referenced": true, + "name": "New Device", + "restricted": false, + "thid": 45, + "tid": 45 + } + ], + "type_label": "Desktop", + "type_name": "desktop" + }, + "mitre_techniques": [ + { + "id": "T1200", + "name": "Hardware Additions Mitigation" + } + ], + "model": { + "actions": { + "is_alerting": true, + "is_breach": true, + "is_priority_set": false, + "is_tag_set": false, + "is_type_set": false, + "model": true + }, + "active_times": { + "type": "exclusions", + "version": 2 + }, + "behaviour": "decreasing", + "category": "Suspicious", + "created": { + "by": "System" + }, + "delay": 0, + "description": "A device is using common penetration testing tools.\\n\\nAction: Review the device to see if it a security device, these can be tagged as such to exclude them from future breaches. Activity from non security devices merit further investigation into what else the device is doing and could be a significant risk within the network.", + "edited": { + "by": "System" + }, + "in_compliance_behavior_category": false, + "interval": 3600, + "is_active": true, + "is_auto_suppress": true, + "is_auto_updatable": true, + "is_auto_update": true, + "is_sequenced": false, + "is_shared_endpoints": false, + "logic": { + "data_weighted_component_list": [ + { + "cid": 2311, + "weight": 1 + }, + { + "cid": 2312, + "weight": 1 + }, + { + "cid": 2315, + "weight": 1 + }, + { + "cid": 2316, + "weight": 1 + }, + { + "cid": 2314, + "weight": 1 + }, + { + "cid": 2310, + "weight": 1 + }, + { + "cid": 2313, + "weight": 1 + } + ], + "target_score": 1, + "type": "weightedComponentList", + "version": 1 + }, + "modified": "2022-07-11T11:47:51.000Z", + "name": "Device::Attack and Recon Tools", + "phid": 1198, + "pid": 135, + "priority": 4, + "tags": [ + "AP: Internal Recon", + "OT Engineer" + ], + "throttle": 604800, + "uuid": "8abcdefg-1234-1234-1234-5abcdefg12", + "version": 77 + }, + "pbid": 6, + "score": 0.871, + "time": "2022-07-13T21:46:48.000Z", + "triggered_components": [ + { + "cbid": 6, + "chid": 2676, + "cid": 2311, + "interval": 300, + "logic": { + "data": "{left={left=A, right={left=H, right={left=J, right=N, operator=AND}, operator=AND}, operator=AND}, right={left={left=B, right={left=H, right={left=J, right={left=N, right=O, operator=AND}, operator=AND}, operator=AND}, operator=AND}, right={left={left=C, right={left=H, right={left=J, right=N, operator=AND}, operator=AND}, operator=AND}, right={left={left=E, right={left=H, right={left=J, right=N, operator=AND}, operator=AND}, operator=AND}, right={left={left=F, right={left=H, right={left=J, right=N, operator=AND}, operator=AND}, operator=AND}, right={left={left=G, right={left=H, right={left=J, right=N, operator=AND}, operator=AND}, operator=AND}, right={left={left=H, right={left=I, right={left=J, right=N, operator=AND}, operator=AND}, operator=AND}, right={left={left=H, right={left=J, right={left=K, right=N, operator=AND}, operator=AND}, operator=AND}, right={left={left=H, right={left=J, right={left=L, right=N, operator=AND}, operator=AND}, operator=AND}, right={left={left=H, right={left=J, right={left=M, right=N, operator=AND}, operator=AND}, operator=AND}, right={left=H, right={left=J, right={left=N, right=P, operator=AND}, operator=AND}, operator=AND}, operator=OR}, operator=OR}, operator=OR}, operator=OR}, operator=OR}, operator=OR}, operator=OR}, operator=OR}, operator=OR}, operator=OR}", + "version": "v0.1" + }, + "metric": { + "label": "Connections", + "mlid": 16, + "name": "connections" + }, + "size": 1, + "threshold": 0, + "time": "2022-07-13T21:46:47.000Z", + "triggered_filters": [ + { + "arguments": { + "value": "out" + }, + "cfid": 26781, + "comparator_type": "is", + "filter_type": "Direction", + "id": "H", + "trigger": { + "value": "out" + } + }, + { + "arguments": { + "value": 12 + }, + "cfid": 26783, + "comparator_type": "does not have tag", + "filter_type": "Tagged internal source", + "id": "J", + "trigger": { + "tag": { + "data": { + "auto": false, + "color": 55, + "visibility": "Public" + }, + "expiry": 0, + "is_referenced": true, + "name": "Security Device", + "restricted": false, + "thid": 12, + "tid": 12 + }, + "value": "12" + } + }, + { + "arguments": { + "value": 12 + }, + "cfid": 26787, + "comparator_type": "does not have tag", + "filter_type": "Tagged internal destination", + "id": "N", + "trigger": { + "tag": { + "data": { + "auto": false, + "color": 55, + "visibility": "Public" + }, + "expiry": 0, + "is_referenced": true, + "name": "Security Device", + "restricted": false, + "thid": 12, + "tid": 12 + }, + "value": "12" + } + }, + { + "arguments": { + "value": "examples" + }, + "cfid": 26788, + "comparator_type": "does not match regular expression", + "filter_type": "User agent", + "id": "O" + }, + { + "arguments": { + "value": "examples" + }, + "cfid": 26789, + "comparator_type": "matches regular expression", + "filter_type": "URI", + "id": "P" + }, + { + "arguments": { + "value": "true" + }, + "cfid": 26790, + "comparator_type": "display", + "filter_type": "Proxied connection", + "id": "d1", + "trigger": { + "value": "false" + } + }, + { + "cfid": 26791, + "comparator_type": "display", + "filter_type": "HTTP response code", + "id": "d10", + "trigger": { + "value": "0" + } + }, + { + "cfid": 26792, + "comparator_type": "display", + "filter_type": "HTTP referrer", + "id": "d2" + }, + { + "cfid": 26793, + "comparator_type": "display", + "filter_type": "HTTP method", + "id": "d3", + "trigger": { + "value": "GET" + } + }, + { + "cfid": 26794, + "comparator_type": "display", + "filter_type": "HTTP X-Forwarded-For", + "id": "d4" + }, + { + "cfid": 26795, + "comparator_type": "display", + "filter_type": "URI", + "id": "d5" + }, + { + "cfid": 26796, + "comparator_type": "display", + "filter_type": "User agent", + "id": "d6" + }, + { + "cfid": 26797, + "comparator_type": "display", + "filter_type": "Destination IP", + "id": "d7", + "trigger": { + "value": "81.2.69.144" + } + }, + { + "cfid": 26798, + "comparator_type": "display", + "filter_type": "Internal destination device name", + "id": "d8", + "trigger": { + "value": "localhost.local" + } + }, + { + "cfid": 26799, + "comparator_type": "display", + "filter_type": "Connection hostname", + "id": "d9" + } + ] + } + ] + } + }, + "ecs": { + "version": "8.4.0" + }, + "event": { + "category": [ + "threat", + "network" + ], + "created": "2022-07-13T21:46:55.000Z", + "kind": "alert", + "original": "{\"model\":{\"name\":\"Device::Attack and Recon Tools\",\"pid\":135,\"phid\":1198,\"uuid\":\"8abcdefg-1234-1234-1234-5abcdefg12\",\"logic\":{\"data\":[{\"cid\":2311,\"weight\":1},{\"cid\":2312,\"weight\":1},{\"cid\":2315,\"weight\":1},{\"cid\":2316,\"weight\":1},{\"cid\":2314,\"weight\":1},{\"cid\":2310,\"weight\":1},{\"cid\":2313,\"weight\":1}],\"targetScore\":1,\"type\":\"weightedComponentList\",\"version\":1},\"throttle\":604800,\"sharedEndpoints\":false,\"actions\":{\"alert\":true,\"antigena\":{},\"breach\":true,\"model\":true,\"setPriority\":false,\"setTag\":false,\"setType\":false},\"tags\":[\"AP: Internal Recon\",\"OT Engineer\"],\"interval\":3600,\"delay\":0,\"sequenced\":false,\"active\":true,\"modified\":\"2022-07-11 11:47:51\",\"activeTimes\":{\"devices\":{},\"tags\":{},\"type\":\"exclusions\",\"version\":2},\"autoUpdatable\":true,\"autoUpdate\":true,\"autoSuppress\":true,\"description\":\"A device is using common penetration testing tools.\\\\n\\\\nAction: Review the device to see if it a security device, these can be tagged as such to exclude them from future breaches. Activity from non security devices merit further investigation into what else the device is doing and could be a significant risk within the network.\",\"behaviour\":\"decreasing\",\"created\":{\"by\":\"System\"},\"edited\":{\"by\":\"System\"},\"version\":77,\"priority\":4,\"category\":\"Suspicious\",\"compliance\":false},\"device\":{\"did\":7,\"ip\":\"81.2.69.192\",\"ips\":[{\"ip\":\"175.16.199.2\",\"timems\":1657746000000,\"time\":\"2022-07-13 21:00:00\",\"sid\":1}],\"sid\":1,\"hostname\":\"175.16.199.1\",\"firstSeen\":1657544891000,\"lastSeen\":1657748638000,\"typename\":\"desktop\",\"typelabel\":\"Desktop\",\"credentials\":[\"dummy\"],\"tags\":[{\"tid\":66,\"expiry\":0,\"thid\":66,\"name\":\"Domain Authenticated\",\"restricted\":false,\"data\":{\"auto\":false,\"color\":183,\"description\":\"\"},\"isReferenced\":true},{\"tid\":29,\"expiry\":0,\"thid\":29,\"name\":\"Linux\",\"restricted\":false,\"data\":{\"auto\":false,\"color\":168,\"description\":\"\",\"visibility\":\"Public\"},\"isReferenced\":true},{\"tid\":45,\"expiry\":0,\"thid\":45,\"name\":\"New Device\",\"restricted\":false,\"data\":{\"auto\":false,\"color\":130,\"description\":\"\",\"visibility\":\"Public\"},\"isReferenced\":true}]},\"triggeredComponents\":[{\"time\":1657748807000,\"cbid\":6,\"cid\":2311,\"chid\":2676,\"size\":1,\"threshold\":0,\"interval\":300,\"logic\":{\"data\":{\"left\":{\"left\":\"A\",\"operator\":\"AND\",\"right\":{\"left\":\"H\",\"operator\":\"AND\",\"right\":{\"left\":\"J\",\"operator\":\"AND\",\"right\":\"N\"}}},\"operator\":\"OR\",\"right\":{\"left\":{\"left\":\"B\",\"operator\":\"AND\",\"right\":{\"left\":\"H\",\"operator\":\"AND\",\"right\":{\"left\":\"J\",\"operator\":\"AND\",\"right\":{\"left\":\"N\",\"operator\":\"AND\",\"right\":\"O\"}}}},\"operator\":\"OR\",\"right\":{\"left\":{\"left\":\"C\",\"operator\":\"AND\",\"right\":{\"left\":\"H\",\"operator\":\"AND\",\"right\":{\"left\":\"J\",\"operator\":\"AND\",\"right\":\"N\"}}},\"operator\":\"OR\",\"right\":{\"left\":{\"left\":\"E\",\"operator\":\"AND\",\"right\":{\"left\":\"H\",\"operator\":\"AND\",\"right\":{\"left\":\"J\",\"operator\":\"AND\",\"right\":\"N\"}}},\"operator\":\"OR\",\"right\":{\"left\":{\"left\":\"F\",\"operator\":\"AND\",\"right\":{\"left\":\"H\",\"operator\":\"AND\",\"right\":{\"left\":\"J\",\"operator\":\"AND\",\"right\":\"N\"}}},\"operator\":\"OR\",\"right\":{\"left\":{\"left\":\"G\",\"operator\":\"AND\",\"right\":{\"left\":\"H\",\"operator\":\"AND\",\"right\":{\"left\":\"J\",\"operator\":\"AND\",\"right\":\"N\"}}},\"operator\":\"OR\",\"right\":{\"left\":{\"left\":\"H\",\"operator\":\"AND\",\"right\":{\"left\":\"I\",\"operator\":\"AND\",\"right\":{\"left\":\"J\",\"operator\":\"AND\",\"right\":\"N\"}}},\"operator\":\"OR\",\"right\":{\"left\":{\"left\":\"H\",\"operator\":\"AND\",\"right\":{\"left\":\"J\",\"operator\":\"AND\",\"right\":{\"left\":\"K\",\"operator\":\"AND\",\"right\":\"N\"}}},\"operator\":\"OR\",\"right\":{\"left\":{\"left\":\"H\",\"operator\":\"AND\",\"right\":{\"left\":\"J\",\"operator\":\"AND\",\"right\":{\"left\":\"L\",\"operator\":\"AND\",\"right\":\"N\"}}},\"operator\":\"OR\",\"right\":{\"left\":{\"left\":\"H\",\"operator\":\"AND\",\"right\":{\"left\":\"J\",\"operator\":\"AND\",\"right\":{\"left\":\"M\",\"operator\":\"AND\",\"right\":\"N\"}}},\"operator\":\"OR\",\"right\":{\"left\":\"H\",\"operator\":\"AND\",\"right\":{\"left\":\"J\",\"operator\":\"AND\",\"right\":{\"left\":\"N\",\"operator\":\"AND\",\"right\":\"P\"}}}}}}}}}}}}},\"version\":\"v0.1\"},\"metric\":{\"mlid\":16,\"name\":\"connections\",\"label\":\"Connections\"},\"triggeredFilters\":[{\"cfid\":26781,\"id\":\"H\",\"filterType\":\"Direction\",\"arguments\":{\"value\":\"out\"},\"comparatorType\":\"is\",\"trigger\":{\"value\":\"out\"}},{\"cfid\":26783,\"id\":\"J\",\"filterType\":\"Tagged internal source\",\"arguments\":{\"value\":12},\"comparatorType\":\"does not have tag\",\"trigger\":{\"value\":\"12\",\"tag\":{\"tid\":12,\"expiry\":0,\"thid\":12,\"name\":\"Security Device\",\"restricted\":false,\"data\":{\"auto\":false,\"color\":55,\"description\":\"\",\"visibility\":\"Public\"},\"isReferenced\":true}}},{\"cfid\":26787,\"id\":\"N\",\"filterType\":\"Tagged internal destination\",\"arguments\":{\"value\":12},\"comparatorType\":\"does not have tag\",\"trigger\":{\"value\":\"12\",\"tag\":{\"tid\":12,\"expiry\":0,\"thid\":12,\"name\":\"Security Device\",\"restricted\":false,\"data\":{\"auto\":false,\"color\":55,\"description\":\"\",\"visibility\":\"Public\"},\"isReferenced\":true}}},{\"cfid\":26788,\"id\":\"O\",\"filterType\":\"User agent\",\"arguments\":{\"value\":\"examples\"},\"comparatorType\":\"does not match regular expression\",\"trigger\":{\"value\":\"\"}},{\"cfid\":26789,\"id\":\"P\",\"filterType\":\"URI\",\"arguments\":{\"value\":\"examples\"},\"comparatorType\":\"matches regular expression\",\"trigger\":{\"value\":\"\"}},{\"cfid\":26790,\"id\":\"d1\",\"filterType\":\"Proxied connection\",\"arguments\":{\"value\":\"true\"},\"comparatorType\":\"display\",\"trigger\":{\"value\":\"false\"}},{\"cfid\":26791,\"id\":\"d10\",\"filterType\":\"HTTP response code\",\"arguments\":{},\"comparatorType\":\"display\",\"trigger\":{\"value\":\"0\"}},{\"cfid\":26792,\"id\":\"d2\",\"filterType\":\"HTTP referrer\",\"arguments\":{},\"comparatorType\":\"display\",\"trigger\":{\"value\":\"\"}},{\"cfid\":26793,\"id\":\"d3\",\"filterType\":\"HTTP method\",\"arguments\":{},\"comparatorType\":\"display\",\"trigger\":{\"value\":\"GET\"}},{\"cfid\":26794,\"id\":\"d4\",\"filterType\":\"HTTP X-Forwarded-For\",\"arguments\":{},\"comparatorType\":\"display\",\"trigger\":{\"value\":\"\"}},{\"cfid\":26795,\"id\":\"d5\",\"filterType\":\"URI\",\"arguments\":{},\"comparatorType\":\"display\",\"trigger\":{\"value\":\"\"}},{\"cfid\":26796,\"id\":\"d6\",\"filterType\":\"User agent\",\"arguments\":{},\"comparatorType\":\"display\",\"trigger\":{\"value\":\"\"}},{\"cfid\":26797,\"id\":\"d7\",\"filterType\":\"Destination IP\",\"arguments\":{},\"comparatorType\":\"display\",\"trigger\":{\"value\":\"81.2.69.144\"}},{\"cfid\":26798,\"id\":\"d8\",\"filterType\":\"Internal destination device name\",\"arguments\":{},\"comparatorType\":\"display\",\"trigger\":{\"value\":\"localhost.local\"}},{\"cfid\":26799,\"id\":\"d9\",\"filterType\":\"Connection hostname\",\"arguments\":{},\"comparatorType\":\"display\",\"trigger\":{\"value\":\"\"}}]}],\"breachUrl\":\"https://example.com/#modelbreach/6\",\"pbid\":6,\"score\":0.871,\"commentCount\":0,\"creationTime\":1657748815000,\"time\":1657748808000,\"mitreTechniques\":[{\"technique\":\"Hardware Additions Mitigation\",\"techniqueID\":\"T1200\"}]}", + "risk_score": 0.871, + "risk_score_norm": 87.1, + "severity": 4, + "start": [ + "2022-07-13T21:46:47.000Z" + ], + "type": [ + "info", + "connection" + ], + "url": "https://example.com/#modelbreach/6" + }, + "host": { + "id": "7", + "ip": [ + "81.2.69.192" + ], + "type": "desktop" + }, + "related": { + "ip": [ + "175.16.199.1", + "81.2.69.192", + "175.16.199.2" + ], + "user": [ + "System" + ] + }, + "rule": { + "author": "System", + "category": "Suspicious", + "description": "A device is using common penetration testing tools.\\n\\nAction: Review the device to see if it a security device, these can be tagged as such to exclude them from future breaches. Activity from non security devices merit further investigation into what else the device is doing and could be a significant risk within the network.", + "name": "Device::Attack and Recon Tools", + "ruleset": [ + "AP: Internal Recon", + "OT Engineer" + ], + "uuid": "8abcdefg-1234-1234-1234-5abcdefg12", + "version": "77" + }, + "tags": [ + "preserve_original_event", + "preserve_duplicate_custom_fields" + ], + "threat": { + "technique": { + "id": [ + "T1200" + ], + "name": [ + "Hardware Additions Mitigation" + ] + } + } + }, + { + "@timestamp": "2022-07-11T13:04:09.000Z", + "darktrace": { + "model_breach_alert": { + "aianalyst_data": [ + { + "related": [ + 1 + ], + "summariser": "BeaconSummary", + "uuid": "1234abcd-1234-1234-1234-123456abcdef" + } + ], + "comment": { + "count": 0 + }, + "creation_time": "2022-07-11T13:04:19.000Z", + "device": { + "did": 3, + "first_seen": "2022-07-11T12:54:49.000Z", + "ip": "81.2.69.142", + "last_seen": "2022-07-11T13:00:18.000Z", + "sid": 1, + "type_label": "Desktop", + "type_name": "desktop" + }, + "model": { + "actions": { + "is_alerting": true, + "is_breach": true, + "is_priority_set": false, + "is_tag_set": false, + "is_type_set": false, + "model": true + }, + "active_times": { + "type": "exclusions", + "version": 2 + }, + "behaviour": "incdec1", + "category": "Informational", + "created": { + "by": "System" + }, + "delay": 0, + "description": "A device has been repeatedly connecting to a rare external location with a beacon score. A beacon score is added when Darktrace identifies that a device is regularly communicating with an endpoint, for example, if a device connects to a rare external endpoint every 12 minutes this would get a beacon score. This model is designed to identify beaconing at a lower threshold and be protocol agnostic.\\n\\nAction: Review the external domains and IPs being connected to to see if they are legitimate and would be expected for business purposes.", + "edited": { + "by": "System" + }, + "in_compliance_behavior_category": false, + "interval": 10800, + "is_active": true, + "is_auto_suppress": true, + "is_auto_updatable": true, + "is_auto_update": true, + "is_sequenced": false, + "is_shared_endpoints": false, + "logic": { + "data_weighted_component_list": [ + { + "cid": 2026, + "weight": 1 + }, + { + "cid": 2024, + "weight": 1 + }, + { + "cid": 2025, + "weight": -100 + } + ], + "target_score": 1, + "type": "weightedComponentList", + "version": 1 + }, + "modified": "2022-07-11T11:47:37.000Z", + "name": "Compromise::Beaconing Activity To External Rare", + "phid": 1072, + "pid": 156, + "priority": 2, + "tags": [ + "AP: C2 Comms" + ], + "throttle": 10800, + "uuid": "1234abcd-1234-1234-1234-123456abcdef", + "version": 23 + }, + "pbid": 1, + "score": 0.674, + "time": "2022-07-11T13:04:09.000Z", + "triggered_components": [ + { + "cbid": 1, + "chid": 2113, + "cid": 2026, + "interval": 3600, + "logic": { + "data": "{left={left=A, right={left=AA, right={left=AC, right={left=AD, right={left=AF, right={left=AG, right={left=AH, right={left=B, right={left=C, right={left=D, right={left=E, right={left=H, right={left=I, right={left=J, right={left=K, right={left=L, right={left=M, right={left=N, right={left=O, right={left=P, right={left=S, right={left=U, right={left=V, right={left=X, right={left=Y, right=Z, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, right={left=A, right={left=AA, right={left=AB, right={left=AE, right={left=AF, right={left=AG, right={left=AH, right={left=C, right={left=D, right={left=E, right={left=H, right={left=I, right={left=J, right={left=K, right={left=L, right={left=M, right={left=N, right={left=O, right={left=P, right={left=S, right={left=U, right={left=V, right={left=X, right={left=Y, right=Z, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=OR}", + "version": "v0.1" + }, + "metric": { + "label": "External Connections", + "mlid": 1, + "name": "externalconnections" + }, + "size": 11, + "threshold": 10, + "time": "2022-07-11T13:04:08.000Z", + "triggered_filters": [ + { + "arguments": { + "value": 60 + }, + "cfid": 23426, + "comparator_type": "\u003e", + "filter_type": "Beaconing score", + "id": "A", + "trigger": { + "value": "100" + } + }, + { + "arguments": { + "value": 0 + }, + "cfid": 23427, + "comparator_type": "\u003e", + "filter_type": "Individual size up", + "id": "AA", + "trigger": { + "value": "4382" + } + }, + { + "arguments": { + "value": 95 + }, + "cfid": 23428, + "comparator_type": "\u003e", + "filter_type": "Rare domain", + "id": "AB", + "trigger": { + "value": "100" + } + }, + { + "arguments": { + "value": 1209600 + }, + "cfid": 23430, + "comparator_type": "\u003c", + "filter_type": "Age of destination", + "id": "AD", + "trigger": { + "value": "558" + } + }, + { + "arguments": { + "value": 1209600 + }, + "cfid": 23431, + "comparator_type": "\u003c", + "filter_type": "Age of external hostname", + "id": "AE", + "trigger": { + "value": "558" + } + }, + { + "arguments": { + "value": "examples" + }, + "cfid": 23432, + "comparator_type": "does not match regular expression", + "filter_type": "Connection hostname", + "id": "AF", + "trigger": { + "value": "example.com" + } + }, + { + "arguments": { + "value": "examples" + }, + "cfid": 23433, + "comparator_type": "does not match regular expression", + "filter_type": "ASN", + "id": "AG", + "trigger": { + "value": "AS12345 LOCAL-02" + } + }, + { + "arguments": { + "value": "5d41402abc4b2a76b9719d911017c592" + }, + "cfid": 23434, + "comparator_type": "does not match", + "filter_type": "JA3 hash", + "id": "AH", + "trigger": { + "value": "5d41402abc4b2a76b9719d911017c592" + } + }, + { + "arguments": { + "value": 95 + }, + "cfid": 23435, + "comparator_type": "\u003e", + "filter_type": "Rare external IP", + "id": "B", + "trigger": { + "value": "100" + } + }, + { + "arguments": { + "value": "1003" + }, + "cfid": 23436, + "comparator_type": "is not", + "filter_type": "Application protocol", + "id": "C", + "trigger": { + "value": "1004" + } + }, + { + "arguments": { + "value": 53 + }, + "cfid": 23437, + "comparator_type": "!=", + "filter_type": "Destination port", + "id": "D", + "trigger": { + "value": "443" + } + }, + { + "arguments": { + "value": "out" + }, + "cfid": 23438, + "comparator_type": "is", + "filter_type": "Direction", + "id": "E", + "trigger": { + "value": "out" + } + }, + { + "arguments": { + "value": 137 + }, + "cfid": 23439, + "comparator_type": "!=", + "filter_type": "Destination port", + "id": "H", + "trigger": { + "value": "443" + } + }, + { + "arguments": { + "value": 161 + }, + "cfid": 23440, + "comparator_type": "!=", + "filter_type": "Destination port", + "id": "I", + "trigger": { + "value": "443" + } + }, + { + "arguments": { + "value": "6" + }, + "cfid": 23441, + "comparator_type": "is", + "filter_type": "Protocol", + "id": "J", + "trigger": { + "value": "6" + } + }, + { + "arguments": { + "value": "Company" + }, + "cfid": 23442, + "comparator_type": "does not contain", + "filter_type": "ASN", + "id": "K", + "trigger": { + "value": "AS12345 LOCAL-02" + } + }, + { + "arguments": { + "value": "Company" + }, + "cfid": 23443, + "comparator_type": "does not contain", + "filter_type": "ASN", + "id": "L", + "trigger": { + "value": "AS12345 LOCAL-02" + } + }, + { + "arguments": { + "value": "13" + }, + "cfid": 23444, + "comparator_type": "is not", + "filter_type": "Internal source device type", + "id": "M", + "trigger": { + "value": "6" + } + }, + { + "arguments": { + "value": "5" + }, + "cfid": 23445, + "comparator_type": "is not", + "filter_type": "Internal source device type", + "id": "N", + "trigger": { + "value": "6" + } + }, + { + "arguments": { + "value": "9" + }, + "cfid": 23446, + "comparator_type": "is not", + "filter_type": "Internal source device type", + "id": "O", + "trigger": { + "value": "6" + } + }, + { + "arguments": { + "value": "12" + }, + "cfid": 23447, + "comparator_type": "is not", + "filter_type": "Internal source device type", + "id": "P", + "trigger": { + "value": "6" + } + }, + { + "arguments": { + "value": "30" + }, + "cfid": 23448, + "comparator_type": "is not", + "filter_type": "Internal source device type", + "id": "S", + "trigger": { + "value": "6" + } + }, + { + "arguments": { + "value": "4" + }, + "cfid": 23449, + "comparator_type": "is not", + "filter_type": "Internal source device type", + "id": "U", + "trigger": { + "value": "6" + } + }, + { + "arguments": { + "value": "3" + }, + "cfid": 23450, + "comparator_type": "is not", + "filter_type": "Internal source device type", + "id": "V", + "trigger": { + "value": "6" + } + }, + { + "arguments": { + "value": "false" + }, + "cfid": 23451, + "comparator_type": "is", + "filter_type": "Trusted hostname", + "id": "X", + "trigger": { + "value": "false" + } + }, + { + "arguments": { + "value": 26 + }, + "cfid": 23452, + "comparator_type": "does not have tag", + "filter_type": "Tagged internal source", + "id": "Y", + "trigger": { + "tag": { + "data": { + "auto": false, + "color": 5, + "visibility": "Public" + }, + "expiry": 0, + "is_referenced": true, + "name": "No Device Tracking", + "restricted": false, + "thid": 26, + "tid": 26 + }, + "value": "26" + } + }, + { + "arguments": { + "value": 0 + }, + "cfid": 23453, + "comparator_type": "\u003e", + "filter_type": "Individual size down", + "id": "Z", + "trigger": { + "value": "5862" + } + }, + { + "cfid": 23454, + "comparator_type": "display", + "filter_type": "JA3 hash", + "id": "d1", + "trigger": { + "value": "5d41402abc4b2a76b9719d911017c592" + } + }, + { + "cfid": 23455, + "comparator_type": "display", + "filter_type": "ASN", + "id": "d2", + "trigger": { + "value": "AS12345 LOCAL-02" + } + }, + { + "cfid": 23456, + "comparator_type": "display", + "filter_type": "Destination IP", + "id": "d3", + "trigger": { + "value": "81.2.69.192" + } + }, + { + "cfid": 23457, + "comparator_type": "display", + "filter_type": "Connection hostname", + "id": "d4", + "trigger": { + "value": "example.com" + } + } + ] + } + ] + } + }, + "ecs": { + "version": "8.4.0" + }, + "event": { + "category": [ + "network" + ], + "created": "2022-07-11T13:04:19.000Z", + "kind": "event", + "original": "{\"commentCount\":0,\"pbid\":1,\"time\":1657544649000,\"creationTime\":1657544659000,\"aianalystData\":[{\"uuid\":\"1234abcd-1234-1234-1234-123456abcdef\",\"related\":[1],\"summariser\":\"BeaconSummary\"}],\"model\":{\"name\":\"Compromise::Beaconing Activity To External Rare\",\"pid\":156,\"phid\":1072,\"uuid\":\"1234abcd-1234-1234-1234-123456abcdef\",\"logic\":{\"data\":[{\"cid\":2026,\"weight\":1},{\"cid\":2024,\"weight\":1},{\"cid\":2025,\"weight\":-100}],\"targetScore\":1,\"type\":\"weightedComponentList\",\"version\":1},\"throttle\":10800,\"sharedEndpoints\":false,\"actions\":{\"alert\":true,\"antigena\":{},\"breach\":true,\"model\":true,\"setPriority\":false,\"setTag\":false,\"setType\":false},\"tags\":[\"AP: C2 Comms\"],\"interval\":10800,\"delay\":0,\"sequenced\":false,\"active\":true,\"modified\":\"2022-07-11 11:47:37\",\"activeTimes\":{\"devices\":{},\"tags\":{},\"type\":\"exclusions\",\"version\":2},\"autoUpdatable\":true,\"autoUpdate\":true,\"autoSuppress\":true,\"description\":\"A device has been repeatedly connecting to a rare external location with a beacon score. A beacon score is added when Darktrace identifies that a device is regularly communicating with an endpoint, for example, if a device connects to a rare external endpoint every 12 minutes this would get a beacon score. This model is designed to identify beaconing at a lower threshold and be protocol agnostic.\\\\n\\\\nAction: Review the external domains and IPs being connected to to see if they are legitimate and would be expected for business purposes.\",\"behaviour\":\"incdec1\",\"created\":{\"by\":\"System\"},\"edited\":{\"by\":\"System\"},\"version\":23,\"priority\":2,\"category\":\"Informational\",\"compliance\":false},\"triggeredComponents\":[{\"time\":1657544648000,\"cbid\":1,\"cid\":2026,\"chid\":2113,\"size\":11,\"threshold\":10,\"interval\":3600,\"logic\":{\"data\":{\"left\":{\"left\":\"A\",\"operator\":\"AND\",\"right\":{\"left\":\"AA\",\"operator\":\"AND\",\"right\":{\"left\":\"AC\",\"operator\":\"AND\",\"right\":{\"left\":\"AD\",\"operator\":\"AND\",\"right\":{\"left\":\"AF\",\"operator\":\"AND\",\"right\":{\"left\":\"AG\",\"operator\":\"AND\",\"right\":{\"left\":\"AH\",\"operator\":\"AND\",\"right\":{\"left\":\"B\",\"operator\":\"AND\",\"right\":{\"left\":\"C\",\"operator\":\"AND\",\"right\":{\"left\":\"D\",\"operator\":\"AND\",\"right\":{\"left\":\"E\",\"operator\":\"AND\",\"right\":{\"left\":\"H\",\"operator\":\"AND\",\"right\":{\"left\":\"I\",\"operator\":\"AND\",\"right\":{\"left\":\"J\",\"operator\":\"AND\",\"right\":{\"left\":\"K\",\"operator\":\"AND\",\"right\":{\"left\":\"L\",\"operator\":\"AND\",\"right\":{\"left\":\"M\",\"operator\":\"AND\",\"right\":{\"left\":\"N\",\"operator\":\"AND\",\"right\":{\"left\":\"O\",\"operator\":\"AND\",\"right\":{\"left\":\"P\",\"operator\":\"AND\",\"right\":{\"left\":\"S\",\"operator\":\"AND\",\"right\":{\"left\":\"U\",\"operator\":\"AND\",\"right\":{\"left\":\"V\",\"operator\":\"AND\",\"right\":{\"left\":\"X\",\"operator\":\"AND\",\"right\":{\"left\":\"Y\",\"operator\":\"AND\",\"right\":\"Z\"}}}}}}}}}}}}}}}}}}}}}}}}},\"operator\":\"OR\",\"right\":{\"left\":\"A\",\"operator\":\"AND\",\"right\":{\"left\":\"AA\",\"operator\":\"AND\",\"right\":{\"left\":\"AB\",\"operator\":\"AND\",\"right\":{\"left\":\"AE\",\"operator\":\"AND\",\"right\":{\"left\":\"AF\",\"operator\":\"AND\",\"right\":{\"left\":\"AG\",\"operator\":\"AND\",\"right\":{\"left\":\"AH\",\"operator\":\"AND\",\"right\":{\"left\":\"C\",\"operator\":\"AND\",\"right\":{\"left\":\"D\",\"operator\":\"AND\",\"right\":{\"left\":\"E\",\"operator\":\"AND\",\"right\":{\"left\":\"H\",\"operator\":\"AND\",\"right\":{\"left\":\"I\",\"operator\":\"AND\",\"right\":{\"left\":\"J\",\"operator\":\"AND\",\"right\":{\"left\":\"K\",\"operator\":\"AND\",\"right\":{\"left\":\"L\",\"operator\":\"AND\",\"right\":{\"left\":\"M\",\"operator\":\"AND\",\"right\":{\"left\":\"N\",\"operator\":\"AND\",\"right\":{\"left\":\"O\",\"operator\":\"AND\",\"right\":{\"left\":\"P\",\"operator\":\"AND\",\"right\":{\"left\":\"S\",\"operator\":\"AND\",\"right\":{\"left\":\"U\",\"operator\":\"AND\",\"right\":{\"left\":\"V\",\"operator\":\"AND\",\"right\":{\"left\":\"X\",\"operator\":\"AND\",\"right\":{\"left\":\"Y\",\"operator\":\"AND\",\"right\":\"Z\"}}}}}}}}}}}}}}}}}}}}}}}}},\"version\":\"v0.1\"},\"metric\":{\"mlid\":1,\"name\":\"externalconnections\",\"label\":\"External Connections\"},\"triggeredFilters\":[{\"cfid\":23426,\"id\":\"A\",\"filterType\":\"Beaconing score\",\"arguments\":{\"value\":60},\"comparatorType\":\"\u003e\",\"trigger\":{\"value\":\"100\"}},{\"cfid\":23427,\"id\":\"AA\",\"filterType\":\"Individual size up\",\"arguments\":{\"value\":0},\"comparatorType\":\"\u003e\",\"trigger\":{\"value\":\"4382\"}},{\"cfid\":23428,\"id\":\"AB\",\"filterType\":\"Rare domain\",\"arguments\":{\"value\":95},\"comparatorType\":\"\u003e\",\"trigger\":{\"value\":\"100\"}},{\"cfid\":23430,\"id\":\"AD\",\"filterType\":\"Age of destination\",\"arguments\":{\"value\":1209600},\"comparatorType\":\"\u003c\",\"trigger\":{\"value\":\"558\"}},{\"cfid\":23431,\"id\":\"AE\",\"filterType\":\"Age of external hostname\",\"arguments\":{\"value\":1209600},\"comparatorType\":\"\u003c\",\"trigger\":{\"value\":\"558\"}},{\"cfid\":23432,\"id\":\"AF\",\"filterType\":\"Connection hostname\",\"arguments\":{\"value\":\"examples\"},\"comparatorType\":\"does not match regular expression\",\"trigger\":{\"value\":\"example.com\"}},{\"cfid\":23433,\"id\":\"AG\",\"filterType\":\"ASN\",\"arguments\":{\"value\":\"examples\"},\"comparatorType\":\"does not match regular expression\",\"trigger\":{\"value\":\"AS12345 LOCAL-02\"}},{\"cfid\":23434,\"id\":\"AH\",\"filterType\":\"JA3 hash\",\"arguments\":{\"value\":\"5d41402abc4b2a76b9719d911017c592\"},\"comparatorType\":\"does not match\",\"trigger\":{\"value\":\"5d41402abc4b2a76b9719d911017c592\"}},{\"cfid\":23435,\"id\":\"B\",\"filterType\":\"Rare external IP\",\"arguments\":{\"value\":95},\"comparatorType\":\"\u003e\",\"trigger\":{\"value\":\"100\"}},{\"cfid\":23436,\"id\":\"C\",\"filterType\":\"Application protocol\",\"arguments\":{\"value\":\"1003\"},\"comparatorType\":\"is not\",\"trigger\":{\"value\":\"1004\"}},{\"cfid\":23437,\"id\":\"D\",\"filterType\":\"Destination port\",\"arguments\":{\"value\":53},\"comparatorType\":\"!=\",\"trigger\":{\"value\":\"443\"}},{\"cfid\":23438,\"id\":\"E\",\"filterType\":\"Direction\",\"arguments\":{\"value\":\"out\"},\"comparatorType\":\"is\",\"trigger\":{\"value\":\"out\"}},{\"cfid\":23439,\"id\":\"H\",\"filterType\":\"Destination port\",\"arguments\":{\"value\":137},\"comparatorType\":\"!=\",\"trigger\":{\"value\":\"443\"}},{\"cfid\":23440,\"id\":\"I\",\"filterType\":\"Destination port\",\"arguments\":{\"value\":161},\"comparatorType\":\"!=\",\"trigger\":{\"value\":\"443\"}},{\"cfid\":23441,\"id\":\"J\",\"filterType\":\"Protocol\",\"arguments\":{\"value\":\"6\"},\"comparatorType\":\"is\",\"trigger\":{\"value\":\"6\"}},{\"cfid\":23442,\"id\":\"K\",\"filterType\":\"ASN\",\"arguments\":{\"value\":\"Company\"},\"comparatorType\":\"does not contain\",\"trigger\":{\"value\":\"AS12345 LOCAL-02\"}},{\"cfid\":23443,\"id\":\"L\",\"filterType\":\"ASN\",\"arguments\":{\"value\":\"Company\"},\"comparatorType\":\"does not contain\",\"trigger\":{\"value\":\"AS12345 LOCAL-02\"}},{\"cfid\":23444,\"id\":\"M\",\"filterType\":\"Internal source device type\",\"arguments\":{\"value\":\"13\"},\"comparatorType\":\"is not\",\"trigger\":{\"value\":\"6\"}},{\"cfid\":23445,\"id\":\"N\",\"filterType\":\"Internal source device type\",\"arguments\":{\"value\":\"5\"},\"comparatorType\":\"is not\",\"trigger\":{\"value\":\"6\"}},{\"cfid\":23446,\"id\":\"O\",\"filterType\":\"Internal source device type\",\"arguments\":{\"value\":\"9\"},\"comparatorType\":\"is not\",\"trigger\":{\"value\":\"6\"}},{\"cfid\":23447,\"id\":\"P\",\"filterType\":\"Internal source device type\",\"arguments\":{\"value\":\"12\"},\"comparatorType\":\"is not\",\"trigger\":{\"value\":\"6\"}},{\"cfid\":23448,\"id\":\"S\",\"filterType\":\"Internal source device type\",\"arguments\":{\"value\":\"30\"},\"comparatorType\":\"is not\",\"trigger\":{\"value\":\"6\"}},{\"cfid\":23449,\"id\":\"U\",\"filterType\":\"Internal source device type\",\"arguments\":{\"value\":\"4\"},\"comparatorType\":\"is not\",\"trigger\":{\"value\":\"6\"}},{\"cfid\":23450,\"id\":\"V\",\"filterType\":\"Internal source device type\",\"arguments\":{\"value\":\"3\"},\"comparatorType\":\"is not\",\"trigger\":{\"value\":\"6\"}},{\"cfid\":23451,\"id\":\"X\",\"filterType\":\"Trusted hostname\",\"arguments\":{\"value\":\"false\"},\"comparatorType\":\"is\",\"trigger\":{\"value\":\"false\"}},{\"cfid\":23452,\"id\":\"Y\",\"filterType\":\"Tagged internal source\",\"arguments\":{\"value\":26},\"comparatorType\":\"does not have tag\",\"trigger\":{\"value\":\"26\",\"tag\":{\"tid\":26,\"expiry\":0,\"thid\":26,\"name\":\"No Device Tracking\",\"restricted\":false,\"data\":{\"auto\":false,\"color\":5,\"description\":\"\",\"visibility\":\"Public\"},\"isReferenced\":true}}},{\"cfid\":23453,\"id\":\"Z\",\"filterType\":\"Individual size down\",\"arguments\":{\"value\":0},\"comparatorType\":\"\u003e\",\"trigger\":{\"value\":\"5862\"}},{\"cfid\":23454,\"id\":\"d1\",\"filterType\":\"JA3 hash\",\"arguments\":{},\"comparatorType\":\"display\",\"trigger\":{\"value\":\"5d41402abc4b2a76b9719d911017c592\"}},{\"cfid\":23455,\"id\":\"d2\",\"filterType\":\"ASN\",\"arguments\":{},\"comparatorType\":\"display\",\"trigger\":{\"value\":\"AS12345 LOCAL-02\"}},{\"cfid\":23456,\"id\":\"d3\",\"filterType\":\"Destination IP\",\"arguments\":{},\"comparatorType\":\"display\",\"trigger\":{\"value\":\"81.2.69.192\"}},{\"cfid\":23457,\"id\":\"d4\",\"filterType\":\"Connection hostname\",\"arguments\":{},\"comparatorType\":\"display\",\"trigger\":{\"value\":\"example.com\"}}]}],\"score\":0.674,\"device\":{\"did\":3,\"ip\":\"81.2.69.142\",\"sid\":1,\"firstSeen\":1657544089000,\"lastSeen\":1657544418000,\"typename\":\"desktop\",\"typelabel\":\"Desktop\"}}", + "risk_score": 0.674, + "risk_score_norm": 67.4, + "severity": 2, + "start": [ + "2022-07-11T13:04:08.000Z" + ], + "type": [ + "info", + "connection" + ] + }, + "host": { + "id": "3", + "ip": [ + "81.2.69.142" + ], + "type": "desktop" + }, + "related": { + "ip": [ + "81.2.69.142" + ], + "user": [ + "System" + ] + }, + "rule": { + "author": "System", + "category": "Informational", + "description": "A device has been repeatedly connecting to a rare external location with a beacon score. A beacon score is added when Darktrace identifies that a device is regularly communicating with an endpoint, for example, if a device connects to a rare external endpoint every 12 minutes this would get a beacon score. This model is designed to identify beaconing at a lower threshold and be protocol agnostic.\\n\\nAction: Review the external domains and IPs being connected to to see if they are legitimate and would be expected for business purposes.", + "name": "Compromise::Beaconing Activity To External Rare", + "ruleset": [ + "AP: C2 Comms" + ], + "uuid": "1234abcd-1234-1234-1234-123456abcdef", + "version": "23" + }, + "tags": [ + "preserve_original_event", + "preserve_duplicate_custom_fields" + ] + }, + { + "@timestamp": "2022-08-05T09:35:45.460Z", + "darktrace": { + "model_breach_alert": { + "creation_time": "2022-08-05T09:35:45.460Z", + "device": { + "hostname": "test-device.example.com", + "ip": "175.16.199.1", + "mac_address": "00-00-5E-00-53-00", + "vendor": "Test Vendor" + }, + "model": { + "name": "Unrestricted Test Model" + }, + "pbid": 123, + "score": 1.0, + "time": "2022-08-05T09:35:45.460Z", + "triggered_components": [ + { + "metric": { + "label": "Test Metric" + }, + "triggered_filters": [ + { + "comparator_type": "display", + "filter_type": "Test Metric Filter", + "trigger": { + "value": "Test filter value" + } + } + ] + } + ] + } + }, + "ecs": { + "version": "8.4.0" + }, + "event": { + "created": "2022-08-05T09:35:45.460Z", + "original": "{\"model\":{\"name\":\"Unrestricted Test Model\"},\"device\":{\"ip\":\"175.16.199.1\",\"hostname\":\"test-device.example.com\",\"macaddress\":\"00:00:5e:00:53:00\",\"vendor\":\"Test Vendor\",\"label\":\"Test Device\"},\"triggeredComponents\":[{\"metric\":{\"label\":\"Test Metric\"},\"triggeredFilters\":[{\"comparatorType\":\"display\",\"filterType\":\"Test Metric Filter\",\"trigger\":{\"value\":\"Test filter value\"}}]}],\"breachUrl\":\"\",\"pbid\":123,\"score\":1,\"creationTime\":1659692145460,\"time\":1659692145460,\"mitreTechniques\":[]}", + "risk_score": 1.0, + "risk_score_norm": 100.0, + "type": [ + "info" + ] + }, + "host": { + "hostname": "test-device.example.com", + "ip": [ + "175.16.199.1" + ], + "mac": "00-00-5E-00-53-00" + }, + "related": { + "hosts": [ + "test-device.example.com" + ], + "ip": [ + "175.16.199.1" + ] + }, + "rule": { + "name": "Unrestricted Test Model" + }, + "tags": [ + "preserve_original_event", + "preserve_duplicate_custom_fields" + ] + }, + { + "@timestamp": "2022-08-05T09:34:46.317Z", + "darktrace": { + "model_breach_alert": { + "creation_time": "2022-08-05T09:34:46.317Z", + "device": { + "hostname": "test-device.example.com", + "ip": "175.16.199.1", + "mac_address": "00-00-5E-00-53-00", + "vendor": "Test Vendor" + }, + "model": { + "created": { + "by": "System" + }, + "description": "Test model used for testing alerting configuration.", + "edited": { + "by": "Nobody" + }, + "name": "Unrestricted Test Model", + "priority": 5 + }, + "pbid": 123, + "score": 1.0, + "time": "2022-08-05T09:34:46.317Z", + "triggered_components": [ + { + "metric": { + "label": "Test Metric" + }, + "triggered_filters": [ + { + "comparator_type": "display", + "filter_type": "Test Metric Filter", + "trigger": { + "value": "Test filter value" + } + } + ] + } + ] + } + }, + "ecs": { + "version": "8.4.0" + }, + "event": { + "created": "2022-08-05T09:34:46.317Z", + "original": "{\"model\":{\"description\":\"Test model used for testing alerting configuration.\",\"created\":{\"by\":\"System\"},\"edited\":{\"by\":\"Nobody\"},\"name\":\"Unrestricted Test Model\",\"priority\":5},\"device\":{\"ip\":\"175.16.199.1\",\"hostname\":\"test-device.example.com\",\"macaddress\":\"00:00:5e:00:53:00\",\"vendor\":\"Test Vendor\",\"label\":\"Test Device\"},\"triggeredComponents\":[{\"metric\":{\"label\":\"Test Metric\"},\"triggeredFilters\":[{\"comparatorType\":\"display\",\"filterType\":\"Test Metric Filter\",\"trigger\":{\"value\":\"Test filter value\"}}]}],\"breachUrl\":\"\",\"pbid\":123,\"score\":1,\"creationTime\":1659692086317,\"time\":1659692086317,\"mitreTechniques\":[]}", + "risk_score": 1.0, + "risk_score_norm": 100.0, + "severity": 5, + "type": [ + "info" + ] + }, + "host": { + "hostname": "test-device.example.com", + "ip": [ + "175.16.199.1" + ], + "mac": "00-00-5E-00-53-00" + }, + "log": { + "syslog": { + "appname": "darktrace" + } + }, + "related": { + "hosts": [ + "test-device.example.com" + ], + "ip": [ + "175.16.199.1" + ], + "user": [ + "System", + "Nobody" + ] + }, + "rule": { + "author": "System", + "description": "Test model used for testing alerting configuration.", + "name": "Unrestricted Test Model" + }, + "tags": [ + "preserve_original_event", + "preserve_duplicate_custom_fields" + ] + } + ] +} \ No newline at end of file diff --git a/packages/darktrace/data_stream/model_breach_alert/_dev/test/system/test-httpjson-config.yml b/packages/darktrace/data_stream/model_breach_alert/_dev/test/system/test-httpjson-config.yml new file mode 100644 index 00000000000..938018884cf --- /dev/null +++ b/packages/darktrace/data_stream/model_breach_alert/_dev/test/system/test-httpjson-config.yml @@ -0,0 +1,10 @@ +input: httpjson +service: darktrace +vars: + url: http://{{Hostname}}:{{Port}} + public_token: xxxx + private_token: xxxx +data_stream: + vars: + preserve_original_event: true + preserve_duplicate_custom_fields: true diff --git a/packages/darktrace/data_stream/model_breach_alert/_dev/test/system/test-tcp-config.yml b/packages/darktrace/data_stream/model_breach_alert/_dev/test/system/test-tcp-config.yml new file mode 100644 index 00000000000..db13b8d1220 --- /dev/null +++ b/packages/darktrace/data_stream/model_breach_alert/_dev/test/system/test-tcp-config.yml @@ -0,0 +1,10 @@ +service: darktrace-model_breach_alert-tcp +service_notify_signal: SIGHUP +input: tcp +vars: + listen_address: 0.0.0.0 +data_stream: + vars: + listen_port: 9572 + preserve_original_event: true + preserve_duplicate_custom_fields: true diff --git a/packages/darktrace/data_stream/model_breach_alert/_dev/test/system/test-tls-config.yml b/packages/darktrace/data_stream/model_breach_alert/_dev/test/system/test-tls-config.yml new file mode 100644 index 00000000000..0343a55d8d8 --- /dev/null +++ b/packages/darktrace/data_stream/model_breach_alert/_dev/test/system/test-tls-config.yml @@ -0,0 +1,62 @@ +service: darktrace-model_breach_alert-tls +service_notify_signal: SIGHUP +input: tcp +vars: + listen_address: 0.0.0.0 + ssl: | + key: | + -----BEGIN PRIVATE KEY----- + MIIEvwIBADANBgkqhkiG9w0BAQEFAASCBKkwggSlAgEAAoIBAQDhCLvLsQAHufsN + U+u1x/CequAUphfXZqLhDo2Eo/holfBS0+ey4bnzPL6lS9NFL5JkLQA2gYESqsXU + /Ru8E76Az1egzMwT3TVAPLVU8NbrxBqeNiQa2m9wC37HQy4qC9OxL28LUoKtFjxS + cD1sa0oikXCJN1a3BSoAf9iiZ/dxz4WVfrNhrzq2JFXjravY84n5ujkZOg45Pg70 + 4vHOeg0rBbIoSNfjDUVZWjwC95K1BMN3msOTL9juv/EDa6BujqCxl+G1nY7JPFDL + SHWis65p+1AAa5xieYDb47vyJ0SSR7lEURTXZOkkM6k5JWfgkATEmGzRxPkOloIT + Xg9ag1OlAgMBAAECggEAEHfPJmzhj68wjB0kFr13AmWG2Hv/Kqg8KzQhbx+AwkaW + u7j+L70NGpvLZ9VQtLNyhxoz9cksZO1SZO/Q48aeHlcOFppmJN3/U6AdtQWa9M35 + FLLpmX16wjxVHsfvzOvopgLOoYl8PqZt66qDFDgVyMnT7na6RdJ+7GJuvBPXq+Bc + vgThvAZitHSAOhnBFYmTMlBi6AzOMMsaFlgE3Xf9v3M0pAKItPRKMhXlC3MyvA/v + jgbra4Ib+0ryohggHheHB3bn3Jgv7iFKoW9OQSePVxacJ+kfr9H+No5g495URzqR + mx/96WCiv3rAh3ct8Sk/C4/3zMC8fUueDJIVjhgw0QKBgQD8NufLINNkIpBrLoCS + 972oFEjZB2u6EusQ7X9raROqpaw26ZSu+zSHeIKCGQ93M3aRb3FpdGeOxgZ095MV + 8a+nlh4stOvHj2Mm5YhTBDUavTC7o9aVR3Od5eTXUpHnaJpNI/uyIcKupeK1UJnV + UlBLeIwo/vJ1gsVrKMMAJkuKbwKBgQDkaWRRd0w2gUIbCTGf203BqXft0VdIiOW7 + +gnkeaNHAf09XljzxMcQzrB8kG63aKVGbJffphEfzxtiJ+HRQVH+7QpKRhU/GHmu + +6OKkxTcxJm5zhoRFxcSi2wG4PWmUGJvc7ss1OJGcaOUxwocCepO7N/jfdDz9Uke + KnA+YWOdKwKBgQDteZkYlojT0QOgF8HyH5gQyUCqMKWLJ0LzxltiPCbLV4Dml1pq + w5Z7M8nWS1hXiTpLx93GSFc1hFkSCwYP9GfK6Lryp0sVtHnMZvTMDbseuSJImwRx + vDwtYQfugg1lEQWwOoBEAiu3m/PxernNtNprpU57T0nlwUK3GkM5QdWAuwKBgQCZ + ZF3GiANapzupxGbbH//8Cr9LqsafI7CEqMpz8WxBh4h16iJ6sq+tDeFgBe8UpOY5 + gTwNKg1d+0w8guQYD3HtbWr3rlEeamVtqfiOW3ArQqyqJ0tCJuuLvK3zgKf35Qv2 + JRaSaPT8sdxVUcXsRoxgLJu+vwPQke1koMN4YRbwuQKBgQDJiZ/WSeqa5oIqkXbn + hjm7RXKaf2oE1U/bNjdSFtdEP7T4vUvvr7Hq2f/jiBLtCE7w16PJjKx9iIq2+jMl + qIY43Sk9bdi5FxtYTHda0hwrbH274P+QVcVs5PXCT0TGktOleHGBlXaaPrxl9iCh + 8tmmxZZYa5aQxEO/lxB9xQKaiQ== + -----END PRIVATE KEY----- + certificate: | + -----BEGIN CERTIFICATE----- + MIIDazCCAlOgAwIBAgIUW5TDu1tJMY2Oa7PsL+BQSmeWqz0wDQYJKoZIhvcNAQEL + BQAwRTELMAkGA1UEBhMCVVMxEzARBgNVBAgMClNvbWUtU3RhdGUxITAfBgNVBAoM + GEludGVybmV0IFdpZGdpdHMgUHR5IEx0ZDAeFw0yMTEwMDEwNTAwMjNaFw0yMTEw + MDIwNTAwMjNaMEUxCzAJBgNVBAYTAlVTMRMwEQYDVQQIDApTb21lLVN0YXRlMSEw + HwYDVQQKDBhJbnRlcm5ldCBXaWRnaXRzIFB0eSBMdGQwggEiMA0GCSqGSIb3DQEB + AQUAA4IBDwAwggEKAoIBAQDhCLvLsQAHufsNU+u1x/CequAUphfXZqLhDo2Eo/ho + lfBS0+ey4bnzPL6lS9NFL5JkLQA2gYESqsXU/Ru8E76Az1egzMwT3TVAPLVU8Nbr + xBqeNiQa2m9wC37HQy4qC9OxL28LUoKtFjxScD1sa0oikXCJN1a3BSoAf9iiZ/dx + z4WVfrNhrzq2JFXjravY84n5ujkZOg45Pg704vHOeg0rBbIoSNfjDUVZWjwC95K1 + BMN3msOTL9juv/EDa6BujqCxl+G1nY7JPFDLSHWis65p+1AAa5xieYDb47vyJ0SS + R7lEURTXZOkkM6k5JWfgkATEmGzRxPkOloITXg9ag1OlAgMBAAGjUzBRMB0GA1Ud + DgQWBBRYUSKDHBBE9Q6fTeTqogicCxcXwDAfBgNVHSMEGDAWgBRYUSKDHBBE9Q6f + TeTqogicCxcXwDAPBgNVHRMBAf8EBTADAQH/MA0GCSqGSIb3DQEBCwUAA4IBAQBc + T8B+GpvPy9NQ700LsywRPY0L9IJCKiu6j3TP1tqqSPjAC/cg9ac+bFXuWOu7V+KJ + s09Q/pItq9SLX6UvnfRzTxu5lCBwwGX9cL131mTIu5SmFo7Eks+sorbiIarWDMoC + e+9An3GFpagW+YhOt4BdIM5lTqoeodzganDBsOUZI9aDAj2Yo5h2O7r6Wd12cb6T + mz8vMfB2eG8BxU20ZMfkdERWjiyXHOSBQqeqfkV8d9370gMu5RcJNcIgnbmTRdho + X3HJFiimZVaNjXATqmC/y2A1KXvJdamPLy3mGXkW2cFLoPCdK2OZFUHqiuc1bigA + qEf55SihFqErRMeURPPF + -----END CERTIFICATE----- +data_stream: + vars: + listen_port: 9572 + preserve_original_event: true + preserve_duplicate_custom_fields: true diff --git a/packages/darktrace/data_stream/model_breach_alert/_dev/test/system/test-udp-config.yml b/packages/darktrace/data_stream/model_breach_alert/_dev/test/system/test-udp-config.yml new file mode 100644 index 00000000000..c18b3883870 --- /dev/null +++ b/packages/darktrace/data_stream/model_breach_alert/_dev/test/system/test-udp-config.yml @@ -0,0 +1,10 @@ +service: darktrace-model_breach_alert-udp +service_notify_signal: SIGHUP +input: udp +vars: + listen_address: 0.0.0.0 +data_stream: + vars: + listen_port: 9575 + preserve_original_event: true + preserve_duplicate_custom_fields: true diff --git a/packages/darktrace/data_stream/model_breach_alert/agent/stream/httpjson.yml.hbs b/packages/darktrace/data_stream/model_breach_alert/agent/stream/httpjson.yml.hbs new file mode 100644 index 00000000000..8201241e037 --- /dev/null +++ b/packages/darktrace/data_stream/model_breach_alert/agent/stream/httpjson.yml.hbs @@ -0,0 +1,53 @@ +config_version: 2 +interval: {{interval}} +request.method: GET +{{#if proxy_url}} +request.proxy_url: {{proxy_url}} +{{/if}} +{{#if ssl}} +request.ssl: {{ssl}} +{{/if}} +request.url: {{url}}/modelbreaches?expandenums=true&historicmodelonly=true&includeacknowledged=true&includebreachurl=true +{{#if http_client_timeout}} +request.timeout: {{http_client_timeout}} +{{/if}} +request.transforms: + - set: + target: header.DTAPI-Token + value: {{public_token}} + - set: + target: header.DTAPI-Date + value: '[[formatDate (now) "20060102T150405"]]' + - set: + target: url.params.group + value: 'device' + - set: + target: url.params.starttime + value: '[[.cursor.last_execution_datetime]]' + default: '[[(now (parseDuration "-{{initial_interval}}")).UnixMilli]]' + - set: + target: url.params.endtime + value: '[[(now).UnixMilli]]' + - set: + target: header.DTAPI-Signature + value: '[[hmac "sha1" "{{private_token}}" (sprintf "%s?%s\n%s\n%s" .url.Path .url.RawQuery "{{public_token}}" (formatDate (now) "20060102T150405"))]]' +cursor: + last_execution_datetime: + value: '[[toInt .last_event.time]]' +tags: +{{#if preserve_original_event}} + - preserve_original_event +{{/if}} +{{#if preserve_duplicate_custom_fields}} + - preserve_duplicate_custom_fields +{{/if}} +{{#each tags as |tag|}} + - {{tag}} +{{/each}} +{{#contains "forwarded" tags}} +publisher_pipeline.disable_host: true +{{/contains}} +{{#if processors}} +processors: +{{processors}} +{{/if}} diff --git a/packages/darktrace/data_stream/model_breach_alert/agent/stream/tcp.yml.hbs b/packages/darktrace/data_stream/model_breach_alert/agent/stream/tcp.yml.hbs new file mode 100644 index 00000000000..b1d260f0f9c --- /dev/null +++ b/packages/darktrace/data_stream/model_breach_alert/agent/stream/tcp.yml.hbs @@ -0,0 +1,26 @@ +host: "{{listen_address}}:{{listen_port}}" +{{#if tcp_options}} +{{tcp_options}} +{{/if}} +{{#if ssl}} +ssl: {{ssl}} +{{/if}} +tags: +{{#if preserve_original_event}} + - preserve_original_event +{{/if}} +{{#if preserve_duplicate_custom_fields}} + - preserve_duplicate_custom_fields +{{/if}} +{{#each tags as |tag|}} + - {{tag}} +{{/each}} +{{#contains "forwarded" tags}} +publisher_pipeline.disable_host: true +{{/contains}} +processors: +- syslog: + field: message +{{#if processors}} +{{processors}} +{{/if}} diff --git a/packages/darktrace/data_stream/model_breach_alert/agent/stream/udp.yml.hbs b/packages/darktrace/data_stream/model_breach_alert/agent/stream/udp.yml.hbs new file mode 100644 index 00000000000..f342c4fa75c --- /dev/null +++ b/packages/darktrace/data_stream/model_breach_alert/agent/stream/udp.yml.hbs @@ -0,0 +1,23 @@ +host: "{{listen_address}}:{{listen_port}}" +{{#if udp_options}} +{{udp_options}} +{{/if}} +tags: +{{#if preserve_original_event}} + - preserve_original_event +{{/if}} +{{#if preserve_duplicate_custom_fields}} + - preserve_duplicate_custom_fields +{{/if}} +{{#each tags as |tag|}} + - {{tag}} +{{/each}} +{{#contains "forwarded" tags}} +publisher_pipeline.disable_host: true +{{/contains}} +processors: +- syslog: + field: message +{{#if processors}} +{{processors}} +{{/if}} diff --git a/packages/darktrace/data_stream/model_breach_alert/elasticsearch/ingest_pipeline/default.yml b/packages/darktrace/data_stream/model_breach_alert/elasticsearch/ingest_pipeline/default.yml new file mode 100644 index 00000000000..126107e391c --- /dev/null +++ b/packages/darktrace/data_stream/model_breach_alert/elasticsearch/ingest_pipeline/default.yml @@ -0,0 +1,1446 @@ +--- +description: Pipeline for processing Model Breach Alert logs. +processors: + - set: + field: ecs.version + value: '8.4.0' + - grok: + field: message + patterns: + - "^%{FIELD:log.syslog.appname}\\s*%{GREEDYDATA:message}$" + pattern_definitions: + FIELD: "[a-zA-Z]*" + - rename: + field: message + target_field: event.original + ignore_missing: true + - json: + field: event.original + target_field: json + - fingerprint: + fields: + - json.time + - json.creationTime + - json.pbid + - json.model.phid + target_field: _id + ignore_missing: true + - set: + field: event.kind + value: alert + if: (['critical','suspicious'].contains(ctx.json?.model?.category?.toLowerCase())) + - set: + field: event.kind + value: event + if: (['compliance','informational'].contains(ctx.json?.model?.category?.toLowerCase())) + - set: + field: event.category + value: [threat] + if: ctx.event?.kind == 'alert' + - set: + field: event.type + value: [info] + - script: + description: Dynamically map event.* fields from metric label. + lang: painless + if: ctx.json?.triggeredComponents instanceof List + ignore_failure: true + source: + for (component in ctx.json.triggeredComponents) { + if (component?.metric?.label?.toLowerCase().contains('connection')) { + ctx.event?.type?.add('connection'); + if (ctx.event.category == null) { + ctx.event.category = new ArrayList(); + } + ctx.event.category.add('network'); + } + } + - foreach: + field: json.aianalystData + if: ctx.json?.aianalystData instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.related + ignore_failure: true + processor: + convert: + field: _ingest._value + type: long + on_failure: + - remove: + field: _ingest._value + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - rename: + field: json.aianalystData + target_field: darktrace.model_breach_alert.aianalyst_data + ignore_missing: true + - uri_parts: + field: json.breachUrl + target_field: darktrace.model_breach_alert.breach_url + if: ctx.json?.breachUrl != null + keep_original: true + on_failure: + - remove: + field: json.breachUrl + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - set: + field: event.url + copy_from: darktrace.model_breach_alert.breach_url.original + ignore_failure: true + - convert: + field: json.commentCount + target_field: darktrace.model_breach_alert.comment.count + type: long + ignore_missing: true + on_failure: + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - date: + field: json.creationTime + target_field: darktrace.model_breach_alert.creation_time + formats: + - ISO8601 + - UNIX_MS + - 'MMM dd HH:mm:ss' + if: ctx.json?.creationTime != null + on_failure: + - remove: + field: json.creationTime + ignore_missing: true + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - set: + field: event.created + copy_from: darktrace.model_breach_alert.creation_time + ignore_failure: true + - convert: + field: json.devicescore + target_field: darktrace.model_breach_alert.device_score + type: double + ignore_missing: true + on_failure: + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - rename: + field: json.device.credentials + target_field: darktrace.model_breach_alert.device.credentials + ignore_missing: true + - convert: + field: json.device.did + target_field: darktrace.model_breach_alert.device.did + type: long + ignore_missing: true + on_failure: + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - remove: + field: darktrace.model_breach_alert.device.did + ignore_missing: true + if: ctx.darktrace?.model_breach_alert?.device?.did != null && ctx.darktrace?.model_breach_alert?.device?.did < 0 + - convert: + field: darktrace.model_breach_alert.device.did + target_field: host.id + type: string + ignore_missing: true + on_failure: + - remove: + field: darktrace.model_breach_alert.device.did + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - date: + field: json.device.firstSeen + target_field: darktrace.model_breach_alert.device.first_seen + formats: + - ISO8601 + - UNIX_MS + - 'MMM dd HH:mm:ss' + if: ctx.json?.device?.firstSeen != null + on_failure: + - remove: + field: json.device.firstseen + ignore_missing: true + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - convert: + field: json.device.hostname + target_field: json.device._temp_.hostname_ip + type: ip + ignore_missing: true + on_failure: + - set: + field: host.hostname + copy_from: json.device.hostname + ignore_failure: true + - append: + field: related.ip + value: '{{{json.device._temp_.hostname_ip}}}' + allow_duplicates: false + ignore_failure: true + - rename: + field: json.device.hostname + target_field: darktrace.model_breach_alert.device.hostname + ignore_missing: true + - append: + field: related.hosts + value: '{{{host.hostname}}}' + allow_duplicates: false + ignore_failure: true + - convert: + field: json.device.ip + target_field: darktrace.model_breach_alert.device._temp_.ip + type: ip + ignore_failure: true + - append: + field: host.ip + value: '{{{darktrace.model_breach_alert.device._temp_.ip}}}' + allow_duplicates: false + ignore_failure: true + - convert: + field: json.device.ip6 + target_field: darktrace.model_breach_alert.device._temp_.ip6 + type: ip + ignore_failure: true + - append: + field: host.ip + value: '{{{darktrace.model_breach_alert.device._temp_.ip6}}}' + allow_duplicates: false + ignore_failure: true + - rename: + field: json.device.ip + target_field: darktrace.model_breach_alert.device.ip + ignore_missing: true + - rename: + field: json.device.ip6 + target_field: darktrace.model_breach_alert.device.ip6 + ignore_missing: true + - remove: + field: + - darktrace.model_breach_alert.device._temp_ + ignore_missing: true + - foreach: + field: host.ip + if: ctx.host?.ip instanceof List + ignore_failure: true + processor: + append: + field: related.ip + value: '{{{_ingest._value}}}' + allow_duplicates: false + ignore_failure: true + - foreach: + field: json.device.ips + if: ctx.json?.device?.ips instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value.ip + target_field: _ingest._value._temp_.ip + type: ip + ignore_failure: true + - foreach: + field: json.device.ips + if: ctx.json?.device?.ips instanceof List + ignore_failure: true + processor: + append: + field: related.ip + value: '{{{_ingest._value._temp_.ip}}}' + allow_duplicates: false + ignore_failure: true + - foreach: + field: json.device.ips + if: ctx.json?.device?.ips instanceof List + ignore_failure: true + processor: + date: + field: _ingest._value.timems + target_field: _ingest._value.timems + formats: + - ISO8601 + - UNIX_MS + - 'MMM dd HH:mm:ss' + on_failure: + - remove: + field: _ingest._value.timems + ignore_missing: true + - foreach: + field: json.device.ips + if: ctx.json?.device?.ips instanceof List + ignore_failure: true + processor: + date: + field: _ingest._value.time + target_field: _ingest._value.time + formats: + - ISO8601 + - UNIX_MS + - 'MMM dd HH:mm:ss' + - 'yyyy-MM-dd HH:mm:ss' + on_failure: + - remove: + field: _ingest._value.time + ignore_missing: true + - foreach: + field: json.device.ips + if: ctx.json?.device?.ips instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value.sid + type: long + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.sid + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - foreach: + field: json.device.ips + if: ctx.json?.device?.ips instanceof List + ignore_failure: true + processor: + remove: + field: _ingest._value._temp_ + ignore_missing: true + - rename: + field: json.device.ips + target_field: darktrace.model_breach_alert.device.ips + ignore_missing: true + - date: + field: json.device.lastSeen + target_field: darktrace.model_breach_alert.device.last_seen + formats: + - ISO8601 + - UNIX_MS + - 'MMM dd HH:mm:ss' + if: ctx.json?.device?.lastSeen != null + on_failure: + - remove: + field: json.device.lastseen + ignore_missing: true + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - gsub: + field: json.device.macaddress + target_field: darktrace.model_breach_alert.device.mac_address + pattern: '[:.]' + replacement: '-' + ignore_missing: true + - uppercase: + field: darktrace.model_breach_alert.device.mac_address + ignore_missing: true + - set: + field: host.mac + copy_from: darktrace.model_breach_alert.device.mac_address + ignore_failure: true + - convert: + field: json.device.sid + target_field: darktrace.model_breach_alert.device.sid + type: long + ignore_missing: true + on_failure: + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - foreach: + field: json.device.tags + if: ctx.json?.device?.tags instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value.tid + type: long + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.tid + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - foreach: + field: json.device.tags + if: ctx.json?.device?.tags instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value.thid + type: long + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.thid + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - foreach: + field: json.device.tags + if: ctx.json?.device?.tags instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value.expiry + type: long + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.expiry + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - foreach: + field: json.device.tags + if: ctx.json?.device?.tags instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value.restricted + type: boolean + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.restricted + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - foreach: + field: json.device.tags + if: ctx.json?.device?.tags instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value.isReferenced + target_field: _ingest._value.is_referenced + type: boolean + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.isReferenced + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - foreach: + field: json.device.tags + if: ctx.json?.device?.tags instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value.data.auto + type: boolean + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.data.auto + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - foreach: + field: json.device.tags + if: ctx.json?.device?.tags instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value.data.color + type: long + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.color + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - foreach: + field: json.device.tags + if: ctx.json?.device?.tags instanceof List + ignore_failure: true + processor: + remove: + field: _ingest._value.isReferenced + ignore_missing: true + - rename: + field: json.device.tags + target_field: darktrace.model_breach_alert.device.tags + ignore_missing: true + - rename: + field: json.device.typelabel + target_field: darktrace.model_breach_alert.device.type_label + ignore_missing: true + - rename: + field: json.device.typename + target_field: darktrace.model_breach_alert.device.type_name + ignore_missing: true + - set: + field: host.type + copy_from: darktrace.model_breach_alert.device.type_name + ignore_failure: true + - rename: + field: json.device.vendor + target_field: darktrace.model_breach_alert.device.vendor + ignore_missing: true + - convert: + field: json.acknowledged + target_field: darktrace.model_breach_alert.is_acknowledged + type: boolean + ignore_missing: true + on_failure: + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - foreach: + field: json.mitreTechniques + if: ctx.json?.mitreTechniques instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.techniqueID + target_field: _ingest._value.id + ignore_missing: true + - foreach: + field: json.mitreTechniques + if: ctx.json?.mitreTechniques instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.technique + target_field: _ingest._value.name + ignore_missing: true + - rename: + field: json.mitreTechniques + target_field: darktrace.model_breach_alert.mitre_techniques + ignore_missing: true + - foreach: + field: darktrace.model_breach_alert.mitre_techniques + if: ctx.darktrace?.model_breach_alert?.mitre_techniques instanceof List + ignore_failure: true + processor: + append: + field: threat.technique.id + value: '{{{_ingest._value.id}}}' + allow_duplicates: false + ignore_failure: true + - foreach: + field: darktrace.model_breach_alert.mitre_techniques + if: ctx.darktrace?.model_breach_alert?.mitre_techniques instanceof List + ignore_failure: true + processor: + append: + field: threat.technique.name + value: '{{{_ingest._value.name}}}' + allow_duplicates: false + ignore_failure: true + - rename: + field: json.model.actions.antigena.action + target_field: darktrace.model_breach_alert.model.actions.antigena.action + ignore_missing: true + - set: + field: event.action + copy_from: darktrace.model_breach_alert.model.actions.antigena.action + ignore_failure: true + - convert: + field: json.model.actions.antigena.duration + target_field: darktrace.model_breach_alert.model.actions.antigena.duration + type: long + ignore_missing: true + on_failure: + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - convert: + field: json.model.actions.antigena.confirm + target_field: darktrace.model_breach_alert.model.actions.antigena.is_confirm_by_human_operator + type: boolean + ignore_missing: true + on_failure: + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - convert: + field: json.model.actions.antigena.threshold + target_field: darktrace.model_breach_alert.model.actions.antigena.threshold + type: long + ignore_missing: true + on_failure: + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - convert: + field: json.model.actions.alert + target_field: darktrace.model_breach_alert.model.actions.is_alerting + type: boolean + ignore_missing: true + on_failure: + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - convert: + field: json.model.actions.breach + target_field: darktrace.model_breach_alert.model.actions.is_breach + type: boolean + ignore_missing: true + on_failure: + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - convert: + field: json.model.actions.setPriority + target_field: darktrace.model_breach_alert.model.actions.is_priority_set + type: boolean + ignore_missing: true + on_failure: + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - convert: + field: json.model.actions.setTag + target_field: darktrace.model_breach_alert.model.actions.is_tag_set + type: boolean + ignore_missing: true + on_failure: + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - convert: + field: json.model.actions.setType + target_field: darktrace.model_breach_alert.model.actions.is_type_set + type: boolean + ignore_missing: true + on_failure: + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - convert: + field: json.model.actions.model + target_field: darktrace.model_breach_alert.model.actions.model + type: boolean + ignore_missing: true + on_failure: + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - convert: + field: json.model.activeTimes.version + type: long + ignore_missing: true + on_failure: + - remove: + field: json.model.activeTimes.version + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - rename: + field: json.model.activeTimes + target_field: darktrace.model_breach_alert.model.active_times + ignore_missing: true + - rename: + field: json.model.behaviour + target_field: darktrace.model_breach_alert.model.behaviour + ignore_missing: true + - rename: + field: json.model.category + target_field: darktrace.model_breach_alert.model.category + ignore_missing: true + - set: + field: rule.category + copy_from: darktrace.model_breach_alert.model.category + ignore_failure: true + - rename: + field: json.model.created.by + target_field: darktrace.model_breach_alert.model.created.by + ignore_missing: true + - append: + field: related.user + value: '{{{darktrace.model_breach_alert.model.created.by}}}' + allow_duplicates: false + ignore_failure: true + - set: + field: rule.author + copy_from: darktrace.model_breach_alert.model.created.by + ignore_failure: true + - foreach: + field: json.model.defeats + if: ctx.json?.model?.defeats instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value.defeatID + target_field: _ingest._value.id + type: long + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.defeatID + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - foreach: + field: json.model.defeats + if: ctx.json?.model?.defeats instanceof List + ignore_failure: true + processor: + remove: + field: _ingest._value.defeatID + ignore_missing: true + - rename: + field: json.model.defeats + target_field: darktrace.model_breach_alert.model.defeats + ignore_missing: true + - convert: + field: json.model.delay + target_field: darktrace.model_breach_alert.model.delay + type: long + ignore_missing: true + on_failure: + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - rename: + field: json.model.description + target_field: darktrace.model_breach_alert.model.description + ignore_missing: true + - set: + field: rule.description + copy_from: darktrace.model_breach_alert.model.description + ignore_failure: true + - rename: + field: json.model.edited.by + target_field: darktrace.model_breach_alert.model.edited.by + ignore_missing: true + - append: + field: related.user + value: '{{{darktrace.model_breach_alert.model.edited.by}}}' + allow_duplicates: false + ignore_failure: true + - convert: + field: json.model.compliance + target_field: darktrace.model_breach_alert.model.in_compliance_behavior_category + type: boolean + ignore_missing: true + on_failure: + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - convert: + field: json.model.interval + target_field: darktrace.model_breach_alert.model.interval + type: long + ignore_missing: true + on_failure: + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - convert: + field: json.model.active + target_field: darktrace.model_breach_alert.model.is_active + type: boolean + ignore_missing: true + on_failure: + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - convert: + field: json.model.autoSuppress + target_field: darktrace.model_breach_alert.model.is_auto_suppress + type: boolean + ignore_missing: true + on_failure: + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - convert: + field: json.model.autoUpdatable + target_field: darktrace.model_breach_alert.model.is_auto_updatable + type: boolean + ignore_missing: true + on_failure: + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - convert: + field: json.model.autoUpdate + target_field: darktrace.model_breach_alert.model.is_auto_update + type: boolean + ignore_missing: true + on_failure: + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - convert: + field: json.model.sequenced + target_field: darktrace.model_breach_alert.model.is_sequenced + type: boolean + ignore_missing: true + on_failure: + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - convert: + field: json.model.sharedEndpoints + target_field: darktrace.model_breach_alert.model.is_shared_endpoints + type: boolean + ignore_missing: true + on_failure: + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - script: + description: Dynamically map model.logic.data array using model.logic.type field. + if: ctx.json?.model?.logic?.data instanceof List + lang: painless + ignore_failure: true + params: + componentList: data_component_list + weightedComponentList: data_weighted_component_list + source: + def data = ctx.json.model.logic.data; + if (ctx.json.model.logic?.type != null) { + if (['componentList', 'weightedComponentList'].contains(ctx.json.model.logic?.type)) { + ctx["json"]["model"]["logic"][params.get(ctx.json.model.logic?.type)] = data; + } else { + ctx["json"]["model"]["logic"]["data_" + ctx.json.model.logic?.type] = data; + } + } + ctx.json.model.logic.remove("data"); + - convert: + field: json.model.logic.data_component_list + type: long + ignore_missing: true + on_failure: + - remove: + field: json.model.logic.data_component_list + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - convert: + field: json.model.logic.data_weighted_component_list.cid + type: long + ignore_missing: true + on_failure: + - remove: + field: json.model.logic.data_weighted_component_list.cid + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - convert: + field: json.model.logic.data_weighted_component_list.weight + type: long + ignore_missing: true + on_failure: + - remove: + field: json.model.logic.data_weighted_component_list.weight + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - convert: + field: json.model.logic.version + type: long + ignore_missing: true + on_failure: + - remove: + field: json.model.logic.version + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - convert: + field: json.model.logic.targetScore + target_field: json.model.logic.target_score + type: long + ignore_missing: true + on_failure: + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - remove: + field: json.model.logic.targetScore + ignore_missing: true + - rename: + field: json.model.logic + target_field: darktrace.model_breach_alert.model.logic + ignore_missing: true + - date: + field: json.model.modified + target_field: darktrace.model_breach_alert.model.modified + formats: + - ISO8601 + - UNIX_MS + - 'MMM dd HH:mm:ss' + - 'yyyy-MM-dd HH:mm:ss' + if: ctx.json?.model?.modified != null + on_failure: + - remove: + field: json.model.modified + ignore_missing: true + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - rename: + field: json.model.name + target_field: darktrace.model_breach_alert.model.name + ignore_missing: true + - set: + field: rule.name + copy_from: darktrace.model_breach_alert.model.name + ignore_failure: true + - convert: + field: json.model.phid + target_field: darktrace.model_breach_alert.model.phid + type: long + ignore_missing: true + on_failure: + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - convert: + field: json.model.pid + target_field: darktrace.model_breach_alert.model.pid + type: long + ignore_missing: true + on_failure: + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - convert: + field: json.model.priority + target_field: darktrace.model_breach_alert.model.priority + type: long + ignore_missing: true + on_failure: + - remove: + field: json.model.priority + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - set: + field: event.severity + copy_from: darktrace.model_breach_alert.model.priority + ignore_failure: true + - rename: + field: json.model.tags + target_field: darktrace.model_breach_alert.model.tags + ignore_missing: true + - set: + field: rule.ruleset + copy_from: darktrace.model_breach_alert.model.tags + ignore_failure: true + - convert: + field: json.model.throttle + target_field: darktrace.model_breach_alert.model.throttle + type: long + ignore_missing: true + on_failure: + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - convert: + field: json.model.userID + target_field: darktrace.model_breach_alert.model.userid + type: long + ignore_missing: true + on_failure: + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - rename: + field: json.model.uuid + target_field: darktrace.model_breach_alert.model.uuid + ignore_missing: true + - set: + field: rule.uuid + copy_from: darktrace.model_breach_alert.model.uuid + ignore_failure: true + - convert: + field: json.model.version + target_field: darktrace.model_breach_alert.model.version + type: long + ignore_missing: true + on_failure: + - remove: + field: json.model.version + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - convert: + field: darktrace.model_breach_alert.model.version + target_field: rule.version + type: string + ignore_missing: true + on_failure: + - remove: + field: darktrace.model_breach_alert.model.version + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - convert: + field: json.pbscore + target_field: darktrace.model_breach_alert.pb_score + type: double + ignore_missing: true + on_failure: + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - convert: + field: json.pbid + target_field: darktrace.model_breach_alert.pbid + type: long + ignore_missing: true + on_failure: + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - convert: + field: json.score + target_field: darktrace.model_breach_alert.score + type: double + ignore_missing: true + on_failure: + - remove: + field: json.score + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - set: + field: event.risk_score + copy_from: darktrace.model_breach_alert.score + ignore_failure: true + - script: + description: Normalize event.risk_score to event.risk_score_norm + lang: painless + if: ctx.event?.risk_score != null + source: + def normalizedRiskScore = ctx.event.risk_score * 100.0; + ctx.event.risk_score_norm = normalizedRiskScore; + - date: + field: json.time + target_field: darktrace.model_breach_alert.time + formats: + - ISO8601 + - UNIX_MS + - 'MMM dd HH:mm:ss' + if: ctx.json?.time != null + on_failure: + - remove: + field: json.time + ignore_missing: true + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - set: + field: '@timestamp' + copy_from: darktrace.model_breach_alert.time + ignore_failure: true + - foreach: + field: json.triggeredComponents + if: ctx.json?.triggeredComponents instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value.cbid + type: long + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.cbid + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - foreach: + field: json.triggeredComponents + if: ctx.json?.triggeredComponents instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value.chid + type: long + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.chid + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - foreach: + field: json.triggeredComponents + if: ctx.json?.triggeredComponents instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value.cid + type: long + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.cid + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - foreach: + field: json.triggeredComponents + if: ctx.json?.triggeredComponents instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value.interval + type: long + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.interval + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - script: + description: Stringify logic.data field of triggeredComponents array. + lang: painless + if: ctx.json?.triggeredComponents instanceof List + ignore_failure: true + source: + for (component in ctx.json.triggeredComponents) { + component.logic.data = component?.logic?.data.toString(); + } + - foreach: + field: json.triggeredComponents + if: ctx.json?.triggeredComponents instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value.metric.mlid + type: long + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.metric.mlid + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - foreach: + field: json.triggeredComponents + if: ctx.json?.triggeredComponents instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value.size + type: long + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.size + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - foreach: + field: json.triggeredComponents + if: ctx.json?.triggeredComponents instanceof List + ignore_failure: true + processor: + convert: + field: _ingest._value.threshold + type: long + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.threshold + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - foreach: + field: json.triggeredComponents + if: ctx.json?.triggeredComponents instanceof List + ignore_failure: true + processor: + date: + field: _ingest._value.time + target_field: _ingest._value.time + formats: + - ISO8601 + - UNIX_MS + - 'MMM dd HH:mm:ss' + on_failure: + - remove: + field: _ingest._value.time + ignore_missing: true + - foreach: + field: json.triggeredComponents + if: ctx.json?.triggeredComponents instanceof List + ignore_failure: true + processor: + append: + field: event.start + value: '{{{_ingest._value.time}}}' + allow_duplicates: false + ignore_failure: true + - foreach: + field: json.triggeredComponents + if: ctx.json?.triggeredComponents instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.triggeredFilters + ignore_failure: true + processor: + convert: + field: _ingest._value.cfid + type: long + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.cfid + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - foreach: + field: json.triggeredComponents + if: ctx.json?.triggeredComponents instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.triggeredFilters + ignore_failure: true + processor: + rename: + field: _ingest._value.comparatorType + target_field: _ingest._value.comparator_type + ignore_missing: true + - foreach: + field: json.triggeredComponents + if: ctx.json?.triggeredComponents instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.triggeredFilters + ignore_failure: true + processor: + rename: + field: _ingest._value.filterType + target_field: _ingest._value.filter_type + ignore_missing: true + - foreach: + field: json.triggeredComponents + if: ctx.json?.triggeredComponents instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.triggeredFilters + ignore_failure: true + processor: + convert: + field: _ingest._value.trigger.tag.data.auto + type: boolean + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.trigger.tag.data.auto + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - foreach: + field: json.triggeredComponents + if: ctx.json?.triggeredComponents instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.triggeredFilters + ignore_failure: true + processor: + convert: + field: _ingest._value.trigger.tag.data.color + type: long + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.trigger.tag.data.color + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - foreach: + field: json.triggeredComponents + if: ctx.json?.triggeredComponents instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.triggeredFilters + ignore_failure: true + processor: + convert: + field: _ingest._value.trigger.tag.expiry + type: long + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.trigger.tag.expiry + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - foreach: + field: json.triggeredComponents + if: ctx.json?.triggeredComponents instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.triggeredFilters + ignore_failure: true + processor: + convert: + field: _ingest._value.trigger.tag.isReferenced + target_field: _ingest._value.trigger.tag.is_referenced + type: boolean + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.trigger.tag.isReferenced + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - foreach: + field: json.triggeredComponents + if: ctx.json?.triggeredComponents instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.triggeredFilters + ignore_failure: true + processor: + convert: + field: _ingest._value.trigger.tag.restricted + type: boolean + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.trigger.tag.restricted + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - foreach: + field: json.triggeredComponents + if: ctx.json?.triggeredComponents instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.triggeredFilters + ignore_failure: true + processor: + convert: + field: _ingest._value.trigger.tag.thid + type: long + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.trigger.tag.thid + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - foreach: + field: json.triggeredComponents + if: ctx.json?.triggeredComponents instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.triggeredFilters + ignore_failure: true + processor: + convert: + field: _ingest._value.trigger.tag.tid + type: long + ignore_missing: true + on_failure: + - remove: + field: _ingest._value.trigger.tag.tid + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - foreach: + field: json.triggeredComponents + if: ctx.json?.triggeredComponents instanceof List + ignore_failure: true + processor: + foreach: + field: _ingest._value.triggeredFilters + ignore_failure: true + processor: + remove: + field: _ingest._value.trigger.tag.isReferenced + ignore_missing: true + - foreach: + field: json.triggeredComponents + if: ctx.json?.triggeredComponents instanceof List + ignore_failure: true + processor: + rename: + field: _ingest._value.triggeredFilters + target_field: _ingest._value.triggered_filters + ignore_missing: true + - rename: + field: json.triggeredComponents + target_field: darktrace.model_breach_alert.triggered_components + ignore_missing: true + - remove: + field: json + ignore_missing: true + - remove: + if: ctx.tags == null || !(ctx.tags.contains('preserve_duplicate_custom_fields')) + field: + - darktrace.model_breach_alert.time + - darktrace.model_breach_alert.model.actions.antigena.action + - darktrace.model_breach_alert.creation_time + - darktrace.model_breach_alert.score + - darktrace.model_breach_alert.model.priority + - darktrace.model_breach_alert.device.did + - darktrace.model_breach_alert.device.mac_address + - darktrace.model_breach_alert.device.type_name + - darktrace.model_breach_alert.model.created.by + - darktrace.model_breach_alert.model.category + - darktrace.model_breach_alert.model.description + - darktrace.model_breach_alert.model.name + - darktrace.model_breach_alert.model.tags + - darktrace.model_breach_alert.model.uuid + - darktrace.model_breach_alert.model.version + - darktrace.model_breach_alert.mitre_techniques + ignore_failure: true + ignore_missing: true + - foreach: + field: darktrace.model_breach_alert.triggered_components + if: ctx.darktrace?.model_breach_alert?.triggered_components instanceof List && (ctx.tags == null || !(ctx.tags.contains('preserve_duplicate_custom_fields'))) + ignore_failure: true + processor: + remove: + field: + - _ingest._value.time + ignore_missing: true + ignore_failure: true + - remove: + field: event.original + if: ctx.tags == null || !(ctx.tags.contains('preserve_original_event')) + ignore_failure: true + ignore_missing: true + - script: + description: Drops null/empty values recursively. + lang: painless + source: + boolean dropEmptyFields(Object object) { + if (object == null || object == "") { + return true; + } else if (object instanceof Map) { + ((Map) object).values().removeIf(value -> dropEmptyFields(value)); + return (((Map) object).size() == 0); + } else if (object instanceof List) { + ((List) object).removeIf(value -> dropEmptyFields(value)); + return (((List) object).length == 0); + } + return false; + } + dropEmptyFields(ctx); +on_failure: + - append: + field: error.message + value: '{{{ _ingest.on_failure_message }}}' diff --git a/packages/darktrace/data_stream/model_breach_alert/fields/agent.yml b/packages/darktrace/data_stream/model_breach_alert/fields/agent.yml new file mode 100644 index 00000000000..1f754679d06 --- /dev/null +++ b/packages/darktrace/data_stream/model_breach_alert/fields/agent.yml @@ -0,0 +1,184 @@ +- name: cloud + title: Cloud + group: 2 + description: Fields related to the cloud or infrastructure the events are coming from. + footnote: 'Examples: If Metricbeat is running on an EC2 host and fetches data from its host, the cloud info contains the data about this machine. If Metricbeat runs on a remote machine outside the cloud and fetches data from a service running in the cloud, the field contains cloud data from the machine the service is running on.' + type: group + fields: + - name: account.id + level: extended + type: keyword + ignore_above: 1024 + description: 'The cloud account or organization id used to identify different entities in a multi-tenant environment. Examples: AWS account id, Google Cloud ORG Id, or other unique identifier.' + example: 666777888999 + - name: availability_zone + level: extended + type: keyword + ignore_above: 1024 + description: Availability zone in which this host is running. + example: us-east-1c + - name: instance.id + level: extended + type: keyword + ignore_above: 1024 + description: Instance ID of the host machine. + example: i-1234567890abcdef0 + - name: instance.name + level: extended + type: keyword + ignore_above: 1024 + description: Instance name of the host machine. + - name: machine.type + level: extended + type: keyword + ignore_above: 1024 + description: Machine type of the host machine. + example: t2.medium + - name: provider + level: extended + type: keyword + ignore_above: 1024 + description: Name of the cloud provider. Example values are aws, azure, gcp, or digitalocean. + example: aws + - name: region + level: extended + type: keyword + ignore_above: 1024 + description: Region in which this host is running. + example: us-east-1 + - name: project.id + type: keyword + description: Name of the project in Google Cloud. + - name: image.id + type: keyword + description: Image ID for the cloud instance. +- name: container + title: Container + group: 2 + description: 'Container fields are used for meta information about the specific container that is the source of information. These fields help correlate data based containers from any runtime.' + type: group + fields: + - name: id + level: core + type: keyword + ignore_above: 1024 + description: Unique container id. + - name: image.name + level: extended + type: keyword + ignore_above: 1024 + description: Name of the image the container was built on. + - name: labels + level: extended + type: object + object_type: keyword + description: Image labels. + - name: name + level: extended + type: keyword + ignore_above: 1024 + description: Container name. +- name: host + title: Host + group: 2 + description: 'A host is defined as a general computing instance. ECS host.* fields should be populated with details about the host on which the event happened, or from which the measurement was taken. Host types include hardware, virtual machines, Docker containers, and Kubernetes nodes.' + type: group + fields: + - name: architecture + level: core + type: keyword + ignore_above: 1024 + description: Operating system architecture. + example: x86_64 + - name: domain + level: extended + type: keyword + ignore_above: 1024 + description: 'Name of the domain of which the host is a member. For example, on Windows this could be the host''s Active Directory domain or NetBIOS domain name. For Linux this could be the domain of the host''s LDAP provider.' + example: CONTOSO + default_field: false + - name: hostname + level: core + type: keyword + ignore_above: 1024 + description: 'Hostname of the host. It normally contains what the `hostname` command returns on the host machine.' + - name: id + level: core + type: keyword + ignore_above: 1024 + description: 'Unique host id. As hostname is not always unique, use values that are meaningful in your environment. Example: The current usage of `beat.name`.' + - name: ip + level: core + type: ip + description: Host ip addresses. + - name: mac + level: core + type: keyword + ignore_above: 1024 + description: Host mac addresses. + - name: name + level: core + type: keyword + ignore_above: 1024 + description: 'Name of the host. It can contain what `hostname` returns on Unix systems, the fully qualified domain name, or a name specified by the user. The sender decides which value to use.' + - name: os.family + level: extended + type: keyword + ignore_above: 1024 + description: OS family (such as redhat, debian, freebsd, windows). + example: debian + - name: os.kernel + level: extended + type: keyword + ignore_above: 1024 + description: Operating system kernel version as a raw string. + example: 4.4.0-112-generic + - name: os.name + level: extended + type: keyword + ignore_above: 1024 + multi_fields: + - name: text + type: text + norms: false + default_field: false + description: Operating system name, without the version. + example: Mac OS X + - name: os.platform + level: extended + type: keyword + ignore_above: 1024 + description: Operating system platform (such centos, ubuntu, windows). + example: darwin + - name: os.version + level: extended + type: keyword + ignore_above: 1024 + description: Operating system version as a raw string. + example: 10.14.1 + - name: type + level: core + type: keyword + ignore_above: 1024 + description: 'Type of host. For Cloud providers this can be the machine type like `t2.medium`. If vm, this could be the container, for example, or other information meaningful in your environment.' + - name: containerized + type: boolean + description: >- + If the host is a container. + - name: os.build + type: keyword + example: '18D109' + description: > + OS build information. + + - name: os.codename + type: keyword + example: 'stretch' + description: >- + OS codename, if any. +- name: input.type + type: keyword + description: Input type +- name: log.offset + type: long + description: Log offset diff --git a/packages/darktrace/data_stream/model_breach_alert/fields/base-fields.yml b/packages/darktrace/data_stream/model_breach_alert/fields/base-fields.yml new file mode 100644 index 00000000000..7dd51b599cc --- /dev/null +++ b/packages/darktrace/data_stream/model_breach_alert/fields/base-fields.yml @@ -0,0 +1,20 @@ +- name: data_stream.type + type: constant_keyword + description: Data stream type. +- name: data_stream.dataset + type: constant_keyword + description: Data stream dataset. +- name: data_stream.namespace + type: constant_keyword + description: Data stream namespace. +- name: event.module + type: constant_keyword + description: Event module. + value: darktrace +- name: event.dataset + type: constant_keyword + description: Event dataset. + value: darktrace.model_breach_alert +- name: '@timestamp' + type: date + description: Event timestamp. diff --git a/packages/darktrace/data_stream/model_breach_alert/fields/ecs.yml b/packages/darktrace/data_stream/model_breach_alert/fields/ecs.yml new file mode 100644 index 00000000000..2b34237b623 --- /dev/null +++ b/packages/darktrace/data_stream/model_breach_alert/fields/ecs.yml @@ -0,0 +1,74 @@ +- external: ecs + name: ecs.version +- external: ecs + name: event.action +- external: ecs + name: event.category +- external: ecs + name: event.created +- external: ecs + name: event.kind +- external: ecs + name: event.original +- external: ecs + name: event.risk_score +- external: ecs + name: event.risk_score_norm +- external: ecs + name: event.severity +- external: ecs + name: event.start +- external: ecs + name: event.type +- external: ecs + name: event.url +- external: ecs + name: host.hostname +- external: ecs + name: host.id +- external: ecs + name: host.ip +- external: ecs + name: host.type +- external: ecs + name: log.syslog.appname +- external: ecs + name: log.syslog.facility.code +- external: ecs + name: log.syslog.facility.name +- external: ecs + name: log.syslog.hostname +- external: ecs + name: log.syslog.priority +- external: ecs + name: log.syslog.severity.code +- external: ecs + name: log.syslog.severity.name +- external: ecs + name: log.syslog.version +- external: ecs + name: related.hosts +- external: ecs + name: related.ip +- external: ecs + name: related.user +- external: ecs + name: rule.author +- external: ecs + name: rule.category +- external: ecs + name: rule.description +- external: ecs + name: rule.name +- external: ecs + name: rule.ruleset +- external: ecs + name: rule.uuid +- external: ecs + name: rule.version +- external: ecs + name: tags +- external: ecs + name: threat.technique.id +- external: ecs + name: threat.technique.name diff --git a/packages/darktrace/data_stream/model_breach_alert/fields/fields.yml b/packages/darktrace/data_stream/model_breach_alert/fields/fields.yml new file mode 100644 index 00000000000..a9723624791 --- /dev/null +++ b/packages/darktrace/data_stream/model_breach_alert/fields/fields.yml @@ -0,0 +1,428 @@ +- name: darktrace.model_breach_alert + type: group + fields: + - name: aianalyst_data + type: group + fields: + - name: related + type: long + - name: summariser + type: keyword + - name: uuid + type: keyword + - name: breach_url + type: group + description: A link to the specific model breach in the Darktrace Threat Visualizer - the configuration option FQDN must be set for this field to appear. + fields: + - name: domain + type: keyword + - name: extension + type: keyword + - name: fragment + type: keyword + - name: full + type: keyword + - name: original + type: keyword + - name: password + type: keyword + - name: path + type: keyword + - name: port + type: long + - name: query + type: keyword + - name: scheme + type: keyword + - name: username + type: keyword + - name: comment + type: group + fields: + - name: count + type: long + description: The number of comments made against this breach. + - name: creation_time + type: date + description: The timestamp that the record of the breach was created. This is distinct from the “time” field. + - name: device_score + type: double + - name: device + type: group + fields: + - name: credentials + type: keyword + - name: did + type: long + description: The “device id”, a unique identifier. + - name: first_seen + type: date + description: The first time the device was seen on the network. + - name: hostname + type: keyword + description: The current device hostname. + - name: ip + type: keyword + description: The current IP associated with the device. + - name: ip6 + type: keyword + description: Current IPv6 address of this device if applicable, otherwise undefined. + - name: ips + type: group + fields: + - name: ip + type: keyword + description: A historic IP associated with the device. + - name: sid + type: long + description: The subnet id for the subnet the IP belongs to. + - name: time + type: date + description: The time the IP was last seen associated with that device in readable format. + - name: timems + type: date + description: The time the IP was last seen associated with that device in epoch time. + - name: last_seen + type: date + description: The last time the device was seen on the network. + - name: mac_address + type: keyword + description: The current MAC address associated with the device. + - name: sid + type: long + description: The subnet id for the subnet the device is currently located in. + - name: tags + type: group + fields: + - name: data + type: group + fields: + - name: auto + type: boolean + - name: color + type: long + - name: description + type: keyword + - name: visibility + type: keyword + - name: expiry + type: long + - name: is_referenced + type: boolean + - name: name + type: keyword + - name: restricted + type: boolean + - name: thid + type: long + - name: tid + type: long + - name: type_label + type: keyword + description: The device type in readable format. + - name: type_name + type: keyword + description: The device type in system format. + - name: vendor + type: keyword + description: The vendor of the device network card as derived by Darktrace from the MAC address. + - name: is_acknowledged + type: boolean + - name: mitre_techniques + type: group + description: Any mapped MITRE ATT&CK techniques the model corresponds to. + fields: + - name: name + type: keyword + - name: id + type: keyword + - name: model + type: group + fields: + - name: actions + type: group + fields: + - name: antigena + type: group + fields: + - name: action + type: keyword + description: The action to be performed. + - name: duration + type: long + description: The duration in seconds that the antigena action should last for. + - name: is_confirm_by_human_operator + type: boolean + description: Whether the action must be confirmed by a human operator, regardless of the global setting for Human Confirmation mode. + - name: threshold + type: long + description: The breach score threshold (out of 100) over which antigena will take an action. + - name: is_alerting + type: boolean + description: If true, an alert turned on will be pushed out to external systems if conditions for such alerting are met. + - name: is_breach + type: boolean + description: If true, an alert turned on will be pushed out to external systems if conditions for such alerting are met. + - name: is_priority_set + type: boolean + description: If the priority is to be changed on breach, the numeric value it should become. If no priority change action, a false boolean. + - name: is_tag_set + type: boolean + description: If a tag is to be applied on model breach, a single number or array of the system ID for the tag(s) to be applied. If no tag action, a false boolean. + - name: is_type_set + type: boolean + description: If a change device type action is to be applied on model breach, the numeric system ID for the label to be applied. If no change device type action is applied to the model, a false boolean. + - name: model + type: boolean + description: If true, creates an event in the device’s event log without creating an alert/ model breach in the threat tray. + - name: active_times + type: group + fields: + - name: devices + type: flattened + description: The device ids for devices on the list. + - name: tags + type: flattened + description: A system field. + - name: type + type: keyword + description: 'The type of list: “restrictions” indicates a blacklist, “exclusions” a whitelist.' + - name: version + type: long + description: A system field. + - name: behaviour + type: keyword + description: The score modulation function as set in the model editor. + - name: category + type: keyword + description: The behavior category of the model that was breached. + - name: created + type: group + fields: + - name: by + type: keyword + description: Username that created the model. + - name: defeats + type: group + fields: + - name: arguments + type: group + fields: + - name: value + type: keyword + description: The value(s) that must match for the defeat to take effect. + - name: comparator + type: keyword + description: The comparator that the value is compared against the create the defeat. + - name: filtertype + type: keyword + description: The filter the defeat is made from. + - name: id + type: long + description: A unique ID for the defeat. + - name: delay + type: long + description: Minimum delay in seconds after a positive-scoring component has fired before the overall model score is calculated. Only applicable in target score models. + - name: description + type: keyword + description: The optional description of the model. + - name: edited + type: group + fields: + - name: by + type: keyword + description: Username that last edited the model. + - name: userid + type: long + - name: in_compliance_behavior_category + type: boolean + description: Whether the model is in the compliance behavior category. + - name: interval + type: long + description: Where a model contains multiple components, this interval represents the time window in seconds in which all the components should fire for this model to be breached. + - name: is_active + type: boolean + description: Whether the model is enabled or disabled. + - name: is_auto_suppress + type: boolean + description: Whether the model will automatically be suppressed in the case of over-breaching. + - name: is_auto_updatable + type: boolean + description: Whether the model is suitable for auto update. + - name: is_auto_update + type: boolean + description: Whether the model is enabled for auto update. + - name: is_sequenced + type: boolean + description: Whether the components are required to fire in the specified order for the model breach to occur. + - name: is_shared_endpoints + type: boolean + description: For models that contain multiple components that reference an endpoint, this value indicates whether all endpoints should be identical for the model to fire. + - name: logic + type: group + fields: + - name: data_component_list + type: long + description: This will be a list of component ID numbers. + - name: data_weighted_component_list + type: group + description: This model is a weighted type this will be a list of component ID, weight object pairs. + fields: + - name: cid + type: long + - name: weight + type: long + - name: target_score + type: long + - name: type + type: keyword + description: The type of model. + - name: version + type: long + description: A number representing the version of model logic. + - name: modified + type: date + description: Timestamp at which the model was last modified, in a readable format. + - name: name + type: keyword + description: Name of the model that was breached. + - name: phid + type: long + description: The model “policy history” id. Increments when the model is modified. + - name: pid + type: long + description: The “policy id” of the model that was breached. + - name: priority + type: long + description: The model’s priority affects the strength with which it breaches (0-5 scale). + - name: tags + type: keyword + description: A list of tags that have been applied to this model in the Threat Visualizer model editor. + - name: throttle + type: long + description: For an individual device, this is the value in seconds for which this model will not fire again. + - name: uuid + type: keyword + description: A unique ID that is generated on creation of the model. + - name: version + type: long + description: The version of the model. Increments on each edit. + - name: pb_score + type: double + description: The model breach score, represented by a value between 0 and 1. + - name: pbid + type: long + description: The “policy breach ID” of the model breach. + - name: score + type: double + description: The model breach score, represented by a value between 0 and 1. + - name: time + type: date + description: The timestamp when the record was created in epoch time. + - name: triggered_components + type: group + fields: + - name: cbid + type: long + description: The “component breach id”. A unique identifier for the component breach. + - name: chid + type: long + description: The “component history id”. Increments when the component is edited. + - name: cid + type: long + description: The “component id”. A unique identifier. + - name: interval + type: long + description: The timeframe in seconds within which the threshold must be satisfied. + - name: logic + type: group + fields: + - name: data + type: text + description: It representing the logical relationship between component filters. Each filter is given an alphabetical reference and the contents of this field describe the relationship between those filters. + - name: version + type: keyword + description: The version of the component logic. + - name: metric + type: group + fields: + - name: label + type: keyword + description: The metric which data is returned for in readable format. + - name: mlid + type: long + description: The “metric logic” id - unique identifier. + - name: name + type: keyword + description: The metric which data is returned for in system format. + - name: size + type: long + description: The size of the value that was compared in the component. + - name: threshold + type: long + description: The threshold value that the size must exceed for the component to breach. + - name: time + type: date + description: A timestamp in Epoch time at which the components were triggered. + - name: triggered_filters + type: group + fields: + - name: arguments + type: group + fields: + - name: value + type: keyword + description: The value the filtertype should be compared against (using the specified comparator) to create the filter. + - name: cfid + type: long + description: The ‘component filter id’. A unique identifier for the filter as part of a the component. + - name: comparator_type + type: keyword + description: The comparator. A full list of comparators available for each filtertype can be found on the /filtertypes endpoint. + - name: filter_type + type: keyword + description: The filtertype that is used in the filter. A full list of filtertypes can be found on the /filtertypes endpoint. + - name: id + type: keyword + description: A filter that is used in the component logic. All filters are given alphabetical identifiers. Display filters - those that appear in the breach notification - can be identified by a lowercase ‘d’ and a numeral. + - name: trigger + type: group + fields: + - name: tag + type: group + fields: + - name: data + type: group + fields: + - name: auto + type: boolean + - name: color + type: long + - name: description + type: keyword + - name: visibility + type: keyword + - name: expiry + type: long + description: nan + - name: isReferenced + type: boolean + description: nan + - name: name + type: keyword + description: nan + - name: restricted + type: boolean + description: nan + - name: thid + type: long + description: nan + - name: tid + type: long + description: nan + - name: value + type: keyword + description: The actual value that triggered the filter. +- name: log.source.address + type: keyword + description: Source address from which the log event was read / sent from. diff --git a/packages/darktrace/data_stream/model_breach_alert/manifest.yml b/packages/darktrace/data_stream/model_breach_alert/manifest.yml new file mode 100644 index 00000000000..1f16664378b --- /dev/null +++ b/packages/darktrace/data_stream/model_breach_alert/manifest.yml @@ -0,0 +1,179 @@ +title: Collect Model Breach Alert logs from Darktrace +type: logs +streams: + - input: httpjson + title: Model Breach Alert logs + description: Collect Model Breach Alert logs via API. + template_path: httpjson.yml.hbs + vars: + - name: initial_interval + type: text + title: Initial Interval + description: How far back to pull the Model Breach Alert logs from Darktrace. NOTE:- Supported units for this parameter are h/m/s. + multi: false + required: true + show_user: true + default: 24h + - name: interval + type: text + title: Interval + description: Duration between requests to the Darktrace API. NOTE:- Supported units for this parameter are h/m/s. + default: 1m + multi: false + required: true + show_user: true + - name: http_client_timeout + type: text + title: HTTP Client Timeout + description: Duration before declaring that the HTTP client connection has timed out. NOTE:- Valid time units are ns, us, ms, s, m, h. + multi: false + required: true + show_user: false + default: 30s + - name: tags + type: text + title: Tags + multi: true + required: true + show_user: false + default: + - forwarded + - darktrace-model_breach_alert + - name: preserve_original_event + required: true + show_user: true + title: Preserve original event + description: Preserves a raw copy of the original event, added to the field `event.original`. + type: bool + multi: false + default: false + - name: preserve_duplicate_custom_fields + required: true + show_user: false + title: Preserve duplicate custom fields + description: Preserve darktrace.model_breach_alert fields that were copied to Elastic Common Schema (ECS) fields. + type: bool + multi: false + default: false + - name: processors + type: yaml + title: Processors + multi: false + required: false + show_user: false + description: >- + Processors are used to reduce the number of fields in the exported event or to enhance the event with metadata. This executes in the agent before the logs are parsed. See [Processors](https://www.elastic.co/guide/en/beats/filebeat/current/filtering-and-enhancing-data.html) for details. + - input: tcp + title: Model Breach Alert logs + description: Collect Model Breach Alert logs via TCP input. + template_path: tcp.yml.hbs + vars: + - name: listen_port + type: integer + title: Listen Port + description: The TCP port number to listen on. + multi: false + required: true + show_user: true + default: 9572 + - name: tcp_options + type: yaml + title: Custom TCP Options + multi: false + required: false + show_user: false + default: | + max_message_size: 50KiB + #max_connections: 1 + #framing: delimiter + #line_delimiter: "\n" + description: Specify custom configuration options for the TCP input. + - name: tags + type: text + title: Tags + multi: true + required: true + show_user: false + default: + - forwarded + - darktrace-model_breach_alert + - name: preserve_original_event + required: true + show_user: true + title: Preserve original event + description: Preserves a raw copy of the original event, added to the field `event.original`. + type: bool + multi: false + default: false + - name: preserve_duplicate_custom_fields + required: true + show_user: false + title: Preserve duplicate custom fields + description: Preserve darktrace.model_breach_alert fields that were copied to Elastic Common Schema (ECS) fields. + type: bool + multi: false + default: false + - name: processors + type: yaml + title: Processors + multi: false + required: false + show_user: false + description: >- + Processors are used to reduce the number of fields in the exported event or to enhance the event with metadata. This executes in the agent before the logs are parsed. See [Processors](https://www.elastic.co/guide/en/beats/filebeat/current/filtering-and-enhancing-data.html) for details. + - input: udp + title: Model Breach Alert logs + description: Collect Model Breach Alert logs via UDP input. + template_path: udp.yml.hbs + vars: + - name: listen_port + type: integer + title: Listen Port + description: The UDP port number to listen on. + multi: false + required: true + show_user: true + default: 9575 + - name: udp_options + type: yaml + title: Custom UDP Options + multi: false + required: false + show_user: false + default: | + max_message_size: 50KiB + #timeout: 300s + description: Specify custom configuration options for the UDP input. + - name: tags + type: text + title: Tags + multi: true + required: true + show_user: false + default: + - forwarded + - darktrace-model_breach_alert + - name: preserve_original_event + required: true + show_user: true + title: Preserve original event + description: Preserves a raw copy of the original event, added to the field `event.original`. + type: bool + multi: false + default: false + - name: preserve_duplicate_custom_fields + required: true + show_user: false + title: Preserve duplicate custom fields + description: Preserve darktrace.model_breach_alert fields that were copied to Elastic Common Schema (ECS) fields. + type: bool + multi: false + default: false + - name: processors + type: yaml + title: Processors + multi: false + required: false + show_user: false + description: >- + Processors are used to reduce the number of fields in the exported event or to enhance the event with metadata. This executes in the agent before the logs are parsed. See [Processors](https://www.elastic.co/guide/en/beats/filebeat/current/filtering-and-enhancing-data.html) for details. diff --git a/packages/darktrace/data_stream/model_breach_alert/sample_event.json b/packages/darktrace/data_stream/model_breach_alert/sample_event.json new file mode 100644 index 00000000000..87660729767 --- /dev/null +++ b/packages/darktrace/data_stream/model_breach_alert/sample_event.json @@ -0,0 +1,583 @@ +{ + "@timestamp": "2022-07-11T13:04:08.000Z", + "agent": { + "ephemeral_id": "572d7663-c480-491f-b06f-96f0330cf942", + "id": "95d2bc73-8bc8-47d9-b36e-a21b58255eec", + "name": "docker-fleet-agent", + "type": "filebeat", + "version": "8.2.1" + }, + "darktrace": { + "model_breach_alert": { + "aianalyst_data": [ + { + "related": [ + 1 + ], + "summariser": "BeaconSummary", + "uuid": "1234abcd-1234-1234-1234-123456abcdef" + } + ], + "comment": { + "count": 0 + }, + "creation_time": "2022-07-11T13:04:19.000Z", + "device": { + "did": 3, + "first_seen": "2022-07-11T12:54:49.000Z", + "ip": "81.2.69.142", + "last_seen": "2022-07-11T13:00:18.000Z", + "sid": 1, + "type_label": "Desktop", + "type_name": "desktop" + }, + "model": { + "actions": { + "is_alerting": true, + "is_breach": true, + "is_priority_set": false, + "is_tag_set": false, + "is_type_set": false, + "model": true + }, + "active_times": { + "type": "exclusions", + "version": 2 + }, + "behaviour": "incdec1", + "category": "Informational", + "created": { + "by": "System" + }, + "delay": 0, + "description": "A device has been repeatedly connecting to a rare external location with a beacon score. A beacon score is added when Darktrace identifies that a device is regularly communicating with an endpoint, for example, if a device connects to a rare external endpoint every 12 minutes this would get a beacon score. This model is designed to identify beaconing at a lower threshold and be protocol agnostic.\\n\\nAction: Review the external domains and IPs being connected to to see if they are legitimate and would be expected for business purposes.", + "edited": { + "by": "System" + }, + "in_compliance_behavior_category": false, + "interval": 10800, + "is_active": true, + "is_auto_suppress": true, + "is_auto_updatable": true, + "is_auto_update": true, + "is_sequenced": false, + "is_shared_endpoints": false, + "logic": { + "data_weighted_component_list": [ + { + "cid": 2026, + "weight": 1 + }, + { + "cid": 2024, + "weight": 1 + }, + { + "cid": 2025, + "weight": -100 + } + ], + "target_score": 1, + "type": "weightedComponentList", + "version": 1 + }, + "modified": "2022-07-11T11:47:37.000Z", + "name": "Compromise::Beaconing Activity To External Rare", + "phid": 1072, + "pid": 156, + "priority": 2, + "tags": [ + "AP: C2 Comms" + ], + "throttle": 10800, + "uuid": "1234abcd-1234-1234-1234-123456abcdef", + "version": 23 + }, + "pbid": 1, + "score": 0.674, + "time": "2022-07-11T13:04:08.000Z", + "triggered_components": [ + { + "cbid": 1, + "chid": 2113, + "cid": 2026, + "interval": 3600, + "logic": { + "data": "{left={left=A, right={left=AA, right={left=AC, right={left=AD, right={left=AF, right={left=AG, right={left=AH, right={left=B, right={left=C, right={left=D, right={left=E, right={left=H, right={left=I, right={left=J, right={left=K, right={left=L, right={left=M, right={left=N, right={left=O, right={left=P, right={left=S, right={left=U, right={left=V, right={left=X, right={left=Y, right=Z, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, right={left=A, right={left=AA, right={left=AB, right={left=AE, right={left=AF, right={left=AG, right={left=AH, right={left=C, right={left=D, right={left=E, right={left=H, right={left=I, right={left=J, right={left=K, right={left=L, right={left=M, right={left=N, right={left=O, right={left=P, right={left=S, right={left=U, right={left=V, right={left=X, right={left=Y, right=Z, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=OR}", + "version": "v0.1" + }, + "metric": { + "label": "External Connections", + "mlid": 1, + "name": "externalconnections" + }, + "size": 11, + "threshold": 10, + "time": "2022-07-11T13:04:08.000Z", + "triggered_filters": [ + { + "arguments": { + "value": 60 + }, + "cfid": 23426, + "comparator_type": "\u003e", + "filter_type": "Beaconing score", + "id": "A", + "trigger": { + "value": "100" + } + }, + { + "arguments": { + "value": 0 + }, + "cfid": 23427, + "comparator_type": "\u003e", + "filter_type": "Individual size up", + "id": "AA", + "trigger": { + "value": "4382" + } + }, + { + "arguments": { + "value": 95 + }, + "cfid": 23428, + "comparator_type": "\u003e", + "filter_type": "Rare domain", + "id": "AB", + "trigger": { + "value": "100" + } + }, + { + "arguments": { + "value": 1209600 + }, + "cfid": 23430, + "comparator_type": "\u003c", + "filter_type": "Age of destination", + "id": "AD", + "trigger": { + "value": "558" + } + }, + { + "arguments": { + "value": 1209600 + }, + "cfid": 23431, + "comparator_type": "\u003c", + "filter_type": "Age of external hostname", + "id": "AE", + "trigger": { + "value": "558" + } + }, + { + "arguments": { + "value": "examples" + }, + "cfid": 23432, + "comparator_type": "does not match regular expression", + "filter_type": "Connection hostname", + "id": "AF", + "trigger": { + "value": "example.com" + } + }, + { + "arguments": { + "value": "examples" + }, + "cfid": 23433, + "comparator_type": "does not match regular expression", + "filter_type": "ASN", + "id": "AG", + "trigger": { + "value": "AS12345 LOCAL-02" + } + }, + { + "arguments": { + "value": "5d41402abc4b2a76b9719d911017c592" + }, + "cfid": 23434, + "comparator_type": "does not match", + "filter_type": "JA3 hash", + "id": "AH", + "trigger": { + "value": "5d41402abc4b2a76b9719d911017c592" + } + }, + { + "arguments": { + "value": 95 + }, + "cfid": 23435, + "comparator_type": "\u003e", + "filter_type": "Rare external IP", + "id": "B", + "trigger": { + "value": "100" + } + }, + { + "arguments": { + "value": "1003" + }, + "cfid": 23436, + "comparator_type": "is not", + "filter_type": "Application protocol", + "id": "C", + "trigger": { + "value": "1004" + } + }, + { + "arguments": { + "value": 53 + }, + "cfid": 23437, + "comparator_type": "!=", + "filter_type": "Destination port", + "id": "D", + "trigger": { + "value": "443" + } + }, + { + "arguments": { + "value": "out" + }, + "cfid": 23438, + "comparator_type": "is", + "filter_type": "Direction", + "id": "E", + "trigger": { + "value": "out" + } + }, + { + "arguments": { + "value": 137 + }, + "cfid": 23439, + "comparator_type": "!=", + "filter_type": "Destination port", + "id": "H", + "trigger": { + "value": "443" + } + }, + { + "arguments": { + "value": 161 + }, + "cfid": 23440, + "comparator_type": "!=", + "filter_type": "Destination port", + "id": "I", + "trigger": { + "value": "443" + } + }, + { + "arguments": { + "value": "6" + }, + "cfid": 23441, + "comparator_type": "is", + "filter_type": "Protocol", + "id": "J", + "trigger": { + "value": "6" + } + }, + { + "arguments": { + "value": "Company" + }, + "cfid": 23442, + "comparator_type": "does not contain", + "filter_type": "ASN", + "id": "K", + "trigger": { + "value": "AS12345 LOCAL-02" + } + }, + { + "arguments": { + "value": "Company" + }, + "cfid": 23443, + "comparator_type": "does not contain", + "filter_type": "ASN", + "id": "L", + "trigger": { + "value": "AS12345 LOCAL-02" + } + }, + { + "arguments": { + "value": "13" + }, + "cfid": 23444, + "comparator_type": "is not", + "filter_type": "Internal source device type", + "id": "M", + "trigger": { + "value": "6" + } + }, + { + "arguments": { + "value": "5" + }, + "cfid": 23445, + "comparator_type": "is not", + "filter_type": "Internal source device type", + "id": "N", + "trigger": { + "value": "6" + } + }, + { + "arguments": { + "value": "9" + }, + "cfid": 23446, + "comparator_type": "is not", + "filter_type": "Internal source device type", + "id": "O", + "trigger": { + "value": "6" + } + }, + { + "arguments": { + "value": "12" + }, + "cfid": 23447, + "comparator_type": "is not", + "filter_type": "Internal source device type", + "id": "P", + "trigger": { + "value": "6" + } + }, + { + "arguments": { + "value": "30" + }, + "cfid": 23448, + "comparator_type": "is not", + "filter_type": "Internal source device type", + "id": "S", + "trigger": { + "value": "6" + } + }, + { + "arguments": { + "value": "4" + }, + "cfid": 23449, + "comparator_type": "is not", + "filter_type": "Internal source device type", + "id": "U", + "trigger": { + "value": "6" + } + }, + { + "arguments": { + "value": "3" + }, + "cfid": 23450, + "comparator_type": "is not", + "filter_type": "Internal source device type", + "id": "V", + "trigger": { + "value": "6" + } + }, + { + "arguments": { + "value": "false" + }, + "cfid": 23451, + "comparator_type": "is", + "filter_type": "Trusted hostname", + "id": "X", + "trigger": { + "value": "false" + } + }, + { + "arguments": { + "value": 26 + }, + "cfid": 23452, + "comparator_type": "does not have tag", + "filter_type": "Tagged internal source", + "id": "Y", + "trigger": { + "tag": { + "data": { + "auto": false, + "color": 5, + "visibility": "Public" + }, + "expiry": 0, + "is_referenced": true, + "name": "No Device Tracking", + "restricted": false, + "thid": 26, + "tid": 26 + }, + "value": "26" + } + }, + { + "arguments": { + "value": 0 + }, + "cfid": 23453, + "comparator_type": "\u003e", + "filter_type": "Individual size down", + "id": "Z", + "trigger": { + "value": "5862" + } + }, + { + "cfid": 23454, + "comparator_type": "display", + "filter_type": "JA3 hash", + "id": "d1", + "trigger": { + "value": "5d41402abc4b2a76b9719d911017c592" + } + }, + { + "cfid": 23455, + "comparator_type": "display", + "filter_type": "ASN", + "id": "d2", + "trigger": { + "value": "AS12345 LOCAL-02" + } + }, + { + "cfid": 23456, + "comparator_type": "display", + "filter_type": "Destination IP", + "id": "d3", + "trigger": { + "value": "81.2.69.192" + } + }, + { + "cfid": 23457, + "comparator_type": "display", + "filter_type": "Connection hostname", + "id": "d4", + "trigger": { + "value": "example.com" + } + } + ] + } + ] + } + }, + "data_stream": { + "dataset": "darktrace.model_breach_alert", + "namespace": "ep", + "type": "logs" + }, + "ecs": { + "version": "8.4.0" + }, + "elastic_agent": { + "id": "95d2bc73-8bc8-47d9-b36e-a21b58255eec", + "snapshot": false, + "version": "8.2.1" + }, + "event": { + "agent_id_status": "verified", + "category": [ + "network" + ], + "created": "2022-07-11T13:04:19.000Z", + "dataset": "darktrace.model_breach_alert", + "ingested": "2022-09-30T11:39:13Z", + "kind": "event", + "original": "{\"commentCount\":0,\"pbid\":1,\"time\":1657544648000,\"creationTime\":1657544659000,\"aianalystData\":[{\"uuid\":\"1234abcd-1234-1234-1234-123456abcdef\",\"related\":[1],\"summariser\":\"BeaconSummary\"}],\"model\":{\"name\":\"Compromise::Beaconing Activity To External Rare\",\"pid\":156,\"phid\":1072,\"uuid\":\"1234abcd-1234-1234-1234-123456abcdef\",\"logic\":{\"data\":[{\"cid\":2026,\"weight\":1},{\"cid\":2024,\"weight\":1},{\"cid\":2025,\"weight\":-100}],\"targetScore\":1,\"type\":\"weightedComponentList\",\"version\":1},\"throttle\":10800,\"sharedEndpoints\":false,\"actions\":{\"alert\":true,\"antigena\":{},\"breach\":true,\"model\":true,\"setPriority\":false,\"setTag\":false,\"setType\":false},\"tags\":[\"AP: C2 Comms\"],\"interval\":10800,\"delay\":0,\"sequenced\":false,\"active\":true,\"modified\":\"2022-07-11 11:47:37\",\"activeTimes\":{\"devices\":{},\"tags\":{},\"type\":\"exclusions\",\"version\":2},\"autoUpdatable\":true,\"autoUpdate\":true,\"autoSuppress\":true,\"description\":\"A device has been repeatedly connecting to a rare external location with a beacon score. A beacon score is added when Darktrace identifies that a device is regularly communicating with an endpoint, for example, if a device connects to a rare external endpoint every 12 minutes this would get a beacon score. This model is designed to identify beaconing at a lower threshold and be protocol agnostic.\\\\n\\\\nAction: Review the external domains and IPs being connected to to see if they are legitimate and would be expected for business purposes.\",\"behaviour\":\"incdec1\",\"created\":{\"by\":\"System\"},\"edited\":{\"by\":\"System\"},\"version\":23,\"priority\":2,\"category\":\"Informational\",\"compliance\":false},\"triggeredComponents\":[{\"time\":1657544648000,\"cbid\":1,\"cid\":2026,\"chid\":2113,\"size\":11,\"threshold\":10,\"interval\":3600,\"logic\":{\"data\":{\"left\":{\"left\":\"A\",\"operator\":\"AND\",\"right\":{\"left\":\"AA\",\"operator\":\"AND\",\"right\":{\"left\":\"AC\",\"operator\":\"AND\",\"right\":{\"left\":\"AD\",\"operator\":\"AND\",\"right\":{\"left\":\"AF\",\"operator\":\"AND\",\"right\":{\"left\":\"AG\",\"operator\":\"AND\",\"right\":{\"left\":\"AH\",\"operator\":\"AND\",\"right\":{\"left\":\"B\",\"operator\":\"AND\",\"right\":{\"left\":\"C\",\"operator\":\"AND\",\"right\":{\"left\":\"D\",\"operator\":\"AND\",\"right\":{\"left\":\"E\",\"operator\":\"AND\",\"right\":{\"left\":\"H\",\"operator\":\"AND\",\"right\":{\"left\":\"I\",\"operator\":\"AND\",\"right\":{\"left\":\"J\",\"operator\":\"AND\",\"right\":{\"left\":\"K\",\"operator\":\"AND\",\"right\":{\"left\":\"L\",\"operator\":\"AND\",\"right\":{\"left\":\"M\",\"operator\":\"AND\",\"right\":{\"left\":\"N\",\"operator\":\"AND\",\"right\":{\"left\":\"O\",\"operator\":\"AND\",\"right\":{\"left\":\"P\",\"operator\":\"AND\",\"right\":{\"left\":\"S\",\"operator\":\"AND\",\"right\":{\"left\":\"U\",\"operator\":\"AND\",\"right\":{\"left\":\"V\",\"operator\":\"AND\",\"right\":{\"left\":\"X\",\"operator\":\"AND\",\"right\":{\"left\":\"Y\",\"operator\":\"AND\",\"right\":\"Z\"}}}}}}}}}}}}}}}}}}}}}}}}},\"operator\":\"OR\",\"right\":{\"left\":\"A\",\"operator\":\"AND\",\"right\":{\"left\":\"AA\",\"operator\":\"AND\",\"right\":{\"left\":\"AB\",\"operator\":\"AND\",\"right\":{\"left\":\"AE\",\"operator\":\"AND\",\"right\":{\"left\":\"AF\",\"operator\":\"AND\",\"right\":{\"left\":\"AG\",\"operator\":\"AND\",\"right\":{\"left\":\"AH\",\"operator\":\"AND\",\"right\":{\"left\":\"C\",\"operator\":\"AND\",\"right\":{\"left\":\"D\",\"operator\":\"AND\",\"right\":{\"left\":\"E\",\"operator\":\"AND\",\"right\":{\"left\":\"H\",\"operator\":\"AND\",\"right\":{\"left\":\"I\",\"operator\":\"AND\",\"right\":{\"left\":\"J\",\"operator\":\"AND\",\"right\":{\"left\":\"K\",\"operator\":\"AND\",\"right\":{\"left\":\"L\",\"operator\":\"AND\",\"right\":{\"left\":\"M\",\"operator\":\"AND\",\"right\":{\"left\":\"N\",\"operator\":\"AND\",\"right\":{\"left\":\"O\",\"operator\":\"AND\",\"right\":{\"left\":\"P\",\"operator\":\"AND\",\"right\":{\"left\":\"S\",\"operator\":\"AND\",\"right\":{\"left\":\"U\",\"operator\":\"AND\",\"right\":{\"left\":\"V\",\"operator\":\"AND\",\"right\":{\"left\":\"X\",\"operator\":\"AND\",\"right\":{\"left\":\"Y\",\"operator\":\"AND\",\"right\":\"Z\"}}}}}}}}}}}}}}}}}}}}}}}}},\"version\":\"v0.1\"},\"metric\":{\"mlid\":1,\"name\":\"externalconnections\",\"label\":\"External Connections\"},\"triggeredFilters\":[{\"cfid\":23426,\"id\":\"A\",\"filterType\":\"Beaconing score\",\"arguments\":{\"value\":60},\"comparatorType\":\"\u003e\",\"trigger\":{\"value\":\"100\"}},{\"cfid\":23427,\"id\":\"AA\",\"filterType\":\"Individual size up\",\"arguments\":{\"value\":0},\"comparatorType\":\"\u003e\",\"trigger\":{\"value\":\"4382\"}},{\"cfid\":23428,\"id\":\"AB\",\"filterType\":\"Rare domain\",\"arguments\":{\"value\":95},\"comparatorType\":\"\u003e\",\"trigger\":{\"value\":\"100\"}},{\"cfid\":23430,\"id\":\"AD\",\"filterType\":\"Age of destination\",\"arguments\":{\"value\":1209600},\"comparatorType\":\"\u003c\",\"trigger\":{\"value\":\"558\"}},{\"cfid\":23431,\"id\":\"AE\",\"filterType\":\"Age of external hostname\",\"arguments\":{\"value\":1209600},\"comparatorType\":\"\u003c\",\"trigger\":{\"value\":\"558\"}},{\"cfid\":23432,\"id\":\"AF\",\"filterType\":\"Connection hostname\",\"arguments\":{\"value\":\"examples\"},\"comparatorType\":\"does not match regular expression\",\"trigger\":{\"value\":\"example.com\"}},{\"cfid\":23433,\"id\":\"AG\",\"filterType\":\"ASN\",\"arguments\":{\"value\":\"examples\"},\"comparatorType\":\"does not match regular expression\",\"trigger\":{\"value\":\"AS12345 LOCAL-02\"}},{\"cfid\":23434,\"id\":\"AH\",\"filterType\":\"JA3 hash\",\"arguments\":{\"value\":\"5d41402abc4b2a76b9719d911017c592\"},\"comparatorType\":\"does not match\",\"trigger\":{\"value\":\"5d41402abc4b2a76b9719d911017c592\"}},{\"cfid\":23435,\"id\":\"B\",\"filterType\":\"Rare external IP\",\"arguments\":{\"value\":95},\"comparatorType\":\"\u003e\",\"trigger\":{\"value\":\"100\"}},{\"cfid\":23436,\"id\":\"C\",\"filterType\":\"Application protocol\",\"arguments\":{\"value\":\"1003\"},\"comparatorType\":\"is not\",\"trigger\":{\"value\":\"1004\"}},{\"cfid\":23437,\"id\":\"D\",\"filterType\":\"Destination port\",\"arguments\":{\"value\":53},\"comparatorType\":\"!=\",\"trigger\":{\"value\":\"443\"}},{\"cfid\":23438,\"id\":\"E\",\"filterType\":\"Direction\",\"arguments\":{\"value\":\"out\"},\"comparatorType\":\"is\",\"trigger\":{\"value\":\"out\"}},{\"cfid\":23439,\"id\":\"H\",\"filterType\":\"Destination port\",\"arguments\":{\"value\":137},\"comparatorType\":\"!=\",\"trigger\":{\"value\":\"443\"}},{\"cfid\":23440,\"id\":\"I\",\"filterType\":\"Destination port\",\"arguments\":{\"value\":161},\"comparatorType\":\"!=\",\"trigger\":{\"value\":\"443\"}},{\"cfid\":23441,\"id\":\"J\",\"filterType\":\"Protocol\",\"arguments\":{\"value\":\"6\"},\"comparatorType\":\"is\",\"trigger\":{\"value\":\"6\"}},{\"cfid\":23442,\"id\":\"K\",\"filterType\":\"ASN\",\"arguments\":{\"value\":\"Company\"},\"comparatorType\":\"does not contain\",\"trigger\":{\"value\":\"AS12345 LOCAL-02\"}},{\"cfid\":23443,\"id\":\"L\",\"filterType\":\"ASN\",\"arguments\":{\"value\":\"Company\"},\"comparatorType\":\"does not contain\",\"trigger\":{\"value\":\"AS12345 LOCAL-02\"}},{\"cfid\":23444,\"id\":\"M\",\"filterType\":\"Internal source device type\",\"arguments\":{\"value\":\"13\"},\"comparatorType\":\"is not\",\"trigger\":{\"value\":\"6\"}},{\"cfid\":23445,\"id\":\"N\",\"filterType\":\"Internal source device type\",\"arguments\":{\"value\":\"5\"},\"comparatorType\":\"is not\",\"trigger\":{\"value\":\"6\"}},{\"cfid\":23446,\"id\":\"O\",\"filterType\":\"Internal source device type\",\"arguments\":{\"value\":\"9\"},\"comparatorType\":\"is not\",\"trigger\":{\"value\":\"6\"}},{\"cfid\":23447,\"id\":\"P\",\"filterType\":\"Internal source device type\",\"arguments\":{\"value\":\"12\"},\"comparatorType\":\"is not\",\"trigger\":{\"value\":\"6\"}},{\"cfid\":23448,\"id\":\"S\",\"filterType\":\"Internal source device type\",\"arguments\":{\"value\":\"30\"},\"comparatorType\":\"is not\",\"trigger\":{\"value\":\"6\"}},{\"cfid\":23449,\"id\":\"U\",\"filterType\":\"Internal source device type\",\"arguments\":{\"value\":\"4\"},\"comparatorType\":\"is not\",\"trigger\":{\"value\":\"6\"}},{\"cfid\":23450,\"id\":\"V\",\"filterType\":\"Internal source device type\",\"arguments\":{\"value\":\"3\"},\"comparatorType\":\"is not\",\"trigger\":{\"value\":\"6\"}},{\"cfid\":23451,\"id\":\"X\",\"filterType\":\"Trusted hostname\",\"arguments\":{\"value\":\"false\"},\"comparatorType\":\"is\",\"trigger\":{\"value\":\"false\"}},{\"cfid\":23452,\"id\":\"Y\",\"filterType\":\"Tagged internal source\",\"arguments\":{\"value\":26},\"comparatorType\":\"does not have tag\",\"trigger\":{\"value\":\"26\",\"tag\":{\"tid\":26,\"expiry\":0,\"thid\":26,\"name\":\"No Device Tracking\",\"restricted\":false,\"data\":{\"auto\":false,\"color\":5,\"description\":\"\",\"visibility\":\"Public\"},\"isReferenced\":true}}},{\"cfid\":23453,\"id\":\"Z\",\"filterType\":\"Individual size down\",\"arguments\":{\"value\":0},\"comparatorType\":\"\u003e\",\"trigger\":{\"value\":\"5862\"}},{\"cfid\":23454,\"id\":\"d1\",\"filterType\":\"JA3 hash\",\"arguments\":{},\"comparatorType\":\"display\",\"trigger\":{\"value\":\"5d41402abc4b2a76b9719d911017c592\"}},{\"cfid\":23455,\"id\":\"d2\",\"filterType\":\"ASN\",\"arguments\":{},\"comparatorType\":\"display\",\"trigger\":{\"value\":\"AS12345 LOCAL-02\"}},{\"cfid\":23456,\"id\":\"d3\",\"filterType\":\"Destination IP\",\"arguments\":{},\"comparatorType\":\"display\",\"trigger\":{\"value\":\"81.2.69.192\"}},{\"cfid\":23457,\"id\":\"d4\",\"filterType\":\"Connection hostname\",\"arguments\":{},\"comparatorType\":\"display\",\"trigger\":{\"value\":\"example.com\"}}]}],\"score\":0.674,\"device\":{\"did\":3,\"ip\":\"81.2.69.142\",\"sid\":1,\"firstSeen\":1657544089000,\"lastSeen\":1657544418000,\"typename\":\"desktop\",\"typelabel\":\"Desktop\"}}", + "risk_score": 0.674, + "risk_score_norm": 67.4, + "severity": 2, + "start": [ + "2022-07-11T13:04:08.000Z" + ], + "type": [ + "info", + "connection" + ] + }, + "host": { + "id": "3", + "ip": [ + "81.2.69.142" + ], + "type": "desktop" + }, + "input": { + "type": "udp" + }, + "log": { + "source": { + "address": "192.168.128.5:60206" + }, + "syslog": { + "facility": { + "code": 20, + "name": "local4" + }, + "hostname": "example.cloud.darktrace.com", + "priority": 165, + "severity": { + "code": 5, + "name": "Notice" + }, + "version": "1" + } + }, + "related": { + "ip": [ + "81.2.69.142" + ], + "user": [ + "System" + ] + }, + "rule": { + "author": "System", + "category": "Informational", + "description": "A device has been repeatedly connecting to a rare external location with a beacon score. A beacon score is added when Darktrace identifies that a device is regularly communicating with an endpoint, for example, if a device connects to a rare external endpoint every 12 minutes this would get a beacon score. This model is designed to identify beaconing at a lower threshold and be protocol agnostic.\\n\\nAction: Review the external domains and IPs being connected to to see if they are legitimate and would be expected for business purposes.", + "name": "Compromise::Beaconing Activity To External Rare", + "ruleset": [ + "AP: C2 Comms" + ], + "uuid": "1234abcd-1234-1234-1234-123456abcdef", + "version": "23" + }, + "tags": [ + "preserve_original_event", + "preserve_duplicate_custom_fields", + "forwarded", + "darktrace-model_breach_alert" + ] +} \ No newline at end of file diff --git a/packages/darktrace/data_stream/system_status_alert/_dev/test/pipeline/test-common-config.yml b/packages/darktrace/data_stream/system_status_alert/_dev/test/pipeline/test-common-config.yml new file mode 100644 index 00000000000..be41bb0d476 --- /dev/null +++ b/packages/darktrace/data_stream/system_status_alert/_dev/test/pipeline/test-common-config.yml @@ -0,0 +1,4 @@ +fields: + tags: + - preserve_original_event + - preserve_duplicate_custom_fields diff --git a/packages/darktrace/data_stream/system_status_alert/_dev/test/pipeline/test-system-status-alert.log b/packages/darktrace/data_stream/system_status_alert/_dev/test/pipeline/test-system-status-alert.log new file mode 100644 index 00000000000..950c12c7b48 --- /dev/null +++ b/packages/darktrace/data_stream/system_status_alert/_dev/test/pipeline/test-system-status-alert.log @@ -0,0 +1,2 @@ +{"last_updated":1618760651,"uuid":"abcdabcd-1234-1234-1234-3abababcdcd3","priority":43,"priority_level":"medium","hostname":"example-vsensor","ip_address":"175.16.199.1","message":"There have been no Advanced Search hits for this instance seen since Sun 18 April 2021 13:20:23 (UTC). If this is not expected behaviour, please open a ticket using the following link or get in touch with your Cyber Technology Specialist. https://example.com/test","name":"advanced_search","acknowledge_timeout":null,"alert_name":"Advanced Search","child_id":1,"last_updated_status":1618760651,"status":"active"} +{"hostname":"local-abc","ip_address":"175.16.199.1","child_id":null,"name":"probe_down-0","priority":98,"priority_level":"critical","alert_name":"Probe Down","status":"Active","message":"The probe 1/175.16.199.1 has lost connection to the Master instance. Please ensure HTTPS bidirectional connectivity exists between the Master and the Probe.\n\nIf you have any issues, please open a ticket using the following link. https://example.com/test","last_updated":1658110810.556194,"last_updated_status":1658110810.556194,"acknowledge_timeout":null,"uuid":"abcd1234-1234-1234-1234-3abababcdcd3","url":"https://example.com/test?value=abcd1234-1234-1234-1234-3abababcdcd3"} diff --git a/packages/darktrace/data_stream/system_status_alert/_dev/test/pipeline/test-system-status-alert.log-expected.json b/packages/darktrace/data_stream/system_status_alert/_dev/test/pipeline/test-system-status-alert.log-expected.json new file mode 100644 index 00000000000..e1c42b4f11c --- /dev/null +++ b/packages/darktrace/data_stream/system_status_alert/_dev/test/pipeline/test-system-status-alert.log-expected.json @@ -0,0 +1,109 @@ +{ + "expected": [ + { + "@timestamp": "2021-04-18T15:44:11.000Z", + "darktrace": { + "system_status_alert": { + "alert_name": "Advanced Search", + "child_id": 1, + "hostname": "example-vsensor", + "ip_address": "175.16.199.1", + "last_updated": "2021-04-18T15:44:11.000Z", + "last_updated_status": "2021-04-18T15:44:11.000Z", + "message": "There have been no Advanced Search hits for this instance seen since Sun 18 April 2021 13:20:23 (UTC). If this is not expected behaviour, please open a ticket using the following link or get in touch with your Cyber Technology Specialist. https://example.com/test", + "name": "advanced_search", + "priority": 43.0, + "priority_level": "medium", + "status": "active", + "uuid": "abcdabcd-1234-1234-1234-3abababcdcd3" + } + }, + "ecs": { + "version": "8.4.0" + }, + "event": { + "id": "abcdabcd-1234-1234-1234-3abababcdcd3", + "kind": "alert", + "original": "{\"last_updated\":1618760651,\"uuid\":\"abcdabcd-1234-1234-1234-3abababcdcd3\",\"priority\":43,\"priority_level\":\"medium\",\"hostname\":\"example-vsensor\",\"ip_address\":\"175.16.199.1\",\"message\":\"There have been no Advanced Search hits for this instance seen since Sun 18 April 2021 13:20:23 (UTC). If this is not expected behaviour, please open a ticket using the following link or get in touch with your Cyber Technology Specialist. https://example.com/test\",\"name\":\"advanced_search\",\"acknowledge_timeout\":null,\"alert_name\":\"Advanced Search\",\"child_id\":1,\"last_updated_status\":1618760651,\"status\":\"active\"}", + "reason": "There have been no Advanced Search hits for this instance seen since Sun 18 April 2021 13:20:23 (UTC). If this is not expected behaviour, please open a ticket using the following link or get in touch with your Cyber Technology Specialist. https://example.com/test", + "risk_score": 43.0, + "risk_score_norm": 43.0, + "type": [ + "info" + ] + }, + "host": { + "hostname": "example-vsensor", + "ip": "175.16.199.1" + }, + "related": { + "hosts": [ + "example-vsensor" + ], + "ip": [ + "175.16.199.1" + ] + }, + "tags": [ + "preserve_original_event", + "preserve_duplicate_custom_fields" + ] + }, + { + "@timestamp": "2022-07-18T02:20:10.556Z", + "darktrace": { + "system_status_alert": { + "alert_name": "Probe Down", + "hostname": "local-abc", + "ip_address": "175.16.199.1", + "last_updated": "2022-07-18T02:20:10.556Z", + "last_updated_status": "2022-07-18T02:20:10.556Z", + "message": "The probe 1/175.16.199.1 has lost connection to the Master instance. Please ensure HTTPS bidirectional connectivity exists between the Master and the Probe.\n\nIf you have any issues, please open a ticket using the following link. https://example.com/test", + "name": "probe_down-0", + "priority": 98.0, + "priority_level": "critical", + "status": "active", + "url": { + "domain": "example.com", + "original": "https://example.com/test?value=abcd1234-1234-1234-1234-3abababcdcd3", + "path": "/test", + "query": "value=abcd1234-1234-1234-1234-3abababcdcd3", + "scheme": "https" + }, + "uuid": "abcd1234-1234-1234-1234-3abababcdcd3" + } + }, + "ecs": { + "version": "8.4.0" + }, + "event": { + "id": "abcd1234-1234-1234-1234-3abababcdcd3", + "kind": "alert", + "original": "{\"hostname\":\"local-abc\",\"ip_address\":\"175.16.199.1\",\"child_id\":null,\"name\":\"probe_down-0\",\"priority\":98,\"priority_level\":\"critical\",\"alert_name\":\"Probe Down\",\"status\":\"Active\",\"message\":\"The probe 1/175.16.199.1 has lost connection to the Master instance. Please ensure HTTPS bidirectional connectivity exists between the Master and the Probe.\\n\\nIf you have any issues, please open a ticket using the following link. https://example.com/test\",\"last_updated\":1658110810.556194,\"last_updated_status\":1658110810.556194,\"acknowledge_timeout\":null,\"uuid\":\"abcd1234-1234-1234-1234-3abababcdcd3\",\"url\":\"https://example.com/test?value=abcd1234-1234-1234-1234-3abababcdcd3\"}", + "reason": "The probe 1/175.16.199.1 has lost connection to the Master instance. Please ensure HTTPS bidirectional connectivity exists between the Master and the Probe.\n\nIf you have any issues, please open a ticket using the following link. https://example.com/test", + "risk_score": 98.0, + "risk_score_norm": 98.0, + "type": [ + "info" + ], + "url": "https://example.com/test?value=abcd1234-1234-1234-1234-3abababcdcd3" + }, + "host": { + "hostname": "local-abc", + "ip": "175.16.199.1" + }, + "related": { + "hosts": [ + "local-abc" + ], + "ip": [ + "175.16.199.1" + ] + }, + "tags": [ + "preserve_original_event", + "preserve_duplicate_custom_fields" + ] + } + ] +} \ No newline at end of file diff --git a/packages/darktrace/data_stream/system_status_alert/_dev/test/system/test-tcp-config.yml b/packages/darktrace/data_stream/system_status_alert/_dev/test/system/test-tcp-config.yml new file mode 100644 index 00000000000..063df1e59b0 --- /dev/null +++ b/packages/darktrace/data_stream/system_status_alert/_dev/test/system/test-tcp-config.yml @@ -0,0 +1,10 @@ +service: darktrace-system_status_alert-tcp +service_notify_signal: SIGHUP +input: tcp +vars: + listen_address: 0.0.0.0 +data_stream: + vars: + listen_port: 9573 + preserve_original_event: true + preserve_duplicate_custom_fields: true diff --git a/packages/darktrace/data_stream/system_status_alert/_dev/test/system/test-tls-config.yml b/packages/darktrace/data_stream/system_status_alert/_dev/test/system/test-tls-config.yml new file mode 100644 index 00000000000..81f051da6d4 --- /dev/null +++ b/packages/darktrace/data_stream/system_status_alert/_dev/test/system/test-tls-config.yml @@ -0,0 +1,62 @@ +service: darktrace-system_status_alert-tls +service_notify_signal: SIGHUP +input: tcp +vars: + listen_address: 0.0.0.0 + ssl: | + key: | + -----BEGIN PRIVATE KEY----- + MIIEvwIBADANBgkqhkiG9w0BAQEFAASCBKkwggSlAgEAAoIBAQDhCLvLsQAHufsN + U+u1x/CequAUphfXZqLhDo2Eo/holfBS0+ey4bnzPL6lS9NFL5JkLQA2gYESqsXU + /Ru8E76Az1egzMwT3TVAPLVU8NbrxBqeNiQa2m9wC37HQy4qC9OxL28LUoKtFjxS + cD1sa0oikXCJN1a3BSoAf9iiZ/dxz4WVfrNhrzq2JFXjravY84n5ujkZOg45Pg70 + 4vHOeg0rBbIoSNfjDUVZWjwC95K1BMN3msOTL9juv/EDa6BujqCxl+G1nY7JPFDL + SHWis65p+1AAa5xieYDb47vyJ0SSR7lEURTXZOkkM6k5JWfgkATEmGzRxPkOloIT + Xg9ag1OlAgMBAAECggEAEHfPJmzhj68wjB0kFr13AmWG2Hv/Kqg8KzQhbx+AwkaW + u7j+L70NGpvLZ9VQtLNyhxoz9cksZO1SZO/Q48aeHlcOFppmJN3/U6AdtQWa9M35 + FLLpmX16wjxVHsfvzOvopgLOoYl8PqZt66qDFDgVyMnT7na6RdJ+7GJuvBPXq+Bc + vgThvAZitHSAOhnBFYmTMlBi6AzOMMsaFlgE3Xf9v3M0pAKItPRKMhXlC3MyvA/v + jgbra4Ib+0ryohggHheHB3bn3Jgv7iFKoW9OQSePVxacJ+kfr9H+No5g495URzqR + mx/96WCiv3rAh3ct8Sk/C4/3zMC8fUueDJIVjhgw0QKBgQD8NufLINNkIpBrLoCS + 972oFEjZB2u6EusQ7X9raROqpaw26ZSu+zSHeIKCGQ93M3aRb3FpdGeOxgZ095MV + 8a+nlh4stOvHj2Mm5YhTBDUavTC7o9aVR3Od5eTXUpHnaJpNI/uyIcKupeK1UJnV + UlBLeIwo/vJ1gsVrKMMAJkuKbwKBgQDkaWRRd0w2gUIbCTGf203BqXft0VdIiOW7 + +gnkeaNHAf09XljzxMcQzrB8kG63aKVGbJffphEfzxtiJ+HRQVH+7QpKRhU/GHmu + +6OKkxTcxJm5zhoRFxcSi2wG4PWmUGJvc7ss1OJGcaOUxwocCepO7N/jfdDz9Uke + KnA+YWOdKwKBgQDteZkYlojT0QOgF8HyH5gQyUCqMKWLJ0LzxltiPCbLV4Dml1pq + w5Z7M8nWS1hXiTpLx93GSFc1hFkSCwYP9GfK6Lryp0sVtHnMZvTMDbseuSJImwRx + vDwtYQfugg1lEQWwOoBEAiu3m/PxernNtNprpU57T0nlwUK3GkM5QdWAuwKBgQCZ + ZF3GiANapzupxGbbH//8Cr9LqsafI7CEqMpz8WxBh4h16iJ6sq+tDeFgBe8UpOY5 + gTwNKg1d+0w8guQYD3HtbWr3rlEeamVtqfiOW3ArQqyqJ0tCJuuLvK3zgKf35Qv2 + JRaSaPT8sdxVUcXsRoxgLJu+vwPQke1koMN4YRbwuQKBgQDJiZ/WSeqa5oIqkXbn + hjm7RXKaf2oE1U/bNjdSFtdEP7T4vUvvr7Hq2f/jiBLtCE7w16PJjKx9iIq2+jMl + qIY43Sk9bdi5FxtYTHda0hwrbH274P+QVcVs5PXCT0TGktOleHGBlXaaPrxl9iCh + 8tmmxZZYa5aQxEO/lxB9xQKaiQ== + -----END PRIVATE KEY----- + certificate: | + -----BEGIN CERTIFICATE----- + MIIDazCCAlOgAwIBAgIUW5TDu1tJMY2Oa7PsL+BQSmeWqz0wDQYJKoZIhvcNAQEL + BQAwRTELMAkGA1UEBhMCVVMxEzARBgNVBAgMClNvbWUtU3RhdGUxITAfBgNVBAoM + GEludGVybmV0IFdpZGdpdHMgUHR5IEx0ZDAeFw0yMTEwMDEwNTAwMjNaFw0yMTEw + MDIwNTAwMjNaMEUxCzAJBgNVBAYTAlVTMRMwEQYDVQQIDApTb21lLVN0YXRlMSEw + HwYDVQQKDBhJbnRlcm5ldCBXaWRnaXRzIFB0eSBMdGQwggEiMA0GCSqGSIb3DQEB + AQUAA4IBDwAwggEKAoIBAQDhCLvLsQAHufsNU+u1x/CequAUphfXZqLhDo2Eo/ho + lfBS0+ey4bnzPL6lS9NFL5JkLQA2gYESqsXU/Ru8E76Az1egzMwT3TVAPLVU8Nbr + xBqeNiQa2m9wC37HQy4qC9OxL28LUoKtFjxScD1sa0oikXCJN1a3BSoAf9iiZ/dx + z4WVfrNhrzq2JFXjravY84n5ujkZOg45Pg704vHOeg0rBbIoSNfjDUVZWjwC95K1 + BMN3msOTL9juv/EDa6BujqCxl+G1nY7JPFDLSHWis65p+1AAa5xieYDb47vyJ0SS + R7lEURTXZOkkM6k5JWfgkATEmGzRxPkOloITXg9ag1OlAgMBAAGjUzBRMB0GA1Ud + DgQWBBRYUSKDHBBE9Q6fTeTqogicCxcXwDAfBgNVHSMEGDAWgBRYUSKDHBBE9Q6f + TeTqogicCxcXwDAPBgNVHRMBAf8EBTADAQH/MA0GCSqGSIb3DQEBCwUAA4IBAQBc + T8B+GpvPy9NQ700LsywRPY0L9IJCKiu6j3TP1tqqSPjAC/cg9ac+bFXuWOu7V+KJ + s09Q/pItq9SLX6UvnfRzTxu5lCBwwGX9cL131mTIu5SmFo7Eks+sorbiIarWDMoC + e+9An3GFpagW+YhOt4BdIM5lTqoeodzganDBsOUZI9aDAj2Yo5h2O7r6Wd12cb6T + mz8vMfB2eG8BxU20ZMfkdERWjiyXHOSBQqeqfkV8d9370gMu5RcJNcIgnbmTRdho + X3HJFiimZVaNjXATqmC/y2A1KXvJdamPLy3mGXkW2cFLoPCdK2OZFUHqiuc1bigA + qEf55SihFqErRMeURPPF + -----END CERTIFICATE----- +data_stream: + vars: + listen_port: 9573 + preserve_original_event: true + preserve_duplicate_custom_fields: true diff --git a/packages/darktrace/data_stream/system_status_alert/_dev/test/system/test-udp-config.yml b/packages/darktrace/data_stream/system_status_alert/_dev/test/system/test-udp-config.yml new file mode 100644 index 00000000000..cc4f6b075d4 --- /dev/null +++ b/packages/darktrace/data_stream/system_status_alert/_dev/test/system/test-udp-config.yml @@ -0,0 +1,10 @@ +service: darktrace-system_status_alert-udp +service_notify_signal: SIGHUP +input: udp +vars: + listen_address: 0.0.0.0 +data_stream: + vars: + listen_port: 9576 + preserve_original_event: true + preserve_duplicate_custom_fields: true diff --git a/packages/darktrace/data_stream/system_status_alert/agent/stream/tcp.yml.hbs b/packages/darktrace/data_stream/system_status_alert/agent/stream/tcp.yml.hbs new file mode 100644 index 00000000000..b1d260f0f9c --- /dev/null +++ b/packages/darktrace/data_stream/system_status_alert/agent/stream/tcp.yml.hbs @@ -0,0 +1,26 @@ +host: "{{listen_address}}:{{listen_port}}" +{{#if tcp_options}} +{{tcp_options}} +{{/if}} +{{#if ssl}} +ssl: {{ssl}} +{{/if}} +tags: +{{#if preserve_original_event}} + - preserve_original_event +{{/if}} +{{#if preserve_duplicate_custom_fields}} + - preserve_duplicate_custom_fields +{{/if}} +{{#each tags as |tag|}} + - {{tag}} +{{/each}} +{{#contains "forwarded" tags}} +publisher_pipeline.disable_host: true +{{/contains}} +processors: +- syslog: + field: message +{{#if processors}} +{{processors}} +{{/if}} diff --git a/packages/darktrace/data_stream/system_status_alert/agent/stream/udp.yml.hbs b/packages/darktrace/data_stream/system_status_alert/agent/stream/udp.yml.hbs new file mode 100644 index 00000000000..f342c4fa75c --- /dev/null +++ b/packages/darktrace/data_stream/system_status_alert/agent/stream/udp.yml.hbs @@ -0,0 +1,23 @@ +host: "{{listen_address}}:{{listen_port}}" +{{#if udp_options}} +{{udp_options}} +{{/if}} +tags: +{{#if preserve_original_event}} + - preserve_original_event +{{/if}} +{{#if preserve_duplicate_custom_fields}} + - preserve_duplicate_custom_fields +{{/if}} +{{#each tags as |tag|}} + - {{tag}} +{{/each}} +{{#contains "forwarded" tags}} +publisher_pipeline.disable_host: true +{{/contains}} +processors: +- syslog: + field: message +{{#if processors}} +{{processors}} +{{/if}} diff --git a/packages/darktrace/data_stream/system_status_alert/elasticsearch/ingest_pipeline/default.yml b/packages/darktrace/data_stream/system_status_alert/elasticsearch/ingest_pipeline/default.yml new file mode 100644 index 00000000000..50abd1f6162 --- /dev/null +++ b/packages/darktrace/data_stream/system_status_alert/elasticsearch/ingest_pipeline/default.yml @@ -0,0 +1,235 @@ +--- +description: Pipeline for processing System Status Alert logs. +processors: + - set: + field: ecs.version + value: '8.4.0' + - grok: + field: message + patterns: + - "^%{FIELD:log.syslog.appname}\\s*%{GREEDYDATA:message}$" + pattern_definitions: + FIELD: "[a-zA-Z]*" + - rename: + field: message + target_field: event.original + ignore_missing: true + - json: + field: event.original + target_field: json + - fingerprint: + fields: + - json.uuid + - json.last_updated + - json.last_updated_status + - json.message + target_field: _id + ignore_missing: true + - set: + field: event.type + value: [info] + - set: + field: event.kind + value: event + - set: + field: event.kind + value: alert + if: (['active','resolved'].contains(ctx.json?.status?.toLowerCase())) + - date: + field: json.last_updated + target_field: darktrace.system_status_alert.last_updated + formats: + - ISO8601 + - UNIX + - UNIX_MS + - 'MMM dd HH:mm:ss' + if: ctx.json?.last_updated != null + on_failure: + - remove: + field: json.last_updated + ignore_missing: true + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - set: + field: '@timestamp' + copy_from: darktrace.system_status_alert.last_updated + ignore_failure: true + - rename: + field: json.uuid + target_field: darktrace.system_status_alert.uuid + ignore_missing: true + - set: + field: event.id + copy_from: darktrace.system_status_alert.uuid + ignore_failure: true + - rename: + field: json.message + target_field: darktrace.system_status_alert.message + ignore_missing: true + - set: + field: event.reason + copy_from: darktrace.system_status_alert.message + ignore_failure: true + - convert: + field: json.priority + target_field: darktrace.system_status_alert.priority + type: double + ignore_missing: true + on_failure: + - remove: + field: json.priority + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - set: + field: event.risk_score + copy_from: darktrace.system_status_alert.priority + ignore_failure: true + - set: + field: event.risk_score_norm + copy_from: darktrace.system_status_alert.priority + ignore_failure: true + - uri_parts: + field: json.url + target_field: darktrace.system_status_alert.url + if: ctx.json?.url != null + keep_original: true + on_failure: + - remove: + field: json.url + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - set: + field: event.url + copy_from: darktrace.system_status_alert.url.original + ignore_failure: true + - rename: + field: json.hostname + target_field: darktrace.system_status_alert.hostname + ignore_missing: true + - convert: + field: darktrace.system_status_alert.hostname + target_field: darktrace.system_status_alert._temp_.hostname_ip + type: ip + ignore_missing: true + on_failure: + - set: + field: host.hostname + copy_from: darktrace.system_status_alert.hostname + ignore_failure: true + - append: + field: related.ip + value: '{{{darktrace.system_status_alert._temp_.hostname_ip}}}' + allow_duplicates: false + ignore_failure: true + - append: + field: related.hosts + value: '{{{host.hostname}}}' + allow_duplicates: false + ignore_failure: true + - convert: + field: json.ip_address + target_field: darktrace.system_status_alert._temp_.ip_address + type: ip + ignore_failure: true + - set: + field: host.ip + copy_from: darktrace.system_status_alert._temp_.ip_address + ignore_failure: true + - append: + field: related.ip + value: '{{{host.ip}}}' + allow_duplicates: false + ignore_failure: true + - rename: + field: json.ip_address + target_field: darktrace.system_status_alert.ip_address + ignore_missing: true + - remove: + field: darktrace.system_status_alert._temp_ + ignore_missing: true + - rename: + field: json.acknowledge_timeout + target_field: darktrace.system_status_alert.acknowledge_timeout + ignore_missing: true + - rename: + field: json.alert_name + target_field: darktrace.system_status_alert.alert_name + ignore_missing: true + - convert: + field: json.child_id + target_field: darktrace.system_status_alert.child_id + type: long + ignore_missing: true + on_failure: + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - date: + field: json.last_updated_status + target_field: darktrace.system_status_alert.last_updated_status + formats: + - ISO8601 + - UNIX + - UNIX_MS + - 'MMM dd HH:mm:ss' + if: ctx.json?.last_updated_status != null + on_failure: + - remove: + field: json.last_updated_status + ignore_missing: true + - append: + field: error.message + value: '{{{_ingest.on_failure_message}}}' + - rename: + field: json.name + target_field: darktrace.system_status_alert.name + ignore_missing: true + - rename: + field: json.priority_level + target_field: darktrace.system_status_alert.priority_level + ignore_missing: true + - lowercase: + field: json.status + target_field: darktrace.system_status_alert.status + ignore_failure: true + - remove: + field: json + ignore_missing: true + - remove: + if: ctx.tags == null || !(ctx.tags.contains('preserve_duplicate_custom_fields')) + field: + - darktrace.system_status_alert.last_updated + - darktrace.system_status_alert.uuid + - darktrace.system_status_alert.message + - darktrace.system_status_alert.priority + ignore_failure: true + ignore_missing: true + - remove: + field: event.original + if: ctx.tags == null || !(ctx.tags.contains('preserve_original_event')) + ignore_failure: true + ignore_missing: true + - script: + description: Drops null/empty values recursively. + lang: painless + source: + boolean dropEmptyFields(Object object) { + if (object == null || object == "") { + return true; + } else if (object instanceof Map) { + ((Map) object).values().removeIf(value -> dropEmptyFields(value)); + return (((Map) object).size() == 0); + } else if (object instanceof List) { + ((List) object).removeIf(value -> dropEmptyFields(value)); + return (((List) object).length == 0); + } + return false; + } + dropEmptyFields(ctx); +on_failure: + - append: + field: error.message + value: '{{{ _ingest.on_failure_message }}}' diff --git a/packages/darktrace/data_stream/system_status_alert/fields/agent.yml b/packages/darktrace/data_stream/system_status_alert/fields/agent.yml new file mode 100644 index 00000000000..10023a11743 --- /dev/null +++ b/packages/darktrace/data_stream/system_status_alert/fields/agent.yml @@ -0,0 +1,183 @@ +- name: cloud + title: Cloud + group: 2 + description: Fields related to the cloud or infrastructure the events are coming from. + footnote: 'Examples: If Metricbeat is running on an EC2 host and fetches data from its host, the cloud info contains the data about this machine. If Metricbeat runs on a remote machine outside the cloud and fetches data from a service running in the cloud, the field contains cloud data from the machine the service is running on.' + type: group + fields: + - name: account.id + level: extended + type: keyword + ignore_above: 1024 + description: 'The cloud account or organization id used to identify different entities in a multi-tenant environment. Examples: AWS account id, Google Cloud ORG Id, or other unique identifier.' + example: 666777888999 + - name: availability_zone + level: extended + type: keyword + ignore_above: 1024 + description: Availability zone in which this host is running. + example: us-east-1c + - name: instance.id + level: extended + type: keyword + ignore_above: 1024 + description: Instance ID of the host machine. + example: i-1234567890abcdef0 + - name: instance.name + level: extended + type: keyword + ignore_above: 1024 + description: Instance name of the host machine. + - name: machine.type + level: extended + type: keyword + ignore_above: 1024 + description: Machine type of the host machine. + example: t2.medium + - name: provider + level: extended + type: keyword + ignore_above: 1024 + description: Name of the cloud provider. Example values are aws, azure, gcp, or digitalocean. + example: aws + - name: region + level: extended + type: keyword + ignore_above: 1024 + description: Region in which this host is running. + example: us-east-1 + - name: project.id + type: keyword + description: Name of the project in Google Cloud. + - name: image.id + type: keyword + description: Image ID for the cloud instance. +- name: container + title: Container + group: 2 + description: 'Container fields are used for meta information about the specific container that is the source of information. These fields help correlate data based containers from any runtime.' + type: group + fields: + - name: id + level: core + type: keyword + ignore_above: 1024 + description: Unique container id. + - name: image.name + level: extended + type: keyword + ignore_above: 1024 + description: Name of the image the container was built on. + - name: labels + level: extended + type: object + object_type: keyword + description: Image labels. + - name: name + level: extended + type: keyword + ignore_above: 1024 + description: Container name. +- name: host + title: Host + group: 2 + description: 'A host is defined as a general computing instance. ECS host.* fields should be populated with details about the host on which the event happened, or from which the measurement was taken. Host types include hardware, virtual machines, Docker containers, and Kubernetes nodes.' + type: group + fields: + - name: architecture + level: core + type: keyword + ignore_above: 1024 + description: Operating system architecture. + example: x86_64 + - name: domain + level: extended + type: keyword + ignore_above: 1024 + description: 'Name of the domain of which the host is a member. For example, on Windows this could be the host''s Active Directory domain or NetBIOS domain name. For Linux this could be the domain of the host''s LDAP provider.' + example: CONTOSO + default_field: false + - name: hostname + level: core + type: keyword + ignore_above: 1024 + description: 'Hostname of the host. It normally contains what the `hostname` command returns on the host machine.' + - name: id + level: core + type: keyword + ignore_above: 1024 + description: 'Unique host id. As hostname is not always unique, use values that are meaningful in your environment. Example: The current usage of `beat.name`.' + - name: ip + level: core + type: ip + description: Host ip addresses. + - name: mac + level: core + type: keyword + ignore_above: 1024 + description: Host mac addresses. + - name: name + level: core + type: keyword + ignore_above: 1024 + description: 'Name of the host. It can contain what `hostname` returns on Unix systems, the fully qualified domain name, or a name specified by the user. The sender decides which value to use.' + - name: os.family + level: extended + type: keyword + ignore_above: 1024 + description: OS family (such as redhat, debian, freebsd, windows). + example: debian + - name: os.kernel + level: extended + type: keyword + ignore_above: 1024 + description: Operating system kernel version as a raw string. + example: 4.4.0-112-generic + - name: os.name + level: extended + type: keyword + ignore_above: 1024 + multi_fields: + - name: text + type: text + norms: false + default_field: false + description: Operating system name, without the version. + example: Mac OS X + - name: os.platform + level: extended + type: keyword + ignore_above: 1024 + description: Operating system platform (such centos, ubuntu, windows). + example: darwin + - name: os.version + level: extended + type: keyword + ignore_above: 1024 + description: Operating system version as a raw string. + example: 10.14.1 + - name: type + level: core + type: keyword + ignore_above: 1024 + description: 'Type of host. For Cloud providers this can be the machine type like `t2.medium`. If vm, this could be the container, for example, or other information meaningful in your environment.' + - name: containerized + type: boolean + description: >- + If the host is a container. + - name: os.build + type: keyword + example: '18D109' + description: >- + OS build information. + - name: os.codename + type: keyword + example: 'stretch' + description: >- + OS codename, if any. +- name: input.type + type: keyword + description: Input type +- name: log.offset + type: long + description: Log offset diff --git a/packages/darktrace/data_stream/system_status_alert/fields/base-fields.yml b/packages/darktrace/data_stream/system_status_alert/fields/base-fields.yml new file mode 100644 index 00000000000..ba05367fe7d --- /dev/null +++ b/packages/darktrace/data_stream/system_status_alert/fields/base-fields.yml @@ -0,0 +1,20 @@ +- name: data_stream.type + type: constant_keyword + description: Data stream type. +- name: data_stream.dataset + type: constant_keyword + description: Data stream dataset. +- name: data_stream.namespace + type: constant_keyword + description: Data stream namespace. +- name: event.module + type: constant_keyword + description: Event module. + value: darktrace +- name: event.dataset + type: constant_keyword + description: Event dataset. + value: darktrace.system_status_alert +- name: '@timestamp' + type: date + description: Event timestamp. diff --git a/packages/darktrace/data_stream/system_status_alert/fields/ecs.yml b/packages/darktrace/data_stream/system_status_alert/fields/ecs.yml new file mode 100644 index 00000000000..f2acb655a91 --- /dev/null +++ b/packages/darktrace/data_stream/system_status_alert/fields/ecs.yml @@ -0,0 +1,48 @@ +- external: ecs + name: ecs.version +- external: ecs + name: event.category +- external: ecs + name: event.created +- external: ecs + name: event.id +- external: ecs + name: event.kind +- external: ecs + name: event.original +- external: ecs + name: event.reason +- external: ecs + name: event.risk_score +- external: ecs + name: event.risk_score_norm +- external: ecs + name: event.type +- external: ecs + name: event.url +- external: ecs + name: host.hostname +- external: ecs + name: host.ip +- external: ecs + name: log.syslog.appname +- external: ecs + name: log.syslog.facility.code +- external: ecs + name: log.syslog.facility.name +- external: ecs + name: log.syslog.hostname +- external: ecs + name: log.syslog.priority +- external: ecs + name: log.syslog.severity.code +- external: ecs + name: log.syslog.severity.name +- external: ecs + name: log.syslog.version +- external: ecs + name: related.hosts +- external: ecs + name: related.ip +- external: ecs + name: tags diff --git a/packages/darktrace/data_stream/system_status_alert/fields/fields.yml b/packages/darktrace/data_stream/system_status_alert/fields/fields.yml new file mode 100644 index 00000000000..5bd54f4a6be --- /dev/null +++ b/packages/darktrace/data_stream/system_status_alert/fields/fields.yml @@ -0,0 +1,70 @@ +- name: darktrace.system_status_alert + type: group + fields: + - name: acknowledge_timeout + type: keyword + description: When acknowledgement of the alert expires. As alerts are sent externally on creation before acknowledgement is possible, this will be null in almost all cases. + - name: alert_name + type: keyword + description: A human readable name of the alert type. + - name: child_id + type: long + description: For probes (physical or virtual), the unique ID associated with the probe. + - name: hostname + type: keyword + description: The hostname (if known) of the host experiencing the system alert. An exception exists for disconnection notices, where the hostname will be of the master from which the instance has disconnected. + - name: ip_address + type: keyword + description: The IP of the host experiencing the system alert. An exception exists for disconnection notices, where the IP will be of the master from which the instance has disconnected. + - name: last_updated + type: date + description: A timestamp in epoch time that the system alert itself was updated. + - name: last_updated_status + type: date + description: A timestamp in epoch time that the status of the system alert was last updated globally. A status update is distinct from a update to the alert itself. + - name: message + type: keyword + description: A textual description of the system event that has triggered the alert. + - name: name + type: keyword + description: A system name of the alert type. + - name: priority + type: double + description: The numeric criticality associated with the alert. + - name: priority_level + type: keyword + description: 'The criticality of the alert. This value is calculated from the priority value: 0 - 40 low, 41 - 60 medium, 61 - 80 high, 81 - 100 critical.' + - name: status + type: keyword + description: The current status of the alert. Active alerts are ongoing, acknowledged events are those acknowledged on the System Status page, resolved alerts are system alerts that are no longer ongoing. Alerts will only be sent when alert enters the “active” or “resolved” state. + - name: url + type: group + fields: + - name: domain + type: keyword + - name: extension + type: keyword + - name: fragment + type: keyword + - name: full + type: keyword + - name: original + type: keyword + - name: password + type: keyword + - name: path + type: keyword + - name: port + type: long + - name: query + type: keyword + - name: scheme + type: keyword + - name: username + type: keyword + - name: uuid + type: keyword + description: A consistent UUID that can be used to navigate to the specific alert in the Threat Visualizer (https://[instance]/sysstatus/[uuid]). Where an alert is reactivated after resolution due to the issue reoccurring, the UUId will remain consistent across alerts. +- name: log.source.address + type: keyword + description: Source address from which the log event was read / sent from. diff --git a/packages/darktrace/data_stream/system_status_alert/manifest.yml b/packages/darktrace/data_stream/system_status_alert/manifest.yml new file mode 100644 index 00000000000..1f115305dc3 --- /dev/null +++ b/packages/darktrace/data_stream/system_status_alert/manifest.yml @@ -0,0 +1,117 @@ +title: Collect System Status Alert logs from Darktrace +type: logs +streams: + - input: tcp + title: System Status Alert logs + description: Collect System Status Alert logs via TCP input. + template_path: tcp.yml.hbs + vars: + - name: listen_port + type: integer + title: Listen Port + description: The TCP port number to listen on. + multi: false + required: true + show_user: true + default: 9573 + - name: tcp_options + type: yaml + title: Custom TCP Options + multi: false + required: false + show_user: false + default: | + max_message_size: 50KiB + #max_connections: 1 + #framing: delimiter + #line_delimiter: "\n" + description: Specify custom configuration options for the TCP input. + - name: tags + type: text + title: Tags + multi: true + required: true + show_user: false + default: + - forwarded + - darktrace-system_status_alert + - name: preserve_original_event + required: true + show_user: true + title: Preserve original event + description: Preserves a raw copy of the original event, added to the field `event.original`. + type: bool + multi: false + default: false + - name: preserve_duplicate_custom_fields + required: true + show_user: false + title: Preserve duplicate custom fields + description: Preserve darktrace.system_status_alert fields that were copied to Elastic Common Schema (ECS) fields. + type: bool + multi: false + default: false + - name: processors + type: yaml + title: Processors + multi: false + required: false + show_user: false + description: >- + Processors are used to reduce the number of fields in the exported event or to enhance the event with metadata. This executes in the agent before the logs are parsed. See [Processors](https://www.elastic.co/guide/en/beats/filebeat/current/filtering-and-enhancing-data.html) for details. + - input: udp + title: System Status Alert logs + description: Collect System Status Alert logs via UDP input. + template_path: udp.yml.hbs + vars: + - name: listen_port + type: integer + title: Listen Port + description: The UDP port number to listen on. + multi: false + required: true + show_user: true + default: 9576 + - name: udp_options + type: yaml + title: Custom UDP Options + multi: false + required: false + show_user: false + default: | + max_message_size: 50KiB + #timeout: 300s + description: Specify custom configuration options for the UDP input. + - name: tags + type: text + title: Tags + multi: true + required: true + show_user: false + default: + - forwarded + - darktrace-system_status_alert + - name: preserve_original_event + required: true + show_user: true + title: Preserve original event + description: Preserves a raw copy of the original event, added to the field `event.original`. + type: bool + multi: false + default: false + - name: preserve_duplicate_custom_fields + required: true + show_user: false + title: Preserve duplicate custom fields + description: Preserve darktrace.system_status_alert fields that were copied to Elastic Common Schema (ECS) fields. + type: bool + multi: false + default: false + - name: processors + type: yaml + title: Processors + multi: false + required: false + show_user: false + description: >- + Processors are used to reduce the number of fields in the exported event or to enhance the event with metadata. This executes in the agent before the logs are parsed. See [Processors](https://www.elastic.co/guide/en/beats/filebeat/current/filtering-and-enhancing-data.html) for details. diff --git a/packages/darktrace/data_stream/system_status_alert/sample_event.json b/packages/darktrace/data_stream/system_status_alert/sample_event.json new file mode 100644 index 00000000000..31283fa03ac --- /dev/null +++ b/packages/darktrace/data_stream/system_status_alert/sample_event.json @@ -0,0 +1,92 @@ +{ + "@timestamp": "2021-04-18T15:44:11.000Z", + "agent": { + "ephemeral_id": "5b042cea-01fa-47a2-ab0f-ac1f7baa6bd2", + "id": "95d2bc73-8bc8-47d9-b36e-a21b58255eec", + "name": "docker-fleet-agent", + "type": "filebeat", + "version": "8.2.1" + }, + "darktrace": { + "system_status_alert": { + "alert_name": "Advanced Search", + "child_id": 1, + "hostname": "example-vsensor", + "ip_address": "175.16.199.1", + "last_updated": "2021-04-18T15:44:11.000Z", + "last_updated_status": "2021-04-18T15:44:11.000Z", + "message": "There have been no Advanced Search hits for this instance seen since Sun 18 April 2021 13:20:23 (UTC). If this is not expected behaviour, please open a ticket using the following link or get in touch with your Cyber Technology Specialist. https://example.com/test", + "name": "advanced_search", + "priority": 43, + "priority_level": "medium", + "status": "active", + "uuid": "abcdabcd-1234-1234-1234-3abababcdcd3" + } + }, + "data_stream": { + "dataset": "darktrace.system_status_alert", + "namespace": "ep", + "type": "logs" + }, + "ecs": { + "version": "8.4.0" + }, + "elastic_agent": { + "id": "95d2bc73-8bc8-47d9-b36e-a21b58255eec", + "snapshot": false, + "version": "8.2.1" + }, + "event": { + "agent_id_status": "verified", + "dataset": "darktrace.system_status_alert", + "id": "abcdabcd-1234-1234-1234-3abababcdcd3", + "ingested": "2022-09-30T11:41:35Z", + "kind": "alert", + "original": "{\"last_updated\":1618760651,\"uuid\":\"abcdabcd-1234-1234-1234-3abababcdcd3\",\"priority\":43,\"priority_level\":\"medium\",\"hostname\":\"example-vsensor\",\"ip_address\":\"175.16.199.1\",\"message\":\"There have been no Advanced Search hits for this instance seen since Sun 18 April 2021 13:20:23 (UTC). If this is not expected behaviour, please open a ticket using the following link or get in touch with your Cyber Technology Specialist. https://example.com/test\",\"name\":\"advanced_search\",\"acknowledge_timeout\":null,\"alert_name\":\"Advanced Search\",\"child_id\":1,\"last_updated_status\":1618760651,\"status\":\"active\"}", + "reason": "There have been no Advanced Search hits for this instance seen since Sun 18 April 2021 13:20:23 (UTC). If this is not expected behaviour, please open a ticket using the following link or get in touch with your Cyber Technology Specialist. https://example.com/test", + "risk_score": 43, + "risk_score_norm": 43, + "type": [ + "info" + ] + }, + "host": { + "hostname": "example-vsensor", + "ip": "175.16.199.1" + }, + "input": { + "type": "udp" + }, + "log": { + "source": { + "address": "192.168.128.5:36197" + }, + "syslog": { + "facility": { + "code": 20, + "name": "local4" + }, + "hostname": "example.cloud.darktrace.com", + "priority": 165, + "severity": { + "code": 5, + "name": "Notice" + }, + "version": "1" + } + }, + "related": { + "hosts": [ + "example-vsensor" + ], + "ip": [ + "175.16.199.1" + ] + }, + "tags": [ + "preserve_original_event", + "preserve_duplicate_custom_fields", + "forwarded", + "darktrace-system_status_alert" + ] +} \ No newline at end of file diff --git a/packages/darktrace/docs/README.md b/packages/darktrace/docs/README.md new file mode 100644 index 00000000000..fe1804a8544 --- /dev/null +++ b/packages/darktrace/docs/README.md @@ -0,0 +1,1462 @@ +# Darktrace + +## Overview + +The [Darktrace](https://darktrace.com/) integration allows you to monitor Alert Logs. Darktrace is a network solution for detecting and investigating emerging cyber-threats that evade traditional security tools. It is powered by Enterprise Immune System technology, which uses machine learning and mathematics to monitor behaviors and detect anomalies in your organization’s network. + +Use the Darktrace integration to collect and parse data from the REST APIs or via Syslog. Then visualise that data in Kibana. + +For example, you could use the data from this integration to know which model is breached and analyse model breaches, and also know about system health, changes in monitored traffic, and any errors experienced by Darktrace Security Modules or probe instances. + +## Data streams + +The Darktrace integration collects logs for three types of events: AI Analyst Alert, Model Breach Alert and System Status Alert. + +**AI Analyst Alert** is generated by investigates, analyzes, and reports upon threats seen within your Darktrace environment; as a starting point, it reviews and investigates all Model Breaches that occur on the system. If behavior which would be of interest to a cyber analyst is detected, an event is created. See Example Schema [here](https://customerportal.darktrace.com/product-guides/main/syslog-aia-json-schema). + +**Model Breach Alert** is generated when a model breach is triggered. A model is used to define a set of conditions which, when met, will alert the system to the occurrence of a particular event or chain of anomalous behavior. Darktrace models are focused on pattern-of-life anomaly detection, potentially malicious behavior, and compliance issues. See Example Schema [here](https://customerportal.darktrace.com/product-guides/main/syslog-json-schema). + +**System Status Alert** keep Darktrace operators informed of system health, changes in monitored traffic, and any errors experienced by Darktrace Security Modules or probe instances. System Status Alerts include details of the originating host, the severity of the event, and links that may be helpful to investigate or resolve the issue. Notifications are sent for active system events and (optionally) on event resolution. See Example Schema [here](https://customerportal.darktrace.com/product-guides/main/syslog-json-system-schema). + +## Requirements + +You need Elasticsearch for storing and searching your data and Kibana for visualizing and managing it. You can use our hosted Elasticsearch Service on Elastic Cloud, which is recommended, or self-manage the Elastic Stack on your own hardware. + +Firewall exceptions to allow communication from the Darktrace master instance to the Syslog server. + +This module has been tested against **Darktrace Threat Visualizer v5.2**. + +## Setup + +### To collect data from Darktrace REST APIs, follow the below steps: + +1. Hostname URL will be your . (Threat Visualizer Console Hostname) +2. Public and Private Token will be generated by following this [Link](https://customerportal.darktrace.com/product-guides/main/api-tokens). + +**Note:** System Status Alert are not supported by REST API. + +### To collect data from Darktrace via Syslog, follow the below steps: + +The user needs to create a different Syslog Forwarder with different ports for each data stream. + +The process for configuring syslog-format alerts is identical for AI Analyst Alerts, Model Breach Alerts and System Status Alerts. Generic configuration guidance is provided below: + +1. Open the Darktrace Threat Visualizer Dashboard and navigate to the **System Config** page. (**Main menu › Admin**). +2. From the left-side menu, select **Modules**, then navigate to the **Workflow Integrations** section and choose **Syslog**. +3. Select **Syslog JSON** tab and click **New** to set up new Syslog Forwarder. +4. Enter the **IP Address**  and **Port** of the Elastic Agent that is running the integration in the **Server** and **Server Port** field respectively. + +For more details, see [Documentation](https://customerportal.darktrace.com/product-guides/main/json-alerts). + +**Note:** + - It is recommended to turn on **Full Timestamps** toggle in **Show Advanced Options** to get the full timestamp instead of the RFC3164-formatted timestamp. + - It is also recommended to turn off **Reduced Message Size** toggle in **Show Advanced Options** to get more information about alerts. + +### After following generic guidance steps, below are the steps for collecting individual logs for all three data streams. + +#### For AI Analyst Alert, below are the suggested configurations to collect all the events of AI Analyst Alert: + +- Configure the following settings in **Show Advanced Options**: + +| Field Name | Value | +| --------------------------------------- | ----------------------------------- | +| Send AI Analyst Alerts | ON | +| Send AI Analyst Alerts Immediately | ON | +| AI Analyst Behavior Filter | Critical, Suspicious and Compliance | +| Minimum AI Analyst Incident Event Score | 0 | +| Minimum AI Analyst Incident Score | 0 | +| Legacy AI Analyst Alerts | OFF | + +#### For Model Breach Alert, below are the suggested configurations to collect all the events of Model Breach Alert: + +- Configure the following settings in **Show Advanced Options**: + +| Field Name | Value | +| ---------------------------- | -------------------------------------------------- | +| Send Model Breach Alerts | ON | +| Model Breach Behavior Filter | Critical, Suspicious, Compliance and Informational | +| Minimum Breach Score | 0 | +| Minimum Breach Priority | 0 | +| Model Expression | N/A | +| Model Tags Expression | N/A | +| Device IP Addresses | N/A | +| Device Tags Addresses | N/A | + +#### For System Status Alert, below are the suggested configurations to collect all the events of System Status Alert: + +- Configure the following settings in **Show Advanced Options**: + +| Field Name | Value | +| ---------------------------------- | ------------- | +| Send System Status Alerts | ON | +| Send Resolved System Status Alerts | ON | +| Minimum System Status Priority | Informational | + +### See more about [Syslog Filters and Optional Settings](https://customerportal.darktrace.com/product-guides/main/syslog-json-alert-settings) + +**Note** : A Fully Qualified Domain Name (FQDN) must be configured for the Darktrace instance in order for links to be included in external alerts. + - An FQDN can be configured from the **System** subsection on the **Settings** tab of the Darktrace **System Config** page. + +## Logs reference + +### ai_analyst_alert + +This is the `ai_analyst_alert` dataset. + +#### Example + +An example event for `ai_analyst_alert` looks as following: + +```json +{ + "@timestamp": "2021-08-03T14:48:09.240Z", + "agent": { + "ephemeral_id": "82482032-e103-4c45-a00e-103ac604f4ae", + "id": "95d2bc73-8bc8-47d9-b36e-a21b58255eec", + "name": "docker-fleet-agent", + "type": "filebeat", + "version": "8.2.1" + }, + "darktrace": { + "ai_analyst_alert": { + "activity_id": "abcd1234", + "aia_score": 98, + "attack_phases": [ + 5 + ], + "breach_devices": [ + { + "did": 10, + "ip": "81.2.69.144", + "sid": 12, + "subnet": "VPN" + } + ], + "category": "critical", + "children": [ + "eabcdef0-1234-1234-1234-cabcdefghij9" + ], + "created_at": "2021-08-03T14:48:09.240Z", + "current_group": "eabc1234-1234-1234-1234-cabcdefg0011", + "details": [ + [ + { + "contents": [ + { + "type": "device", + "values": [ + { + "did": 10, + "ip": "175.16.199.1", + "sid": 12, + "subnet": "VPN" + } + ] + } + ], + "header": "Breaching Device" + } + ], + [ + { + "contents": [ + { + "key": "Time", + "type": "timestampRange", + "values": [ + { + "end": 1628000141220, + "start": 1627985298683 + } + ] + }, + { + "key": "Number of unique IPs", + "type": "integer", + "values": [ + 16 + ] + }, + { + "key": "Targeted IP ranges include", + "type": "device", + "values": [ + { + "ip": "81.2.69.192" + }, + { + "ip": "175.16.199.1" + }, + { + "ip": "175.16.199.3" + } + ] + }, + { + "key": "Destination port", + "type": "integer", + "values": [ + 22 + ] + }, + { + "key": "Connection count", + "type": "integer", + "values": [ + 40 + ] + }, + { + "key": "Percentage successful", + "type": "percentage", + "values": [ + 100 + ] + } + ], + "header": "SSH Activity" + } + ] + ], + "group_by_activity": false, + "group_category": "critical", + "group_score": 72.9174234, + "grouping_ids": [ + "abcdef12" + ], + "id": "eabc0011-1234-1234-1234-cabcdefg0011", + "is_acknowledged": false, + "is_external_triggered": false, + "is_pinned": true, + "is_user_triggered": false, + "periods": [ + { + "end": "2021-08-03T14:15:41.220Z", + "start": "2021-08-03T10:08:18.683Z" + } + ], + "related_breaches": [ + { + "model_name": "Unusual Activity / Unusual Activity from Re-Activated Device", + "pbid": 1234, + "threat_score": 37, + "timestamp": "2021-08-03T13:25:57.000Z" + } + ], + "summariser": "AdminConnSummary", + "summary": "The device 175.16.199.1 was observed making unusual internal SSH connections to a wide range of devices.\n\nThough this behaviour could be the result of legitimate remote access or administration, it could also be a sign of attempted lateral movement by a compromised machine.\n\nConsequently, if this activity was not expected, the security team may wish to investigate further.", + "title": "Extensive Unusual SSH Connections" + } + }, + "data_stream": { + "dataset": "darktrace.ai_analyst_alert", + "namespace": "ep", + "type": "logs" + }, + "ecs": { + "version": "8.4.0" + }, + "elastic_agent": { + "id": "95d2bc73-8bc8-47d9-b36e-a21b58255eec", + "snapshot": false, + "version": "8.2.1" + }, + "event": { + "agent_id_status": "verified", + "category": [ + "threat" + ], + "dataset": "darktrace.ai_analyst_alert", + "duration": [ + 14842537000000 + ], + "end": [ + "2021-08-03T14:15:41.220Z" + ], + "id": "eabc0011-1234-1234-1234-cabcdefg0011", + "ingested": "2022-09-30T11:36:06Z", + "kind": "alert", + "original": "{\"summariser\":\"AdminConnSummary\",\"acknowledged\":false,\"pinned\":true,\"createdAt\":1628002089240,\"attackPhases\":[5],\"title\":\"Extensive Unusual SSH Connections\",\"id\":\"eabc0011-1234-1234-1234-cabcdefg0011\",\"children\":[\"eabcdef0-1234-1234-1234-cabcdefghij9\"],\"category\":\"critical\",\"currentGroup\":\"eabc1234-1234-1234-1234-cabcdefg0011\",\"groupCategory\":\"critical\",\"groupScore\":\"72.9174234\",\"groupPreviousGroups\":null,\"activityId\":\"abcd1234\",\"groupingIds\":[\"abcdef12\"],\"groupByActivity\":false,\"userTriggered\":false,\"externalTriggered\":false,\"aiaScore\":98,\"summary\":\"The device 175.16.199.1 was observed making unusual internal SSH connections to a wide range of devices.\\n\\nThough this behaviour could be the result of legitimate remote access or administration, it could also be a sign of attempted lateral movement by a compromised machine.\\n\\nConsequently, if this activity was not expected, the security team may wish to investigate further.\",\"periods\":[{\"start\":1627985298683,\"end\":1628000141220}],\"breachDevices\":[{\"identifier\":null,\"hostname\":null,\"ip\":\"81.2.69.144\",\"mac\":null,\"subnet\":\"VPN\",\"did\":10,\"sid\":12}],\"relatedBreaches\":[{\"modelName\":\"Unusual Activity / Unusual Activity from Re-Activated Device\",\"pbid\":1234,\"threatScore\":37,\"timestamp\":1627997157000}],\"details\":[[{\"header\":\"Breaching Device\",\"contents\":[{\"key\":null,\"type\":\"device\",\"values\":[{\"identifier\":null,\"hostname\":null,\"ip\":\"175.16.199.1\",\"mac\":null,\"subnet\":\"VPN\",\"did\":10,\"sid\":12}]}]}],[{\"header\":\"SSH Activity\",\"contents\":[{\"key\":\"Time\",\"type\":\"timestampRange\",\"values\":[{\"start\":1627985298683,\"end\":1628000141220}]},{\"key\":\"Number of unique IPs\",\"type\":\"integer\",\"values\":[16]},{\"key\":\"Targeted IP ranges include\",\"type\":\"device\",\"values\":[{\"identifier\":null,\"hostname\":null,\"ip\":\"81.2.69.192\",\"mac\":null,\"subnet\":null,\"did\":null,\"sid\":null},{\"identifier\":null,\"hostname\":null,\"ip\":\"175.16.199.1\",\"mac\":null,\"subnet\":null,\"did\":null,\"sid\":null},{\"identifier\":null,\"hostname\":null,\"ip\":\"175.16.199.3\",\"mac\":null,\"subnet\":null,\"did\":null,\"sid\":null}]},{\"key\":\"Destination port\",\"type\":\"integer\",\"values\":[22]},{\"key\":\"Connection count\",\"type\":\"integer\",\"values\":[40]},{\"key\":\"Percentage successful\",\"type\":\"percentage\",\"values\":[100]}]}]]}", + "reason": "Extensive Unusual SSH Connections", + "risk_score": 98, + "risk_score_norm": 98, + "start": [ + "2021-08-03T10:08:18.683Z" + ], + "type": [ + "info" + ] + }, + "host": { + "id": [ + "10" + ], + "ip": [ + "81.2.69.144" + ] + }, + "input": { + "type": "udp" + }, + "log": { + "source": { + "address": "192.168.128.5:49066" + }, + "syslog": { + "facility": { + "code": 20, + "name": "local4" + }, + "hostname": "example.cloud.darktrace.com", + "priority": 165, + "severity": { + "code": 5, + "name": "Notice" + }, + "version": "1" + } + }, + "message": "The device 175.16.199.1 was observed making unusual internal SSH connections to a wide range of devices.\n\nThough this behaviour could be the result of legitimate remote access or administration, it could also be a sign of attempted lateral movement by a compromised machine.\n\nConsequently, if this activity was not expected, the security team may wish to investigate further.", + "related": { + "ip": [ + "81.2.69.144", + "175.16.199.1", + "81.2.69.192", + "175.16.199.3" + ] + }, + "rule": { + "name": [ + "Unusual Activity / Unusual Activity from Re-Activated Device" + ] + }, + "tags": [ + "preserve_original_event", + "preserve_duplicate_custom_fields", + "forwarded", + "darktrace-ai_analyst_alert" + ], + "threat": { + "enrichments": { + "matched": { + "id": [ + "eabcdef0-1234-1234-1234-cabcdefghij9" + ] + } + }, + "group": { + "id": "eabc1234-1234-1234-1234-cabcdefg0011" + } + } +} +``` + +**Exported fields** + +| Field | Description | Type | +|---|---|---| +| @timestamp | Event timestamp. | date | +| cloud.account.id | The cloud account or organization id used to identify different entities in a multi-tenant environment. Examples: AWS account id, Google Cloud ORG Id, or other unique identifier. | keyword | +| cloud.availability_zone | Availability zone in which this host is running. | keyword | +| cloud.image.id | Image ID for the cloud instance. | keyword | +| cloud.instance.id | Instance ID of the host machine. | keyword | +| cloud.instance.name | Instance name of the host machine. | keyword | +| cloud.machine.type | Machine type of the host machine. | keyword | +| cloud.project.id | Name of the project in Google Cloud. | keyword | +| cloud.provider | Name of the cloud provider. Example values are aws, azure, gcp, or digitalocean. | keyword | +| cloud.region | Region in which this host is running. | keyword | +| container.id | Unique container id. | keyword | +| container.image.name | Name of the image the container was built on. | keyword | +| container.labels | Image labels. | object | +| container.name | Container name. | keyword | +| darktrace.ai_analyst_alert.activity_id | An identifier for the specific activity detected by AI Analyst. If groupByActivity=true , this field should be used to group events together into an incident. | keyword | +| darktrace.ai_analyst_alert.aia_score | The score of the event as classified by AI Analyst - out of 100. | double | +| darktrace.ai_analyst_alert.attack_phases | Of the six attack phases, which phases are applicable to the activity. | long | +| darktrace.ai_analyst_alert.breach_devices.did | The unique device id identifier for the device that triggered the breach. This field is used to group events into device-based incidents within the Threat Visualizer. | long | +| darktrace.ai_analyst_alert.breach_devices.hostname | The hostname associated with the device, if available. | keyword | +| darktrace.ai_analyst_alert.breach_devices.identifier | An identifier for the device used when constructing summaries or reports. May be the device label, hostname or IP, depending on availability. | keyword | +| darktrace.ai_analyst_alert.breach_devices.ip | The IP associated with the device. | keyword | +| darktrace.ai_analyst_alert.breach_devices.mac_address | The MAC address associated with the device. | keyword | +| darktrace.ai_analyst_alert.breach_devices.sid | The subnet id for the subnet the device is currently located in. | long | +| darktrace.ai_analyst_alert.breach_devices.subnet | The subnet label for the corresponding subnet, if available. | keyword | +| darktrace.ai_analyst_alert.category | The behavior category associated with the incident event. | keyword | +| darktrace.ai_analyst_alert.children | One or more unique identifiers that can be used to request this AI Analyst event via the UI or API. Where there is more than one uuid, requests can be made with comma-separated values. | keyword | +| darktrace.ai_analyst_alert.created_at | Timestamp for event creation in epoch time. | date | +| darktrace.ai_analyst_alert.current_group | The UUID of the current incident this event belongs to. | keyword | +| darktrace.ai_analyst_alert.details | An array of multiple sections (sub-arrays) of event information. | flattened | +| darktrace.ai_analyst_alert.group_by_activity | Used by pre-v5.2 legacy incident construction. Indicates whether the event should be aggregated by activity or by device to create an incident. When true, the event should be aggregated by activityID, and when false, aggregated by groupingID(s). | boolean | +| darktrace.ai_analyst_alert.group_category | The behavior category associated with the incident overall. Relevant for v5.2+ incident construction only. | keyword | +| darktrace.ai_analyst_alert.group_previous_groups | If the incident event was part of an incident which was later merged with another, the UUIDs of the incidents before they were merged. | keyword | +| darktrace.ai_analyst_alert.group_score | The current overall score of the incident this event is part of. | double | +| darktrace.ai_analyst_alert.grouping_ids | Used by pre-v5.2 legacy incident construction. Each entry in the groupingIDs array refers to a device that triggered the activity detection. In single events, should only contain one ID. If groupByActivity=false , this field should be used to group events together into an incident. | keyword | +| darktrace.ai_analyst_alert.id | A system field. | keyword | +| darktrace.ai_analyst_alert.incident_event_url.domain | | keyword | +| darktrace.ai_analyst_alert.incident_event_url.extension | | keyword | +| darktrace.ai_analyst_alert.incident_event_url.fragment | | keyword | +| darktrace.ai_analyst_alert.incident_event_url.full | | keyword | +| darktrace.ai_analyst_alert.incident_event_url.original | | keyword | +| darktrace.ai_analyst_alert.incident_event_url.password | | keyword | +| darktrace.ai_analyst_alert.incident_event_url.path | | keyword | +| darktrace.ai_analyst_alert.incident_event_url.port | | long | +| darktrace.ai_analyst_alert.incident_event_url.query | | keyword | +| darktrace.ai_analyst_alert.incident_event_url.scheme | | keyword | +| darktrace.ai_analyst_alert.incident_event_url.username | | keyword | +| darktrace.ai_analyst_alert.is_acknowledged | Whether the event has been acknowledged. | boolean | +| darktrace.ai_analyst_alert.is_external_triggered | Whether the event was created as a result of an externally triggered AI Analyst investigation. | boolean | +| darktrace.ai_analyst_alert.is_pinned | Whether the event, or an incident that the event is associated with, is pinned within the Threat Visualizer user interface. Pinned events will always return regardless of the timeframe specified. | boolean | +| darktrace.ai_analyst_alert.is_user_triggered | Whether the event was created as a result of a user-triggered AI Analyst investigation. | boolean | +| darktrace.ai_analyst_alert.periods.end | A timestamp for the end of the activity period in epoch time. | date | +| darktrace.ai_analyst_alert.periods.start | A timestamp for the start of the activity period in epoch time. | date | +| darktrace.ai_analyst_alert.related_breaches.model_name | The name of the model that breached. | keyword | +| darktrace.ai_analyst_alert.related_breaches.pbid | The policy breach ID unique identifier of the model breach. | long | +| darktrace.ai_analyst_alert.related_breaches.threat_score | The breach score of the associated model breach - out of 100. | long | +| darktrace.ai_analyst_alert.related_breaches.timestamp | The timestamp at which the model breach occurred in epoch time. | date | +| darktrace.ai_analyst_alert.summariser | A system field. | keyword | +| darktrace.ai_analyst_alert.summary | A textual summary of the suspicious activity. This example is abbreviated. | keyword | +| darktrace.ai_analyst_alert.title | A title describing the activity that occurred. | keyword | +| data_stream.dataset | Data stream dataset. | constant_keyword | +| data_stream.namespace | Data stream namespace. | constant_keyword | +| data_stream.type | Data stream type. | constant_keyword | +| ecs.version | ECS version this event conforms to. `ecs.version` is a required field and must exist in all events. When querying across multiple indices -- which may conform to slightly different ECS versions -- this field lets integrations adjust to the schema version of the events. | keyword | +| event.category | This is one of four ECS Categorization Fields, and indicates the second level in the ECS category hierarchy. `event.category` represents the "big buckets" of ECS categories. For example, filtering on `event.category:process` yields all events relating to process activity. This field is closely related to `event.type`, which is used as a subcategory. This field is an array. This will allow proper categorization of some events that fall in multiple categories. | keyword | +| event.created | event.created contains the date/time when the event was first read by an agent, or by your pipeline. This field is distinct from @timestamp in that @timestamp typically contain the time extracted from the original event. In most situations, these two timestamps will be slightly different. The difference can be used to calculate the delay between your source generating an event, and the time when your agent first processed it. This can be used to monitor your agent's or pipeline's ability to keep up with your event source. In case the two timestamps are identical, @timestamp should be used. | date | +| event.dataset | Event dataset. | constant_keyword | +| event.end | event.end contains the date when the event ended or when the activity was last observed. | date | +| event.id | Unique ID to describe the event. | keyword | +| event.kind | This is one of four ECS Categorization Fields, and indicates the highest level in the ECS category hierarchy. `event.kind` gives high-level information about what type of information the event contains, without being specific to the contents of the event. For example, values of this field distinguish alert events from metric events. The value of this field can be used to inform how these kinds of events should be handled. They may warrant different retention, different access control, it may also help understand whether the data coming in at a regular interval or not. | keyword | +| event.module | Event module. | constant_keyword | +| event.original | Raw text message of entire event. Used to demonstrate log integrity or where the full log message (before splitting it up in multiple parts) may be required, e.g. for reindex. This field is not indexed and doc_values are disabled. It cannot be searched, but it can be retrieved from `_source`. If users wish to override this and index this field, please see `Field data types` in the `Elasticsearch Reference`. | keyword | +| event.reason | Reason why this event happened, according to the source. This describes the why of a particular action or outcome captured in the event. Where `event.action` captures the action from the event, `event.reason` describes why that action was taken. For example, a web proxy with an `event.action` which denied the request may also populate `event.reason` with the reason why (e.g. `blocked site`). | keyword | +| event.risk_score | Risk score or priority of the event (e.g. security solutions). Use your system's original value here. | float | +| event.risk_score_norm | Normalized risk score or priority of the event, on a scale of 0 to 100. This is mainly useful if you use more than one system that assigns risk scores, and you want to see a normalized value across all systems. | float | +| event.start | event.start contains the date when the event started or when the activity was first observed. | date | +| event.type | This is one of four ECS Categorization Fields, and indicates the third level in the ECS category hierarchy. `event.type` represents a categorization "sub-bucket" that, when used along with the `event.category` field values, enables filtering events down to a level appropriate for single visualization. This field is an array. This will allow proper categorization of some events that fall in multiple event types. | keyword | +| event.url | URL linking to an external system to continue investigation of this event. This URL links to another system where in-depth investigation of the specific occurrence of this event can take place. Alert events, indicated by `event.kind:alert`, are a common use case for this field. | keyword | +| host.architecture | Operating system architecture. | keyword | +| host.containerized | If the host is a container. | boolean | +| host.domain | Name of the domain of which the host is a member. For example, on Windows this could be the host's Active Directory domain or NetBIOS domain name. For Linux this could be the domain of the host's LDAP provider. | keyword | +| host.hostname | Hostname of the host. It normally contains what the `hostname` command returns on the host machine. | keyword | +| host.id | Unique host id. As hostname is not always unique, use values that are meaningful in your environment. Example: The current usage of `beat.name`. | keyword | +| host.ip | Host ip addresses. | ip | +| host.mac | Host mac addresses. | keyword | +| host.name | Name of the host. It can contain what `hostname` returns on Unix systems, the fully qualified domain name, or a name specified by the user. The sender decides which value to use. | keyword | +| host.os.build | OS build information. | keyword | +| host.os.codename | OS codename, if any. | keyword | +| host.os.family | OS family (such as redhat, debian, freebsd, windows). | keyword | +| host.os.kernel | Operating system kernel version as a raw string. | keyword | +| host.os.name | Operating system name, without the version. | keyword | +| host.os.name.text | Multi-field of `host.os.name`. | text | +| host.os.platform | Operating system platform (such centos, ubuntu, windows). | keyword | +| host.os.version | Operating system version as a raw string. | keyword | +| host.type | Type of host. For Cloud providers this can be the machine type like `t2.medium`. If vm, this could be the container, for example, or other information meaningful in your environment. | keyword | +| input.type | Input type | keyword | +| log.offset | Log offset | long | +| log.source.address | Source address from which the log event was read / sent from. | keyword | +| log.syslog.appname | The device or application that originated the Syslog message, if available. | keyword | +| log.syslog.facility.code | The Syslog numeric facility of the log event, if available. According to RFCs 5424 and 3164, this value should be an integer between 0 and 23. | long | +| log.syslog.facility.name | The Syslog text-based facility of the log event, if available. | keyword | +| log.syslog.hostname | The hostname, FQDN, or IP of the machine that originally sent the Syslog message. This is sourced from the hostname field of the syslog header. Depending on the environment, this value may be different from the host that handled the event, especially if the host handling the events is acting as a collector. | keyword | +| log.syslog.priority | Syslog numeric priority of the event, if available. According to RFCs 5424 and 3164, the priority is 8 \* facility + severity. This number is therefore expected to contain a value between 0 and 191. | long | +| log.syslog.severity.code | The Syslog numeric severity of the log event, if available. If the event source publishing via Syslog provides a different numeric severity value (e.g. firewall, IDS), your source's numeric severity should go to `event.severity`. If the event source does not specify a distinct severity, you can optionally copy the Syslog severity to `event.severity`. | long | +| log.syslog.severity.name | The Syslog numeric severity of the log event, if available. If the event source publishing via Syslog provides a different severity value (e.g. firewall, IDS), your source's text severity should go to `log.level`. If the event source does not specify a distinct severity, you can optionally copy the Syslog severity to `log.level`. | keyword | +| log.syslog.version | The version of the Syslog protocol specification. Only applicable for RFC 5424 messages. | keyword | +| message | For log events the message field contains the log message, optimized for viewing in a log viewer. For structured logs without an original message field, other fields can be concatenated to form a human-readable summary of the event. If multiple messages exist, they can be combined into one message. | match_only_text | +| related.hosts | All hostnames or other host identifiers seen on your event. Example identifiers include FQDNs, domain names, workstation names, or aliases. | keyword | +| related.ip | All of the IPs seen on your event. | ip | +| rule.name | The name of the rule or signature generating the event. | keyword | +| tags | List of keywords used to tag each event. | keyword | +| threat.enrichments.matched.id | Identifies the _id of the indicator document enriching the event. | keyword | +| threat.group.id | The id of the group for a set of related intrusion activity that are tracked by a common name in the security community. While not required, you can use a MITRE ATT&CK® group id. | keyword | + + +### model_breach_alert + +This is the `model_breach_alert` dataset. + +#### Example + +An example event for `model_breach_alert` looks as following: + +```json +{ + "@timestamp": "2022-07-11T13:04:08.000Z", + "agent": { + "ephemeral_id": "572d7663-c480-491f-b06f-96f0330cf942", + "id": "95d2bc73-8bc8-47d9-b36e-a21b58255eec", + "name": "docker-fleet-agent", + "type": "filebeat", + "version": "8.2.1" + }, + "darktrace": { + "model_breach_alert": { + "aianalyst_data": [ + { + "related": [ + 1 + ], + "summariser": "BeaconSummary", + "uuid": "1234abcd-1234-1234-1234-123456abcdef" + } + ], + "comment": { + "count": 0 + }, + "creation_time": "2022-07-11T13:04:19.000Z", + "device": { + "did": 3, + "first_seen": "2022-07-11T12:54:49.000Z", + "ip": "81.2.69.142", + "last_seen": "2022-07-11T13:00:18.000Z", + "sid": 1, + "type_label": "Desktop", + "type_name": "desktop" + }, + "model": { + "actions": { + "is_alerting": true, + "is_breach": true, + "is_priority_set": false, + "is_tag_set": false, + "is_type_set": false, + "model": true + }, + "active_times": { + "type": "exclusions", + "version": 2 + }, + "behaviour": "incdec1", + "category": "Informational", + "created": { + "by": "System" + }, + "delay": 0, + "description": "A device has been repeatedly connecting to a rare external location with a beacon score. A beacon score is added when Darktrace identifies that a device is regularly communicating with an endpoint, for example, if a device connects to a rare external endpoint every 12 minutes this would get a beacon score. This model is designed to identify beaconing at a lower threshold and be protocol agnostic.\\n\\nAction: Review the external domains and IPs being connected to to see if they are legitimate and would be expected for business purposes.", + "edited": { + "by": "System" + }, + "in_compliance_behavior_category": false, + "interval": 10800, + "is_active": true, + "is_auto_suppress": true, + "is_auto_updatable": true, + "is_auto_update": true, + "is_sequenced": false, + "is_shared_endpoints": false, + "logic": { + "data_weighted_component_list": [ + { + "cid": 2026, + "weight": 1 + }, + { + "cid": 2024, + "weight": 1 + }, + { + "cid": 2025, + "weight": -100 + } + ], + "target_score": 1, + "type": "weightedComponentList", + "version": 1 + }, + "modified": "2022-07-11T11:47:37.000Z", + "name": "Compromise::Beaconing Activity To External Rare", + "phid": 1072, + "pid": 156, + "priority": 2, + "tags": [ + "AP: C2 Comms" + ], + "throttle": 10800, + "uuid": "1234abcd-1234-1234-1234-123456abcdef", + "version": 23 + }, + "pbid": 1, + "score": 0.674, + "time": "2022-07-11T13:04:08.000Z", + "triggered_components": [ + { + "cbid": 1, + "chid": 2113, + "cid": 2026, + "interval": 3600, + "logic": { + "data": "{left={left=A, right={left=AA, right={left=AC, right={left=AD, right={left=AF, right={left=AG, right={left=AH, right={left=B, right={left=C, right={left=D, right={left=E, right={left=H, right={left=I, right={left=J, right={left=K, right={left=L, right={left=M, right={left=N, right={left=O, right={left=P, right={left=S, right={left=U, right={left=V, right={left=X, right={left=Y, right=Z, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, right={left=A, right={left=AA, right={left=AB, right={left=AE, right={left=AF, right={left=AG, right={left=AH, right={left=C, right={left=D, right={left=E, right={left=H, right={left=I, right={left=J, right={left=K, right={left=L, right={left=M, right={left=N, right={left=O, right={left=P, right={left=S, right={left=U, right={left=V, right={left=X, right={left=Y, right=Z, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=AND}, operator=OR}", + "version": "v0.1" + }, + "metric": { + "label": "External Connections", + "mlid": 1, + "name": "externalconnections" + }, + "size": 11, + "threshold": 10, + "time": "2022-07-11T13:04:08.000Z", + "triggered_filters": [ + { + "arguments": { + "value": 60 + }, + "cfid": 23426, + "comparator_type": "\u003e", + "filter_type": "Beaconing score", + "id": "A", + "trigger": { + "value": "100" + } + }, + { + "arguments": { + "value": 0 + }, + "cfid": 23427, + "comparator_type": "\u003e", + "filter_type": "Individual size up", + "id": "AA", + "trigger": { + "value": "4382" + } + }, + { + "arguments": { + "value": 95 + }, + "cfid": 23428, + "comparator_type": "\u003e", + "filter_type": "Rare domain", + "id": "AB", + "trigger": { + "value": "100" + } + }, + { + "arguments": { + "value": 1209600 + }, + "cfid": 23430, + "comparator_type": "\u003c", + "filter_type": "Age of destination", + "id": "AD", + "trigger": { + "value": "558" + } + }, + { + "arguments": { + "value": 1209600 + }, + "cfid": 23431, + "comparator_type": "\u003c", + "filter_type": "Age of external hostname", + "id": "AE", + "trigger": { + "value": "558" + } + }, + { + "arguments": { + "value": "examples" + }, + "cfid": 23432, + "comparator_type": "does not match regular expression", + "filter_type": "Connection hostname", + "id": "AF", + "trigger": { + "value": "example.com" + } + }, + { + "arguments": { + "value": "examples" + }, + "cfid": 23433, + "comparator_type": "does not match regular expression", + "filter_type": "ASN", + "id": "AG", + "trigger": { + "value": "AS12345 LOCAL-02" + } + }, + { + "arguments": { + "value": "5d41402abc4b2a76b9719d911017c592" + }, + "cfid": 23434, + "comparator_type": "does not match", + "filter_type": "JA3 hash", + "id": "AH", + "trigger": { + "value": "5d41402abc4b2a76b9719d911017c592" + } + }, + { + "arguments": { + "value": 95 + }, + "cfid": 23435, + "comparator_type": "\u003e", + "filter_type": "Rare external IP", + "id": "B", + "trigger": { + "value": "100" + } + }, + { + "arguments": { + "value": "1003" + }, + "cfid": 23436, + "comparator_type": "is not", + "filter_type": "Application protocol", + "id": "C", + "trigger": { + "value": "1004" + } + }, + { + "arguments": { + "value": 53 + }, + "cfid": 23437, + "comparator_type": "!=", + "filter_type": "Destination port", + "id": "D", + "trigger": { + "value": "443" + } + }, + { + "arguments": { + "value": "out" + }, + "cfid": 23438, + "comparator_type": "is", + "filter_type": "Direction", + "id": "E", + "trigger": { + "value": "out" + } + }, + { + "arguments": { + "value": 137 + }, + "cfid": 23439, + "comparator_type": "!=", + "filter_type": "Destination port", + "id": "H", + "trigger": { + "value": "443" + } + }, + { + "arguments": { + "value": 161 + }, + "cfid": 23440, + "comparator_type": "!=", + "filter_type": "Destination port", + "id": "I", + "trigger": { + "value": "443" + } + }, + { + "arguments": { + "value": "6" + }, + "cfid": 23441, + "comparator_type": "is", + "filter_type": "Protocol", + "id": "J", + "trigger": { + "value": "6" + } + }, + { + "arguments": { + "value": "Company" + }, + "cfid": 23442, + "comparator_type": "does not contain", + "filter_type": "ASN", + "id": "K", + "trigger": { + "value": "AS12345 LOCAL-02" + } + }, + { + "arguments": { + "value": "Company" + }, + "cfid": 23443, + "comparator_type": "does not contain", + "filter_type": "ASN", + "id": "L", + "trigger": { + "value": "AS12345 LOCAL-02" + } + }, + { + "arguments": { + "value": "13" + }, + "cfid": 23444, + "comparator_type": "is not", + "filter_type": "Internal source device type", + "id": "M", + "trigger": { + "value": "6" + } + }, + { + "arguments": { + "value": "5" + }, + "cfid": 23445, + "comparator_type": "is not", + "filter_type": "Internal source device type", + "id": "N", + "trigger": { + "value": "6" + } + }, + { + "arguments": { + "value": "9" + }, + "cfid": 23446, + "comparator_type": "is not", + "filter_type": "Internal source device type", + "id": "O", + "trigger": { + "value": "6" + } + }, + { + "arguments": { + "value": "12" + }, + "cfid": 23447, + "comparator_type": "is not", + "filter_type": "Internal source device type", + "id": "P", + "trigger": { + "value": "6" + } + }, + { + "arguments": { + "value": "30" + }, + "cfid": 23448, + "comparator_type": "is not", + "filter_type": "Internal source device type", + "id": "S", + "trigger": { + "value": "6" + } + }, + { + "arguments": { + "value": "4" + }, + "cfid": 23449, + "comparator_type": "is not", + "filter_type": "Internal source device type", + "id": "U", + "trigger": { + "value": "6" + } + }, + { + "arguments": { + "value": "3" + }, + "cfid": 23450, + "comparator_type": "is not", + "filter_type": "Internal source device type", + "id": "V", + "trigger": { + "value": "6" + } + }, + { + "arguments": { + "value": "false" + }, + "cfid": 23451, + "comparator_type": "is", + "filter_type": "Trusted hostname", + "id": "X", + "trigger": { + "value": "false" + } + }, + { + "arguments": { + "value": 26 + }, + "cfid": 23452, + "comparator_type": "does not have tag", + "filter_type": "Tagged internal source", + "id": "Y", + "trigger": { + "tag": { + "data": { + "auto": false, + "color": 5, + "visibility": "Public" + }, + "expiry": 0, + "is_referenced": true, + "name": "No Device Tracking", + "restricted": false, + "thid": 26, + "tid": 26 + }, + "value": "26" + } + }, + { + "arguments": { + "value": 0 + }, + "cfid": 23453, + "comparator_type": "\u003e", + "filter_type": "Individual size down", + "id": "Z", + "trigger": { + "value": "5862" + } + }, + { + "cfid": 23454, + "comparator_type": "display", + "filter_type": "JA3 hash", + "id": "d1", + "trigger": { + "value": "5d41402abc4b2a76b9719d911017c592" + } + }, + { + "cfid": 23455, + "comparator_type": "display", + "filter_type": "ASN", + "id": "d2", + "trigger": { + "value": "AS12345 LOCAL-02" + } + }, + { + "cfid": 23456, + "comparator_type": "display", + "filter_type": "Destination IP", + "id": "d3", + "trigger": { + "value": "81.2.69.192" + } + }, + { + "cfid": 23457, + "comparator_type": "display", + "filter_type": "Connection hostname", + "id": "d4", + "trigger": { + "value": "example.com" + } + } + ] + } + ] + } + }, + "data_stream": { + "dataset": "darktrace.model_breach_alert", + "namespace": "ep", + "type": "logs" + }, + "ecs": { + "version": "8.4.0" + }, + "elastic_agent": { + "id": "95d2bc73-8bc8-47d9-b36e-a21b58255eec", + "snapshot": false, + "version": "8.2.1" + }, + "event": { + "agent_id_status": "verified", + "category": [ + "network" + ], + "created": "2022-07-11T13:04:19.000Z", + "dataset": "darktrace.model_breach_alert", + "ingested": "2022-09-30T11:39:13Z", + "kind": "event", + "original": "{\"commentCount\":0,\"pbid\":1,\"time\":1657544648000,\"creationTime\":1657544659000,\"aianalystData\":[{\"uuid\":\"1234abcd-1234-1234-1234-123456abcdef\",\"related\":[1],\"summariser\":\"BeaconSummary\"}],\"model\":{\"name\":\"Compromise::Beaconing Activity To External Rare\",\"pid\":156,\"phid\":1072,\"uuid\":\"1234abcd-1234-1234-1234-123456abcdef\",\"logic\":{\"data\":[{\"cid\":2026,\"weight\":1},{\"cid\":2024,\"weight\":1},{\"cid\":2025,\"weight\":-100}],\"targetScore\":1,\"type\":\"weightedComponentList\",\"version\":1},\"throttle\":10800,\"sharedEndpoints\":false,\"actions\":{\"alert\":true,\"antigena\":{},\"breach\":true,\"model\":true,\"setPriority\":false,\"setTag\":false,\"setType\":false},\"tags\":[\"AP: C2 Comms\"],\"interval\":10800,\"delay\":0,\"sequenced\":false,\"active\":true,\"modified\":\"2022-07-11 11:47:37\",\"activeTimes\":{\"devices\":{},\"tags\":{},\"type\":\"exclusions\",\"version\":2},\"autoUpdatable\":true,\"autoUpdate\":true,\"autoSuppress\":true,\"description\":\"A device has been repeatedly connecting to a rare external location with a beacon score. A beacon score is added when Darktrace identifies that a device is regularly communicating with an endpoint, for example, if a device connects to a rare external endpoint every 12 minutes this would get a beacon score. This model is designed to identify beaconing at a lower threshold and be protocol agnostic.\\\\n\\\\nAction: Review the external domains and IPs being connected to to see if they are legitimate and would be expected for business purposes.\",\"behaviour\":\"incdec1\",\"created\":{\"by\":\"System\"},\"edited\":{\"by\":\"System\"},\"version\":23,\"priority\":2,\"category\":\"Informational\",\"compliance\":false},\"triggeredComponents\":[{\"time\":1657544648000,\"cbid\":1,\"cid\":2026,\"chid\":2113,\"size\":11,\"threshold\":10,\"interval\":3600,\"logic\":{\"data\":{\"left\":{\"left\":\"A\",\"operator\":\"AND\",\"right\":{\"left\":\"AA\",\"operator\":\"AND\",\"right\":{\"left\":\"AC\",\"operator\":\"AND\",\"right\":{\"left\":\"AD\",\"operator\":\"AND\",\"right\":{\"left\":\"AF\",\"operator\":\"AND\",\"right\":{\"left\":\"AG\",\"operator\":\"AND\",\"right\":{\"left\":\"AH\",\"operator\":\"AND\",\"right\":{\"left\":\"B\",\"operator\":\"AND\",\"right\":{\"left\":\"C\",\"operator\":\"AND\",\"right\":{\"left\":\"D\",\"operator\":\"AND\",\"right\":{\"left\":\"E\",\"operator\":\"AND\",\"right\":{\"left\":\"H\",\"operator\":\"AND\",\"right\":{\"left\":\"I\",\"operator\":\"AND\",\"right\":{\"left\":\"J\",\"operator\":\"AND\",\"right\":{\"left\":\"K\",\"operator\":\"AND\",\"right\":{\"left\":\"L\",\"operator\":\"AND\",\"right\":{\"left\":\"M\",\"operator\":\"AND\",\"right\":{\"left\":\"N\",\"operator\":\"AND\",\"right\":{\"left\":\"O\",\"operator\":\"AND\",\"right\":{\"left\":\"P\",\"operator\":\"AND\",\"right\":{\"left\":\"S\",\"operator\":\"AND\",\"right\":{\"left\":\"U\",\"operator\":\"AND\",\"right\":{\"left\":\"V\",\"operator\":\"AND\",\"right\":{\"left\":\"X\",\"operator\":\"AND\",\"right\":{\"left\":\"Y\",\"operator\":\"AND\",\"right\":\"Z\"}}}}}}}}}}}}}}}}}}}}}}}}},\"operator\":\"OR\",\"right\":{\"left\":\"A\",\"operator\":\"AND\",\"right\":{\"left\":\"AA\",\"operator\":\"AND\",\"right\":{\"left\":\"AB\",\"operator\":\"AND\",\"right\":{\"left\":\"AE\",\"operator\":\"AND\",\"right\":{\"left\":\"AF\",\"operator\":\"AND\",\"right\":{\"left\":\"AG\",\"operator\":\"AND\",\"right\":{\"left\":\"AH\",\"operator\":\"AND\",\"right\":{\"left\":\"C\",\"operator\":\"AND\",\"right\":{\"left\":\"D\",\"operator\":\"AND\",\"right\":{\"left\":\"E\",\"operator\":\"AND\",\"right\":{\"left\":\"H\",\"operator\":\"AND\",\"right\":{\"left\":\"I\",\"operator\":\"AND\",\"right\":{\"left\":\"J\",\"operator\":\"AND\",\"right\":{\"left\":\"K\",\"operator\":\"AND\",\"right\":{\"left\":\"L\",\"operator\":\"AND\",\"right\":{\"left\":\"M\",\"operator\":\"AND\",\"right\":{\"left\":\"N\",\"operator\":\"AND\",\"right\":{\"left\":\"O\",\"operator\":\"AND\",\"right\":{\"left\":\"P\",\"operator\":\"AND\",\"right\":{\"left\":\"S\",\"operator\":\"AND\",\"right\":{\"left\":\"U\",\"operator\":\"AND\",\"right\":{\"left\":\"V\",\"operator\":\"AND\",\"right\":{\"left\":\"X\",\"operator\":\"AND\",\"right\":{\"left\":\"Y\",\"operator\":\"AND\",\"right\":\"Z\"}}}}}}}}}}}}}}}}}}}}}}}}},\"version\":\"v0.1\"},\"metric\":{\"mlid\":1,\"name\":\"externalconnections\",\"label\":\"External Connections\"},\"triggeredFilters\":[{\"cfid\":23426,\"id\":\"A\",\"filterType\":\"Beaconing score\",\"arguments\":{\"value\":60},\"comparatorType\":\"\u003e\",\"trigger\":{\"value\":\"100\"}},{\"cfid\":23427,\"id\":\"AA\",\"filterType\":\"Individual size up\",\"arguments\":{\"value\":0},\"comparatorType\":\"\u003e\",\"trigger\":{\"value\":\"4382\"}},{\"cfid\":23428,\"id\":\"AB\",\"filterType\":\"Rare domain\",\"arguments\":{\"value\":95},\"comparatorType\":\"\u003e\",\"trigger\":{\"value\":\"100\"}},{\"cfid\":23430,\"id\":\"AD\",\"filterType\":\"Age of destination\",\"arguments\":{\"value\":1209600},\"comparatorType\":\"\u003c\",\"trigger\":{\"value\":\"558\"}},{\"cfid\":23431,\"id\":\"AE\",\"filterType\":\"Age of external hostname\",\"arguments\":{\"value\":1209600},\"comparatorType\":\"\u003c\",\"trigger\":{\"value\":\"558\"}},{\"cfid\":23432,\"id\":\"AF\",\"filterType\":\"Connection hostname\",\"arguments\":{\"value\":\"examples\"},\"comparatorType\":\"does not match regular expression\",\"trigger\":{\"value\":\"example.com\"}},{\"cfid\":23433,\"id\":\"AG\",\"filterType\":\"ASN\",\"arguments\":{\"value\":\"examples\"},\"comparatorType\":\"does not match regular expression\",\"trigger\":{\"value\":\"AS12345 LOCAL-02\"}},{\"cfid\":23434,\"id\":\"AH\",\"filterType\":\"JA3 hash\",\"arguments\":{\"value\":\"5d41402abc4b2a76b9719d911017c592\"},\"comparatorType\":\"does not match\",\"trigger\":{\"value\":\"5d41402abc4b2a76b9719d911017c592\"}},{\"cfid\":23435,\"id\":\"B\",\"filterType\":\"Rare external IP\",\"arguments\":{\"value\":95},\"comparatorType\":\"\u003e\",\"trigger\":{\"value\":\"100\"}},{\"cfid\":23436,\"id\":\"C\",\"filterType\":\"Application protocol\",\"arguments\":{\"value\":\"1003\"},\"comparatorType\":\"is not\",\"trigger\":{\"value\":\"1004\"}},{\"cfid\":23437,\"id\":\"D\",\"filterType\":\"Destination port\",\"arguments\":{\"value\":53},\"comparatorType\":\"!=\",\"trigger\":{\"value\":\"443\"}},{\"cfid\":23438,\"id\":\"E\",\"filterType\":\"Direction\",\"arguments\":{\"value\":\"out\"},\"comparatorType\":\"is\",\"trigger\":{\"value\":\"out\"}},{\"cfid\":23439,\"id\":\"H\",\"filterType\":\"Destination port\",\"arguments\":{\"value\":137},\"comparatorType\":\"!=\",\"trigger\":{\"value\":\"443\"}},{\"cfid\":23440,\"id\":\"I\",\"filterType\":\"Destination port\",\"arguments\":{\"value\":161},\"comparatorType\":\"!=\",\"trigger\":{\"value\":\"443\"}},{\"cfid\":23441,\"id\":\"J\",\"filterType\":\"Protocol\",\"arguments\":{\"value\":\"6\"},\"comparatorType\":\"is\",\"trigger\":{\"value\":\"6\"}},{\"cfid\":23442,\"id\":\"K\",\"filterType\":\"ASN\",\"arguments\":{\"value\":\"Company\"},\"comparatorType\":\"does not contain\",\"trigger\":{\"value\":\"AS12345 LOCAL-02\"}},{\"cfid\":23443,\"id\":\"L\",\"filterType\":\"ASN\",\"arguments\":{\"value\":\"Company\"},\"comparatorType\":\"does not contain\",\"trigger\":{\"value\":\"AS12345 LOCAL-02\"}},{\"cfid\":23444,\"id\":\"M\",\"filterType\":\"Internal source device type\",\"arguments\":{\"value\":\"13\"},\"comparatorType\":\"is not\",\"trigger\":{\"value\":\"6\"}},{\"cfid\":23445,\"id\":\"N\",\"filterType\":\"Internal source device type\",\"arguments\":{\"value\":\"5\"},\"comparatorType\":\"is not\",\"trigger\":{\"value\":\"6\"}},{\"cfid\":23446,\"id\":\"O\",\"filterType\":\"Internal source device type\",\"arguments\":{\"value\":\"9\"},\"comparatorType\":\"is not\",\"trigger\":{\"value\":\"6\"}},{\"cfid\":23447,\"id\":\"P\",\"filterType\":\"Internal source device type\",\"arguments\":{\"value\":\"12\"},\"comparatorType\":\"is not\",\"trigger\":{\"value\":\"6\"}},{\"cfid\":23448,\"id\":\"S\",\"filterType\":\"Internal source device type\",\"arguments\":{\"value\":\"30\"},\"comparatorType\":\"is not\",\"trigger\":{\"value\":\"6\"}},{\"cfid\":23449,\"id\":\"U\",\"filterType\":\"Internal source device type\",\"arguments\":{\"value\":\"4\"},\"comparatorType\":\"is not\",\"trigger\":{\"value\":\"6\"}},{\"cfid\":23450,\"id\":\"V\",\"filterType\":\"Internal source device type\",\"arguments\":{\"value\":\"3\"},\"comparatorType\":\"is not\",\"trigger\":{\"value\":\"6\"}},{\"cfid\":23451,\"id\":\"X\",\"filterType\":\"Trusted hostname\",\"arguments\":{\"value\":\"false\"},\"comparatorType\":\"is\",\"trigger\":{\"value\":\"false\"}},{\"cfid\":23452,\"id\":\"Y\",\"filterType\":\"Tagged internal source\",\"arguments\":{\"value\":26},\"comparatorType\":\"does not have tag\",\"trigger\":{\"value\":\"26\",\"tag\":{\"tid\":26,\"expiry\":0,\"thid\":26,\"name\":\"No Device Tracking\",\"restricted\":false,\"data\":{\"auto\":false,\"color\":5,\"description\":\"\",\"visibility\":\"Public\"},\"isReferenced\":true}}},{\"cfid\":23453,\"id\":\"Z\",\"filterType\":\"Individual size down\",\"arguments\":{\"value\":0},\"comparatorType\":\"\u003e\",\"trigger\":{\"value\":\"5862\"}},{\"cfid\":23454,\"id\":\"d1\",\"filterType\":\"JA3 hash\",\"arguments\":{},\"comparatorType\":\"display\",\"trigger\":{\"value\":\"5d41402abc4b2a76b9719d911017c592\"}},{\"cfid\":23455,\"id\":\"d2\",\"filterType\":\"ASN\",\"arguments\":{},\"comparatorType\":\"display\",\"trigger\":{\"value\":\"AS12345 LOCAL-02\"}},{\"cfid\":23456,\"id\":\"d3\",\"filterType\":\"Destination IP\",\"arguments\":{},\"comparatorType\":\"display\",\"trigger\":{\"value\":\"81.2.69.192\"}},{\"cfid\":23457,\"id\":\"d4\",\"filterType\":\"Connection hostname\",\"arguments\":{},\"comparatorType\":\"display\",\"trigger\":{\"value\":\"example.com\"}}]}],\"score\":0.674,\"device\":{\"did\":3,\"ip\":\"81.2.69.142\",\"sid\":1,\"firstSeen\":1657544089000,\"lastSeen\":1657544418000,\"typename\":\"desktop\",\"typelabel\":\"Desktop\"}}", + "risk_score": 0.674, + "risk_score_norm": 67.4, + "severity": 2, + "start": [ + "2022-07-11T13:04:08.000Z" + ], + "type": [ + "info", + "connection" + ] + }, + "host": { + "id": "3", + "ip": [ + "81.2.69.142" + ], + "type": "desktop" + }, + "input": { + "type": "udp" + }, + "log": { + "source": { + "address": "192.168.128.5:60206" + }, + "syslog": { + "facility": { + "code": 20, + "name": "local4" + }, + "hostname": "example.cloud.darktrace.com", + "priority": 165, + "severity": { + "code": 5, + "name": "Notice" + }, + "version": "1" + } + }, + "related": { + "ip": [ + "81.2.69.142" + ], + "user": [ + "System" + ] + }, + "rule": { + "author": "System", + "category": "Informational", + "description": "A device has been repeatedly connecting to a rare external location with a beacon score. A beacon score is added when Darktrace identifies that a device is regularly communicating with an endpoint, for example, if a device connects to a rare external endpoint every 12 minutes this would get a beacon score. This model is designed to identify beaconing at a lower threshold and be protocol agnostic.\\n\\nAction: Review the external domains and IPs being connected to to see if they are legitimate and would be expected for business purposes.", + "name": "Compromise::Beaconing Activity To External Rare", + "ruleset": [ + "AP: C2 Comms" + ], + "uuid": "1234abcd-1234-1234-1234-123456abcdef", + "version": "23" + }, + "tags": [ + "preserve_original_event", + "preserve_duplicate_custom_fields", + "forwarded", + "darktrace-model_breach_alert" + ] +} +``` + +**Exported fields** + +| Field | Description | Type | +|---|---|---| +| @timestamp | Event timestamp. | date | +| cloud.account.id | The cloud account or organization id used to identify different entities in a multi-tenant environment. Examples: AWS account id, Google Cloud ORG Id, or other unique identifier. | keyword | +| cloud.availability_zone | Availability zone in which this host is running. | keyword | +| cloud.image.id | Image ID for the cloud instance. | keyword | +| cloud.instance.id | Instance ID of the host machine. | keyword | +| cloud.instance.name | Instance name of the host machine. | keyword | +| cloud.machine.type | Machine type of the host machine. | keyword | +| cloud.project.id | Name of the project in Google Cloud. | keyword | +| cloud.provider | Name of the cloud provider. Example values are aws, azure, gcp, or digitalocean. | keyword | +| cloud.region | Region in which this host is running. | keyword | +| container.id | Unique container id. | keyword | +| container.image.name | Name of the image the container was built on. | keyword | +| container.labels | Image labels. | object | +| container.name | Container name. | keyword | +| darktrace.model_breach_alert.aianalyst_data.related | | long | +| darktrace.model_breach_alert.aianalyst_data.summariser | | keyword | +| darktrace.model_breach_alert.aianalyst_data.uuid | | keyword | +| darktrace.model_breach_alert.breach_url.domain | | keyword | +| darktrace.model_breach_alert.breach_url.extension | | keyword | +| darktrace.model_breach_alert.breach_url.fragment | | keyword | +| darktrace.model_breach_alert.breach_url.full | | keyword | +| darktrace.model_breach_alert.breach_url.original | | keyword | +| darktrace.model_breach_alert.breach_url.password | | keyword | +| darktrace.model_breach_alert.breach_url.path | | keyword | +| darktrace.model_breach_alert.breach_url.port | | long | +| darktrace.model_breach_alert.breach_url.query | | keyword | +| darktrace.model_breach_alert.breach_url.scheme | | keyword | +| darktrace.model_breach_alert.breach_url.username | | keyword | +| darktrace.model_breach_alert.comment.count | The number of comments made against this breach. | long | +| darktrace.model_breach_alert.creation_time | The timestamp that the record of the breach was created. This is distinct from the “time” field. | date | +| darktrace.model_breach_alert.device.credentials | | keyword | +| darktrace.model_breach_alert.device.did | The “device id”, a unique identifier. | long | +| darktrace.model_breach_alert.device.first_seen | The first time the device was seen on the network. | date | +| darktrace.model_breach_alert.device.hostname | The current device hostname. | keyword | +| darktrace.model_breach_alert.device.ip | The current IP associated with the device. | keyword | +| darktrace.model_breach_alert.device.ip6 | Current IPv6 address of this device if applicable, otherwise undefined. | keyword | +| darktrace.model_breach_alert.device.ips.ip | A historic IP associated with the device. | keyword | +| darktrace.model_breach_alert.device.ips.sid | The subnet id for the subnet the IP belongs to. | long | +| darktrace.model_breach_alert.device.ips.time | The time the IP was last seen associated with that device in readable format. | date | +| darktrace.model_breach_alert.device.ips.timems | The time the IP was last seen associated with that device in epoch time. | date | +| darktrace.model_breach_alert.device.last_seen | The last time the device was seen on the network. | date | +| darktrace.model_breach_alert.device.mac_address | The current MAC address associated with the device. | keyword | +| darktrace.model_breach_alert.device.sid | The subnet id for the subnet the device is currently located in. | long | +| darktrace.model_breach_alert.device.tags.data.auto | | boolean | +| darktrace.model_breach_alert.device.tags.data.color | | long | +| darktrace.model_breach_alert.device.tags.data.description | | keyword | +| darktrace.model_breach_alert.device.tags.data.visibility | | keyword | +| darktrace.model_breach_alert.device.tags.expiry | | long | +| darktrace.model_breach_alert.device.tags.is_referenced | | boolean | +| darktrace.model_breach_alert.device.tags.name | | keyword | +| darktrace.model_breach_alert.device.tags.restricted | | boolean | +| darktrace.model_breach_alert.device.tags.thid | | long | +| darktrace.model_breach_alert.device.tags.tid | | long | +| darktrace.model_breach_alert.device.type_label | The device type in readable format. | keyword | +| darktrace.model_breach_alert.device.type_name | The device type in system format. | keyword | +| darktrace.model_breach_alert.device.vendor | The vendor of the device network card as derived by Darktrace from the MAC address. | keyword | +| darktrace.model_breach_alert.device_score | | double | +| darktrace.model_breach_alert.is_acknowledged | | boolean | +| darktrace.model_breach_alert.mitre_techniques.id | | keyword | +| darktrace.model_breach_alert.mitre_techniques.name | | keyword | +| darktrace.model_breach_alert.model.actions.antigena.action | The action to be performed. | keyword | +| darktrace.model_breach_alert.model.actions.antigena.duration | The duration in seconds that the antigena action should last for. | long | +| darktrace.model_breach_alert.model.actions.antigena.is_confirm_by_human_operator | Whether the action must be confirmed by a human operator, regardless of the global setting for Human Confirmation mode. | boolean | +| darktrace.model_breach_alert.model.actions.antigena.threshold | The breach score threshold (out of 100) over which antigena will take an action. | long | +| darktrace.model_breach_alert.model.actions.is_alerting | If true, an alert turned on will be pushed out to external systems if conditions for such alerting are met. | boolean | +| darktrace.model_breach_alert.model.actions.is_breach | If true, an alert turned on will be pushed out to external systems if conditions for such alerting are met. | boolean | +| darktrace.model_breach_alert.model.actions.is_priority_set | If the priority is to be changed on breach, the numeric value it should become. If no priority change action, a false boolean. | boolean | +| darktrace.model_breach_alert.model.actions.is_tag_set | If a tag is to be applied on model breach, a single number or array of the system ID for the tag(s) to be applied. If no tag action, a false boolean. | boolean | +| darktrace.model_breach_alert.model.actions.is_type_set | If a change device type action is to be applied on model breach, the numeric system ID for the label to be applied. If no change device type action is applied to the model, a false boolean. | boolean | +| darktrace.model_breach_alert.model.actions.model | If true, creates an event in the device’s event log without creating an alert/ model breach in the threat tray. | boolean | +| darktrace.model_breach_alert.model.active_times.devices | The device ids for devices on the list. | flattened | +| darktrace.model_breach_alert.model.active_times.tags | A system field. | flattened | +| darktrace.model_breach_alert.model.active_times.type | The type of list: “restrictions” indicates a blacklist, “exclusions” a whitelist. | keyword | +| darktrace.model_breach_alert.model.active_times.version | A system field. | long | +| darktrace.model_breach_alert.model.behaviour | The score modulation function as set in the model editor. | keyword | +| darktrace.model_breach_alert.model.category | The behavior category of the model that was breached. | keyword | +| darktrace.model_breach_alert.model.created.by | Username that created the model. | keyword | +| darktrace.model_breach_alert.model.defeats.arguments.value | The value(s) that must match for the defeat to take effect. | keyword | +| darktrace.model_breach_alert.model.defeats.comparator | The comparator that the value is compared against the create the defeat. | keyword | +| darktrace.model_breach_alert.model.defeats.filtertype | The filter the defeat is made from. | keyword | +| darktrace.model_breach_alert.model.defeats.id | A unique ID for the defeat. | long | +| darktrace.model_breach_alert.model.delay | Minimum delay in seconds after a positive-scoring component has fired before the overall model score is calculated. Only applicable in target score models. | long | +| darktrace.model_breach_alert.model.description | The optional description of the model. | keyword | +| darktrace.model_breach_alert.model.edited.by | Username that last edited the model. | keyword | +| darktrace.model_breach_alert.model.edited.userid | | long | +| darktrace.model_breach_alert.model.in_compliance_behavior_category | Whether the model is in the compliance behavior category. | boolean | +| darktrace.model_breach_alert.model.interval | Where a model contains multiple components, this interval represents the time window in seconds in which all the components should fire for this model to be breached. | long | +| darktrace.model_breach_alert.model.is_active | Whether the model is enabled or disabled. | boolean | +| darktrace.model_breach_alert.model.is_auto_suppress | Whether the model will automatically be suppressed in the case of over-breaching. | boolean | +| darktrace.model_breach_alert.model.is_auto_updatable | Whether the model is suitable for auto update. | boolean | +| darktrace.model_breach_alert.model.is_auto_update | Whether the model is enabled for auto update. | boolean | +| darktrace.model_breach_alert.model.is_sequenced | Whether the components are required to fire in the specified order for the model breach to occur. | boolean | +| darktrace.model_breach_alert.model.is_shared_endpoints | For models that contain multiple components that reference an endpoint, this value indicates whether all endpoints should be identical for the model to fire. | boolean | +| darktrace.model_breach_alert.model.logic.data_component_list | This will be a list of component ID numbers. | long | +| darktrace.model_breach_alert.model.logic.data_weighted_component_list.cid | | long | +| darktrace.model_breach_alert.model.logic.data_weighted_component_list.weight | | long | +| darktrace.model_breach_alert.model.logic.target_score | | long | +| darktrace.model_breach_alert.model.logic.type | The type of model. | keyword | +| darktrace.model_breach_alert.model.logic.version | A number representing the version of model logic. | long | +| darktrace.model_breach_alert.model.modified | Timestamp at which the model was last modified, in a readable format. | date | +| darktrace.model_breach_alert.model.name | Name of the model that was breached. | keyword | +| darktrace.model_breach_alert.model.phid | The model “policy history” id. Increments when the model is modified. | long | +| darktrace.model_breach_alert.model.pid | The “policy id” of the model that was breached. | long | +| darktrace.model_breach_alert.model.priority | The model’s priority affects the strength with which it breaches (0-5 scale). | long | +| darktrace.model_breach_alert.model.tags | A list of tags that have been applied to this model in the Threat Visualizer model editor. | keyword | +| darktrace.model_breach_alert.model.throttle | For an individual device, this is the value in seconds for which this model will not fire again. | long | +| darktrace.model_breach_alert.model.uuid | A unique ID that is generated on creation of the model. | keyword | +| darktrace.model_breach_alert.model.version | The version of the model. Increments on each edit. | long | +| darktrace.model_breach_alert.pb_score | The model breach score, represented by a value between 0 and 1. | double | +| darktrace.model_breach_alert.pbid | The “policy breach ID” of the model breach. | long | +| darktrace.model_breach_alert.score | The model breach score, represented by a value between 0 and 1. | double | +| darktrace.model_breach_alert.time | The timestamp when the record was created in epoch time. | date | +| darktrace.model_breach_alert.triggered_components.cbid | The “component breach id”. A unique identifier for the component breach. | long | +| darktrace.model_breach_alert.triggered_components.chid | The “component history id”. Increments when the component is edited. | long | +| darktrace.model_breach_alert.triggered_components.cid | The “component id”. A unique identifier. | long | +| darktrace.model_breach_alert.triggered_components.interval | The timeframe in seconds within which the threshold must be satisfied. | long | +| darktrace.model_breach_alert.triggered_components.logic.data | It representing the logical relationship between component filters. Each filter is given an alphabetical reference and the contents of this field describe the relationship between those filters. | text | +| darktrace.model_breach_alert.triggered_components.logic.version | The version of the component logic. | keyword | +| darktrace.model_breach_alert.triggered_components.metric.label | The metric which data is returned for in readable format. | keyword | +| darktrace.model_breach_alert.triggered_components.metric.mlid | The “metric logic” id - unique identifier. | long | +| darktrace.model_breach_alert.triggered_components.metric.name | The metric which data is returned for in system format. | keyword | +| darktrace.model_breach_alert.triggered_components.size | The size of the value that was compared in the component. | long | +| darktrace.model_breach_alert.triggered_components.threshold | The threshold value that the size must exceed for the component to breach. | long | +| darktrace.model_breach_alert.triggered_components.time | A timestamp in Epoch time at which the components were triggered. | date | +| darktrace.model_breach_alert.triggered_components.triggered_filters.arguments.value | The value the filtertype should be compared against (using the specified comparator) to create the filter. | keyword | +| darktrace.model_breach_alert.triggered_components.triggered_filters.cfid | The ‘component filter id’. A unique identifier for the filter as part of a the component. | long | +| darktrace.model_breach_alert.triggered_components.triggered_filters.comparator_type | The comparator. A full list of comparators available for each filtertype can be found on the /filtertypes endpoint. | keyword | +| darktrace.model_breach_alert.triggered_components.triggered_filters.filter_type | The filtertype that is used in the filter. A full list of filtertypes can be found on the /filtertypes endpoint. | keyword | +| darktrace.model_breach_alert.triggered_components.triggered_filters.id | A filter that is used in the component logic. All filters are given alphabetical identifiers. Display filters - those that appear in the breach notification - can be identified by a lowercase ‘d’ and a numeral. | keyword | +| darktrace.model_breach_alert.triggered_components.triggered_filters.trigger.tag.data.auto | | boolean | +| darktrace.model_breach_alert.triggered_components.triggered_filters.trigger.tag.data.color | | long | +| darktrace.model_breach_alert.triggered_components.triggered_filters.trigger.tag.data.description | | keyword | +| darktrace.model_breach_alert.triggered_components.triggered_filters.trigger.tag.data.visibility | | keyword | +| darktrace.model_breach_alert.triggered_components.triggered_filters.trigger.tag.expiry | nan | long | +| darktrace.model_breach_alert.triggered_components.triggered_filters.trigger.tag.isReferenced | nan | boolean | +| darktrace.model_breach_alert.triggered_components.triggered_filters.trigger.tag.name | nan | keyword | +| darktrace.model_breach_alert.triggered_components.triggered_filters.trigger.tag.restricted | nan | boolean | +| darktrace.model_breach_alert.triggered_components.triggered_filters.trigger.tag.thid | nan | long | +| darktrace.model_breach_alert.triggered_components.triggered_filters.trigger.tag.tid | nan | long | +| darktrace.model_breach_alert.triggered_components.triggered_filters.trigger.value | The actual value that triggered the filter. | keyword | +| data_stream.dataset | Data stream dataset. | constant_keyword | +| data_stream.namespace | Data stream namespace. | constant_keyword | +| data_stream.type | Data stream type. | constant_keyword | +| ecs.version | ECS version this event conforms to. `ecs.version` is a required field and must exist in all events. When querying across multiple indices -- which may conform to slightly different ECS versions -- this field lets integrations adjust to the schema version of the events. | keyword | +| event.action | The action captured by the event. This describes the information in the event. It is more specific than `event.category`. Examples are `group-add`, `process-started`, `file-created`. The value is normally defined by the implementer. | keyword | +| event.category | This is one of four ECS Categorization Fields, and indicates the second level in the ECS category hierarchy. `event.category` represents the "big buckets" of ECS categories. For example, filtering on `event.category:process` yields all events relating to process activity. This field is closely related to `event.type`, which is used as a subcategory. This field is an array. This will allow proper categorization of some events that fall in multiple categories. | keyword | +| event.created | event.created contains the date/time when the event was first read by an agent, or by your pipeline. This field is distinct from @timestamp in that @timestamp typically contain the time extracted from the original event. In most situations, these two timestamps will be slightly different. The difference can be used to calculate the delay between your source generating an event, and the time when your agent first processed it. This can be used to monitor your agent's or pipeline's ability to keep up with your event source. In case the two timestamps are identical, @timestamp should be used. | date | +| event.dataset | Event dataset. | constant_keyword | +| event.kind | This is one of four ECS Categorization Fields, and indicates the highest level in the ECS category hierarchy. `event.kind` gives high-level information about what type of information the event contains, without being specific to the contents of the event. For example, values of this field distinguish alert events from metric events. The value of this field can be used to inform how these kinds of events should be handled. They may warrant different retention, different access control, it may also help understand whether the data coming in at a regular interval or not. | keyword | +| event.module | Event module. | constant_keyword | +| event.original | Raw text message of entire event. Used to demonstrate log integrity or where the full log message (before splitting it up in multiple parts) may be required, e.g. for reindex. This field is not indexed and doc_values are disabled. It cannot be searched, but it can be retrieved from `_source`. If users wish to override this and index this field, please see `Field data types` in the `Elasticsearch Reference`. | keyword | +| event.risk_score | Risk score or priority of the event (e.g. security solutions). Use your system's original value here. | float | +| event.risk_score_norm | Normalized risk score or priority of the event, on a scale of 0 to 100. This is mainly useful if you use more than one system that assigns risk scores, and you want to see a normalized value across all systems. | float | +| event.severity | The numeric severity of the event according to your event source. What the different severity values mean can be different between sources and use cases. It's up to the implementer to make sure severities are consistent across events from the same source. The Syslog severity belongs in `log.syslog.severity.code`. `event.severity` is meant to represent the severity according to the event source (e.g. firewall, IDS). If the event source does not publish its own severity, you may optionally copy the `log.syslog.severity.code` to `event.severity`. | long | +| event.start | event.start contains the date when the event started or when the activity was first observed. | date | +| event.type | This is one of four ECS Categorization Fields, and indicates the third level in the ECS category hierarchy. `event.type` represents a categorization "sub-bucket" that, when used along with the `event.category` field values, enables filtering events down to a level appropriate for single visualization. This field is an array. This will allow proper categorization of some events that fall in multiple event types. | keyword | +| event.url | URL linking to an external system to continue investigation of this event. This URL links to another system where in-depth investigation of the specific occurrence of this event can take place. Alert events, indicated by `event.kind:alert`, are a common use case for this field. | keyword | +| host.architecture | Operating system architecture. | keyword | +| host.containerized | If the host is a container. | boolean | +| host.domain | Name of the domain of which the host is a member. For example, on Windows this could be the host's Active Directory domain or NetBIOS domain name. For Linux this could be the domain of the host's LDAP provider. | keyword | +| host.hostname | Hostname of the host. It normally contains what the `hostname` command returns on the host machine. | keyword | +| host.id | Unique host id. As hostname is not always unique, use values that are meaningful in your environment. Example: The current usage of `beat.name`. | keyword | +| host.ip | Host ip addresses. | ip | +| host.mac | Host mac addresses. | keyword | +| host.name | Name of the host. It can contain what `hostname` returns on Unix systems, the fully qualified domain name, or a name specified by the user. The sender decides which value to use. | keyword | +| host.os.build | OS build information. | keyword | +| host.os.codename | OS codename, if any. | keyword | +| host.os.family | OS family (such as redhat, debian, freebsd, windows). | keyword | +| host.os.kernel | Operating system kernel version as a raw string. | keyword | +| host.os.name | Operating system name, without the version. | keyword | +| host.os.name.text | Multi-field of `host.os.name`. | text | +| host.os.platform | Operating system platform (such centos, ubuntu, windows). | keyword | +| host.os.version | Operating system version as a raw string. | keyword | +| host.type | Type of host. For Cloud providers this can be the machine type like `t2.medium`. If vm, this could be the container, for example, or other information meaningful in your environment. | keyword | +| input.type | Input type | keyword | +| log.offset | Log offset | long | +| log.source.address | Source address from which the log event was read / sent from. | keyword | +| log.syslog.appname | The device or application that originated the Syslog message, if available. | keyword | +| log.syslog.facility.code | The Syslog numeric facility of the log event, if available. According to RFCs 5424 and 3164, this value should be an integer between 0 and 23. | long | +| log.syslog.facility.name | The Syslog text-based facility of the log event, if available. | keyword | +| log.syslog.hostname | The hostname, FQDN, or IP of the machine that originally sent the Syslog message. This is sourced from the hostname field of the syslog header. Depending on the environment, this value may be different from the host that handled the event, especially if the host handling the events is acting as a collector. | keyword | +| log.syslog.priority | Syslog numeric priority of the event, if available. According to RFCs 5424 and 3164, the priority is 8 \* facility + severity. This number is therefore expected to contain a value between 0 and 191. | long | +| log.syslog.severity.code | The Syslog numeric severity of the log event, if available. If the event source publishing via Syslog provides a different numeric severity value (e.g. firewall, IDS), your source's numeric severity should go to `event.severity`. If the event source does not specify a distinct severity, you can optionally copy the Syslog severity to `event.severity`. | long | +| log.syslog.severity.name | The Syslog numeric severity of the log event, if available. If the event source publishing via Syslog provides a different severity value (e.g. firewall, IDS), your source's text severity should go to `log.level`. If the event source does not specify a distinct severity, you can optionally copy the Syslog severity to `log.level`. | keyword | +| log.syslog.version | The version of the Syslog protocol specification. Only applicable for RFC 5424 messages. | keyword | +| related.hosts | All hostnames or other host identifiers seen on your event. Example identifiers include FQDNs, domain names, workstation names, or aliases. | keyword | +| related.ip | All of the IPs seen on your event. | ip | +| related.user | All the user names or other user identifiers seen on the event. | keyword | +| rule.author | Name, organization, or pseudonym of the author or authors who created the rule used to generate this event. | keyword | +| rule.category | A categorization value keyword used by the entity using the rule for detection of this event. | keyword | +| rule.description | The description of the rule generating the event. | keyword | +| rule.name | The name of the rule or signature generating the event. | keyword | +| rule.ruleset | Name of the ruleset, policy, group, or parent category in which the rule used to generate this event is a member. | keyword | +| rule.uuid | A rule ID that is unique within the scope of a set or group of agents, observers, or other entities using the rule for detection of this event. | keyword | +| rule.version | The version / revision of the rule being used for analysis. | keyword | +| tags | List of keywords used to tag each event. | keyword | +| threat.technique.id | The id of technique used by this threat. You can use a MITRE ATT&CK® technique, for example. (ex. https://attack.mitre.org/techniques/T1059/) | keyword | +| threat.technique.name | The name of technique used by this threat. You can use a MITRE ATT&CK® technique, for example. (ex. https://attack.mitre.org/techniques/T1059/) | keyword | +| threat.technique.name.text | Multi-field of `threat.technique.name`. | match_only_text | + + +### system_status_alert + +This is the `system_status_alert` dataset. + +#### Example + +An example event for `system_status_alert` looks as following: + +```json +{ + "@timestamp": "2021-04-18T15:44:11.000Z", + "agent": { + "ephemeral_id": "5b042cea-01fa-47a2-ab0f-ac1f7baa6bd2", + "id": "95d2bc73-8bc8-47d9-b36e-a21b58255eec", + "name": "docker-fleet-agent", + "type": "filebeat", + "version": "8.2.1" + }, + "darktrace": { + "system_status_alert": { + "alert_name": "Advanced Search", + "child_id": 1, + "hostname": "example-vsensor", + "ip_address": "175.16.199.1", + "last_updated": "2021-04-18T15:44:11.000Z", + "last_updated_status": "2021-04-18T15:44:11.000Z", + "message": "There have been no Advanced Search hits for this instance seen since Sun 18 April 2021 13:20:23 (UTC). If this is not expected behaviour, please open a ticket using the following link or get in touch with your Cyber Technology Specialist. https://example.com/test", + "name": "advanced_search", + "priority": 43, + "priority_level": "medium", + "status": "active", + "uuid": "abcdabcd-1234-1234-1234-3abababcdcd3" + } + }, + "data_stream": { + "dataset": "darktrace.system_status_alert", + "namespace": "ep", + "type": "logs" + }, + "ecs": { + "version": "8.4.0" + }, + "elastic_agent": { + "id": "95d2bc73-8bc8-47d9-b36e-a21b58255eec", + "snapshot": false, + "version": "8.2.1" + }, + "event": { + "agent_id_status": "verified", + "dataset": "darktrace.system_status_alert", + "id": "abcdabcd-1234-1234-1234-3abababcdcd3", + "ingested": "2022-09-30T11:41:35Z", + "kind": "alert", + "original": "{\"last_updated\":1618760651,\"uuid\":\"abcdabcd-1234-1234-1234-3abababcdcd3\",\"priority\":43,\"priority_level\":\"medium\",\"hostname\":\"example-vsensor\",\"ip_address\":\"175.16.199.1\",\"message\":\"There have been no Advanced Search hits for this instance seen since Sun 18 April 2021 13:20:23 (UTC). If this is not expected behaviour, please open a ticket using the following link or get in touch with your Cyber Technology Specialist. https://example.com/test\",\"name\":\"advanced_search\",\"acknowledge_timeout\":null,\"alert_name\":\"Advanced Search\",\"child_id\":1,\"last_updated_status\":1618760651,\"status\":\"active\"}", + "reason": "There have been no Advanced Search hits for this instance seen since Sun 18 April 2021 13:20:23 (UTC). If this is not expected behaviour, please open a ticket using the following link or get in touch with your Cyber Technology Specialist. https://example.com/test", + "risk_score": 43, + "risk_score_norm": 43, + "type": [ + "info" + ] + }, + "host": { + "hostname": "example-vsensor", + "ip": "175.16.199.1" + }, + "input": { + "type": "udp" + }, + "log": { + "source": { + "address": "192.168.128.5:36197" + }, + "syslog": { + "facility": { + "code": 20, + "name": "local4" + }, + "hostname": "example.cloud.darktrace.com", + "priority": 165, + "severity": { + "code": 5, + "name": "Notice" + }, + "version": "1" + } + }, + "related": { + "hosts": [ + "example-vsensor" + ], + "ip": [ + "175.16.199.1" + ] + }, + "tags": [ + "preserve_original_event", + "preserve_duplicate_custom_fields", + "forwarded", + "darktrace-system_status_alert" + ] +} +``` + +**Exported fields** + +| Field | Description | Type | +|---|---|---| +| @timestamp | Event timestamp. | date | +| cloud.account.id | The cloud account or organization id used to identify different entities in a multi-tenant environment. Examples: AWS account id, Google Cloud ORG Id, or other unique identifier. | keyword | +| cloud.availability_zone | Availability zone in which this host is running. | keyword | +| cloud.image.id | Image ID for the cloud instance. | keyword | +| cloud.instance.id | Instance ID of the host machine. | keyword | +| cloud.instance.name | Instance name of the host machine. | keyword | +| cloud.machine.type | Machine type of the host machine. | keyword | +| cloud.project.id | Name of the project in Google Cloud. | keyword | +| cloud.provider | Name of the cloud provider. Example values are aws, azure, gcp, or digitalocean. | keyword | +| cloud.region | Region in which this host is running. | keyword | +| container.id | Unique container id. | keyword | +| container.image.name | Name of the image the container was built on. | keyword | +| container.labels | Image labels. | object | +| container.name | Container name. | keyword | +| darktrace.system_status_alert.acknowledge_timeout | When acknowledgement of the alert expires. As alerts are sent externally on creation before acknowledgement is possible, this will be null in almost all cases. | keyword | +| darktrace.system_status_alert.alert_name | A human readable name of the alert type. | keyword | +| darktrace.system_status_alert.child_id | For probes (physical or virtual), the unique ID associated with the probe. | long | +| darktrace.system_status_alert.hostname | The hostname (if known) of the host experiencing the system alert. An exception exists for disconnection notices, where the hostname will be of the master from which the instance has disconnected. | keyword | +| darktrace.system_status_alert.ip_address | The IP of the host experiencing the system alert. An exception exists for disconnection notices, where the IP will be of the master from which the instance has disconnected. | keyword | +| darktrace.system_status_alert.last_updated | A timestamp in epoch time that the system alert itself was updated. | date | +| darktrace.system_status_alert.last_updated_status | A timestamp in epoch time that the status of the system alert was last updated globally. A status update is distinct from a update to the alert itself. | date | +| darktrace.system_status_alert.message | A textual description of the system event that has triggered the alert. | keyword | +| darktrace.system_status_alert.name | A system name of the alert type. | keyword | +| darktrace.system_status_alert.priority | The numeric criticality associated with the alert. | double | +| darktrace.system_status_alert.priority_level | The criticality of the alert. This value is calculated from the priority value: 0 - 40 low, 41 - 60 medium, 61 - 80 high, 81 - 100 critical. | keyword | +| darktrace.system_status_alert.status | The current status of the alert. Active alerts are ongoing, acknowledged events are those acknowledged on the System Status page, resolved alerts are system alerts that are no longer ongoing. Alerts will only be sent when alert enters the “active” or “resolved” state. | keyword | +| darktrace.system_status_alert.url.domain | | keyword | +| darktrace.system_status_alert.url.extension | | keyword | +| darktrace.system_status_alert.url.fragment | | keyword | +| darktrace.system_status_alert.url.full | | keyword | +| darktrace.system_status_alert.url.original | | keyword | +| darktrace.system_status_alert.url.password | | keyword | +| darktrace.system_status_alert.url.path | | keyword | +| darktrace.system_status_alert.url.port | | long | +| darktrace.system_status_alert.url.query | | keyword | +| darktrace.system_status_alert.url.scheme | | keyword | +| darktrace.system_status_alert.url.username | | keyword | +| darktrace.system_status_alert.uuid | A consistent UUID that can be used to navigate to the specific alert in the Threat Visualizer (https://[instance]/sysstatus/[uuid]). Where an alert is reactivated after resolution due to the issue reoccurring, the UUId will remain consistent across alerts. | keyword | +| data_stream.dataset | Data stream dataset. | constant_keyword | +| data_stream.namespace | Data stream namespace. | constant_keyword | +| data_stream.type | Data stream type. | constant_keyword | +| ecs.version | ECS version this event conforms to. `ecs.version` is a required field and must exist in all events. When querying across multiple indices -- which may conform to slightly different ECS versions -- this field lets integrations adjust to the schema version of the events. | keyword | +| event.category | This is one of four ECS Categorization Fields, and indicates the second level in the ECS category hierarchy. `event.category` represents the "big buckets" of ECS categories. For example, filtering on `event.category:process` yields all events relating to process activity. This field is closely related to `event.type`, which is used as a subcategory. This field is an array. This will allow proper categorization of some events that fall in multiple categories. | keyword | +| event.created | event.created contains the date/time when the event was first read by an agent, or by your pipeline. This field is distinct from @timestamp in that @timestamp typically contain the time extracted from the original event. In most situations, these two timestamps will be slightly different. The difference can be used to calculate the delay between your source generating an event, and the time when your agent first processed it. This can be used to monitor your agent's or pipeline's ability to keep up with your event source. In case the two timestamps are identical, @timestamp should be used. | date | +| event.dataset | Event dataset. | constant_keyword | +| event.id | Unique ID to describe the event. | keyword | +| event.kind | This is one of four ECS Categorization Fields, and indicates the highest level in the ECS category hierarchy. `event.kind` gives high-level information about what type of information the event contains, without being specific to the contents of the event. For example, values of this field distinguish alert events from metric events. The value of this field can be used to inform how these kinds of events should be handled. They may warrant different retention, different access control, it may also help understand whether the data coming in at a regular interval or not. | keyword | +| event.module | Event module. | constant_keyword | +| event.original | Raw text message of entire event. Used to demonstrate log integrity or where the full log message (before splitting it up in multiple parts) may be required, e.g. for reindex. This field is not indexed and doc_values are disabled. It cannot be searched, but it can be retrieved from `_source`. If users wish to override this and index this field, please see `Field data types` in the `Elasticsearch Reference`. | keyword | +| event.reason | Reason why this event happened, according to the source. This describes the why of a particular action or outcome captured in the event. Where `event.action` captures the action from the event, `event.reason` describes why that action was taken. For example, a web proxy with an `event.action` which denied the request may also populate `event.reason` with the reason why (e.g. `blocked site`). | keyword | +| event.risk_score | Risk score or priority of the event (e.g. security solutions). Use your system's original value here. | float | +| event.risk_score_norm | Normalized risk score or priority of the event, on a scale of 0 to 100. This is mainly useful if you use more than one system that assigns risk scores, and you want to see a normalized value across all systems. | float | +| event.type | This is one of four ECS Categorization Fields, and indicates the third level in the ECS category hierarchy. `event.type` represents a categorization "sub-bucket" that, when used along with the `event.category` field values, enables filtering events down to a level appropriate for single visualization. This field is an array. This will allow proper categorization of some events that fall in multiple event types. | keyword | +| event.url | URL linking to an external system to continue investigation of this event. This URL links to another system where in-depth investigation of the specific occurrence of this event can take place. Alert events, indicated by `event.kind:alert`, are a common use case for this field. | keyword | +| host.architecture | Operating system architecture. | keyword | +| host.containerized | If the host is a container. | boolean | +| host.domain | Name of the domain of which the host is a member. For example, on Windows this could be the host's Active Directory domain or NetBIOS domain name. For Linux this could be the domain of the host's LDAP provider. | keyword | +| host.hostname | Hostname of the host. It normally contains what the `hostname` command returns on the host machine. | keyword | +| host.id | Unique host id. As hostname is not always unique, use values that are meaningful in your environment. Example: The current usage of `beat.name`. | keyword | +| host.ip | Host ip addresses. | ip | +| host.mac | Host mac addresses. | keyword | +| host.name | Name of the host. It can contain what `hostname` returns on Unix systems, the fully qualified domain name, or a name specified by the user. The sender decides which value to use. | keyword | +| host.os.build | OS build information. | keyword | +| host.os.codename | OS codename, if any. | keyword | +| host.os.family | OS family (such as redhat, debian, freebsd, windows). | keyword | +| host.os.kernel | Operating system kernel version as a raw string. | keyword | +| host.os.name | Operating system name, without the version. | keyword | +| host.os.name.text | Multi-field of `host.os.name`. | text | +| host.os.platform | Operating system platform (such centos, ubuntu, windows). | keyword | +| host.os.version | Operating system version as a raw string. | keyword | +| host.type | Type of host. For Cloud providers this can be the machine type like `t2.medium`. If vm, this could be the container, for example, or other information meaningful in your environment. | keyword | +| input.type | Input type | keyword | +| log.offset | Log offset | long | +| log.source.address | Source address from which the log event was read / sent from. | keyword | +| log.syslog.appname | The device or application that originated the Syslog message, if available. | keyword | +| log.syslog.facility.code | The Syslog numeric facility of the log event, if available. According to RFCs 5424 and 3164, this value should be an integer between 0 and 23. | long | +| log.syslog.facility.name | The Syslog text-based facility of the log event, if available. | keyword | +| log.syslog.hostname | The hostname, FQDN, or IP of the machine that originally sent the Syslog message. This is sourced from the hostname field of the syslog header. Depending on the environment, this value may be different from the host that handled the event, especially if the host handling the events is acting as a collector. | keyword | +| log.syslog.priority | Syslog numeric priority of the event, if available. According to RFCs 5424 and 3164, the priority is 8 \* facility + severity. This number is therefore expected to contain a value between 0 and 191. | long | +| log.syslog.severity.code | The Syslog numeric severity of the log event, if available. If the event source publishing via Syslog provides a different numeric severity value (e.g. firewall, IDS), your source's numeric severity should go to `event.severity`. If the event source does not specify a distinct severity, you can optionally copy the Syslog severity to `event.severity`. | long | +| log.syslog.severity.name | The Syslog numeric severity of the log event, if available. If the event source publishing via Syslog provides a different severity value (e.g. firewall, IDS), your source's text severity should go to `log.level`. If the event source does not specify a distinct severity, you can optionally copy the Syslog severity to `log.level`. | keyword | +| log.syslog.version | The version of the Syslog protocol specification. Only applicable for RFC 5424 messages. | keyword | +| related.hosts | All hostnames or other host identifiers seen on your event. Example identifiers include FQDNs, domain names, workstation names, or aliases. | keyword | +| related.ip | All of the IPs seen on your event. | ip | +| tags | List of keywords used to tag each event. | keyword | diff --git a/packages/darktrace/img/darktrace-logo.svg b/packages/darktrace/img/darktrace-logo.svg new file mode 100644 index 00000000000..dd926f62920 --- /dev/null +++ b/packages/darktrace/img/darktrace-logo.svg @@ -0,0 +1 @@ + \ No newline at end of file diff --git a/packages/darktrace/img/darktrace-screenshot.png b/packages/darktrace/img/darktrace-screenshot.png new file mode 100644 index 00000000000..c78dc5bd31e Binary files /dev/null and b/packages/darktrace/img/darktrace-screenshot.png differ diff --git a/packages/darktrace/kibana/dashboard/darktrace-6bd3c320-13b2-11ed-bdc1-9f13147efcf8.json b/packages/darktrace/kibana/dashboard/darktrace-6bd3c320-13b2-11ed-bdc1-9f13147efcf8.json new file mode 100644 index 00000000000..fdabfd6f8c1 --- /dev/null +++ b/packages/darktrace/kibana/dashboard/darktrace-6bd3c320-13b2-11ed-bdc1-9f13147efcf8.json @@ -0,0 +1,513 @@ +{ + "attributes": { + "description": "Darktrace System Status Alerts Overview.", + "hits": 0, + "kibanaSavedObjectMeta": { + "searchSourceJSON": { + "filter": [], + "query": { + "language": "kuery", + "query": "data_stream.dataset : \"darktrace.system_status_alert\"" + } + } + }, + "optionsJSON": { + "hidePanelTitles": false, + "syncColors": false, + "useMargins": true + }, + "panelsJSON": [ + { + "embeddableConfig": { + "attributes": { + "references": [ + { + "id": "logs-*", + "name": "indexpattern-datasource-current-indexpattern", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "indexpattern-datasource-layer-1b85280d-b235-4523-b782-fd77e9046901", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "3702c81f-57cb-4f31-bb86-97827dab7021", + "type": "index-pattern" + } + ], + "state": { + "datasourceStates": { + "indexpattern": { + "layers": { + "1b85280d-b235-4523-b782-fd77e9046901": { + "columnOrder": [ + "426426da-2361-40d0-a759-2591bdf082c9" + ], + "columns": { + "426426da-2361-40d0-a759-2591bdf082c9": { + "customLabel": true, + "dataType": "number", + "isBucketed": false, + "label": "Count", + "operationType": "unique_count", + "params": { + "emptyAsNull": true + }, + "scale": "ratio", + "sourceField": "event.id" + } + }, + "incompleteColumns": {} + } + } + } + }, + "filters": [ + { + "$state": { + "store": "appState" + }, + "meta": { + "alias": null, + "disabled": false, + "index": "3702c81f-57cb-4f31-bb86-97827dab7021", + "key": "darktrace.system_status_alert.status", + "negate": false, + "params": { + "query": "active" + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "darktrace.system_status_alert.status": "active" + } + } + } + ], + "query": { + "language": "kuery", + "query": "data_stream.dataset : \"darktrace.system_status_alert\"" + }, + "visualization": { + "accessor": "426426da-2361-40d0-a759-2591bdf082c9", + "layerId": "1b85280d-b235-4523-b782-fd77e9046901", + "layerType": "data" + } + }, + "title": "", + "type": "lens", + "visualizationType": "lnsMetric" + }, + "enhancements": {}, + "hidePanelTitles": false + }, + "gridData": { + "h": 15, + "i": "5f64c3c5-4d59-4abb-a6ab-234a1ee66151", + "w": 24, + "x": 0, + "y": 0 + }, + "panelIndex": "5f64c3c5-4d59-4abb-a6ab-234a1ee66151", + "title": "Number of Active Alerts [Logs Darktrace]", + "type": "lens", + "version": "8.2.1" + }, + { + "embeddableConfig": { + "attributes": { + "references": [ + { + "id": "logs-*", + "name": "indexpattern-datasource-current-indexpattern", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "indexpattern-datasource-layer-f27d6430-9a24-4f7b-86b0-43950b6f2393", + "type": "index-pattern" + } + ], + "state": { + "datasourceStates": { + "indexpattern": { + "layers": { + "f27d6430-9a24-4f7b-86b0-43950b6f2393": { + "columnOrder": [ + "ecdeb1b2-48c5-4966-bca9-0f228a2916f3", + "11c181af-dff4-4a0a-ad2e-0846bd66affe" + ], + "columns": { + "11c181af-dff4-4a0a-ad2e-0846bd66affe": { + "customLabel": true, + "dataType": "number", + "isBucketed": false, + "label": "Count", + "operationType": "unique_count", + "params": { + "emptyAsNull": true + }, + "scale": "ratio", + "sourceField": "event.id" + }, + "ecdeb1b2-48c5-4966-bca9-0f228a2916f3": { + "customLabel": true, + "dataType": "string", + "isBucketed": true, + "label": "Priority Level", + "operationType": "terms", + "params": { + "missingBucket": false, + "orderBy": { + "columnId": "11c181af-dff4-4a0a-ad2e-0846bd66affe", + "type": "column" + }, + "orderDirection": "desc", + "otherBucket": true, + "parentFormat": { + "id": "terms" + }, + "size": 5 + }, + "scale": "ordinal", + "sourceField": "darktrace.system_status_alert.priority_level" + } + }, + "incompleteColumns": {} + } + } + } + }, + "filters": [], + "query": { + "language": "kuery", + "query": "data_stream.dataset : \"darktrace.system_status_alert\"" + }, + "visualization": { + "layers": [ + { + "categoryDisplay": "default", + "groups": [ + "ecdeb1b2-48c5-4966-bca9-0f228a2916f3" + ], + "layerId": "f27d6430-9a24-4f7b-86b0-43950b6f2393", + "layerType": "data", + "legendDisplay": "default", + "metric": "11c181af-dff4-4a0a-ad2e-0846bd66affe", + "nestedLegend": false, + "numberDisplay": "percent" + } + ], + "shape": "pie" + } + }, + "title": "", + "type": "lens", + "visualizationType": "lnsPie" + }, + "enhancements": {}, + "hidePanelTitles": false + }, + "gridData": { + "h": 15, + "i": "e7b10ecb-271a-4010-9947-9597225acd58", + "w": 24, + "x": 24, + "y": 0 + }, + "panelIndex": "e7b10ecb-271a-4010-9947-9597225acd58", + "title": "Distribution of System Status Alerts by Priority Level [Logs Darktrace]", + "type": "lens", + "version": "8.2.1" + }, + { + "embeddableConfig": { + "attributes": { + "references": [ + { + "id": "logs-*", + "name": "indexpattern-datasource-current-indexpattern", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "indexpattern-datasource-layer-b1042ac5-75bd-48e1-9c8c-4ab507402159", + "type": "index-pattern" + } + ], + "state": { + "datasourceStates": { + "indexpattern": { + "layers": { + "b1042ac5-75bd-48e1-9c8c-4ab507402159": { + "columnOrder": [ + "7deb2674-d025-43e9-b627-7c8e4a3d3ba6", + "72135a2c-712d-421e-8e29-8a5c82f557be" + ], + "columns": { + "72135a2c-712d-421e-8e29-8a5c82f557be": { + "customLabel": true, + "dataType": "number", + "isBucketed": false, + "label": "Count", + "operationType": "unique_count", + "params": { + "emptyAsNull": true + }, + "scale": "ratio", + "sourceField": "event.id" + }, + "7deb2674-d025-43e9-b627-7c8e4a3d3ba6": { + "customLabel": true, + "dataType": "string", + "isBucketed": true, + "label": "Hostname", + "operationType": "terms", + "params": { + "missingBucket": false, + "orderBy": { + "columnId": "72135a2c-712d-421e-8e29-8a5c82f557be", + "type": "column" + }, + "orderDirection": "desc", + "otherBucket": false, + "parentFormat": { + "id": "terms" + }, + "size": 10 + }, + "scale": "ordinal", + "sourceField": "host.hostname" + } + }, + "incompleteColumns": {} + } + } + } + }, + "filters": [], + "query": { + "language": "kuery", + "query": "data_stream.dataset : \"darktrace.system_status_alert\"" + }, + "visualization": { + "columns": [ + { + "columnId": "7deb2674-d025-43e9-b627-7c8e4a3d3ba6", + "isTransposed": false + }, + { + "columnId": "72135a2c-712d-421e-8e29-8a5c82f557be", + "isTransposed": false + } + ], + "layerId": "b1042ac5-75bd-48e1-9c8c-4ab507402159", + "layerType": "data" + } + }, + "title": "", + "type": "lens", + "visualizationType": "lnsDatatable" + }, + "enhancements": {}, + "hidePanelTitles": false + }, + "gridData": { + "h": 15, + "i": "77e3df19-769a-414a-b96b-dbb37169629d", + "w": 24, + "x": 24, + "y": 15 + }, + "panelIndex": "77e3df19-769a-414a-b96b-dbb37169629d", + "title": "Top 10 Hostname with Highest System Status Alerts [Logs Darktrace]", + "type": "lens", + "version": "8.2.1" + }, + { + "embeddableConfig": { + "attributes": { + "references": [ + { + "id": "logs-*", + "name": "indexpattern-datasource-current-indexpattern", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "indexpattern-datasource-layer-607d2de2-df5d-4503-90e0-4ac42323c46e", + "type": "index-pattern" + } + ], + "state": { + "datasourceStates": { + "indexpattern": { + "layers": { + "607d2de2-df5d-4503-90e0-4ac42323c46e": { + "columnOrder": [ + "dafa285e-d83f-4d93-af67-4b6b7a7437f3", + "4ba339dd-9edb-445a-a121-43092d3b33a5" + ], + "columns": { + "4ba339dd-9edb-445a-a121-43092d3b33a5": { + "customLabel": true, + "dataType": "number", + "isBucketed": false, + "label": "Count", + "operationType": "unique_count", + "params": { + "emptyAsNull": true + }, + "scale": "ratio", + "sourceField": "event.id" + }, + "dafa285e-d83f-4d93-af67-4b6b7a7437f3": { + "customLabel": true, + "dataType": "string", + "isBucketed": true, + "label": "Alert Name", + "operationType": "terms", + "params": { + "missingBucket": false, + "orderBy": { + "columnId": "4ba339dd-9edb-445a-a121-43092d3b33a5", + "type": "column" + }, + "orderDirection": "desc", + "otherBucket": false, + "parentFormat": { + "id": "terms" + }, + "size": 10 + }, + "scale": "ordinal", + "sourceField": "darktrace.system_status_alert.name" + } + }, + "incompleteColumns": {} + } + } + } + }, + "filters": [], + "query": { + "language": "kuery", + "query": "data_stream.dataset : \"darktrace.system_status_alert\"" + }, + "visualization": { + "columns": [ + { + "columnId": "dafa285e-d83f-4d93-af67-4b6b7a7437f3", + "isTransposed": false + }, + { + "columnId": "4ba339dd-9edb-445a-a121-43092d3b33a5", + "isTransposed": false + } + ], + "layerId": "607d2de2-df5d-4503-90e0-4ac42323c46e", + "layerType": "data" + } + }, + "title": "", + "type": "lens", + "visualizationType": "lnsDatatable" + }, + "enhancements": {}, + "hidePanelTitles": false + }, + "gridData": { + "h": 15, + "i": "7d794103-85bd-4669-b9bc-b9223d2eba5c", + "w": 24, + "x": 0, + "y": 15 + }, + "panelIndex": "7d794103-85bd-4669-b9bc-b9223d2eba5c", + "title": "Top 10 Alert Name with Highest System Status Alerts [Logs Darktrace]", + "type": "lens", + "version": "8.2.1" + }, + { + "embeddableConfig": { + "enhancements": {} + }, + "gridData": { + "h": 23, + "i": "00e77d89-2b5e-4f2d-bc08-f7d1ce5165dd", + "w": 48, + "x": 0, + "y": 30 + }, + "panelIndex": "00e77d89-2b5e-4f2d-bc08-f7d1ce5165dd", + "panelRefName": "panel_00e77d89-2b5e-4f2d-bc08-f7d1ce5165dd", + "type": "search", + "version": "8.2.1" + } + ], + "timeRestore": false, + "title": "[Logs Darktrace] System Status Alerts Overview", + "version": 1 + }, + "coreMigrationVersion": "8.2.1", + "id": "darktrace-6bd3c320-13b2-11ed-bdc1-9f13147efcf8", + "migrationVersion": { + "dashboard": "8.2.0" + }, + "references": [ + { + "id": "logs-*", + "name": "5f64c3c5-4d59-4abb-a6ab-234a1ee66151:indexpattern-datasource-current-indexpattern", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "5f64c3c5-4d59-4abb-a6ab-234a1ee66151:indexpattern-datasource-layer-1b85280d-b235-4523-b782-fd77e9046901", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "5f64c3c5-4d59-4abb-a6ab-234a1ee66151:3702c81f-57cb-4f31-bb86-97827dab7021", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "e7b10ecb-271a-4010-9947-9597225acd58:indexpattern-datasource-current-indexpattern", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "e7b10ecb-271a-4010-9947-9597225acd58:indexpattern-datasource-layer-f27d6430-9a24-4f7b-86b0-43950b6f2393", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "77e3df19-769a-414a-b96b-dbb37169629d:indexpattern-datasource-current-indexpattern", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "77e3df19-769a-414a-b96b-dbb37169629d:indexpattern-datasource-layer-b1042ac5-75bd-48e1-9c8c-4ab507402159", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "7d794103-85bd-4669-b9bc-b9223d2eba5c:indexpattern-datasource-current-indexpattern", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "7d794103-85bd-4669-b9bc-b9223d2eba5c:indexpattern-datasource-layer-607d2de2-df5d-4503-90e0-4ac42323c46e", + "type": "index-pattern" + }, + { + "id": "darktrace-fbf9cfc0-13b3-11ed-bdc1-9f13147efcf8", + "name": "00e77d89-2b5e-4f2d-bc08-f7d1ce5165dd:panel_00e77d89-2b5e-4f2d-bc08-f7d1ce5165dd", + "type": "search" + } + ], + "type": "dashboard" +} \ No newline at end of file diff --git a/packages/darktrace/kibana/dashboard/darktrace-da768d80-1399-11ed-bdc1-9f13147efcf8.json b/packages/darktrace/kibana/dashboard/darktrace-da768d80-1399-11ed-bdc1-9f13147efcf8.json new file mode 100644 index 00000000000..bf5cb00e86e --- /dev/null +++ b/packages/darktrace/kibana/dashboard/darktrace-da768d80-1399-11ed-bdc1-9f13147efcf8.json @@ -0,0 +1,1528 @@ +{ + "attributes": { + "description": "Darktrace Model Breach Alerts Overview.", + "hits": 0, + "kibanaSavedObjectMeta": { + "searchSourceJSON": { + "filter": [], + "query": { + "language": "kuery", + "query": "data_stream.dataset : \"darktrace.model_breach_alert\"" + } + } + }, + "optionsJSON": { + "hidePanelTitles": false, + "syncColors": false, + "useMargins": true + }, + "panelsJSON": [ + { + "embeddableConfig": { + "attributes": { + "references": [ + { + "id": "logs-*", + "name": "indexpattern-datasource-current-indexpattern", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "indexpattern-datasource-layer-16c69f2e-ffe0-4393-9d91-dece311e3f0f", + "type": "index-pattern" + } + ], + "state": { + "datasourceStates": { + "indexpattern": { + "layers": { + "16c69f2e-ffe0-4393-9d91-dece311e3f0f": { + "columnOrder": [ + "099298f5-fc58-4473-860e-84bc44f2e387" + ], + "columns": { + "099298f5-fc58-4473-860e-84bc44f2e387": { + "customLabel": true, + "dataType": "number", + "isBucketed": false, + "label": "Count", + "operationType": "unique_count", + "params": { + "emptyAsNull": true + }, + "scale": "ratio", + "sourceField": "darktrace.model_breach_alert.pbid" + } + }, + "incompleteColumns": {} + } + } + } + }, + "filters": [], + "query": { + "language": "kuery", + "query": "data_stream.dataset : \"darktrace.model_breach_alert\"" + }, + "visualization": { + "accessor": "099298f5-fc58-4473-860e-84bc44f2e387", + "layerId": "16c69f2e-ffe0-4393-9d91-dece311e3f0f", + "layerType": "data" + } + }, + "title": "", + "type": "lens", + "visualizationType": "lnsMetric" + }, + "enhancements": {}, + "hidePanelTitles": false + }, + "gridData": { + "h": 15, + "i": "14e3bf5d-011f-48d2-83a9-fc62d707cdd1", + "w": 15, + "x": 0, + "y": 0 + }, + "panelIndex": "14e3bf5d-011f-48d2-83a9-fc62d707cdd1", + "title": "Number of Alerts [Logs Darktrace]", + "type": "lens", + "version": "8.2.1" + }, + { + "embeddableConfig": { + "attributes": { + "references": [ + { + "id": "logs-*", + "name": "indexpattern-datasource-current-indexpattern", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "indexpattern-datasource-layer-8d4cd3ff-fd36-462e-ae82-826554dc847d", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "88df79e3-51ce-46c3-b8da-6522f6dc9e40", + "type": "index-pattern" + } + ], + "state": { + "datasourceStates": { + "indexpattern": { + "layers": { + "8d4cd3ff-fd36-462e-ae82-826554dc847d": { + "columnOrder": [ + "861dc1ff-427e-4512-bb2c-e28d3f7564b2" + ], + "columns": { + "861dc1ff-427e-4512-bb2c-e28d3f7564b2": { + "customLabel": true, + "dataType": "number", + "isBucketed": false, + "label": "Count", + "operationType": "unique_count", + "params": { + "emptyAsNull": true + }, + "scale": "ratio", + "sourceField": "rule.uuid" + } + }, + "incompleteColumns": {} + } + } + } + }, + "filters": [ + { + "$state": { + "store": "appState" + }, + "meta": { + "alias": null, + "disabled": false, + "index": "88df79e3-51ce-46c3-b8da-6522f6dc9e40", + "key": "darktrace.model_breach_alert.model.is_active", + "negate": false, + "params": { + "query": true + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "darktrace.model_breach_alert.model.is_active": true + } + } + } + ], + "query": { + "language": "kuery", + "query": "data_stream.dataset : \"darktrace.model_breach_alert\"" + }, + "visualization": { + "accessor": "861dc1ff-427e-4512-bb2c-e28d3f7564b2", + "layerId": "8d4cd3ff-fd36-462e-ae82-826554dc847d", + "layerType": "data" + } + }, + "title": "", + "type": "lens", + "visualizationType": "lnsMetric" + }, + "enhancements": {}, + "hidePanelTitles": false + }, + "gridData": { + "h": 15, + "i": "07f13cdd-3a86-40e5-914f-8f50c695b6ee", + "w": 15, + "x": 15, + "y": 0 + }, + "panelIndex": "07f13cdd-3a86-40e5-914f-8f50c695b6ee", + "title": "Number of Active Models [Logs Darktrace]", + "type": "lens", + "version": "8.2.1" + }, + { + "embeddableConfig": { + "attributes": { + "references": [ + { + "id": "logs-*", + "name": "indexpattern-datasource-current-indexpattern", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "indexpattern-datasource-layer-a4c3d027-4533-411a-b9f1-26f0a4fedb66", + "type": "index-pattern" + } + ], + "state": { + "datasourceStates": { + "indexpattern": { + "layers": { + "a4c3d027-4533-411a-b9f1-26f0a4fedb66": { + "columnOrder": [ + "1ea9479b-4db9-4215-97d9-1d7a275176ab", + "36c1f412-cfb7-4ea0-b9c9-a323c72e800d" + ], + "columns": { + "1ea9479b-4db9-4215-97d9-1d7a275176ab": { + "customLabel": true, + "dataType": "string", + "isBucketed": true, + "label": "Category", + "operationType": "terms", + "params": { + "missingBucket": false, + "orderBy": { + "columnId": "36c1f412-cfb7-4ea0-b9c9-a323c72e800d", + "type": "column" + }, + "orderDirection": "desc", + "otherBucket": true, + "parentFormat": { + "id": "terms" + }, + "size": 5 + }, + "scale": "ordinal", + "sourceField": "rule.category" + }, + "36c1f412-cfb7-4ea0-b9c9-a323c72e800d": { + "customLabel": true, + "dataType": "number", + "isBucketed": false, + "label": "Count", + "operationType": "unique_count", + "params": { + "emptyAsNull": true + }, + "scale": "ratio", + "sourceField": "darktrace.model_breach_alert.pbid" + } + }, + "incompleteColumns": {} + } + } + } + }, + "filters": [], + "query": { + "language": "kuery", + "query": "data_stream.dataset : \"darktrace.model_breach_alert\"" + }, + "visualization": { + "layers": [ + { + "categoryDisplay": "default", + "groups": [ + "1ea9479b-4db9-4215-97d9-1d7a275176ab" + ], + "layerId": "a4c3d027-4533-411a-b9f1-26f0a4fedb66", + "layerType": "data", + "legendDisplay": "default", + "metric": "36c1f412-cfb7-4ea0-b9c9-a323c72e800d", + "nestedLegend": false, + "numberDisplay": "percent" + } + ], + "shape": "pie" + } + }, + "title": "", + "type": "lens", + "visualizationType": "lnsPie" + }, + "enhancements": {}, + "hidePanelTitles": false + }, + "gridData": { + "h": 15, + "i": "1fafffde-be8a-4e46-bc58-a52db1e94931", + "w": 18, + "x": 30, + "y": 0 + }, + "panelIndex": "1fafffde-be8a-4e46-bc58-a52db1e94931", + "title": "Distribution of Model Breach Alerts by Model Category [Logs Darktrace]", + "type": "lens", + "version": "8.2.1" + }, + { + "embeddableConfig": { + "attributes": { + "references": [ + { + "id": "logs-*", + "name": "indexpattern-datasource-current-indexpattern", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "indexpattern-datasource-layer-8a0016c8-0623-4e96-a007-240f0bfe88c2", + "type": "index-pattern" + } + ], + "state": { + "datasourceStates": { + "indexpattern": { + "layers": { + "8a0016c8-0623-4e96-a007-240f0bfe88c2": { + "columnOrder": [ + "55131f02-db30-408f-8795-9cfee8f6758b", + "b5b19414-8d46-4957-b69a-7a57518551fe" + ], + "columns": { + "55131f02-db30-408f-8795-9cfee8f6758b": { + "customLabel": true, + "dataType": "number", + "isBucketed": true, + "label": "Model Priority", + "operationType": "terms", + "params": { + "missingBucket": false, + "orderBy": { + "columnId": "b5b19414-8d46-4957-b69a-7a57518551fe", + "type": "column" + }, + "orderDirection": "desc", + "otherBucket": true, + "parentFormat": { + "id": "terms" + }, + "size": 10 + }, + "scale": "ordinal", + "sourceField": "event.severity" + }, + "b5b19414-8d46-4957-b69a-7a57518551fe": { + "customLabel": true, + "dataType": "number", + "isBucketed": false, + "label": "Count", + "operationType": "unique_count", + "params": { + "emptyAsNull": true + }, + "scale": "ratio", + "sourceField": "darktrace.model_breach_alert.pbid" + } + }, + "incompleteColumns": {} + } + } + } + }, + "filters": [], + "query": { + "language": "kuery", + "query": "data_stream.dataset : \"darktrace.model_breach_alert\"" + }, + "visualization": { + "axisTitlesVisibilitySettings": { + "x": true, + "yLeft": true, + "yRight": true + }, + "fittingFunction": "None", + "gridlinesVisibilitySettings": { + "x": true, + "yLeft": true, + "yRight": true + }, + "labelsOrientation": { + "x": 0, + "yLeft": 0, + "yRight": 0 + }, + "layers": [ + { + "accessors": [ + "b5b19414-8d46-4957-b69a-7a57518551fe" + ], + "layerId": "8a0016c8-0623-4e96-a007-240f0bfe88c2", + "layerType": "data", + "seriesType": "bar_stacked", + "xAccessor": "55131f02-db30-408f-8795-9cfee8f6758b" + } + ], + "legend": { + "isVisible": true, + "position": "right" + }, + "preferredSeriesType": "bar_stacked", + "tickLabelsVisibilitySettings": { + "x": true, + "yLeft": true, + "yRight": true + }, + "valueLabels": "hide" + } + }, + "title": "", + "type": "lens", + "visualizationType": "lnsXY" + }, + "enhancements": {}, + "hidePanelTitles": false + }, + "gridData": { + "h": 15, + "i": "ddcd6a80-5ab0-4522-b984-022b7da2d4b0", + "w": 24, + "x": 0, + "y": 15 + }, + "panelIndex": "ddcd6a80-5ab0-4522-b984-022b7da2d4b0", + "title": "Distribution of Model Breach Alerts by Model Priority [Logs Darktrace]", + "type": "lens", + "version": "8.2.1" + }, + { + "embeddableConfig": { + "attributes": { + "references": [ + { + "id": "logs-*", + "name": "indexpattern-datasource-current-indexpattern", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "indexpattern-datasource-layer-267ebe2d-c964-48cf-9c9a-1a1fb09f6e3c", + "type": "index-pattern" + } + ], + "state": { + "datasourceStates": { + "indexpattern": { + "layers": { + "267ebe2d-c964-48cf-9c9a-1a1fb09f6e3c": { + "columnOrder": [ + "c2ee5623-973c-416f-80b0-bae47d66f83b", + "8630c019-3e7e-4734-b1c2-1a82f39fb7fc" + ], + "columns": { + "8630c019-3e7e-4734-b1c2-1a82f39fb7fc": { + "customLabel": true, + "dataType": "number", + "isBucketed": false, + "label": "Count", + "operationType": "unique_count", + "params": { + "emptyAsNull": true + }, + "scale": "ratio", + "sourceField": "darktrace.model_breach_alert.pbid" + }, + "c2ee5623-973c-416f-80b0-bae47d66f83b": { + "customLabel": true, + "dataType": "string", + "isBucketed": true, + "label": "Model Behaviour", + "operationType": "terms", + "params": { + "missingBucket": false, + "orderBy": { + "columnId": "8630c019-3e7e-4734-b1c2-1a82f39fb7fc", + "type": "column" + }, + "orderDirection": "desc", + "otherBucket": true, + "parentFormat": { + "id": "terms" + }, + "size": 5 + }, + "scale": "ordinal", + "sourceField": "darktrace.model_breach_alert.model.behaviour" + } + }, + "incompleteColumns": {} + } + } + } + }, + "filters": [], + "query": { + "language": "kuery", + "query": "data_stream.dataset : \"darktrace.model_breach_alert\"" + }, + "visualization": { + "layers": [ + { + "categoryDisplay": "default", + "groups": [ + "c2ee5623-973c-416f-80b0-bae47d66f83b" + ], + "layerId": "267ebe2d-c964-48cf-9c9a-1a1fb09f6e3c", + "layerType": "data", + "legendDisplay": "default", + "metric": "8630c019-3e7e-4734-b1c2-1a82f39fb7fc", + "nestedLegend": false, + "numberDisplay": "percent" + } + ], + "shape": "pie" + } + }, + "title": "", + "type": "lens", + "visualizationType": "lnsPie" + }, + "enhancements": {}, + "hidePanelTitles": false + }, + "gridData": { + "h": 15, + "i": "44710442-b7b8-413a-9e52-4d7ba519a296", + "w": 24, + "x": 24, + "y": 15 + }, + "panelIndex": "44710442-b7b8-413a-9e52-4d7ba519a296", + "title": "Distribution of Model Breach Alerts by Model Behaviour [Logs Darktrace]", + "type": "lens", + "version": "8.2.1" + }, + { + "embeddableConfig": { + "attributes": { + "references": [ + { + "id": "logs-*", + "name": "indexpattern-datasource-current-indexpattern", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "indexpattern-datasource-layer-26e0acea-9274-411a-91a3-8537b1e00aff", + "type": "index-pattern" + } + ], + "state": { + "datasourceStates": { + "indexpattern": { + "layers": { + "26e0acea-9274-411a-91a3-8537b1e00aff": { + "columnOrder": [ + "ac9a1bc1-8890-4297-a82e-6f975d9175aa", + "b451b0a8-806d-4d37-85c6-85c98330a533" + ], + "columns": { + "ac9a1bc1-8890-4297-a82e-6f975d9175aa": { + "customLabel": true, + "dataType": "string", + "isBucketed": true, + "label": "Model Name", + "operationType": "terms", + "params": { + "missingBucket": false, + "orderBy": { + "columnId": "b451b0a8-806d-4d37-85c6-85c98330a533", + "type": "column" + }, + "orderDirection": "desc", + "otherBucket": false, + "parentFormat": { + "id": "terms" + }, + "secondaryFields": [], + "size": 10 + }, + "scale": "ordinal", + "sourceField": "rule.name" + }, + "b451b0a8-806d-4d37-85c6-85c98330a533": { + "customLabel": true, + "dataType": "number", + "isBucketed": false, + "label": "Model Breach Score", + "operationType": "max", + "params": { + "emptyAsNull": true + }, + "scale": "ratio", + "sourceField": "event.risk_score" + } + }, + "incompleteColumns": {} + } + } + } + }, + "filters": [], + "query": { + "language": "kuery", + "query": "data_stream.dataset : \"darktrace.model_breach_alert\"" + }, + "visualization": { + "columns": [ + { + "alignment": "left", + "columnId": "ac9a1bc1-8890-4297-a82e-6f975d9175aa", + "isTransposed": false, + "summaryRow": "none" + }, + { + "columnId": "b451b0a8-806d-4d37-85c6-85c98330a533", + "isTransposed": false + } + ], + "layerId": "26e0acea-9274-411a-91a3-8537b1e00aff", + "layerType": "data" + } + }, + "title": "", + "type": "lens", + "visualizationType": "lnsDatatable" + }, + "enhancements": {}, + "hidePanelTitles": false + }, + "gridData": { + "h": 15, + "i": "747c1919-e215-4b97-9d8b-8ee528c1deaa", + "w": 24, + "x": 0, + "y": 30 + }, + "panelIndex": "747c1919-e215-4b97-9d8b-8ee528c1deaa", + "title": "Top 10 Model Breach Alerts by Highest Model Breach Score [Logs Darktrace]", + "type": "lens", + "version": "8.2.1" + }, + { + "embeddableConfig": { + "attributes": { + "references": [ + { + "id": "logs-*", + "name": "indexpattern-datasource-current-indexpattern", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "indexpattern-datasource-layer-adde69bc-fda5-4560-8a54-202ca975652f", + "type": "index-pattern" + } + ], + "state": { + "datasourceStates": { + "indexpattern": { + "layers": { + "adde69bc-fda5-4560-8a54-202ca975652f": { + "columnOrder": [ + "c78a709f-ef66-4dbc-a1f2-070cb2116e4d", + "ead17241-a253-4c78-917d-8ff1249061df" + ], + "columns": { + "c78a709f-ef66-4dbc-a1f2-070cb2116e4d": { + "customLabel": true, + "dataType": "string", + "isBucketed": true, + "label": "Model Name", + "operationType": "terms", + "params": { + "missingBucket": false, + "orderBy": { + "columnId": "ead17241-a253-4c78-917d-8ff1249061df", + "type": "column" + }, + "orderDirection": "desc", + "otherBucket": false, + "parentFormat": { + "id": "terms" + }, + "secondaryFields": [], + "size": 10 + }, + "scale": "ordinal", + "sourceField": "rule.name" + }, + "ead17241-a253-4c78-917d-8ff1249061df": { + "customLabel": true, + "dataType": "number", + "isBucketed": false, + "label": "Count", + "operationType": "unique_count", + "params": { + "emptyAsNull": true + }, + "scale": "ratio", + "sourceField": "darktrace.model_breach_alert.pbid" + } + }, + "incompleteColumns": {} + } + } + } + }, + "filters": [], + "query": { + "language": "kuery", + "query": "data_stream.dataset : \"darktrace.model_breach_alert\"" + }, + "visualization": { + "columns": [ + { + "alignment": "left", + "columnId": "c78a709f-ef66-4dbc-a1f2-070cb2116e4d", + "isTransposed": false, + "width": 574 + }, + { + "columnId": "ead17241-a253-4c78-917d-8ff1249061df", + "isTransposed": false + } + ], + "layerId": "adde69bc-fda5-4560-8a54-202ca975652f", + "layerType": "data" + } + }, + "title": "", + "type": "lens", + "visualizationType": "lnsDatatable" + }, + "enhancements": {}, + "hidePanelTitles": false + }, + "gridData": { + "h": 15, + "i": "aca1678c-d3d8-478e-a09c-dfdd86a5b3f7", + "w": 24, + "x": 24, + "y": 30 + }, + "panelIndex": "aca1678c-d3d8-478e-a09c-dfdd86a5b3f7", + "title": "Top 10 Model Name with Highest Model Breach [Logs Darktrace]", + "type": "lens", + "version": "8.2.1" + }, + { + "embeddableConfig": { + "attributes": { + "references": [ + { + "id": "logs-*", + "name": "indexpattern-datasource-current-indexpattern", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "indexpattern-datasource-layer-0c7a50df-8359-42ff-806d-a22eb35b597a", + "type": "index-pattern" + } + ], + "state": { + "datasourceStates": { + "indexpattern": { + "layers": { + "0c7a50df-8359-42ff-806d-a22eb35b597a": { + "columnOrder": [ + "729d6a4f-b1ba-47be-817a-2f2bf8b6f39c", + "a0c3e6c0-52a2-4fc6-ac35-ff0ee67d1515" + ], + "columns": { + "729d6a4f-b1ba-47be-817a-2f2bf8b6f39c": { + "customLabel": true, + "dataType": "string", + "isBucketed": true, + "label": "Device Type", + "operationType": "terms", + "params": { + "missingBucket": false, + "orderBy": { + "columnId": "a0c3e6c0-52a2-4fc6-ac35-ff0ee67d1515", + "type": "column" + }, + "orderDirection": "desc", + "otherBucket": false, + "parentFormat": { + "id": "terms" + }, + "size": 10 + }, + "scale": "ordinal", + "sourceField": "host.type" + }, + "a0c3e6c0-52a2-4fc6-ac35-ff0ee67d1515": { + "customLabel": true, + "dataType": "number", + "isBucketed": false, + "label": "Count", + "operationType": "unique_count", + "params": { + "emptyAsNull": true + }, + "scale": "ratio", + "sourceField": "darktrace.model_breach_alert.pbid" + } + }, + "incompleteColumns": {} + } + } + } + }, + "filters": [], + "query": { + "language": "kuery", + "query": "data_stream.dataset : \"darktrace.model_breach_alert\"" + }, + "visualization": { + "columns": [ + { + "columnId": "729d6a4f-b1ba-47be-817a-2f2bf8b6f39c", + "isTransposed": false + }, + { + "columnId": "a0c3e6c0-52a2-4fc6-ac35-ff0ee67d1515", + "isTransposed": false + } + ], + "layerId": "0c7a50df-8359-42ff-806d-a22eb35b597a", + "layerType": "data" + } + }, + "title": "", + "type": "lens", + "visualizationType": "lnsDatatable" + }, + "enhancements": {}, + "hidePanelTitles": false + }, + "gridData": { + "h": 15, + "i": "19b3fa09-6280-430a-9046-a613dfde3696", + "w": 24, + "x": 0, + "y": 45 + }, + "panelIndex": "19b3fa09-6280-430a-9046-a613dfde3696", + "title": "Top 10 Device Type with Highest Model Breach [Logs Darktrace]", + "type": "lens", + "version": "8.2.1" + }, + { + "embeddableConfig": { + "attributes": { + "references": [ + { + "id": "logs-*", + "name": "indexpattern-datasource-current-indexpattern", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "indexpattern-datasource-layer-7bd679f9-8a5b-4906-beaa-750102e3a26f", + "type": "index-pattern" + } + ], + "state": { + "datasourceStates": { + "indexpattern": { + "layers": { + "7bd679f9-8a5b-4906-beaa-750102e3a26f": { + "columnOrder": [ + "13359fdf-964f-441c-8d49-dcacd44d74a9", + "d0c28963-3b20-44ed-bd81-668ccef65e64" + ], + "columns": { + "13359fdf-964f-441c-8d49-dcacd44d74a9": { + "customLabel": true, + "dataType": "string", + "isBucketed": true, + "label": "Vendor", + "operationType": "terms", + "params": { + "missingBucket": false, + "orderBy": { + "columnId": "d0c28963-3b20-44ed-bd81-668ccef65e64", + "type": "column" + }, + "orderDirection": "desc", + "otherBucket": true, + "parentFormat": { + "id": "terms" + }, + "size": 5 + }, + "scale": "ordinal", + "sourceField": "darktrace.model_breach_alert.device.vendor" + }, + "d0c28963-3b20-44ed-bd81-668ccef65e64": { + "customLabel": true, + "dataType": "number", + "isBucketed": false, + "label": "Count", + "operationType": "unique_count", + "params": { + "emptyAsNull": true + }, + "scale": "ratio", + "sourceField": "darktrace.model_breach_alert.pbid" + } + }, + "incompleteColumns": {} + } + } + } + }, + "filters": [], + "query": { + "language": "kuery", + "query": "data_stream.dataset : \"darktrace.model_breach_alert\"" + }, + "visualization": { + "layers": [ + { + "categoryDisplay": "default", + "groups": [ + "13359fdf-964f-441c-8d49-dcacd44d74a9" + ], + "layerId": "7bd679f9-8a5b-4906-beaa-750102e3a26f", + "layerType": "data", + "legendDisplay": "default", + "metric": "d0c28963-3b20-44ed-bd81-668ccef65e64", + "nestedLegend": false, + "numberDisplay": "percent" + } + ], + "shape": "pie" + } + }, + "title": "", + "type": "lens", + "visualizationType": "lnsPie" + }, + "enhancements": {}, + "hidePanelTitles": false + }, + "gridData": { + "h": 15, + "i": "185e6cd3-4cf8-45fd-937e-77abd9e6aad7", + "w": 24, + "x": 24, + "y": 45 + }, + "panelIndex": "185e6cd3-4cf8-45fd-937e-77abd9e6aad7", + "title": "Distribution of Model Breach Alerts by Targeted Vendor [Logs Darktrace]", + "type": "lens", + "version": "8.2.1" + }, + { + "embeddableConfig": { + "attributes": { + "references": [ + { + "id": "logs-*", + "name": "indexpattern-datasource-current-indexpattern", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "indexpattern-datasource-layer-c8ea502e-ae28-47dd-9b90-484d50083243", + "type": "index-pattern" + } + ], + "state": { + "datasourceStates": { + "indexpattern": { + "layers": { + "c8ea502e-ae28-47dd-9b90-484d50083243": { + "columnOrder": [ + "0e81fce6-dd05-4dcb-9cc1-1bfedfd001d1", + "dbd63d7d-3048-4f3e-a068-d891e14f517b" + ], + "columns": { + "0e81fce6-dd05-4dcb-9cc1-1bfedfd001d1": { + "customLabel": true, + "dataType": "string", + "isBucketed": true, + "label": "Device Host ID", + "operationType": "terms", + "params": { + "missingBucket": false, + "orderBy": { + "columnId": "dbd63d7d-3048-4f3e-a068-d891e14f517b", + "type": "column" + }, + "orderDirection": "desc", + "otherBucket": false, + "parentFormat": { + "id": "terms" + }, + "size": 10 + }, + "scale": "ordinal", + "sourceField": "host.id" + }, + "dbd63d7d-3048-4f3e-a068-d891e14f517b": { + "customLabel": true, + "dataType": "number", + "isBucketed": false, + "label": "Count", + "operationType": "unique_count", + "params": { + "emptyAsNull": true + }, + "scale": "ratio", + "sourceField": "darktrace.model_breach_alert.pbid" + } + }, + "incompleteColumns": {} + } + } + } + }, + "filters": [], + "query": { + "language": "kuery", + "query": "data_stream.dataset : \"darktrace.model_breach_alert\"" + }, + "visualization": { + "columns": [ + { + "alignment": "left", + "columnId": "0e81fce6-dd05-4dcb-9cc1-1bfedfd001d1", + "isTransposed": false + }, + { + "columnId": "dbd63d7d-3048-4f3e-a068-d891e14f517b", + "isTransposed": false + } + ], + "layerId": "c8ea502e-ae28-47dd-9b90-484d50083243", + "layerType": "data" + } + }, + "title": "", + "type": "lens", + "visualizationType": "lnsDatatable" + }, + "enhancements": {}, + "hidePanelTitles": false + }, + "gridData": { + "h": 15, + "i": "7c6faaf4-5d0c-49a1-b1d2-605f18e675b0", + "w": 24, + "x": 24, + "y": 60 + }, + "panelIndex": "7c6faaf4-5d0c-49a1-b1d2-605f18e675b0", + "title": "Top 10 Device Host ID with Highest Model Breach [Logs Darktrace]", + "type": "lens", + "version": "8.2.1" + }, + { + "embeddableConfig": { + "attributes": { + "references": [ + { + "id": "logs-*", + "name": "indexpattern-datasource-current-indexpattern", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "indexpattern-datasource-layer-88c07c59-c625-4652-8156-54991d0869d8", + "type": "index-pattern" + } + ], + "state": { + "datasourceStates": { + "indexpattern": { + "layers": { + "88c07c59-c625-4652-8156-54991d0869d8": { + "columnOrder": [ + "3fdb34e6-9e66-42b8-8705-ce15282352a8", + "0f644c53-93f7-450a-ab4a-2d08a26251a7" + ], + "columns": { + "0f644c53-93f7-450a-ab4a-2d08a26251a7": { + "customLabel": true, + "dataType": "number", + "isBucketed": false, + "label": "Count", + "operationType": "unique_count", + "params": { + "emptyAsNull": true + }, + "scale": "ratio", + "sourceField": "darktrace.model_breach_alert.pbid" + }, + "3fdb34e6-9e66-42b8-8705-ce15282352a8": { + "customLabel": true, + "dataType": "string", + "isBucketed": true, + "label": "Antigena Action", + "operationType": "terms", + "params": { + "missingBucket": false, + "orderBy": { + "columnId": "0f644c53-93f7-450a-ab4a-2d08a26251a7", + "type": "column" + }, + "orderDirection": "desc", + "otherBucket": true, + "parentFormat": { + "id": "terms" + }, + "secondaryFields": [], + "size": 10 + }, + "scale": "ordinal", + "sourceField": "event.action" + } + }, + "incompleteColumns": {} + } + } + } + }, + "filters": [], + "query": { + "language": "kuery", + "query": "data_stream.dataset : \"darktrace.model_breach_alert\"" + }, + "visualization": { + "layers": [ + { + "accessors": [ + "0f644c53-93f7-450a-ab4a-2d08a26251a7" + ], + "layerId": "88c07c59-c625-4652-8156-54991d0869d8", + "layerType": "data", + "position": "top", + "seriesType": "bar_stacked", + "showGridlines": false, + "xAccessor": "3fdb34e6-9e66-42b8-8705-ce15282352a8" + } + ], + "legend": { + "isVisible": true, + "position": "right" + }, + "preferredSeriesType": "bar_stacked", + "title": "Empty XY chart", + "valueLabels": "hide" + } + }, + "title": "", + "type": "lens", + "visualizationType": "lnsXY" + }, + "enhancements": {}, + "hidePanelTitles": false + }, + "gridData": { + "h": 15, + "i": "889bb859-0938-46a4-b078-30f5fedd10a7", + "w": 24, + "x": 0, + "y": 60 + }, + "panelIndex": "889bb859-0938-46a4-b078-30f5fedd10a7", + "title": "Distribution of Model Breach Alerts by Antigena Action [Logs Darktrace]", + "type": "lens", + "version": "8.2.1" + }, + { + "embeddableConfig": { + "attributes": { + "references": [ + { + "id": "logs-*", + "name": "indexpattern-datasource-current-indexpattern", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "indexpattern-datasource-layer-68e57e92-bad9-44bd-8022-16b46d218096", + "type": "index-pattern" + } + ], + "state": { + "datasourceStates": { + "indexpattern": { + "layers": { + "68e57e92-bad9-44bd-8022-16b46d218096": { + "columnOrder": [ + "e320a021-5c16-4d5f-889a-f88e29cc8fd2", + "e9f48d2b-6578-4b41-afdb-3070764712b2" + ], + "columns": { + "e320a021-5c16-4d5f-889a-f88e29cc8fd2": { + "customLabel": true, + "dataType": "date", + "isBucketed": true, + "label": "Timestamp", + "operationType": "date_histogram", + "params": { + "dropPartials": false, + "ignoreTimeRange": false, + "includeEmptyRows": true, + "interval": "auto" + }, + "scale": "interval", + "sourceField": "@timestamp" + }, + "e9f48d2b-6578-4b41-afdb-3070764712b2": { + "customLabel": true, + "dataType": "number", + "isBucketed": false, + "label": "Median of Model Throttle", + "operationType": "median", + "params": { + "emptyAsNull": true + }, + "scale": "ratio", + "sourceField": "darktrace.model_breach_alert.model.throttle" + } + }, + "incompleteColumns": {} + } + } + } + }, + "filters": [], + "query": { + "language": "kuery", + "query": "data_stream.dataset : \"darktrace.model_breach_alert\"" + }, + "visualization": { + "layers": [ + { + "accessors": [ + "e9f48d2b-6578-4b41-afdb-3070764712b2" + ], + "layerId": "68e57e92-bad9-44bd-8022-16b46d218096", + "layerType": "data", + "position": "top", + "seriesType": "line", + "showGridlines": false, + "xAccessor": "e320a021-5c16-4d5f-889a-f88e29cc8fd2", + "yConfig": [ + { + "axisMode": "auto", + "forAccessor": "e9f48d2b-6578-4b41-afdb-3070764712b2" + } + ] + } + ], + "legend": { + "isVisible": true, + "position": "right" + }, + "preferredSeriesType": "line", + "title": "Empty XY chart", + "valueLabels": "hide" + } + }, + "title": "", + "type": "lens", + "visualizationType": "lnsXY" + }, + "enhancements": {}, + "hidePanelTitles": false + }, + "gridData": { + "h": 19, + "i": "c6560c58-be58-4718-abed-0356a2ba3b09", + "w": 48, + "x": 0, + "y": 75 + }, + "panelIndex": "c6560c58-be58-4718-abed-0356a2ba3b09", + "title": "Model Throttle Over Time [Logs Darktrace]", + "type": "lens", + "version": "8.2.1" + }, + { + "embeddableConfig": { + "enhancements": {}, + "hidePanelTitles": false, + "savedVis": { + "data": { + "aggs": [ + { + "enabled": true, + "id": "1", + "params": { + "customLabel": "Tag Cloud", + "emptyAsNull": false, + "field": "darktrace.model_breach_alert.pbid" + }, + "schema": "metric", + "type": "cardinality" + }, + { + "enabled": true, + "id": "2", + "params": { + "customLabel": "Mitre Techniques", + "field": "threat.technique.name", + "missingBucket": false, + "missingBucketLabel": "Missing", + "order": "desc", + "orderBy": "1", + "otherBucket": false, + "otherBucketLabel": "Other", + "size": 10 + }, + "schema": "segment", + "type": "terms" + } + ], + "searchSource": { + "filter": [], + "indexRefName": "kibanaSavedObjectMeta.searchSourceJSON.index", + "query": { + "language": "kuery", + "query": "data_stream.dataset : \"darktrace.model_breach_alert\"" + } + } + }, + "description": "", + "id": "", + "params": { + "maxFontSize": 72, + "minFontSize": 18, + "orientation": "single", + "palette": { + "name": "default", + "type": "palette" + }, + "scale": "linear", + "showLabel": true + }, + "title": "", + "type": "tagcloud", + "uiState": {} + } + }, + "gridData": { + "h": 23, + "i": "12736f17-d97c-4f4c-a66b-5eba7c2fec9c", + "w": 48, + "x": 0, + "y": 94 + }, + "panelIndex": "12736f17-d97c-4f4c-a66b-5eba7c2fec9c", + "title": "Top Mitre Techniques [Logs Darktrace]", + "type": "visualization", + "version": "8.2.1" + }, + { + "embeddableConfig": { + "enhancements": {} + }, + "gridData": { + "h": 23, + "i": "e9b4f5f5-d478-403d-a78e-9e39ad3486f0", + "w": 48, + "x": 0, + "y": 117 + }, + "panelIndex": "e9b4f5f5-d478-403d-a78e-9e39ad3486f0", + "panelRefName": "panel_e9b4f5f5-d478-403d-a78e-9e39ad3486f0", + "type": "search", + "version": "8.2.1" + } + ], + "timeRestore": false, + "title": "[Logs Darktrace] Model Breach Alerts Overview", + "version": 1 + }, + "coreMigrationVersion": "8.2.1", + "id": "darktrace-da768d80-1399-11ed-bdc1-9f13147efcf8", + "migrationVersion": { + "dashboard": "8.2.0" + }, + "references": [ + { + "id": "logs-*", + "name": "14e3bf5d-011f-48d2-83a9-fc62d707cdd1:indexpattern-datasource-current-indexpattern", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "14e3bf5d-011f-48d2-83a9-fc62d707cdd1:indexpattern-datasource-layer-16c69f2e-ffe0-4393-9d91-dece311e3f0f", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "07f13cdd-3a86-40e5-914f-8f50c695b6ee:indexpattern-datasource-current-indexpattern", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "07f13cdd-3a86-40e5-914f-8f50c695b6ee:indexpattern-datasource-layer-8d4cd3ff-fd36-462e-ae82-826554dc847d", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "07f13cdd-3a86-40e5-914f-8f50c695b6ee:88df79e3-51ce-46c3-b8da-6522f6dc9e40", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "1fafffde-be8a-4e46-bc58-a52db1e94931:indexpattern-datasource-current-indexpattern", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "1fafffde-be8a-4e46-bc58-a52db1e94931:indexpattern-datasource-layer-a4c3d027-4533-411a-b9f1-26f0a4fedb66", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "ddcd6a80-5ab0-4522-b984-022b7da2d4b0:indexpattern-datasource-current-indexpattern", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "ddcd6a80-5ab0-4522-b984-022b7da2d4b0:indexpattern-datasource-layer-8a0016c8-0623-4e96-a007-240f0bfe88c2", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "44710442-b7b8-413a-9e52-4d7ba519a296:indexpattern-datasource-current-indexpattern", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "44710442-b7b8-413a-9e52-4d7ba519a296:indexpattern-datasource-layer-267ebe2d-c964-48cf-9c9a-1a1fb09f6e3c", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "747c1919-e215-4b97-9d8b-8ee528c1deaa:indexpattern-datasource-current-indexpattern", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "747c1919-e215-4b97-9d8b-8ee528c1deaa:indexpattern-datasource-layer-26e0acea-9274-411a-91a3-8537b1e00aff", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "aca1678c-d3d8-478e-a09c-dfdd86a5b3f7:indexpattern-datasource-current-indexpattern", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "aca1678c-d3d8-478e-a09c-dfdd86a5b3f7:indexpattern-datasource-layer-adde69bc-fda5-4560-8a54-202ca975652f", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "19b3fa09-6280-430a-9046-a613dfde3696:indexpattern-datasource-current-indexpattern", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "19b3fa09-6280-430a-9046-a613dfde3696:indexpattern-datasource-layer-0c7a50df-8359-42ff-806d-a22eb35b597a", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "185e6cd3-4cf8-45fd-937e-77abd9e6aad7:indexpattern-datasource-current-indexpattern", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "185e6cd3-4cf8-45fd-937e-77abd9e6aad7:indexpattern-datasource-layer-7bd679f9-8a5b-4906-beaa-750102e3a26f", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "7c6faaf4-5d0c-49a1-b1d2-605f18e675b0:indexpattern-datasource-current-indexpattern", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "7c6faaf4-5d0c-49a1-b1d2-605f18e675b0:indexpattern-datasource-layer-c8ea502e-ae28-47dd-9b90-484d50083243", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "889bb859-0938-46a4-b078-30f5fedd10a7:indexpattern-datasource-current-indexpattern", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "889bb859-0938-46a4-b078-30f5fedd10a7:indexpattern-datasource-layer-88c07c59-c625-4652-8156-54991d0869d8", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "c6560c58-be58-4718-abed-0356a2ba3b09:indexpattern-datasource-current-indexpattern", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "c6560c58-be58-4718-abed-0356a2ba3b09:indexpattern-datasource-layer-68e57e92-bad9-44bd-8022-16b46d218096", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "12736f17-d97c-4f4c-a66b-5eba7c2fec9c:kibanaSavedObjectMeta.searchSourceJSON.index", + "type": "index-pattern" + }, + { + "id": "darktrace-31a3f8a0-13a3-11ed-bdc1-9f13147efcf8", + "name": "e9b4f5f5-d478-403d-a78e-9e39ad3486f0:panel_e9b4f5f5-d478-403d-a78e-9e39ad3486f0", + "type": "search" + } + ], + "type": "dashboard" +} \ No newline at end of file diff --git a/packages/darktrace/kibana/dashboard/darktrace-eb643d20-13a5-11ed-bdc1-9f13147efcf8.json b/packages/darktrace/kibana/dashboard/darktrace-eb643d20-13a5-11ed-bdc1-9f13147efcf8.json new file mode 100644 index 00000000000..b0cec601476 --- /dev/null +++ b/packages/darktrace/kibana/dashboard/darktrace-eb643d20-13a5-11ed-bdc1-9f13147efcf8.json @@ -0,0 +1,981 @@ +{ + "attributes": { + "description": "Darktrace AI Analyst Alerts Overview.", + "hits": 0, + "kibanaSavedObjectMeta": { + "searchSourceJSON": { + "filter": [], + "query": { + "language": "kuery", + "query": "data_stream.dataset : \"darktrace.ai_analyst_alert\"" + } + } + }, + "optionsJSON": { + "hidePanelTitles": false, + "syncColors": false, + "useMargins": true + }, + "panelsJSON": [ + { + "embeddableConfig": { + "attributes": { + "references": [ + { + "id": "logs-*", + "name": "indexpattern-datasource-current-indexpattern", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "indexpattern-datasource-layer-1f84b818-192c-4dca-b929-1884e060576b", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "6f5adda0-d13e-48e5-aead-37e6448b922a", + "type": "index-pattern" + } + ], + "state": { + "datasourceStates": { + "indexpattern": { + "layers": { + "1f84b818-192c-4dca-b929-1884e060576b": { + "columnOrder": [ + "367e5418-6e25-45f2-b5fc-6ddd3618b869" + ], + "columns": { + "367e5418-6e25-45f2-b5fc-6ddd3618b869": { + "customLabel": true, + "dataType": "number", + "isBucketed": false, + "label": "Count", + "operationType": "unique_count", + "params": { + "emptyAsNull": true + }, + "scale": "ratio", + "sourceField": "event.id" + } + }, + "incompleteColumns": {} + } + } + } + }, + "filters": [ + { + "$state": { + "store": "appState" + }, + "meta": { + "alias": null, + "disabled": false, + "index": "6f5adda0-d13e-48e5-aead-37e6448b922a", + "key": "darktrace.ai_analyst_alert.is_user_triggered", + "negate": false, + "params": { + "query": true + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "darktrace.ai_analyst_alert.is_user_triggered": true + } + } + } + ], + "query": { + "language": "kuery", + "query": "data_stream.dataset : \"darktrace.ai_analyst_alert\"" + }, + "visualization": { + "accessor": "367e5418-6e25-45f2-b5fc-6ddd3618b869", + "layerId": "1f84b818-192c-4dca-b929-1884e060576b", + "layerType": "data" + } + }, + "title": "", + "type": "lens", + "visualizationType": "lnsMetric" + }, + "enhancements": {}, + "hidePanelTitles": false + }, + "gridData": { + "h": 13, + "i": "e28c7c69-2ae8-46fd-b361-38be020491a8", + "w": 16, + "x": 0, + "y": 0 + }, + "panelIndex": "e28c7c69-2ae8-46fd-b361-38be020491a8", + "title": "Count of User Triggered AI Analyst Investigation [Logs Darktrace]", + "type": "lens", + "version": "8.2.1" + }, + { + "embeddableConfig": { + "attributes": { + "references": [ + { + "id": "logs-*", + "name": "indexpattern-datasource-current-indexpattern", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "indexpattern-datasource-layer-1f84b818-192c-4dca-b929-1884e060576b", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "68dafc9f-9ed2-4ef9-8587-14dba4241364", + "type": "index-pattern" + } + ], + "state": { + "datasourceStates": { + "indexpattern": { + "layers": { + "1f84b818-192c-4dca-b929-1884e060576b": { + "columnOrder": [ + "367e5418-6e25-45f2-b5fc-6ddd3618b869" + ], + "columns": { + "367e5418-6e25-45f2-b5fc-6ddd3618b869": { + "customLabel": true, + "dataType": "number", + "isBucketed": false, + "label": "Count", + "operationType": "unique_count", + "params": { + "emptyAsNull": true + }, + "scale": "ratio", + "sourceField": "event.id" + } + }, + "incompleteColumns": {} + } + } + } + }, + "filters": [ + { + "$state": { + "store": "appState" + }, + "meta": { + "alias": null, + "disabled": false, + "index": "68dafc9f-9ed2-4ef9-8587-14dba4241364", + "key": "darktrace.ai_analyst_alert.is_external_triggered", + "negate": false, + "params": { + "query": true + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "darktrace.ai_analyst_alert.is_external_triggered": true + } + } + } + ], + "query": { + "language": "kuery", + "query": "data_stream.dataset : \"darktrace.ai_analyst_alert\"" + }, + "visualization": { + "accessor": "367e5418-6e25-45f2-b5fc-6ddd3618b869", + "layerId": "1f84b818-192c-4dca-b929-1884e060576b", + "layerType": "data" + } + }, + "title": "", + "type": "lens", + "visualizationType": "lnsMetric" + }, + "enhancements": {}, + "hidePanelTitles": false + }, + "gridData": { + "h": 13, + "i": "be1b9c5a-2ea0-48ac-8ad6-221769ff83f9", + "w": 16, + "x": 16, + "y": 0 + }, + "panelIndex": "be1b9c5a-2ea0-48ac-8ad6-221769ff83f9", + "title": "Count of Externally Triggered AI Analyst Investigation [Logs Darktrace]", + "type": "lens", + "version": "8.2.1" + }, + { + "embeddableConfig": { + "attributes": { + "references": [ + { + "id": "logs-*", + "name": "indexpattern-datasource-current-indexpattern", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "indexpattern-datasource-layer-1f84b818-192c-4dca-b929-1884e060576b", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "fb69f35b-439b-47fc-b942-15dc9d439f8b", + "type": "index-pattern" + } + ], + "state": { + "datasourceStates": { + "indexpattern": { + "layers": { + "1f84b818-192c-4dca-b929-1884e060576b": { + "columnOrder": [ + "367e5418-6e25-45f2-b5fc-6ddd3618b869" + ], + "columns": { + "367e5418-6e25-45f2-b5fc-6ddd3618b869": { + "customLabel": true, + "dataType": "number", + "isBucketed": false, + "label": "Count", + "operationType": "unique_count", + "params": { + "emptyAsNull": true + }, + "scale": "ratio", + "sourceField": "event.id" + } + }, + "incompleteColumns": {} + } + } + } + }, + "filters": [ + { + "$state": { + "store": "appState" + }, + "meta": { + "alias": null, + "disabled": false, + "index": "fb69f35b-439b-47fc-b942-15dc9d439f8b", + "key": "darktrace.ai_analyst_alert.is_acknowledged", + "negate": false, + "params": { + "query": true + }, + "type": "phrase" + }, + "query": { + "match_phrase": { + "darktrace.ai_analyst_alert.is_acknowledged": true + } + } + } + ], + "query": { + "language": "kuery", + "query": "data_stream.dataset : \"darktrace.ai_analyst_alert\"" + }, + "visualization": { + "accessor": "367e5418-6e25-45f2-b5fc-6ddd3618b869", + "layerId": "1f84b818-192c-4dca-b929-1884e060576b", + "layerType": "data" + } + }, + "title": "", + "type": "lens", + "visualizationType": "lnsMetric" + }, + "enhancements": {}, + "hidePanelTitles": false + }, + "gridData": { + "h": 13, + "i": "034d5870-b571-4276-9fad-1495a3665eed", + "w": 16, + "x": 32, + "y": 0 + }, + "panelIndex": "034d5870-b571-4276-9fad-1495a3665eed", + "title": "Count of Acknowledged AI Analyst Alerts [Logs Darktrace]", + "type": "lens", + "version": "8.2.1" + }, + { + "embeddableConfig": { + "attributes": { + "references": [ + { + "id": "logs-*", + "name": "indexpattern-datasource-current-indexpattern", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "indexpattern-datasource-layer-66afac91-ca1e-4a4a-ab0d-e18a2903ace7", + "type": "index-pattern" + } + ], + "state": { + "datasourceStates": { + "indexpattern": { + "layers": { + "66afac91-ca1e-4a4a-ab0d-e18a2903ace7": { + "columnOrder": [ + "6e2b5d5b-0584-412f-a87f-b60279d2173d", + "8f546d14-cc1d-4d80-8cec-8e326bfd19d1" + ], + "columns": { + "6e2b5d5b-0584-412f-a87f-b60279d2173d": { + "customLabel": true, + "dataType": "string", + "isBucketed": true, + "label": "Behavior Category", + "operationType": "terms", + "params": { + "missingBucket": false, + "orderBy": { + "columnId": "8f546d14-cc1d-4d80-8cec-8e326bfd19d1", + "type": "column" + }, + "orderDirection": "desc", + "otherBucket": true, + "parentFormat": { + "id": "terms" + }, + "size": 5 + }, + "scale": "ordinal", + "sourceField": "darktrace.ai_analyst_alert.category" + }, + "8f546d14-cc1d-4d80-8cec-8e326bfd19d1": { + "customLabel": true, + "dataType": "number", + "isBucketed": false, + "label": "Count", + "operationType": "unique_count", + "params": { + "emptyAsNull": true + }, + "scale": "ratio", + "sourceField": "event.id" + } + }, + "incompleteColumns": {} + } + } + } + }, + "filters": [], + "query": { + "language": "kuery", + "query": "data_stream.dataset : \"darktrace.ai_analyst_alert\"" + }, + "visualization": { + "layers": [ + { + "categoryDisplay": "default", + "groups": [ + "6e2b5d5b-0584-412f-a87f-b60279d2173d" + ], + "layerId": "66afac91-ca1e-4a4a-ab0d-e18a2903ace7", + "layerType": "data", + "legendDisplay": "default", + "metric": "8f546d14-cc1d-4d80-8cec-8e326bfd19d1", + "nestedLegend": false, + "numberDisplay": "percent" + } + ], + "shape": "pie" + } + }, + "title": "", + "type": "lens", + "visualizationType": "lnsPie" + }, + "enhancements": {}, + "hidePanelTitles": false + }, + "gridData": { + "h": 15, + "i": "65f35405-87eb-4a98-a0c2-2e3c7426cb28", + "w": 24, + "x": 0, + "y": 13 + }, + "panelIndex": "65f35405-87eb-4a98-a0c2-2e3c7426cb28", + "title": "Distribution of AI Analyst Alerts by Behavior Category [Logs Darktrace]", + "type": "lens", + "version": "8.2.1" + }, + { + "embeddableConfig": { + "attributes": { + "references": [ + { + "id": "logs-*", + "name": "indexpattern-datasource-current-indexpattern", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "indexpattern-datasource-layer-effe003f-604a-49a3-a903-d4d2c75df944", + "type": "index-pattern" + } + ], + "state": { + "datasourceStates": { + "indexpattern": { + "layers": { + "effe003f-604a-49a3-a903-d4d2c75df944": { + "columnOrder": [ + "937bae71-7159-4e35-87cf-dc372875ad59", + "049804ee-f3a4-474f-8e76-d4c3e0eb77af" + ], + "columns": { + "049804ee-f3a4-474f-8e76-d4c3e0eb77af": { + "customLabel": true, + "dataType": "number", + "isBucketed": false, + "label": "Count", + "operationType": "unique_count", + "params": { + "emptyAsNull": true + }, + "scale": "ratio", + "sourceField": "event.id" + }, + "937bae71-7159-4e35-87cf-dc372875ad59": { + "customLabel": true, + "dataType": "string", + "isBucketed": true, + "label": "Summariser", + "operationType": "terms", + "params": { + "missingBucket": false, + "orderBy": { + "columnId": "049804ee-f3a4-474f-8e76-d4c3e0eb77af", + "type": "column" + }, + "orderDirection": "desc", + "otherBucket": true, + "parentFormat": { + "id": "terms" + }, + "size": 10 + }, + "scale": "ordinal", + "sourceField": "darktrace.ai_analyst_alert.summariser" + } + }, + "incompleteColumns": {} + } + } + } + }, + "filters": [], + "query": { + "language": "kuery", + "query": "data_stream.dataset : \"darktrace.ai_analyst_alert\"" + }, + "visualization": { + "layers": [ + { + "accessors": [ + "049804ee-f3a4-474f-8e76-d4c3e0eb77af" + ], + "layerId": "effe003f-604a-49a3-a903-d4d2c75df944", + "layerType": "data", + "position": "top", + "seriesType": "bar_stacked", + "showGridlines": false, + "xAccessor": "937bae71-7159-4e35-87cf-dc372875ad59" + } + ], + "legend": { + "isVisible": true, + "position": "right" + }, + "preferredSeriesType": "bar_stacked", + "title": "Empty XY chart", + "valueLabels": "hide" + } + }, + "title": "", + "type": "lens", + "visualizationType": "lnsXY" + }, + "enhancements": {}, + "hidePanelTitles": false + }, + "gridData": { + "h": 15, + "i": "8882d78e-7df8-4d33-b7b5-e21f5d25dfe7", + "w": 24, + "x": 24, + "y": 13 + }, + "panelIndex": "8882d78e-7df8-4d33-b7b5-e21f5d25dfe7", + "title": "Distribution of AI Analyst Alerts by Summariser [Logs Darktrace]", + "type": "lens", + "version": "8.2.1" + }, + { + "embeddableConfig": { + "attributes": { + "references": [ + { + "id": "logs-*", + "name": "indexpattern-datasource-current-indexpattern", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "indexpattern-datasource-layer-dea45bd8-269e-48c4-98d3-fc47717ae139", + "type": "index-pattern" + } + ], + "state": { + "datasourceStates": { + "indexpattern": { + "layers": { + "dea45bd8-269e-48c4-98d3-fc47717ae139": { + "columnOrder": [ + "71a3581e-24ae-48d8-958d-c574488b2f48", + "e6083dcb-9465-4007-a133-569f31fe732d" + ], + "columns": { + "71a3581e-24ae-48d8-958d-c574488b2f48": { + "customLabel": true, + "dataType": "string", + "isBucketed": true, + "label": "Alert Title", + "operationType": "terms", + "params": { + "missingBucket": false, + "orderBy": { + "columnId": "e6083dcb-9465-4007-a133-569f31fe732d", + "type": "column" + }, + "orderDirection": "desc", + "otherBucket": false, + "parentFormat": { + "id": "terms" + }, + "secondaryFields": [], + "size": 10 + }, + "scale": "ordinal", + "sourceField": "event.reason" + }, + "e6083dcb-9465-4007-a133-569f31fe732d": { + "customLabel": true, + "dataType": "number", + "isBucketed": false, + "label": "Risk Score", + "operationType": "max", + "params": { + "emptyAsNull": true + }, + "scale": "ratio", + "sourceField": "event.risk_score" + } + }, + "incompleteColumns": {} + } + } + } + }, + "filters": [], + "query": { + "language": "kuery", + "query": "data_stream.dataset : \"darktrace.ai_analyst_alert\"" + }, + "visualization": { + "columns": [ + { + "alignment": "left", + "columnId": "71a3581e-24ae-48d8-958d-c574488b2f48", + "isTransposed": false + }, + { + "columnId": "e6083dcb-9465-4007-a133-569f31fe732d", + "isTransposed": false + } + ], + "layerId": "dea45bd8-269e-48c4-98d3-fc47717ae139", + "layerType": "data" + } + }, + "title": "", + "type": "lens", + "visualizationType": "lnsDatatable" + }, + "enhancements": {}, + "hidePanelTitles": false + }, + "gridData": { + "h": 15, + "i": "6b003410-fd00-4dc5-b9c7-8bd1f711ffbe", + "w": 24, + "x": 0, + "y": 28 + }, + "panelIndex": "6b003410-fd00-4dc5-b9c7-8bd1f711ffbe", + "title": "Top 10 AI Analyst Alerts with Highest Score [Logs Darktrace]", + "type": "lens", + "version": "8.2.1" + }, + { + "embeddableConfig": { + "attributes": { + "references": [ + { + "id": "logs-*", + "name": "indexpattern-datasource-current-indexpattern", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "indexpattern-datasource-layer-3bb3b1dd-30aa-46d6-8a14-32c14c706f47", + "type": "index-pattern" + } + ], + "state": { + "datasourceStates": { + "indexpattern": { + "layers": { + "3bb3b1dd-30aa-46d6-8a14-32c14c706f47": { + "columnOrder": [ + "266f7f3a-5f46-40c5-a716-b2aab1d49d51", + "36e2011c-141b-412b-a5fd-e5e9c62183ad" + ], + "columns": { + "266f7f3a-5f46-40c5-a716-b2aab1d49d51": { + "customLabel": true, + "dataType": "string", + "isBucketed": true, + "label": "Hostname", + "operationType": "terms", + "params": { + "missingBucket": false, + "orderBy": { + "columnId": "36e2011c-141b-412b-a5fd-e5e9c62183ad", + "type": "column" + }, + "orderDirection": "desc", + "otherBucket": false, + "parentFormat": { + "id": "terms" + }, + "size": 10 + }, + "scale": "ordinal", + "sourceField": "host.hostname" + }, + "36e2011c-141b-412b-a5fd-e5e9c62183ad": { + "customLabel": true, + "dataType": "number", + "isBucketed": false, + "label": "Count", + "operationType": "unique_count", + "params": { + "emptyAsNull": true + }, + "scale": "ratio", + "sourceField": "event.id" + } + }, + "incompleteColumns": {} + } + } + } + }, + "filters": [], + "query": { + "language": "kuery", + "query": "data_stream.dataset : \"darktrace.ai_analyst_alert\"" + }, + "visualization": { + "columns": [ + { + "columnId": "266f7f3a-5f46-40c5-a716-b2aab1d49d51", + "isTransposed": false + }, + { + "columnId": "36e2011c-141b-412b-a5fd-e5e9c62183ad", + "isTransposed": false + } + ], + "layerId": "3bb3b1dd-30aa-46d6-8a14-32c14c706f47", + "layerType": "data" + } + }, + "title": "", + "type": "lens", + "visualizationType": "lnsDatatable" + }, + "enhancements": {}, + "hidePanelTitles": false + }, + "gridData": { + "h": 15, + "i": "930d2983-f872-4001-ba45-b44aee791167", + "w": 24, + "x": 24, + "y": 28 + }, + "panelIndex": "930d2983-f872-4001-ba45-b44aee791167", + "title": "Top 10 BreachDevices Hostname with Highest AI Analyst Alerts [Logs Darktrace]", + "type": "lens", + "version": "8.2.1" + }, + { + "embeddableConfig": { + "attributes": { + "references": [ + { + "id": "logs-*", + "name": "indexpattern-datasource-current-indexpattern", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "indexpattern-datasource-layer-9eda772e-1fbd-4296-a543-8bbd18b2359a", + "type": "index-pattern" + } + ], + "state": { + "datasourceStates": { + "indexpattern": { + "layers": { + "9eda772e-1fbd-4296-a543-8bbd18b2359a": { + "columnOrder": [ + "45d996c5-b696-4fff-8f83-78473cc7798f", + "252b0567-f0b1-4677-b6d8-e9d7a229431a" + ], + "columns": { + "252b0567-f0b1-4677-b6d8-e9d7a229431a": { + "customLabel": true, + "dataType": "number", + "isBucketed": false, + "label": "Group Score", + "operationType": "max", + "params": { + "emptyAsNull": true + }, + "scale": "ratio", + "sourceField": "darktrace.ai_analyst_alert.group_score" + }, + "45d996c5-b696-4fff-8f83-78473cc7798f": { + "customLabel": true, + "dataType": "string", + "isBucketed": true, + "label": "Alert Title", + "operationType": "terms", + "params": { + "missingBucket": false, + "orderBy": { + "columnId": "252b0567-f0b1-4677-b6d8-e9d7a229431a", + "type": "column" + }, + "orderDirection": "desc", + "otherBucket": false, + "parentFormat": { + "id": "terms" + }, + "secondaryFields": [], + "size": 10 + }, + "scale": "ordinal", + "sourceField": "event.reason" + } + }, + "incompleteColumns": {} + } + } + } + }, + "filters": [], + "query": { + "language": "kuery", + "query": "data_stream.dataset : \"darktrace.ai_analyst_alert\"" + }, + "visualization": { + "columns": [ + { + "columnId": "252b0567-f0b1-4677-b6d8-e9d7a229431a", + "isTransposed": false + }, + { + "columnId": "45d996c5-b696-4fff-8f83-78473cc7798f", + "isTransposed": false, + "width": 551 + } + ], + "layerId": "9eda772e-1fbd-4296-a543-8bbd18b2359a", + "layerType": "data" + } + }, + "title": "", + "type": "lens", + "visualizationType": "lnsDatatable" + }, + "enhancements": {}, + "hidePanelTitles": false + }, + "gridData": { + "h": 18, + "i": "7e4d0098-0cc8-403d-aaca-92758d697950", + "w": 48, + "x": 0, + "y": 43 + }, + "panelIndex": "7e4d0098-0cc8-403d-aaca-92758d697950", + "title": "Top 10 AI Analyst Alerts with Highest Group Score [Logs Darktrace]", + "type": "lens", + "version": "8.2.1" + }, + { + "embeddableConfig": { + "enhancements": {} + }, + "gridData": { + "h": 23, + "i": "4ce4eb50-af35-423a-b20f-61a715aa4388", + "w": 48, + "x": 0, + "y": 61 + }, + "panelIndex": "4ce4eb50-af35-423a-b20f-61a715aa4388", + "panelRefName": "panel_4ce4eb50-af35-423a-b20f-61a715aa4388", + "type": "search", + "version": "8.2.1" + } + ], + "timeRestore": false, + "title": "[Logs Darktrace] AI Analyst Alerts Overview", + "version": 1 + }, + "coreMigrationVersion": "8.2.1", + "id": "darktrace-eb643d20-13a5-11ed-bdc1-9f13147efcf8", + "migrationVersion": { + "dashboard": "8.2.0" + }, + "references": [ + { + "id": "logs-*", + "name": "e28c7c69-2ae8-46fd-b361-38be020491a8:indexpattern-datasource-current-indexpattern", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "e28c7c69-2ae8-46fd-b361-38be020491a8:indexpattern-datasource-layer-1f84b818-192c-4dca-b929-1884e060576b", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "e28c7c69-2ae8-46fd-b361-38be020491a8:6f5adda0-d13e-48e5-aead-37e6448b922a", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "be1b9c5a-2ea0-48ac-8ad6-221769ff83f9:indexpattern-datasource-current-indexpattern", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "be1b9c5a-2ea0-48ac-8ad6-221769ff83f9:indexpattern-datasource-layer-1f84b818-192c-4dca-b929-1884e060576b", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "be1b9c5a-2ea0-48ac-8ad6-221769ff83f9:68dafc9f-9ed2-4ef9-8587-14dba4241364", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "034d5870-b571-4276-9fad-1495a3665eed:indexpattern-datasource-current-indexpattern", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "034d5870-b571-4276-9fad-1495a3665eed:indexpattern-datasource-layer-1f84b818-192c-4dca-b929-1884e060576b", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "034d5870-b571-4276-9fad-1495a3665eed:fb69f35b-439b-47fc-b942-15dc9d439f8b", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "65f35405-87eb-4a98-a0c2-2e3c7426cb28:indexpattern-datasource-current-indexpattern", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "65f35405-87eb-4a98-a0c2-2e3c7426cb28:indexpattern-datasource-layer-66afac91-ca1e-4a4a-ab0d-e18a2903ace7", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "8882d78e-7df8-4d33-b7b5-e21f5d25dfe7:indexpattern-datasource-current-indexpattern", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "8882d78e-7df8-4d33-b7b5-e21f5d25dfe7:indexpattern-datasource-layer-effe003f-604a-49a3-a903-d4d2c75df944", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "6b003410-fd00-4dc5-b9c7-8bd1f711ffbe:indexpattern-datasource-current-indexpattern", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "6b003410-fd00-4dc5-b9c7-8bd1f711ffbe:indexpattern-datasource-layer-dea45bd8-269e-48c4-98d3-fc47717ae139", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "930d2983-f872-4001-ba45-b44aee791167:indexpattern-datasource-current-indexpattern", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "930d2983-f872-4001-ba45-b44aee791167:indexpattern-datasource-layer-3bb3b1dd-30aa-46d6-8a14-32c14c706f47", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "7e4d0098-0cc8-403d-aaca-92758d697950:indexpattern-datasource-current-indexpattern", + "type": "index-pattern" + }, + { + "id": "logs-*", + "name": "7e4d0098-0cc8-403d-aaca-92758d697950:indexpattern-datasource-layer-9eda772e-1fbd-4296-a543-8bbd18b2359a", + "type": "index-pattern" + }, + { + "id": "darktrace-c0e40350-13aa-11ed-bdc1-9f13147efcf8", + "name": "4ce4eb50-af35-423a-b20f-61a715aa4388:panel_4ce4eb50-af35-423a-b20f-61a715aa4388", + "type": "search" + } + ], + "type": "dashboard" +} \ No newline at end of file diff --git a/packages/darktrace/kibana/search/darktrace-31a3f8a0-13a3-11ed-bdc1-9f13147efcf8.json b/packages/darktrace/kibana/search/darktrace-31a3f8a0-13a3-11ed-bdc1-9f13147efcf8.json new file mode 100644 index 00000000000..b520d9b6d92 --- /dev/null +++ b/packages/darktrace/kibana/search/darktrace-31a3f8a0-13a3-11ed-bdc1-9f13147efcf8.json @@ -0,0 +1,44 @@ +{ + "attributes": { + "columns": [ + "darktrace.model_breach_alert.pbid", + "rule.category", + "rule.name", + "event.risk_score", + "host.id" + ], + "description": "", + "grid": {}, + "hideChart": false, + "kibanaSavedObjectMeta": { + "searchSourceJSON": { + "filter": [], + "indexRefName": "kibanaSavedObjectMeta.searchSourceJSON.index", + "query": { + "language": "kuery", + "query": "data_stream.dataset : \"darktrace.model_breach_alert\"" + } + } + }, + "sort": [ + [ + "@timestamp", + "desc" + ] + ], + "title": "Model Breach Alerts Essential Details [Logs Darktrace]" + }, + "coreMigrationVersion": "8.2.1", + "id": "darktrace-31a3f8a0-13a3-11ed-bdc1-9f13147efcf8", + "migrationVersion": { + "search": "8.0.0" + }, + "references": [ + { + "id": "logs-*", + "name": "kibanaSavedObjectMeta.searchSourceJSON.index", + "type": "index-pattern" + } + ], + "type": "search" +} \ No newline at end of file diff --git a/packages/darktrace/kibana/search/darktrace-c0e40350-13aa-11ed-bdc1-9f13147efcf8.json b/packages/darktrace/kibana/search/darktrace-c0e40350-13aa-11ed-bdc1-9f13147efcf8.json new file mode 100644 index 00000000000..726cefb46b2 --- /dev/null +++ b/packages/darktrace/kibana/search/darktrace-c0e40350-13aa-11ed-bdc1-9f13147efcf8.json @@ -0,0 +1,44 @@ +{ + "attributes": { + "columns": [ + "event.id", + "event.reason", + "darktrace.ai_analyst_alert.related_breaches.pbid", + "darktrace.ai_analyst_alert.attack_phases", + "event.risk_score" + ], + "description": "", + "grid": {}, + "hideChart": false, + "kibanaSavedObjectMeta": { + "searchSourceJSON": { + "filter": [], + "indexRefName": "kibanaSavedObjectMeta.searchSourceJSON.index", + "query": { + "language": "kuery", + "query": "data_stream.dataset : \"darktrace.ai_analyst_alert\"" + } + } + }, + "sort": [ + [ + "@timestamp", + "desc" + ] + ], + "title": "AI Analyst Alerts Essential Details [Logs Darktrace]" + }, + "coreMigrationVersion": "8.2.1", + "id": "darktrace-c0e40350-13aa-11ed-bdc1-9f13147efcf8", + "migrationVersion": { + "search": "8.0.0" + }, + "references": [ + { + "id": "logs-*", + "name": "kibanaSavedObjectMeta.searchSourceJSON.index", + "type": "index-pattern" + } + ], + "type": "search" +} \ No newline at end of file diff --git a/packages/darktrace/kibana/search/darktrace-fbf9cfc0-13b3-11ed-bdc1-9f13147efcf8.json b/packages/darktrace/kibana/search/darktrace-fbf9cfc0-13b3-11ed-bdc1-9f13147efcf8.json new file mode 100644 index 00000000000..49cb80e2e20 --- /dev/null +++ b/packages/darktrace/kibana/search/darktrace-fbf9cfc0-13b3-11ed-bdc1-9f13147efcf8.json @@ -0,0 +1,45 @@ +{ + "attributes": { + "columns": [ + "event.id", + "darktrace.system_status_alert.last_updated_status", + "host.ip", + "darktrace.system_status_alert.alert_name", + "event.risk_score", + "darktrace.system_status_alert.status" + ], + "description": "", + "grid": {}, + "hideChart": false, + "kibanaSavedObjectMeta": { + "searchSourceJSON": { + "filter": [], + "indexRefName": "kibanaSavedObjectMeta.searchSourceJSON.index", + "query": { + "language": "kuery", + "query": "data_stream.dataset : \"darktrace.system_status_alert\"" + } + } + }, + "sort": [ + [ + "@timestamp", + "desc" + ] + ], + "title": "System Status Alerts Essential Details [Logs Darktrace]" + }, + "coreMigrationVersion": "8.2.1", + "id": "darktrace-fbf9cfc0-13b3-11ed-bdc1-9f13147efcf8", + "migrationVersion": { + "search": "8.0.0" + }, + "references": [ + { + "id": "logs-*", + "name": "kibanaSavedObjectMeta.searchSourceJSON.index", + "type": "index-pattern" + } + ], + "type": "search" +} \ No newline at end of file diff --git a/packages/darktrace/manifest.yml b/packages/darktrace/manifest.yml new file mode 100644 index 00000000000..c3b0197f6b7 --- /dev/null +++ b/packages/darktrace/manifest.yml @@ -0,0 +1,136 @@ +format_version: 1.0.0 +name: darktrace +title: Darktrace +version: 0.1.0 +license: basic +description: Collect logs from Darktrace with Elastic Agent. +type: integration +categories: + - security +conditions: + kibana.version: ^8.2.1 +screenshots: + - src: /img/darktrace-screenshot.png + title: Darktrace Model Breach Alert Dashboard Screenshot + size: 600x600 + type: image/png +icons: + - src: /img/darktrace-logo.svg + title: Darktrace Logo + size: 32x32 + type: image/svg+xml +policy_templates: + - name: darktrace + title: Darktrace logs + description: Collect logs from Darktrace. + inputs: + - type: httpjson + title: Collect Darktrace logs via API + description: Collecting Darktrace logs via API. + vars: + - name: url + type: text + title: URL + description: Darktrace console URL. + required: true + - name: public_token + type: password + title: Public API Token + description: Public API Token. + required: true + - name: private_token + type: password + title: Private API Token + description: Private API Token. + required: true + - name: proxy_url + type: text + title: Proxy URL + multi: false + required: false + show_user: false + description: URL to proxy connections in the form of http[s]://:@:. Please ensure your username and password are in URL encoded format. + - name: ssl + type: yaml + title: SSL Configuration + description: i.e. certificate_authorities, supported_protocols, verification_mode etc. + multi: false + required: false + show_user: false + default: | + #certificate_authorities: + # - | + # -----BEGIN CERTIFICATE----- + # MIIDCjCCAfKgAwIBAgITJ706Mu2wJlKckpIvkWxEHvEyijANBgkqhkiG9w0BAQsF + # ADAUMRIwEAYDVQQDDAlsb2NhbGhvc3QwIBcNMTkwNzIyMTkyOTA0WhgPMjExOTA2 + # MjgxOTI5MDRaMBQxEjAQBgNVBAMMCWxvY2FsaG9zdDCCASIwDQYJKoZIhvcNAQEB + # BQADggEPADCCAQoCggEBANce58Y/JykI58iyOXpxGfw0/gMvF0hUQAcUrSMxEO6n + # fZRA49b4OV4SwWmA3395uL2eB2NB8y8qdQ9muXUdPBWE4l9rMZ6gmfu90N5B5uEl + # 94NcfBfYOKi1fJQ9i7WKhTjlRkMCgBkWPkUokvBZFRt8RtF7zI77BSEorHGQCk9t + # /D7BS0GJyfVEhftbWcFEAG3VRcoMhF7kUzYwp+qESoriFRYLeDWv68ZOvG7eoWnP + # PsvZStEVEimjvK5NSESEQa9xWyJOmlOKXhkdymtcUd/nXnx6UTCFgnkgzSdTWV41 + # CI6B6aJ9svCTI2QuoIq2HxX/ix7OvW1huVmcyHVxyUECAwEAAaNTMFEwHQYDVR0O + # BBYEFPwN1OceFGm9v6ux8G+DZ3TUDYxqMB8GA1UdIwQYMBaAFPwN1OceFGm9v6ux + # 8G+DZ3TUDYxqMA8GA1UdEwEB/wQFMAMBAf8wDQYJKoZIhvcNAQELBQADggEBAG5D + # 874A4YI7YUwOVsVAdbWtgp1d0zKcPRR+r2OdSbTAV5/gcS3jgBJ3i1BN34JuDVFw + # 3DeJSYT3nxy2Y56lLnxDeF8CUTUtVQx3CuGkRg1ouGAHpO/6OqOhwLLorEmxi7tA + # H2O8mtT0poX5AnOAhzVy7QW0D/k4WaoLyckM5hUa6RtvgvLxOwA0U+VGurCDoctu + # 8F4QOgTAWyh8EZIwaKCliFRSynDpv3JTUwtfZkxo6K6nce1RhCWFAsMvDZL8Dgc0 + # yvgJ38BRsFOtkRuAGSf6ZUwTO8JJRRIFnpUzXflAnGivK9M13D5GEQMmIl6U9Pvk + # sxSmbIUfc2SGJGCJD4I= + # -----END CERTIFICATE----- + - type: tcp + title: Collect Darktrace logs via TCP + description: Collecting Darktrace logs via TCP. + vars: + - name: listen_address + type: text + title: Listen Address + description: The bind address to listen for TCP connections. Set to `0.0.0.0` to bind to all available interfaces. + multi: false + required: true + show_user: true + default: localhost + - name: ssl + type: yaml + title: SSL Configuration + description: i.e. certificate_authorities, supported_protocols, verification_mode etc. + multi: false + required: false + show_user: false + default: | + #certificate_authorities: + # - | + # -----BEGIN CERTIFICATE----- + # MIIDCjCCAfKgAwIBAgITJ706Mu2wJlKckpIvkWxEHvEyijANBgkqhkiG9w0BAQsF + # ADAUMRIwEAYDVQQDDAlsb2NhbGhvc3QwIBcNMTkwNzIyMTkyOTA0WhgPMjExOTA2 + # MjgxOTI5MDRaMBQxEjAQBgNVBAMMCWxvY2FsaG9zdDCCASIwDQYJKoZIhvcNAQEB + # BQADggEPADCCAQoCggEBANce58Y/JykI58iyOXpxGfw0/gMvF0hUQAcUrSMxEO6n + # fZRA49b4OV4SwWmA3395uL2eB2NB8y8qdQ9muXUdPBWE4l9rMZ6gmfu90N5B5uEl + # 94NcfBfYOKi1fJQ9i7WKhTjlRkMCgBkWPkUokvBZFRt8RtF7zI77BSEorHGQCk9t + # /D7BS0GJyfVEhftbWcFEAG3VRcoMhF7kUzYwp+qESoriFRYLeDWv68ZOvG7eoWnP + # PsvZStEVEimjvK5NSESEQa9xWyJOmlOKXhkdymtcUd/nXnx6UTCFgnkgzSdTWV41 + # CI6B6aJ9svCTI2QuoIq2HxX/ix7OvW1huVmcyHVxyUECAwEAAaNTMFEwHQYDVR0O + # BBYEFPwN1OceFGm9v6ux8G+DZ3TUDYxqMB8GA1UdIwQYMBaAFPwN1OceFGm9v6ux + # 8G+DZ3TUDYxqMA8GA1UdEwEB/wQFMAMBAf8wDQYJKoZIhvcNAQELBQADggEBAG5D + # 874A4YI7YUwOVsVAdbWtgp1d0zKcPRR+r2OdSbTAV5/gcS3jgBJ3i1BN34JuDVFw + # 3DeJSYT3nxy2Y56lLnxDeF8CUTUtVQx3CuGkRg1ouGAHpO/6OqOhwLLorEmxi7tA + # H2O8mtT0poX5AnOAhzVy7QW0D/k4WaoLyckM5hUa6RtvgvLxOwA0U+VGurCDoctu + # 8F4QOgTAWyh8EZIwaKCliFRSynDpv3JTUwtfZkxo6K6nce1RhCWFAsMvDZL8Dgc0 + # yvgJ38BRsFOtkRuAGSf6ZUwTO8JJRRIFnpUzXflAnGivK9M13D5GEQMmIl6U9Pvk + # sxSmbIUfc2SGJGCJD4I= + # -----END CERTIFICATE----- + - type: udp + title: Collect Darktrace logs via UDP + description: Collecting Darktrace logs via UDP. + vars: + - name: listen_address + type: text + title: Listen Address + description: The bind address to listen for UDP connections. Set to `0.0.0.0` to bind to all available interfaces. + multi: false + required: true + show_user: true + default: localhost +owner: + github: elastic/security-external-integrations