[mimecast] Cleanup field mappings and package description

packages/mimecast/_dev/build/docs/README.md

@@ -4,73 +4,111 @@ The Mimecast integration collects events from the Mimecast API.
 
 ## Configuration
 
-Authorization parameters for the Mimecast API (`Application Key`, `Application ID`, `Access Key`, and `Secret Key`), should be provided by a Mimecast representative for this integration.
-Under `Advanced options` you can set the time interval between two API requests as well as the API URL. A Mimecast representative should also be able to give you with this information in case you need to change the defaults. 
+Authorization parameters for the Mimecast API (`Application Key`, `Application
+ID`, `Access Key`, and `Secret Key`) should be provided by a Mimecast
+representative for this integration. Under `Advanced options` you can set the
+time interval between two API requests as well as the API URL. A Mimecast
+representative should also be able to give you this information in case you need
+to change the defaults.
 
-Note that rate limit quotas may require you to set up different credentials for the different available log types.
+Note that rate limit quotas may require you to set up different credentials for
+the different available log types.
 
 ## Logs
 
 ### Audit Events
 
-This is the `mimecast.audit_events` dataset. These logs contain Mimecast audit events with the following details: audit type, event category and detailed information about the event. More information about these logs [here] (https://integrations.mimecast.com/documentation/endpoint-reference/logs-and-statistics/get-audit-events/).
+This is the `mimecast.audit_events` dataset. These logs contain Mimecast audit
+events with the following details: audit type, event category, and detailed
+information about the event. More information about these logs [here]
+(https://integrations.mimecast.com/documentation/endpoint-reference/logs-and-statistics/get-audit-events/).
 
 {{event "audit_events"}}
 
 {{fields "audit_events"}}
 
 ### DLP Logs
 
-This is the `mimecast.dlp_logs` dataset. These logs contain information about messages that triggered a DLP or Content Examination policy. More information about these logs [here] (https://integrations.mimecast.com/documentation/endpoint-reference/logs-and-statistics/get-dlp-logs/). 
+This is the `mimecast.dlp_logs` dataset. These logs contain information about
+messages that triggered a DLP or Content Examination policy. More information
+about these logs [here]
+(https://integrations.mimecast.com/documentation/endpoint-reference/logs-and-statistics/get-dlp-logs/).
 
 {{event "dlp_logs"}}
 
 {{fields "dlp_logs"}}
 
 ### SIEM Logs
 
-This is the `mimecast.siem_logs` dataset. These logs contain information about messages that contains MTA logs (MTA = message transfer agent) – all Inbound, outbound and internal messages. More about these logs [here](https://integrations.mimecast.com/documentation/tutorials/understanding-siem-logs/).
+This is the `mimecast.siem_logs` dataset. These logs contain information about
+messages that contains MTA (message transfer agent) log – all inbound,
+outbound, and internal messages. More about these logs
+[here](https://integrations.mimecast.com/documentation/tutorials/understanding-siem-logs/).
 
 {{event "siem_logs"}}
 
 {{fields "siem_logs"}}
 
-### TTP Impersonation Logs
+### Threat Intel Feed Malware: Customer
 
-This is the `mimecast.ttp_ip_logs` dataset. These logs contain information about messages containing information flagged by an Impersonation Protection configuration. Learn more about these logs [here] (https://integrations.mimecast.com/documentation/endpoint-reference/logs-and-statistics/get-ttp-impersonation-protect-logs/). 
+This is the `mimecast.threat_intel_malware_customer` dataset. These logs contain
+information about messages that return identified malware threats at a customer
+level. More about these logs
+[here](https://integrations.mimecast.com/documentation/endpoint-reference/threat-intel/get-feed/).
 
-{{event "ttp_ip_logs"}}
+{{event "threat_intel_malware_customer"}}
 
-{{fields "ttp_ip_logs"}}
+{{fields "threat_intel_malware_customer"}}
 
-### TTP Attachment Logs
+### Threat Intel Feed Malware: Grid
 
-This is the `mimecast.ttp_ap_logs` dataset. These logs contain Mimecast TTP attachment protection logs with the following details: result of attachment analysis (if it is malicious or not etc.), date when file is released, sender and recipient address, filename and type, action triggered for the attachment, the route of the original email containing the attachment and details. Learn more about these logs [here] (https://integrations.mimecast.com/documentation/endpoint-reference/logs-and-statistics/get-ttp-attachment-protection-logs/).
+This is the `mimecast.threat_intel_malware_grid` dataset. These logs contain
+information about messages that return identified malware threats at a regional
+grid level. More about these logs
+[here](https://integrations.mimecast.com/documentation/endpoint-reference/threat-intel/get-feed/).
 
-{{event "ttp_ap_logs"}}
+{{event "threat_intel_malware_grid"}}
 
-{{fields "ttp_ap_logs"}}
+{{fields "threat_intel_malware_grid"}}
 
-### TTP URL Logs
+### TTP Attachment Logs
 
-This is the `mimecast.ttp_url_logs` dataset. These logs contain Mimecast TTP attachment protection logs with the following details: the category of the URL clicked, the email address of the user who clicked the link, the url clicked, the action taken by the user if user awareness was applied, the route of the email that contained the link, the action defined by the administrator for the URL, the date that the URL was clicked, url scan result, the action that was taken for the click, the description of the definition that triggered the URL to be rewritten by Mimecast, the action requested by the user, an array of components of the message where the URL was found. More about these logs [here](https://integrations.mimecast.com/documentation/endpoint-reference/logs-and-statistics/get-ttp-url-logs/). 
+This is the `mimecast.ttp_ap_logs` dataset. These logs contain Mimecast TTP
+attachment protection logs with the following details: result of attachment
+analysis (if it is malicious or not etc.), date when file is released, sender
+and recipient address, filename and type, action triggered for the attachment,
+the route of the original email containing the attachment and details. Learn
+more about these logs [here]
+(https://integrations.mimecast.com/documentation/endpoint-reference/logs-and-statistics/get-ttp-attachment-protection-logs/).
 
-{{event "ttp_url_logs"}}
+{{event "ttp_ap_logs"}}
 
-{{fields "ttp_url_logs"}}
+{{fields "ttp_ap_logs"}}
 
-### Threat Intel Feed Malware: Customer
+### TTP Impersonation Logs
 
-This is the `mimecast.threat_intel_malware_customer` dataset. These logs contain information about messages that return identified malware threats at a customer level. More about these logs [here](https://integrations.mimecast.com/documentation/endpoint-reference/threat-intel/get-feed/). 
+This is the `mimecast.ttp_ip_logs` dataset. These logs contain information about
+messages containing information flagged by an Impersonation Protection
+configuration. Learn more about these logs [here]
+(https://integrations.mimecast.com/documentation/endpoint-reference/logs-and-statistics/get-ttp-impersonation-protect-logs/).
 
-{{event "threat_intel_malware_customer"}}
+{{event "ttp_ip_logs"}}
 
-{{fields "threat_intel_malware_customer"}}
+{{fields "ttp_ip_logs"}}
 
-### Threat Intel Feed Malware: Grid
+### TTP URL Logs
 
-This is the `mimecast.threat_intel_malware_grid` dataset. These logs contain information about messages that return identified malware threats at a regional grid level. More about these logs [here](https://integrations.mimecast.com/documentation/endpoint-reference/threat-intel/get-feed/). 
+This is the `mimecast.ttp_url_logs` dataset. These logs contain Mimecast TTP
+attachment protection logs with the following details: the category of the URL
+clicked, the email address of the user who clicked the link, the url clicked,
+the action taken by the user if user awareness was applied, the route of the
+email that contained the link, the action defined by the administrator for the
+URL, the date that the URL was clicked, url scan result, the action that was
+taken for the click, the description of the definition that triggered the URL to
+be rewritten by Mimecast, the action requested by the user, an array of
+components of the message where the URL was found. More about these logs
+[here](https://integrations.mimecast.com/documentation/endpoint-reference/logs-and-statistics/get-ttp-url-logs/).
 
-{{event "threat_intel_malware_grid"}}
+{{event "ttp_url_logs"}}
 
-{{fields "threat_intel_malware_grid"}}
+{{fields "ttp_url_logs"}}

packages/mimecast/changelog.yml

@@ -1,3 +1,11 @@
+- version: "0.0.11"
+  changes:
+    - description: Update integration description for consistency with other integrations.
+      type: enhancement
+      link: https://github.com/elastic/integrations/pull/3193
+    - description: Add missing ECS event.* field mappings.
+      type: bugfix
+      link: https://github.com/elastic/integrations/pull/3193
 - version: "0.0.10"
   changes:
     - description: Add more use cases to audit_events pipeline

packages/mimecast/data_stream/audit_events/_dev/test/pipeline/test-audit-events.log-expected.json

@@ -1338,8 +1338,8 @@
                 "version": "8.2.0"
             },
             "event": {
-                "created": "2022-01-11T21:48:01.000Z",
                 "action": "logon-authentication-failed",
+                "created": "2022-01-11T21:48:01.000Z",
                 "id": "eNqrVipOTS4tSs1MUbJSMvCrMHX2MzL1yLFITjJNd8rO9wiJyAlL89c2DXZ1C3eP9AyvijKL9I7Rd_WOzC0ztMg2dzFM1M73s6w09CqoDA1T0lFKLE3JLMnJTwcZaGxoaWxsaGRkoKOUXFpckp-bWpScn5IKtMnZxMzR3BSovCy1qDgzP0_JyrAWAPktKzg",
                 "original": "{\"id\":\"eNqrVipOTS4tSs1MUbJSMvCrMHX2MzL1yLFITjJNd8rO9wiJyAlL89c2DXZ1C3eP9AyvijKL9I7Rd_WOzC0ztMg2dzFM1M73s6w09CqoDA1T0lFKLE3JLMnJTwcZaGxoaWxsaGRkoKOUXFpckp-bWpScn5IKtMnZxMzR3BSovCy1qDgzP0_JyrAWAPktKzg\",\"auditType\":\"Logon Authentication Failed\",\"user\":\"johndoe@example.com\",\"eventTime\":\"2021-10-12T08:47:55+0000\",\"eventInfo\":\"Failed authentication for johndoe@example.com \u003cJohn Doe\u003e, Date: 2022-01-11, Time: 21:48:01 GMT, IP: 67.43.156.15, Application: POP-POP2, Method: Cloud, Reason: Wrong Password\",\"category\":\"authentication_logs\"}",
                 "reason": "Wrong Password"

packages/mimecast/data_stream/audit_events/elasticsearch/ingest_pipeline/default.yml

@@ -1,7 +1,7 @@
 ---
-description: Pipeline for processing sample logs
+description: Pipeline for processing Mimecast audit_events.
 processors:
-  #  # Generic event/ecs fields we always want to populate
+  # Generic event/ecs fields we always want to populate.
   - set:
       field: ecs.version
       value: "8.2.0"

packages/mimecast/data_stream/audit_events/fields/ecs.yml

@@ -1,72 +1,58 @@
 - external: ecs
-  name: event.original
+  name: client.as.number
 - external: ecs
-  name: event.action
+  name: client.as.organization.name
 - external: ecs
-  name: user.email
+  name: client.geo.city_name
 - external: ecs
-  name: event.id
+  name: client.geo.continent_name
 - external: ecs
-  name: tags
+  name: client.geo.country_iso_code
 - external: ecs
-  name: ecs.version
+  name: client.geo.country_name
+- external: ecs
+  name: client.geo.location
+- external: ecs
+  name: client.geo.region_iso_code
+- external: ecs
+  name: client.geo.region_name
 - external: ecs
   name: client.ip
 - external: ecs
-  name: file.name
+  name: ecs.version
 - external: ecs
-  name: user.name
+  name: email.from.address
 - external: ecs
-  name: user.domain
+  name: email.origination_timestamp
 - external: ecs
-  name: file.extension
+  name: email.subject
 - external: ecs
-  name: client.geo.city_name
+  name: email.to.address
 - external: ecs
-  name: client.geo.continent_name
+  name: event.action
 - external: ecs
-  name: client.geo.country_iso_code
+  name: event.created
 - external: ecs
-  name: client.geo.country_name
-- description: Longitude and latitude.
-  level: core
-  name: client.geo.location
-  type: geo_point
+  name: event.id
 - external: ecs
-  name: client.geo.region_iso_code
+  name: event.original
 - external: ecs
-  name: client.geo.region_name
-- description: Client ASN number.
-  name: client.as.asn
-  type: long
-- description: Client Organization name.
-  name: client.as.organization_name
-  type: keyword
+  name: event.reason
 - external: ecs
-  name: client.as.number
+  name: file.extension
 - external: ecs
-  name: client.as.organization.name
-- description: The email address(es) of the message recipient(s)
-  type: keyword
-  name: email.to.address
-- description: Stores the from email address from the RFC5322 From - header field.
-  type: keyword
-  name: email.from.address
-- description: A brief summary of the topic of the message
-  type: keyword
-  name: email.subject
-  ignore_above: 1024
-  multi_fields:
-    - name: text
-      type: text
-      norms: false
-      default_field: false
-- description: The date and time the email message was composed. Many email clients will fill this in automatically when the message is sent by a user.
-  type: date
-  name: email.origination_timestamp
+  name: file.name
 - external: ecs
   name: file.size
 - external: ecs
   name: related.ip
 - external: ecs
   name: related.user
+- external: ecs
+  name: tags
+- external: ecs
+  name: user.domain
+- external: ecs
+  name: user.email
+- external: ecs
+  name: user.name

packages/mimecast/data_stream/dlp_logs/elasticsearch/ingest_pipeline/default.yml

@@ -1,7 +1,7 @@
 ---
-description: Pipeline for processing sample logs
+description: Pipeline for processing Mimecast dlp_logs.
 processors:
-  # Generic event/ecs fields we always want to populated
+  # Generic event/ecs fields we always want to populate.
   - set:
       field: ecs.version
       value: "8.2.0"

packages/mimecast/data_stream/dlp_logs/fields/ecs.yml

@@ -1,36 +1,22 @@
 - external: ecs
-  name: event.original
+  name: ecs.version
 - external: ecs
-  name: event.action
-- description: Identifier from the RFC5322 Message-ID - header field that refers to a particular version of a particular message.
-  type: wildcard
-  name: email.message_id
-  multi_fields:
-    - name: text
-      type: text
-      norms: false
-      default_field: false
-- description: Direction of the message based on the sending and receiving domains
-  type: keyword
   name: email.direction
 - external: ecs
-  name: rule.name
+  name: email.from.address
 - external: ecs
-  name: tags
+  name: email.message_id
 - external: ecs
-  name: ecs.version
-- description: The email address(es) of the message recipient(s)
-  type: keyword
-  name: email.to.address
-- description: Stores the from email address from the RFC5322 From - header field.
-  type: keyword
-  name: email.from.address
-- description: A brief summary of the topic of the message
-  type: keyword
   name: email.subject
-  ignore_above: 1024
-  multi_fields:
-    - name: text
-      type: text
-      norms: false
-      default_field: false
+- external: ecs
+  name: email.to.address
+- external: ecs
+  name: event.action
+- external: ecs
+  name: event.created
+- external: ecs
+  name: event.original
+- external: ecs
+  name: rule.name
+- external: ecs
+  name: tags