Add Syslog fields to ECS (#525)

Here's the intended mapping in different situations

1. Syslog event / event source specifies different severity

* log.syslog.severity.code: Syslog severity number
* log.syslog.severity.name: Syslog severity label
* event.severity: Source severity number
* log.level: Source severity label

2. Syslog event / event source does not specify different severity

* log.syslog.severity.code: Syslog severity number
* log.syslog.severity.name: Syslog severity label
* event.severity: (optional) Syslog severity number
* log.level: (optional) Syslog severity label

3. Non-syslog event, event source specifies severity

* event.severity: Source severity number
* log.level: Source severity label

Author: Mathieu Martin

CHANGELOG.next.md

@@ -11,7 +11,8 @@ Thanks, you're awesome :-) -->
 
 ### Added
 
-* Add group.domain field #547 
+* Added fields in `log.*` to allow for full Syslog mapping. #525
+* Add group.domain field #547
 * Added `error.stack_trace` field. #562
 * Added `log.origin.file.name`, `log.origin.function` and `log.origin.file.line` fields. #563
 * Added `service.node.name` to allow distinction between different nodes of the same service running on the same host. #565

code/go/ecs/event.go

@@ -1277,7 +1277,11 @@ type: long
 // ===============================================================
 
 | event.severity
-| Severity describes the original severity of the event. What the different severity values mean can very different between use cases. It's up to the implementer to make sure severities are consistent across events.
+| The numeric severity of the event according to your event source.
+
+What the different severity values mean can be different between sources and use cases. It's up to the implementer to make sure severities are consistent across events from the same source.
+
+The Syslog severity belongs in `log.syslog.severity.code`. `event.severity` is meant to represent the severity according to the event source (e.g. firewall, IDS). If the event source does not publish its own severity, you may optionally copy the `log.syslog.severity.code` to `event.severity`.
 
 type: long
 
@@ -2112,11 +2116,13 @@ Fields which are specific to log events.
 | log.level
 | Original log level of the log event.
 
-Some examples are `warn`, `error`, `i`.
+If the source of the event provides a log level or textual severity, this is the one that goes in `log.level`. If your source doesn't specify one, you may put your event transport's severity here (e.g. Syslog severity).
+
+Some examples are `warn`, `err`, `i`, `informational`.
 
 type: keyword
 
-example: `err`
+example: `error`
 
 | core
 
@@ -2181,6 +2187,80 @@ example: `Sep 19 08:26:10 localhost My log`
 
 // ===============================================================
 
+| log.syslog
+| The Syslog metadata of the event, if the event was transmitted via Syslog. Please see RFCs 5424 or 3164.
+
+type: object
+
+
+
+| extended
+
+// ===============================================================
+
+| log.syslog.facility.code
+| The Syslog numeric facility of the log event, if available.
+
+According to RFCs 5424 and 3164, this value should be an integer between 0 and 23.
+
+type: long
+
+example: `23`
+
+| extended
+
+// ===============================================================
+
+| log.syslog.facility.name
+| The Syslog text-based facility of the log event, if available.
+
+type: keyword
+
+example: `local7`
+
+| extended
+
+// ===============================================================
+
+| log.syslog.priority
+| Syslog numeric priority of the event, if available.
+
+According to RFCs 5424 and 3164, the priority is 8 * facility + severity. This number is therefore expected to contain a value between 0 and 191.
+
+type: long
+
+example: `135`
+
+| extended
+
+// ===============================================================
+
+| log.syslog.severity.code
+| The Syslog numeric severity of the log event, if available.
+
+If the event source publishing via Syslog provides a different numeric severity value (e.g. firewall, IDS), your source's numeric severity should go to `event.severity`. If the event source does not specify a distinct severity, you can optionally copy the Syslog severity to `event.severity`.
+
+type: long
+
+example: `3`
+
+| extended
+
+// ===============================================================
+
+| log.syslog.severity.name
+| The Syslog numeric severity of the log event, if available.
+
+If the event source publishing via Syslog provides a different severity value (e.g. firewall, IDS), your source's text severity should go to `log.level`. If the event source does not specify a distinct severity, you can optionally copy the Syslog severity to `log.level`.
+
+type: keyword
+
+example: `Error`
+
+| extended
+
+// ===============================================================
+
 |=====
 
 [[ecs-network]]

code/go/ecs/log.go

@@ -1020,10 +1020,17 @@
       level: core
       type: long
       format: string
-      description: Severity describes the original severity of the event. What the
-        different severity values mean can very different between use cases. It's
-        up to the implementer to make sure severities are consistent across events.
-      example: '7'
+      description: 'The numeric severity of the event according to your event source.
+
+        What the different severity values mean can be different between sources and
+        use cases. It''s up to the implementer to make sure severities are consistent
+        across events from the same source.
+
+        The Syslog severity belongs in `log.syslog.severity.code`. `event.severity`
+        is meant to represent the severity according to the event source (e.g. firewall,
+        IDS). If the event source does not publish its own severity, you may optionally
+        copy the `log.syslog.severity.code` to `event.severity`.'
+      example: 7
     - name: start
       level: extended
       type: date
@@ -1592,8 +1599,12 @@
       ignore_above: 1024
       description: 'Original log level of the log event.
 
-        Some examples are `warn`, `error`, `i`.'
-      example: err
+        If the source of the event provides a log level or textual severity, this
+        is the one that goes in `log.level`. If your source doesn''t specify one,
+        you may put your event transport''s severity here (e.g. Syslog severity).
+
+        Some examples are `warn`, `err`, `i`, `informational`.'
+      example: error
     - name: logger
       level: core
       type: keyword
@@ -1635,6 +1646,57 @@
         This field is not indexed and doc_values are disabled so it can''t be queried
         but the value can be retrieved from `_source`.'
       example: Sep 19 08:26:10 localhost My log
+    - name: syslog
+      level: extended
+      type: object
+      object_type: keyword
+      description: The Syslog metadata of the event, if the event was transmitted
+        via Syslog. Please see RFCs 5424 or 3164.
+    - name: syslog.facility.code
+      level: extended
+      type: long
+      format: string
+      description: 'The Syslog numeric facility of the log event, if available.
+
+        According to RFCs 5424 and 3164, this value should be an integer between 0
+        and 23.'
+      example: 23
+    - name: syslog.facility.name
+      level: extended
+      type: keyword
+      ignore_above: 1024
+      description: The Syslog text-based facility of the log event, if available.
+      example: local7
+    - name: syslog.priority
+      level: extended
+      type: long
+      format: string
+      description: 'Syslog numeric priority of the event, if available.
+
+        According to RFCs 5424 and 3164, the priority is 8 * facility + severity.
+        This number is therefore expected to contain a value between 0 and 191.'
+      example: 135
+    - name: syslog.severity.code
+      level: extended
+      type: long
+      description: 'The Syslog numeric severity of the log event, if available.
+
+        If the event source publishing via Syslog provides a different numeric severity
+        value (e.g. firewall, IDS), your source''s numeric severity should go to `event.severity`.
+        If the event source does not specify a distinct severity, you can optionally
+        copy the Syslog severity to `event.severity`.'
+      example: 3
+    - name: syslog.severity.name
+      level: extended
+      type: keyword
+      ignore_above: 1024
+      description: 'The Syslog numeric severity of the log event, if available.
+
+        If the event source publishing via Syslog provides a different severity value
+        (e.g. firewall, IDS), your source''s text severity should go to `log.level`.
+        If the event source does not specify a distinct severity, you can optionally
+        copy the Syslog severity to `log.level`.'
+      example: Error
   - name: network
     title: Network
     group: 2

docs/field-details.asciidoc

@@ -202,12 +202,18 @@ http.response.body.content,keyword,extended,Hello world,1.2.0-dev
 http.response.bytes,long,extended,1437,1.2.0-dev
 http.response.status_code,long,extended,404,1.2.0-dev
 http.version,keyword,extended,1.1,1.2.0-dev
-log.level,keyword,core,err,1.2.0-dev
+log.level,keyword,core,error,1.2.0-dev
 log.logger,keyword,core,org.elasticsearch.bootstrap.Bootstrap,1.2.0-dev
 log.origin.file.line,integer,extended,42,1.2.0-dev
 log.origin.file.name,keyword,extended,Bootstrap.java,1.2.0-dev
 log.origin.function,keyword,extended,init,1.2.0-dev
 log.original,keyword,core,Sep 19 08:26:10 localhost My log,1.2.0-dev
+log.syslog,object,extended,,1.2.0-dev
+log.syslog.facility.code,long,extended,23,1.2.0-dev
+log.syslog.facility.name,keyword,extended,local7,1.2.0-dev
+log.syslog.priority,long,extended,135,1.2.0-dev
+log.syslog.severity.code,long,extended,3,1.2.0-dev
+log.syslog.severity.name,keyword,extended,Error,1.2.0-dev
 network.application,keyword,extended,aim,1.2.0-dev
 network.bytes,long,core,368,1.2.0-dev
 network.community_id,keyword,extended,1:hO+sN4H+MG5MY/8hIrXPqc4ZQz0=,1.2.0-dev