New fileset for googlecloud firewall logs

CHANGELOG.next.asciidoc

@@ -404,6 +404,7 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d
 - Remove beta flag for some filebeat modules. {pull}14374[14374]
 - Add attack_pattern_kql field to MISP threat indicators. {pull}14470[14470]
 - Add fileset to the Zeek module for the intel.log. {pull}14404[14404]
+- New fileset googlecloud/firewall for ingesting Google Cloud Firewall logs. {pull}14553[14553]
 
 *Heartbeat*
 - Add non-privileged icmp on linux and darwin(mac). {pull}13795[13795] {issue}11498[11498]

filebeat/docs/fields.asciidoc

@@ -8700,40 +8700,87 @@ Fields from Google Cloud logs.
 
 
 [float]
-=== vpcflow
+=== destination.instance
 
-Fields for Google Cloud VPC flow logs.
+If the destination of the connection was a VM located on the same VPC, this field is populated with VM instance details. In a Shared VPC configuration, project_id corresponds to the project that owns the instance, usually the service project.
 
 
 
-*`googlecloud.vpcflow.reporter`*::
+*`googlecloud.destination.instance.project_id`*::
 +
 --
-The side which reported the flow. Can be either 'SRC' or 'DEST'.
+ID of the project containing the VM.
 
 
 type: keyword
 
 --
 
-*`googlecloud.vpcflow.rtt.ms`*::
+*`googlecloud.destination.instance.region`*::
 +
 --
-Latency as measured (for TCP flows only) during the time interval. This is the time elapsed between sending a SEQ and receiving a corresponding ACK and it contains the network RTT as well as the application related delay.
+Region of the VM.
 
 
-type: long
+type: keyword
+
+--
+
+*`googlecloud.destination.instance.zone`*::
++
+--
+Zone of the VM.
+
+
+type: keyword
 
 --
 
 [float]
-=== destination.instance
+=== destination.vpc
 
-If the destination of the connection was a VM located on the same VPC, this field is populated with VM instance details. In a Shared VPC configuration, project_id corresponds to the project that owns the instance, usually the service project.
+If the destination of the connection was a VM located on the same VPC, this field is populated with VPC network details. In a Shared VPC configuration, project_id corresponds to that of the host project.
+
+
+
+*`googlecloud.destination.vpc.project_id`*::
++
+--
+ID of the project containing the VM.
+
+
+type: keyword
 
+--
+
+*`googlecloud.destination.vpc.vpc_name`*::
++
+--
+VPC on which the VM is operating.
+
+
+type: keyword
+
+--
+
+*`googlecloud.destination.vpc.subnetwork_name`*::
++
+--
+Subnetwork on which the VM is operating.
+
+
+type: keyword
 
+--
 
-*`googlecloud.vpcflow.destination.instance.project_id`*::
+[float]
+=== source.instance
+
+If the source of the connection was a VM located on the same VPC, this field is populated with VM instance details. In a Shared VPC configuration, project_id corresponds to the project that owns the instance, usually the service project.
+
+
+
+*`googlecloud.source.instance.project_id`*::
 +
 --
 ID of the project containing the VM.
@@ -8743,7 +8790,7 @@ type: keyword
 
 --
 
-*`googlecloud.vpcflow.destination.instance.region`*::
+*`googlecloud.source.instance.region`*::
 +
 --
 Region of the VM.
@@ -8753,7 +8800,7 @@ type: keyword
 
 --
 
-*`googlecloud.vpcflow.destination.instance.zone`*::
+*`googlecloud.source.instance.zone`*::
 +
 --
 Zone of the VM.
@@ -8764,13 +8811,13 @@ type: keyword
 --
 
 [float]
-=== destination.vpc
+=== source.vpc
 
-If the destination of the connection was a VM located on the same VPC, this field is populated with VPC network details. In a Shared VPC configuration, project_id corresponds to that of the host project.
+If the source of the connection was a VM located on the same VPC, this field is populated with VPC network details. In a Shared VPC configuration, project_id corresponds to that of the host project.
 
 
 
-*`googlecloud.vpcflow.destination.vpc.project_id`*::
+*`googlecloud.source.vpc.project_id`*::
 +
 --
 ID of the project containing the VM.
@@ -8780,7 +8827,7 @@ type: keyword
 
 --
 
-*`googlecloud.vpcflow.destination.vpc.vpc_name`*::
+*`googlecloud.source.vpc.vpc_name`*::
 +
 --
 VPC on which the VM is operating.
@@ -8790,7 +8837,7 @@ type: keyword
 
 --
 
-*`googlecloud.vpcflow.destination.vpc.subnetwork_name`*::
+*`googlecloud.source.vpc.subnetwork_name`*::
 +
 --
 Subnetwork on which the VM is operating.
@@ -8801,79 +8848,150 @@ type: keyword
 --
 
 [float]
-=== source.instance
+=== firewall
 
-If the source of the connection was a VM located on the same VPC, this field is populated with VM instance details. In a Shared VPC configuration, project_id corresponds to the project that owns the instance, usually the service project.
+Fields for Google Cloud Firewall logs.
+
+
+
+[float]
+=== rule_details
+
+Description of the firewall rule that matched this connection.
 
 
 
-*`googlecloud.vpcflow.source.instance.project_id`*::
+*`googlecloud.firewall.rule_details.priority`*::
 +
 --
-ID of the project containing the VM.
+The priority for the firewall rule.
+
+type: long
 
+--
+
+*`googlecloud.firewall.rule_details.action`*::
++
+--
+Action that the rule performs on match.
 
 type: keyword
 
 --
 
-*`googlecloud.vpcflow.source.instance.region`*::
+*`googlecloud.firewall.rule_details.direction`*::
 +
 --
-Region of the VM.
+Direction of traffic that matches this rule.
+
+type: keyword
+
+--
 
+*`googlecloud.firewall.rule_details.reference`*::
++
+--
+Reference to the firewall rule.
 
 type: keyword
 
 --
 
-*`googlecloud.vpcflow.source.instance.zone`*::
+*`googlecloud.firewall.rule_details.source_range`*::
 +
 --
-Zone of the VM.
+List of source ranges that the firewall rule applies to.
 
+type: keyword
+
+--
+
+*`googlecloud.firewall.rule_details.destination_range`*::
++
+--
+List of destination ranges that the firewall applies to.
 
 type: keyword
 
 --
 
-[float]
-=== source.vpc
+*`googlecloud.firewall.rule_details.source_tag`*::
++
+--
+List of all the source tags that the firewall rule applies to.
 
-If the source of the connection was a VM located on the same VPC, this field is populated with VPC network details. In a Shared VPC configuration, project_id corresponds to that of the host project.
 
+type: keyword
 
+--
 
-*`googlecloud.vpcflow.source.vpc.project_id`*::
+*`googlecloud.firewall.rule_details.target_tag`*::
 +
 --
-ID of the project containing the VM.
+List of all the target tags that the firewall rule applies to.
 
 
 type: keyword
 
 --
 
-*`googlecloud.vpcflow.source.vpc.vpc_name`*::
+*`googlecloud.firewall.rule_details.ip_port_info`*::
 +
 --
-VPC on which the VM is operating.
+List of ip protocols and applicable port ranges for rules.
+
+
+type: array
+
+--
+
+*`googlecloud.firewall.rule_details.source_service_account`*::
++
+--
+List of all the source service accounts that the firewall rule applies to.
 
 
 type: keyword
 
 --
 
-*`googlecloud.vpcflow.source.vpc.subnetwork_name`*::
+*`googlecloud.firewall.rule_details.target_service_account`*::
 +
 --
-Subnetwork on which the VM is operating.
+List of all the target service accounts that the firewall rule applies to.
+
+
+type: keyword
+
+--
+
+[float]
+=== vpcflow
+
+Fields for Google Cloud VPC flow logs.
+
+
+
+*`googlecloud.vpcflow.reporter`*::
++
+--
+The side which reported the flow. Can be either 'SRC' or 'DEST'.
 
 
 type: keyword
 
 --
 
+*`googlecloud.vpcflow.rtt.ms`*::
++
+--
+Latency as measured (for TCP flows only) during the time interval. This is the time elapsed between sending a SEQ and receiving a corresponding ACK and it contains the network RTT as well as the application related delay.
+
+
+type: long
+
+--
+
 [[exported-fields-haproxy]]
 == haproxy fields
 

filebeat/docs/modules/googlecloud.asciidoc

@@ -12,8 +12,9 @@ This file is generated! See scripts/docs_collector.py
 
 beta[]
 
-This is a module for Google Cloud logs. It supports reading VPC flow logs that
-have been exported from Stackdriver to a Google Pub/Sub topic sink.
+This is a module for Google Cloud logs. It supports reading VPC flow
+and firewall logs that have been exported from Stackdriver to a
+Google Pub/Sub topic sink.
 
 include::../include/what-happens.asciidoc[]
 
@@ -60,6 +61,51 @@ exist it will be created.
 
 Path to a JSON file containing the credentials and key used to subscribe.
 
+:fileset_ex!:
+
+:fileset_ex: firewall
+
+[float]
+==== `firewall` fileset settings
+
+Example config:
+
+[source,yaml]
+----
+- module: googleclcoud
+  firewall:
+    enabled: true
+    var.project_id: my-gcp-project-id
+    var.topic: googlecloud-vpc-flowlogs
+    var.subscription_name: filebeat-googlecloud-vpc-flowlogs-sub
+    var.credentials_file: ${path.config}/gcp-service-account-xyz.json
+    var.keep_original_message: false
+----
+
+include::../include/var-paths.asciidoc[]
+
+*`var.project_id`*::
+
+Google Cloud project ID.
+
+*`var.topic`*::
+
+Google Cloud Pub/Sub topic name.
+
+*`var.subscription_name`*::
+
+Google Cloud Pub/Sub topic subscription name. If the subscription does not
+exist it will be created.
+
+*`var.credentials_file`*::
+
+Path to a JSON file containing the credentials and key used to subscribe.
+
+*`var.keep_original_message`*::
+
+Flag to control whether the original message is stored in the `log.original`
+field. Defaults to `false`, meaning the original message is not saved.
+
 :has-dashboards!:
 
 :fileset_ex!: