New integration for SonicWall firewall by adriansr · Pull Request #3365 · elastic/integrations

adriansr · 2022-05-17T10:15:09Z

What does this PR do?

This integration (sonicwall_firewall) allows to receive syslog messages from SonicOS devices.

Replaces the existing rsa2elk integration (sonicwall).

Uses logs and some ideas from #2729 by @jhaugh0.

Checklist

I have reviewed tips for building integrations and this pull request is aligned with them.
I have verified that all data streams collect metrics or logs.
I have added an entry to my package's changelog.yml file.
I have verified that Kibana version constraints are current according to guidelines.

Author's Checklist

Add system tests (file & udp)
Deprecate existing integration.
Dashboard
Document supported event types

Related issues

Closes Sonicwall #1207
Supersedes [sonicwall] Add new fields to pipeline #2729

Screenshots

Dashboard:

Deprecated package:

elasticmachine · 2022-05-17T10:15:11Z

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

elasticmachine · 2022-05-17T10:19:26Z

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

Start Time: 2022-05-24T07:16:41.771+0000
Duration: 16 min 47 sec

Test stats 🧪

Test	Results
Failed	0
Passed	18
Skipped	0
Total	18

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

/test : Re-trigger the build.

elasticmachine · 2022-05-17T10:33:12Z

🌐 Coverage report

Name	Metrics % (`covered/total`)	Diff
Packages	100.0% (`2/2`)	💚
Files	100.0% (`2/2`)	💚 3.778
Classes	100.0% (`2/2`)	💚 3.778
Methods	88.889% (`24/27`)	👍 0.265
Lines	97.487% (`1164/1194`)	👍 6.707
Conditionals	100.0% (`0/0`)	💚

jamiehynds · 2022-05-18T12:47:33Z

@adriansr is it possible to add a dashboard to avoid having to revisit down the line? Some potential resources on the dashboard:

Top 10 Source IPs
Activity by Protocol
Activity by Port
Denied Traffic on WAN Interface by Port
Denied Traffic on WAN Interface by Source Location
Top 10 Event Types (including count)
Opened and Closed Connections Over Time
VPN Logons by Username (successful and failed)

jamiehynds · 2022-05-18T12:50:28Z

@adriansr is it also possible to include the support log types within the integration description, to set expectations as to which events are covered? e.g. System, Users, VPN, Firewall. Full list available on page 5 here: https://www.sonicwall.com/techdocs/pdf/sonicos-6-5-4-log-events-reference-guide.pdf

adriansr · 2022-05-18T13:45:43Z

sure @jamiehynds , will do

This integration allows to receive syslog messages from SonicOS devices. Replaces the existing rsa2elk integration. Co-authored-by: jhaugh0 <jfhaugh@wol.org>

efd6

Minor comments only, otherwise LGTM.

efd6 · 2022-05-20T04:24:58Z

packages/sonicwall_firewall/data_stream/log/agent/stream/logfile.yml.hbs

+- add_locale: ~
+{{#if processors}}
+{{processors}}
+{{/if}}


nit: final newline

efd6 · 2022-05-20T04:26:23Z

packages/sonicwall_firewall/data_stream/log/fields/package-fields.yml

+- name: sonicwall
+  type: group
+  fields:
+    - name: firewall


Worth a description of the data that would be here?

efd6 · 2022-05-20T04:40:55Z

packages/sonicwall_firewall/data_stream/log/elasticsearch/ingest_pipeline/default.yml

+      description: Extracts optional port number from src nat field
+      ignore_missing: true
+      patterns:
+        - '^%{IPV4:source.nat.ip}:%{POSINT:source.nat.port}$'


Could we see IPv6 IPs here?

This one is there to handle the (undocumented) case of the nat field being ip:port instead of ip. I've only observed this with IPv4, not sure if that can happen with IPv6 and what the format would be.

Just in case I added support for [ipv6]:port, which in my understanding is a common way of appending a port to an IPv6 address.

I have a set of patterns that work for this, but it can happen in a later PR. (

integrations/packages/cisco_duo/data_stream/auth/elasticsearch/ingest_pipeline/default.yml

Lines 54 to 56 in 090e703

- "^\\[%{IPV6:json.access_device.ip}\\]:%{PORT:json.access_device.port}$"

- "^%{IPV6NOCOMPRESS:json.access_device.ip}:%{PORT:json.access_device.port}$"

- "^%{IPV6:json.access_device.ip}%{IPV6PORTSEP}%{PORT:json.access_device.port}$"

)

efd6 · 2022-05-20T04:47:10Z

packages/sonicwall_firewall/_dev/build/docs/README.md

@@ -0,0 +1,78 @@
+# SonicWall Firewall Integration
+
+This integration collects syslog messages from SonicWall firewalls. It has been tested with SonicOS 6.5 and 7.0.


Add a link to the vendor docs?

efd6 · 2022-05-20T04:48:08Z

packages/sonicwall_firewall/data_stream/log/elasticsearch/ingest_pipeline/default.yml

+              - creation
+            outcome: success
+
+        message_codes:


For future maintenance a link to the origin for these would be helpful.

efd6 · 2022-05-20T04:52:30Z

packages/sonicwall_firewall/data_stream/log/elasticsearch/ingest_pipeline/default.yml

+        }
+
+#
+# Extends message field with note


efd6 · 2022-05-20T04:55:40Z

packages/sonicwall_firewall/data_stream/log/_dev/test/pipeline/test-general.log-expected.json

+            },
+            "event": {
+                "code": "97",
+                "original": "10.0.0.1 id=firewall sn=12345678 time=\"2022-03-11 14:17:52 UTC\" fw=10.0.0.2 pri=6 c=1024 gcat=2 m=97 srcMac=12:34:56:78:90:ab src=10.0.0.4:41856:X0 srcZone=Trusted natSrc=10.0.0.2:8689 dstMac=ab:09:87:65:43:21 dst=89.160.20.112:443:X1 dstZone=Untrusted natDst=89.160.20.112:443 usr=\"Unknown (SSO failed)\" proto=tcp/https sent=104 rcvd=230 rule=\"15 (LAN-\u003eWAN)\" app=5 af_polid=4 ipscat=N/A appcat=\"PROXY-ACCESS\" appid=2900 dstname=89.160.20.112 arg=/ code=64 Category=\"Not Rated\" note=\"Policy: cfsZonePolicy0, Info: 6148 \" n=2520325 fw_action=\"NA\" dpi=1 op=0",


It looks like no message is written when there is a note, but no msg. Maybe promote the note to a message in this case?

good catch, done

P1llus

LGTM, added some comments that are more questions rather than fixes.

Awesome job on the documentation and dashboarding as well! :)

P1llus · 2022-05-23T12:45:39Z

packages/sonicwall_firewall/data_stream/log/agent/stream/udp.yml.hbs

@@ -0,0 +1,20 @@
+host: "{{syslog_host}}:{{syslog_port}}"


I know that @taylor-swanson has been doing some changes to allow for both TCP and UDP for all our firewall integrations, would it be worthwhile to add a similar one here?

In this case, the SonicWall only supports UDP so I kept it simple. I see they've received requests for tcp+ssl but the latest version is still udp-only.

https://community.sonicwall.com/technology-and-support/discussion/1774/syslog-over-secure-tcp

P1llus · 2022-05-23T12:48:21Z

packages/sonicwall_firewall/changelog.yml

@@ -0,0 +1,6 @@
+# newer versions go on top
+- version: "0.6.0"


Is it correct for the initial version to be 0.6.0 rather than 0.1.0?

You're right. The integrations docs require the use of 0.1.0 for a new package.
https://github.com/elastic/integrations/blob/main/docs/tips_for_building_integrations.md#manifest-files

P1llus · 2022-05-23T12:50:06Z

packages/sonicwall_firewall/data_stream/log/fields/base-fields.yml

+  type: constant_keyword
+  description: Data stream dataset.
+- name: data_stream.namespace
+  type: constant_keyword


Do we not add in event.module and event.dataset anymore? For example:

- name: event.module type: constant_keyword description: Event module value: o365 - name: event.dataset type: constant_keyword description: Event dataset value: o365.audit

Given that elastic-package doesn't add those when creating a data_stream, I though those were not needed anymore now that we have the data_stream.* fields. But it's worth adding them

adriansr added enhancement New feature or request Team:Security-External Integrations Integration:sonicwall_firewall SonicWall Firewall labels May 17, 2022

adriansr requested a review from a team May 17, 2022 10:16

adriansr mentioned this pull request May 17, 2022

[sonicwall] Add new fields to pipeline #2729

Closed

4 tasks

adriansr and others added 17 commits May 20, 2022 01:08

New integration for SonicWall firewall

6bf8260

This integration allows to receive syslog messages from SonicOS devices. Replaces the existing rsa2elk integration. Co-authored-by: jhaugh0 <jfhaugh@wol.org>

Get rid of zone config

f8b2b55

Adjust PR number

7bbb957

CODEOWNERS update

2a5db3d

Deprecate sonicwall rsa2elk package

624e889

Missing event.outcome values

5a1005c

Simplify painless script

d8f9f88

System test

35b8ebf

Logfile system test

a62b37e

Format

7b7e6e6

Build readme

0c9a4e5

Make compatible with 8.2.0

3898f94

Add dashboard

ebfa33b

Update version

2ff3b78

Add list of enriched messages

873372b

Update version

bbc4a06

Missing dashboard image

fd6a56d

efd6 reviewed May 20, 2022

View reviewed changes

Add final newline

3f2b5b3

adriansr added 5 commits May 20, 2022 09:22

Document and map known vendor fields

71e8648

Support [ipv6]:port format in NAT fields

6933525

Add link to docs in README

4b1e213

Document origin of CSV

26dd611

Populate message with note if not present

f248f22

adriansr requested review from a team and efd6 May 23, 2022 11:58

efd6 approved these changes May 23, 2022

View reviewed changes

P1llus approved these changes May 23, 2022

View reviewed changes

adriansr added 3 commits May 23, 2022 15:26

Reset initial version to 0.1.0

4dba8eb

Add event.{module,dataset} constant_keyword fields

d99dd23

Update README

a5691d9

adriansr merged commit 88b3085 into elastic:main May 24, 2022

adriansr deleted the new_sonicwall branch May 24, 2022 09:09

andrewkroh mentioned this pull request May 31, 2022

[production] Promote many packages from snapshot elastic/package-storage#4504

Merged

drizzthacker mentioned this pull request Jun 9, 2022

[Filebeat] Dissect Parsing Error with Sonicwall Module elastic/beats#24124

Closed

andrewkroh added the New Integration Issue or pull request for creating a new integration package. label Aug 13, 2024

	- "^\\[%{IPV6:json.access_device.ip}\\]:%{PORT:json.access_device.port}$"
	- "^%{IPV6NOCOMPRESS:json.access_device.ip}:%{PORT:json.access_device.port}$"
	- "^%{IPV6:json.access_device.ip}%{IPV6PORTSEP}%{PORT:json.access_device.port}$"

		@@ -0,0 +1,78 @@
		# SonicWall Firewall Integration

		This integration collects syslog messages from SonicWall firewalls. It has been tested with SonicOS 6.5 and 7.0.

Conversation

adriansr commented May 17, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Checklist

Author's Checklist

Related issues

Screenshots

Uh oh!

elasticmachine commented May 17, 2022

Uh oh!

elasticmachine commented May 17, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

💚 Build Succeeded

Build stats

Test stats 🧪

🤖 GitHub comments

Uh oh!

elasticmachine commented May 17, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🌐 Coverage report

Uh oh!

jamiehynds commented May 18, 2022

Uh oh!

jamiehynds commented May 18, 2022

Uh oh!

adriansr commented May 18, 2022

Uh oh!

efd6 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

efd6 May 23, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

P1llus left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

adriansr commented May 17, 2022 •

edited

Loading

elasticmachine commented May 17, 2022 •

edited

Loading

elasticmachine commented May 17, 2022 •

edited

Loading

efd6 May 23, 2022 •

edited

Loading