Skip to content

[Snort] Add Snort 3 JSON alert file compatibility #3876

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
andrewkroh merged 8 commits into from
Aug 15, 2022
Merged

[Snort] Add Snort 3 JSON alert file compatibility #3876

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
4bd9715
Add SNort 3 JSON alert file compatibility
legoguy1000 Jul 28, 2022
3a0418d
Update changelog.yml
legoguy1000 Jul 28, 2022
fe10e6c
update readme
legoguy1000 Jul 28, 2022
a5c83b9
update per comments
legoguy1000 Jul 28, 2022
31caf0d
Fix TCP grok matching, add Additional JSON parsing
legoguy1000 Jul 28, 2022
2cb5088
update docs
legoguy1000 Jul 28, 2022
fc25453
update generated files
legoguy1000 Aug 2, 2022
63c12d0
revert snort.gid back to long
legoguy1000 Aug 3, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions packages/snort/_dev/build/docs/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,9 @@ This integration is for [Snort](https://www.snort.org/).

## Compatibility

This module has been developed against Snort v2.9, but is expected to work
with other versions of Snort. This package is designed to read from the PFsense CSV output
and the Alert Fast output either via reading a local logfile or receiving messages via syslog
This module has been developed against Snort v2.9 and v3, but is expected to work
with other versions of Snort. This package is designed to read from the PFsense CSV output,
the Alert Fast output either via reading a local logfile or receiving messages via syslog and the Snort 3 JSON log file.

## Log

Expand Down
5 changes: 5 additions & 0 deletions packages/snort/changelog.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,9 @@
# newer versions go on top
- version: "1.1.0"
changes:
- description: Add Snort 3 JSON support.
type: enhancement
link: https://github.com/elastic/integrations/pull/3876
- version: "1.0.0"
changes:
- description: Make GA
Expand Down
2 changes: 0 additions & 2 deletions packages/snort/data_stream/log/_dev/test/pipeline/test-common-config.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,4 @@
dynamic_fields:
event.created: "^[0-9]{4}(-[0-9]{2}){2}T[0-9]{2}(:[0-9]{2}){2}\\.[0-9]{3}-[0-9]{2}:[0-9]{2}$"
event.ingested: ".*"
"@timestamp": "^[0-9]{4}(-[0-9]{2}){2}T[0-9]{2}(:[0-9]{2}){2}\\.[0-9]{3}-[0-9]{2}:[0-9]{2}$"
fields:
"@timestamp": "2020-04-28T11:07:58.223Z"
Expand Down
24 changes: 16 additions & 8 deletions packages/snort/data_stream/log/_dev/test/pipeline/test-log-csv.log-expected.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,14 +15,15 @@
"category": [
"network"
],
"created": "2022-09-04T21:45:37.536-05:00",
"created": "2020-04-28T11:07:58.223Z",
"kind": "alert",
"original": "09/04-21:45:37.536335 ,1,1000006,0,\"TCP connection\",TCP,10.100.20.59,57263,10.100.10.190,22,00:25:90:3A:05:13,00:50:56:9D:A5:BE,0x72,***AP***,0x688F0DFC,0xBC763516,,0x80C,127,0,55665,100,102400,,,,",
"timezone": "America/Chicago"
},
"network": {
"community_id": "1:mnJdrIaujYJdP8lXFem/hodYAt0=",
"direction": "internal",
"iana_number": "6",
"transport": "tcp",
"type": "ipv4"
},
Expand Down Expand Up @@ -88,14 +89,15 @@
"category": [
"network"
],
"created": "2022-09-04T21:45:37.553-05:00",
"created": "2020-04-28T11:07:58.223Z",
"kind": "alert",
"original": "09/04-21:45:37.553882 ,1,1000006,0,\"TCP connection\",TCP,10.100.20.59,57263,10.100.10.190,22,00:25:90:3A:05:13,00:50:56:9D:A5:BE,0x72,***AP***,0x688F0E38,0xBC763552,,0x80C,127,0,55666,100,102400,,,,",
"timezone": "America/Chicago"
},
"network": {
"community_id": "1:mnJdrIaujYJdP8lXFem/hodYAt0=",
"direction": "internal",
"iana_number": "6",
"transport": "tcp",
"type": "ipv4"
},
Expand Down Expand Up @@ -161,14 +163,15 @@
"category": [
"network"
],
"created": "2022-09-04T21:50:40.017-05:00",
"created": "2020-04-28T11:07:58.223Z",
"kind": "alert",
"original": "09/04-21:50:40.017935 ,1,1000005,0,\"UDP Connection\",UDP,10.100.10.1,53,10.100.10.190,55475,00:25:90:3A:05:13,00:50:56:9D:A5:BE,0xC1,,,,,,64,0,56094,179,183296,,,,",
"timezone": "America/Chicago"
},
"network": {
"community_id": "1:wvunc3EtDmKBjBft1PFlQ2pSLzw=",
"direction": "internal",
"iana_number": "17",
"transport": "udp",
"type": "ipv4"
},
Expand Down Expand Up @@ -231,14 +234,15 @@
"category": [
"network"
],
"created": "2022-09-04T21:50:39.947-05:00",
"created": "2020-04-28T11:07:58.223Z",
"kind": "alert",
"original": "09/04-21:50:39.947383 ,1,1000005,0,\"UDP Connection\",UDP,10.100.10.1,53,10.100.10.190,55333,00:25:90:3A:05:13,00:50:56:9D:A5:BE,0xB1,,,,,,64,0,26112,163,166912,,,,",
"timezone": "America/Chicago"
},
"network": {
"community_id": "1:IcqpMEB/fJpNhZgyJVhx8VHROwY=",
"direction": "internal",
"iana_number": "17",
"transport": "udp",
"type": "ipv4"
},
Expand Down Expand Up @@ -301,14 +305,15 @@
"category": [
"network"
],
"created": "2022-09-04T21:50:40.666-05:00",
"created": "2020-04-28T11:07:58.223Z",
"kind": "alert",
"original": "09/04-21:50:40.666095 ,1,1000005,0,\"UDP Connection\",UDP,10.100.10.75,55776,10.100.10.255,32414,00:0C:29:B8:43:CE,FF:FF:FF:FF:FF:FF,0x3F,,,,,,64,0,37712,49,50176,,,,",
"timezone": "America/Chicago"
},
"network": {
"community_id": "1:NW0wNEOLThLuO4EsoJXFbyp6zII=",
"direction": "internal",
"iana_number": "17",
"transport": "udp",
"type": "ipv4"
},
Expand Down Expand Up @@ -382,14 +387,15 @@
"category": [
"network"
],
"created": "2022-09-04T21:49:55.900-05:00",
"created": "2020-04-28T11:07:58.223Z",
"kind": "alert",
"original": "09/04-21:49:55.900215 ,1,1000004,0,\"Pinging...\",ICMP,10.100.10.190,,175.16.199.1,,00:50:56:9D:A5:BE,00:25:90:3A:05:13,0x62,,,,,,64,0,37607,84,86016,8,0,83,1",
"timezone": "America/Chicago"
},
"network": {
"community_id": "1:jEaJsIOzda45Q8FjN0LeZEATihA=",
"direction": "outbound",
"iana_number": "1",
"transport": "icmp",
"type": "ipv4"
},
Expand Down Expand Up @@ -456,14 +462,15 @@
"category": [
"network"
],
"created": "2022-09-04T21:49:55.911-05:00",
"created": "2020-04-28T11:07:58.223Z",
"kind": "alert",
"original": "09/04-21:49:55.911592 ,1,1000004,0,\"Pinging...\",ICMP,175.16.199.1,,10.100.10.190,,00:25:90:3A:05:13,00:50:56:9D:A5:BE,0x62,,,,,,54,0,23522,84,86016,0,0,83,1",
"timezone": "America/Chicago"
},
"network": {
"community_id": "1:PsO7nB0G1KDZ1IDLocfXBmcxiaA=",
"direction": "inbound",
"iana_number": "1",
"transport": "icmp",
"type": "ipv4"
},
Expand Down Expand Up @@ -554,14 +561,15 @@
"category": [
"network"
],
"created": "2022-09-04T21:49:56.900-05:00",
"created": "2020-04-28T11:07:58.223Z",
"kind": "alert",
"original": "09/04-21:49:56.900997 ,1,1000004,0,\"Pinging...\",ICMP,10.100.10.190,,175.16.199.1,,00:50:56:9D:A5:BE,00:25:90:3A:05:13,0x62,,,,,,64,0,37636,84,86016,8,0,83,2",
"timezone": "America/Chicago"
},
"network": {
"community_id": "1:jEaJsIOzda45Q8FjN0LeZEATihA=",
"direction": "outbound",
"iana_number": "1",
"transport": "icmp",
"type": "ipv4"
},
Expand Down
30 changes: 20 additions & 10 deletions packages/snort/data_stream/log/_dev/test/pipeline/test-log-fast.log-expected.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
"category": [
"network"
],
"created": "2022-05-30T19:09:10.917-05:00",
"created": "2020-04-28T11:07:58.223Z",
"kind": "alert",
"original": "05/30-19:09:10.917356 [**] [1:527:8] BAD-TRAFFIC same SRC/DST [**] [Classification: Potentially Bad Traffic] [Priority: 2] {UDP} 0.0.0.0:68 -\u003e 255.255.255.255:67",
"severity": 2,
Expand All @@ -23,6 +23,7 @@
"network": {
"community_id": "1:t9O1j0qj71O4wJM7gnaHtgmfev8=",
"direction": "external",
"iana_number": "17",
"transport": "udp",
"type": "ipv4"
},
Expand Down Expand Up @@ -81,7 +82,7 @@
"category": [
"network"
],
"created": "2022-05-30T19:09:28.472-05:00",
"created": "2020-04-28T11:07:58.223Z",
"kind": "alert",
"original": "05/30-19:09:28.472094 [**] [1:2012811:2] ET DNS DNS Query to a .tk domain - Likely Hostile [**] [Classification: Potentially Bad Traffic] [Priority: 2] {UDP} 192.168.88.10:1029 -\u003e 175.16.199.1:53",
"severity": 2,
Expand All @@ -90,6 +91,7 @@
"network": {
"community_id": "1:RZ4iVwBzp5juqzQJiu5WebaF9J4=",
"direction": "outbound",
"iana_number": "17",
"transport": "udp",
"type": "ipv4"
},
Expand Down Expand Up @@ -147,7 +149,7 @@
"category": [
"network"
],
"created": "2022-05-30T19:09:10.917-05:00",
"created": "2020-04-28T11:07:58.223Z",
"kind": "alert",
"original": "05/30-19:09:10.917356 [**] [1:477:3] ICMP Packet [**] [Priority: 0] {ICMP} 175.16.199.1 -\u003e 175.16.199.1",
"severity": 0,
Expand All @@ -156,6 +158,7 @@
"network": {
"community_id": "1:ae//KI+huidgn9Nxeaibd8SUiVA=",
"direction": "external",
"iana_number": "1",
"transport": "icmp",
"type": "ipv4"
},
Expand Down Expand Up @@ -223,7 +226,7 @@
"category": [
"network"
],
"created": "2022-12-30T14:09:21.116-06:00",
"created": "2020-04-28T11:07:58.223Z",
"kind": "alert",
"original": "12/30-14:09:21.116402 [**] [1:1917:6] SCAN UPnP service discover attempt [**] [Classification: Detection of a Network Scan] [Priority: 3] {TCP} 192.168.15.10:1035 -\u003e 175.16.199.1:1900",
"severity": 3,
Expand All @@ -232,6 +235,7 @@
"network": {
"community_id": "1:lTRw3g8ZdxItqss80+SSa07uVWc=",
"direction": "outbound",
"iana_number": "6",
"transport": "tcp",
"type": "ipv4"
},
Expand Down Expand Up @@ -278,7 +282,7 @@
"category": [
"network"
],
"created": "2022-01-21T02:23:42.327-06:00",
"created": "2020-04-28T11:07:58.223Z",
"kind": "alert",
"original": "01/21-02:23:42.327730 [**] [1:2014520:2] ET INFO EXE - Served Attached HTTP [**] [Classification: Misc activity] [Priority: 3] {TCP} 175.16.199.1:80 -\u003e 192.168.115.10:1051",
"severity": 3,
Expand All @@ -287,6 +291,7 @@
"network": {
"community_id": "1:qSaSgRpopkbN/a7ST5y66ztJl8U=",
"direction": "inbound",
"iana_number": "6",
"transport": "tcp",
"type": "ipv4"
},
Expand Down Expand Up @@ -344,7 +349,7 @@
"category": [
"network"
],
"created": "2022-01-21T02:23:42.208-06:00",
"created": "2020-04-28T11:07:58.223Z",
"kind": "alert",
"original": "01/21-02:23:42.208605 [**] [1:408:5] ICMP Echo Reply [**] [Classification: Misc activity] [Priority: 3] {ICMP} 175.16.199.1 -\u003e 192.168.115.10",
"severity": 3,
Expand All @@ -353,6 +358,7 @@
"network": {
"community_id": "1:EtB/zlC1JmfdF0An9MzN1EDqn7o=",
"direction": "inbound",
"iana_number": "1",
"transport": "icmp",
"type": "ipv4"
},
Expand Down Expand Up @@ -410,7 +416,7 @@
"category": [
"network"
],
"created": "2022-09-04T21:55:02.041-05:00",
"created": "2020-04-28T11:07:58.223Z",
"kind": "alert",
"original": "09/04-21:55:02.041364 [**] [1:1000005:0] UDP Connection [**] [Classification: A Network Trojan was Detected] [Priority: 1] {UDP} 175.16.199.1:53 -\u003e 10.100.10.190:54757",
"severity": 1,
Expand All @@ -419,6 +425,7 @@
"network": {
"community_id": "1:Rj/XwIFirLCUpBLJSDip5ZzpVZY=",
"direction": "inbound",
"iana_number": "17",
"transport": "udp",
"type": "ipv4"
},
Expand Down Expand Up @@ -477,7 +484,7 @@
"category": [
"network"
],
"created": "2022-09-04T21:55:02.118-05:00",
"created": "2020-04-28T11:07:58.223Z",
"kind": "alert",
"original": "09/04-21:55:02.118427 [**] [1:1000005:0] UDP Connection [**] [Classification: A Network Trojan was Detected] [Priority: 1] {UDP} 175.16.199.1:53 -\u003e 10.100.10.190:36312",
"severity": 1,
Expand All @@ -486,6 +493,7 @@
"network": {
"community_id": "1:lFRQEVyjqFCLDyAOzC3sRuoFLkI=",
"direction": "inbound",
"iana_number": "17",
"transport": "udp",
"type": "ipv4"
},
Expand Down Expand Up @@ -555,7 +563,7 @@
"category": [
"network"
],
"created": "2022-09-04T21:54:43.216-05:00",
"created": "2020-04-28T11:07:58.223Z",
"kind": "alert",
"original": "09/04-21:54:43.216486 [**] [1:1000004:0] Pinging... [**] [Classification: Attempted Information Leak] [Priority: 2] {ICMP} 10.100.10.190 -\u003e 175.16.199.1",
"severity": 2,
Expand All @@ -564,6 +572,7 @@
"network": {
"community_id": "1:jEaJsIOzda45Q8FjN0LeZEATihA=",
"direction": "outbound",
"iana_number": "1",
"transport": "icmp",
"type": "ipv4"
},
Expand Down Expand Up @@ -608,7 +617,7 @@
"category": [
"network"
],
"created": "2022-09-04T21:54:43.227-05:00",
"created": "2020-04-28T11:07:58.223Z",
"kind": "alert",
"original": "09/04-21:54:43.227117 [**] [1:1000004:0] Pinging... [**] [Classification: Attempted Information Leak] [Priority: 2] {ICMP} 175.16.199.1 -\u003e 10.100.10.190",
"severity": 2,
Expand All @@ -617,6 +626,7 @@
"network": {
"community_id": "1:PsO7nB0G1KDZ1IDLocfXBmcxiaA=",
"direction": "inbound",
"iana_number": "1",
"transport": "icmp",
"type": "ipv4"
},
Expand Down
2 changes: 0 additions & 2 deletions packages/snort/data_stream/log/_dev/test/pipeline/test-log-full.log-config.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,4 @@
dynamic_fields:
event.created: "^[0-9]{4}(-[0-9]{2}){2}T[0-9]{2}(:[0-9]{2}){2}\\.[0-9]{3}-[0-9]{2}:[0-9]{2}$"
event.ingested: ".*"
"@timestamp": "^[0-9]{4}(-[0-9]{2}){2}T[0-9]{2}(:[0-9]{2}){2}\\.[0-9]{3}-[0-9]{2}:[0-9]{2}$"
fields:
"@timestamp": "2020-04-28T11:07:58.223Z"
Expand Down
Loading