Skip to content

Cherry-pick #18525 to 7.7: Fix PANW bad mapping of client/source and server/dest packets and bytes #18533

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
May 15, 2020
Merged

Cherry-pick #18525 to 7.7: Fix PANW bad mapping of client/source and server/dest packets and bytes #18533

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
1365180
Fix PANW bad mapping of client/source and server/dest packets and byt…
adriansr May 14, 2020
a0c50d9
Merge branch '7.7' into backport_18525_7.7
adriansr May 15, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.next.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -131,6 +131,7 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d
- Fix `googlecloud.audit` pipeline to only take in fields that are explicitly defined by the dataset. {issue}18465[18465] {pull}18472[18472]
- Fix a rate limit related issue in httpjson input for Okta module. {issue}18530[18530] {pull}18534[18534]
- Fixed ingestion of some Cisco ASA and FTD messages when a hostname was used instead of an IP for NAT fields. {issue}14034[14034] {pull}18376[18376]
- Fix PANW module wrong mappings for bytes and packets counters. {issue}18522[18522] {pull}18525[18525]

*Heartbeat*

Expand Down
28 changes: 22 additions & 6 deletions x-pack/filebeat/module/panw/panos/config/input.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,41 +50,49 @@ processors:
destination.ip: 8
destination.address: 8
source.nat.ip: 9
client.nat.ip: 9
destination.nat.ip: 10
server.nat.ip: 10
panw.panos.ruleset: 11
client.user.name: 12
source.user.name: 12
server.user.name: 13
destination.user.name: 13
network.application: 14
panw.panos.source.zone: 16
observer.ingress.zone: 16
panw.panos.destination.zone: 17
observer.egress.zone: 17
panw.panos.source.interface: 18
observer.ingress.interface.name: 18
panw.panos.destination.interface: 19
observer.egress.interface.name: 19
panw.panos.flow_id: 22
client.port: 24
source.port: 24
destination.port: 25
server.port: 25
source.nat.port: 26
client.nat.port: 26
destination.nat.port: 27
server.nat.port: 27
_temp_.labels: 28
network.transport: 29
event.outcome: 30
network.bytes: 31
client.bytes: 32
destination.bytes: 32
source.bytes: 32
server.bytes: 33
source.bytes: 33
destination.bytes: 33
network.packets: 34
event.start: 35
event.duration: 36
panw.panos.url.category: 37
panw.panos.sequence_number: 39
server.packets: 44
destination.packets: 44
client.packets: 45
source.packets: 45
client.packets: 44
source.packets: 44
server.packets: 45
destination.packets: 45
observer.hostname: 52

- extract_array:
Expand All @@ -103,24 +111,32 @@ processors:
destination.ip: 8
destination.address: 8
source.nat.ip: 9
client.nat.ip: 9
destination.nat.ip: 10
server.nat.ip: 10
panw.panos.ruleset: 11
client.user.name: 12
source.user.name: 12
server.user.name: 13
destination.user.name: 13
network.application: 14
panw.panos.source.zone: 16
observer.ingress.zone: 16
panw.panos.destination.zone: 17
observer.egress.zone: 17
panw.panos.source.interface: 18
observer.ingress.interface.name: 18
panw.panos.destination.interface: 19
observer.egress.interface.name: 19
panw.panos.flow_id: 22
client.port: 24
source.port: 24
destination.port: 25
server.port: 25
source.nat.port: 26
client.nat.port: 26
destination.nat.port: 27
server.nat.port: 27
_temp_.labels: 28
network.transport: 29
event.outcome: 30
Expand Down
18 changes: 18 additions & 0 deletions x-pack/filebeat/module/panw/panos/ingest/pipeline.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,8 @@ processors:
- convert: { type: long, ignore_missing: true, field: panw.panos.sequence_number }
- convert: { type: long, ignore_missing: true, field: source.nat.port }
- convert: { type: long, ignore_missing: true, field: destination.nat.port }
- convert: { type: long, ignore_missing: true, field: client.nat.port }
- convert: { type: long, ignore_missing: true, field: server.nat.port }

# Remove PCAP ID when zero (no packet capture).
- remove:
Expand Down Expand Up @@ -397,6 +399,22 @@ processors:
- _temp_
ignore_missing: true

# Remove NAT fields when translation was not done.
- remove:
field:
- source.nat.ip
- source.nat.port
- client.nat.ip
- client.nat.port
if: 'ctx?.source?.nat?.ip == "0.0.0.0" && ctx?.source?.nat?.port == 0'
- remove:
field:
- destination.nat.ip
- destination.nat.port
- server.nat.ip
- server.nat.port
if: 'ctx?.destination?.nat?.ip == "0.0.0.0" && ctx?.destination?.nat?.port == 0'

on_failure:
- set:
field: "error.message"
Expand Down
20 changes: 10 additions & 10 deletions x-pack/filebeat/module/panw/panos/test/pan_inc_other.log-expected.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,13 +3,13 @@
"@timestamp": "2012-04-10T04:39:56.000-02:00",
"client.bytes": 78,
"client.ip": "192.168.0.2",
"client.packets": 0,
"client.packets": 1,
"client.port": 59309,
"client.user.name": "crusher",
"destination.address": "204.232.231.46",
"destination.as.number": 27357,
"destination.as.organization.name": "Rackspace Hosting",
"destination.bytes": 78,
"destination.bytes": 0,
"destination.geo.city_name": "Fort Lauderdale",
"destination.geo.continent_name": "North America",
"destination.geo.country_iso_code": "US",
Expand All @@ -18,9 +18,7 @@
"destination.geo.region_iso_code": "US-FL",
"destination.geo.region_name": "Florida",
"destination.ip": "204.232.231.46",
"destination.nat.ip": "0.0.0.0",
"destination.nat.port": 0,
"destination.packets": 1,
"destination.packets": 0,
"destination.port": 80,
"event.action": "flow_started",
"event.category": "network_traffic",
Expand All @@ -43,6 +41,10 @@
"network.packets": 1,
"network.transport": "tcp",
"network.type": "ipv4",
"observer.egress.interface.name": "ethernet1/1",
"observer.egress.zone": "untrust",
"observer.ingress.interface.name": "ethernet1/2",
"observer.ingress.zone": "trust",
"observer.serial_number": "01606001116",
"panw.panos.destination.interface": "ethernet1/1",
"panw.panos.destination.nat.ip": "0.0.0.0",
Expand All @@ -64,15 +66,13 @@
],
"server.bytes": 0,
"server.ip": "204.232.231.46",
"server.packets": 1,
"server.packets": 0,
"server.port": 80,
"service.type": "panw",
"source.address": "192.168.0.2",
"source.bytes": 0,
"source.bytes": 78,
"source.ip": "192.168.0.2",
"source.nat.ip": "0.0.0.0",
"source.nat.port": 0,
"source.packets": 0,
"source.packets": 1,
"source.port": 59309,
"source.user.name": "crusher",
"tags": [
Expand Down
Loading