Skip to content

x-filebeat/modules/cisco: fix syslog priority handling#32025

Merged
efd6 merged 3 commits intoelastic:mainfrom
efd6:3507int-cisco_asa
Jun 23, 2022
Merged

x-filebeat/modules/cisco: fix syslog priority handling#32025
efd6 merged 3 commits intoelastic:mainfrom
efd6:3507int-cisco_asa

Conversation

@efd6
Copy link
Contributor

@efd6 efd6 commented Jun 21, 2022

What does this PR do?

This fixes the mapping of syslog priority and its derived values and their extraction.

Why is it important?

The mapping is currently incorrect and syslog written facility values are wrong.

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Author's Checklist

  • [ ]

How to test this PR locally

Related issues

Use cases

Screenshots

Logs

@efd6 efd6 added bug Team:Security-External Integrations backport-v8.3.0 Automated backport with mergify backport-7.17 Automated backport to the 7.17 branch with mergify 8.4-candidate labels Jun 21, 2022
@efd6 efd6 requested a review from a team as a code owner June 21, 2022 23:53
@efd6 efd6 self-assigned this Jun 21, 2022
@elasticmachine
Copy link
Contributor

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

@botelastic botelastic bot added needs_team Indicates that the issue/PR needs a Team:* label and removed needs_team Indicates that the issue/PR needs a Team:* label labels Jun 21, 2022
@efd6 efd6 force-pushed the 3507int-cisco_asa branch from 3525b9d to 8060c82 Compare June 22, 2022 01:23
@efd6 efd6 changed the title x-filebeat/modules/cisco/asa: fix syslog priority handling x-filebeat/modules/cisco: fix syslog priority handling Jun 22, 2022
@elasticmachine
Copy link
Contributor

elasticmachine commented Jun 22, 2022

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2022-06-23T01:40:23.634+0000

  • Duration: 75 min 27 sec

Test stats 🧪

Test Results
Failed 0
Passed 2130
Skipped 166
Total 2296

💚 Flaky test report

Tests succeeded.

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

  • /package : Generate the packages and run the E2E tests.

  • /beats-tester : Run the installation tests with beats-tester.

  • run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

@efd6 efd6 force-pushed the 3507int-cisco_asa branch from 1e5dc16 to 4f79ef0 Compare June 22, 2022 05:15
@efd6
Copy link
Contributor Author

efd6 commented Jun 22, 2022

Windows failures look to be unrelated.

@efd6
Copy link
Contributor Author

efd6 commented Jun 22, 2022

/test

@efd6 efd6 force-pushed the 3507int-cisco_asa branch from 4f79ef0 to 91277ad Compare June 22, 2022 08:31
@efd6
Copy link
Contributor Author

efd6 commented Jun 22, 2022

@andrewkroh What is your preferred approach to dealing with the test case?

@efd6 efd6 force-pushed the 3507int-cisco_asa branch from 91277ad to 1c20ce5 Compare June 22, 2022 22:24
efd6 added 2 commits June 23, 2022 07:55
This is a temporary work around for the test case line

 <14.2>Aug 14 2019 14:02:38 platformSettingEdit.cgi: siem-management: admin@10.0.255.31, Devices > Platform Settings > Audit Log Settings >  Modified: Send Audit Log to Syslog enabled > Disabled

which includes an invalid syslog priority. This format does not appear anywhere
that I can find in the Cisco documentation and I cannot find another similar
example.

The original grok pattern definition was

 <%{NONNEGINT:syslog.facility:int}(?:.%{NONNEGINT:syslog.priority:int})?>

which gives a result for this input that does not make sense since pri > facility
according to the specs. Perhaps facility.severity? But this is inconsistent with
the remaining log lines and also with the semantics of the log line here. The
alternative interpretations:

- 14.2 → facility 14 severity 2 → "log alert", "Critical"
- 14 → facility 1 severity 6 → "user-level messages", "Informational"

My suspicion is that it is a corrupted input.
@efd6 efd6 force-pushed the 3507int-cisco_asa branch from 1c20ce5 to e0e2a98 Compare June 22, 2022 22:26
@andrewkroh
Copy link
Member

What is your preferred approach to dealing with the test case?

My suspicion is that it is a corrupted input.

I also tried to find outside examples that had this format and it turned up nothing. So removing it is good with me. If we're wrong then we can add it back.

@efd6
Copy link
Contributor Author

efd6 commented Jun 23, 2022

It would be nice to know the provenance of the test case file since it is text with \0a\00\0a\00 line termination, which means it's opaque to review changes. My guess is that it was captured from a syslog network conn.

diff --git a/x-pack/filebeat/module/cisco/ftd/test/firepower-management.log2 b/x-pack/filebeat/module/cisco/ftd/test/firepower-management.log2
index cf204804bb..e362bb7f2d 100644
--- a/x-pack/filebeat/module/cisco/ftd/test/firepower-management.log
+++ b/x-pack/filebeat/module/cisco/ftd/test/firepower-management.log
@@ -31,4 +31,4 @@
 <14>Aug 14 2019 14:02:19 mojo_server.pl: siem-management: admin@10.0.255.31, System > Configuration > Configuration, Page View
 <14>Aug 14 2019 14:02:31 platformSettingEdit.cgi: siem-management: admin@10.0.255.31, System > Configuration > Configuration > /platinum/platformSettingEdit.cgi?type=AuditLog, Page View
 <14>Aug 14 2019 14:02:38 platformSettingEdit.cgi: siem-management: admin@10.0.255.31, Devices > Platform Settings > Local System Configuration, Save Local System Configuration
-<14.2>Aug 14 2019 14:02:38 platformSettingEdit.cgi: siem-management: admin@10.0.255.31, Devices > Platform Settings > Audit Log Settings >  Modified: Send Audit Log to Syslog enabled > Disabled
+<14>Aug 14 2019 14:02:38 platformSettingEdit.cgi: siem-management: admin@10.0.255.31, Devices > Platform Settings > Audit Log Settings >  Modified: Send Audit Log to Syslog enabled > Disabled
@efd6 efd6 merged commit ea65707 into elastic:main Jun 23, 2022
mergify bot pushed a commit that referenced this pull request Jun 23, 2022
With this change one of the existing test cases fails because of a non-
standard syslog pri value.

 <14.2>Aug 14 2019 14:02:38 platformSettingEdit.cgi: siem-management: admin@10.0.255.31, Devices > Platform Settings > Audit Log Settings >  Modified: Send Audit Log to Syslog enabled > Disabled

This format does not appear anywhere that I can find in the Cisco
documentation and I cannot find another similar example.

The original grok pattern definition was

 <%{NONNEGINT:syslog.facility:int}(?:.%{NONNEGINT:syslog.priority:int})?>

which gives a result for this input that does not make sense since pri > facility
according to the specs. Perhaps facility.severity? But this is inconsistent with
the remaining log lines and also with the semantics of the log line here. The
alternative interpretations are:

- 14.2 → facility 14 severity 2 → "log alert", "Critical"
- 14 → facility 1 severity 6 → "user-level messages", "Informational"

Note also that the pattern definition uses an unescaped dot rather than
a dot literal which is surprising.

My suspicion is that it is a corrupted input and I have edited it to prevent
the failure. Since the input is line terminated by "\00x0a\00x00" it shows
us as a binary file in the diff, so the change is shown here:

diff --git a/x-pack/filebeat/module/cisco/ftd/test/firepower-management.log2 b/x-pack/filebeat/module/cisco/ftd/test/firepower-management.log2
index cf204804bb..e362bb7f2d 100644
--- a/x-pack/filebeat/module/cisco/ftd/test/firepower-management.log
+++ b/x-pack/filebeat/module/cisco/ftd/test/firepower-management.log
@@ -31,4 +31,4 @@
 <14>Aug 14 2019 14:02:19 mojo_server.pl: siem-management: admin@10.0.255.31, System > Configuration > Configuration, Page View
 <14>Aug 14 2019 14:02:31 platformSettingEdit.cgi: siem-management: admin@10.0.255.31, System > Configuration > Configuration > /platinum/platformSettingEdit.cgi?type=AuditLog, Page View
 <14>Aug 14 2019 14:02:38 platformSettingEdit.cgi: siem-management: admin@10.0.255.31, Devices > Platform Settings > Local System Configuration, Save Local System Configuration
-<14.2>Aug 14 2019 14:02:38 platformSettingEdit.cgi: siem-management: admin@10.0.255.31, Devices > Platform Settings > Audit Log Settings >  Modified: Send Audit Log to Syslog enabled > Disabled
+<14>Aug 14 2019 14:02:38 platformSettingEdit.cgi: siem-management: admin@10.0.255.31, Devices > Platform Settings > Audit Log Settings >  Modified: Send Audit Log to Syslog enabled > Disabled

(cherry picked from commit ea65707)

# Conflicts:
#	x-pack/filebeat/module/cisco/asa/test/sample.log-expected.json
mergify bot pushed a commit that referenced this pull request Jun 23, 2022
With this change one of the existing test cases fails because of a non-
standard syslog pri value.

 <14.2>Aug 14 2019 14:02:38 platformSettingEdit.cgi: siem-management: admin@10.0.255.31, Devices > Platform Settings > Audit Log Settings >  Modified: Send Audit Log to Syslog enabled > Disabled

This format does not appear anywhere that I can find in the Cisco
documentation and I cannot find another similar example.

The original grok pattern definition was

 <%{NONNEGINT:syslog.facility:int}(?:.%{NONNEGINT:syslog.priority:int})?>

which gives a result for this input that does not make sense since pri > facility
according to the specs. Perhaps facility.severity? But this is inconsistent with
the remaining log lines and also with the semantics of the log line here. The
alternative interpretations are:

- 14.2 → facility 14 severity 2 → "log alert", "Critical"
- 14 → facility 1 severity 6 → "user-level messages", "Informational"

Note also that the pattern definition uses an unescaped dot rather than
a dot literal which is surprising.

My suspicion is that it is a corrupted input and I have edited it to prevent
the failure. Since the input is line terminated by "\00x0a\00x00" it shows
us as a binary file in the diff, so the change is shown here:

diff --git a/x-pack/filebeat/module/cisco/ftd/test/firepower-management.log2 b/x-pack/filebeat/module/cisco/ftd/test/firepower-management.log2
index cf204804bb..e362bb7f2d 100644
--- a/x-pack/filebeat/module/cisco/ftd/test/firepower-management.log
+++ b/x-pack/filebeat/module/cisco/ftd/test/firepower-management.log
@@ -31,4 +31,4 @@
 <14>Aug 14 2019 14:02:19 mojo_server.pl: siem-management: admin@10.0.255.31, System > Configuration > Configuration, Page View
 <14>Aug 14 2019 14:02:31 platformSettingEdit.cgi: siem-management: admin@10.0.255.31, System > Configuration > Configuration > /platinum/platformSettingEdit.cgi?type=AuditLog, Page View
 <14>Aug 14 2019 14:02:38 platformSettingEdit.cgi: siem-management: admin@10.0.255.31, Devices > Platform Settings > Local System Configuration, Save Local System Configuration
-<14.2>Aug 14 2019 14:02:38 platformSettingEdit.cgi: siem-management: admin@10.0.255.31, Devices > Platform Settings > Audit Log Settings >  Modified: Send Audit Log to Syslog enabled > Disabled
+<14>Aug 14 2019 14:02:38 platformSettingEdit.cgi: siem-management: admin@10.0.255.31, Devices > Platform Settings > Audit Log Settings >  Modified: Send Audit Log to Syslog enabled > Disabled

(cherry picked from commit ea65707)
efd6 added a commit that referenced this pull request Jun 23, 2022
With this change one of the existing test cases fails because of a non-
standard syslog pri value.

 <14.2>Aug 14 2019 14:02:38 platformSettingEdit.cgi: siem-management: admin@10.0.255.31, Devices > Platform Settings > Audit Log Settings >  Modified: Send Audit Log to Syslog enabled > Disabled

This format does not appear anywhere that I can find in the Cisco
documentation and I cannot find another similar example.

The original grok pattern definition was

 <%{NONNEGINT:syslog.facility:int}(?:.%{NONNEGINT:syslog.priority:int})?>

which gives a result for this input that does not make sense since pri > facility
according to the specs. Perhaps facility.severity? But this is inconsistent with
the remaining log lines and also with the semantics of the log line here. The
alternative interpretations are:

- 14.2 → facility 14 severity 2 → "log alert", "Critical"
- 14 → facility 1 severity 6 → "user-level messages", "Informational"

Note also that the pattern definition uses an unescaped dot rather than
a dot literal which is surprising.

My suspicion is that it is a corrupted input and I have edited it to prevent
the failure. Since the input is line terminated by "\00x0a\00x00" it shows
us as a binary file in the diff, so the change is shown here:

diff --git a/x-pack/filebeat/module/cisco/ftd/test/firepower-management.log2 b/x-pack/filebeat/module/cisco/ftd/test/firepower-management.log2
index cf204804bb..e362bb7f2d 100644
--- a/x-pack/filebeat/module/cisco/ftd/test/firepower-management.log
+++ b/x-pack/filebeat/module/cisco/ftd/test/firepower-management.log
@@ -31,4 +31,4 @@
 <14>Aug 14 2019 14:02:19 mojo_server.pl: siem-management: admin@10.0.255.31, System > Configuration > Configuration, Page View
 <14>Aug 14 2019 14:02:31 platformSettingEdit.cgi: siem-management: admin@10.0.255.31, System > Configuration > Configuration > /platinum/platformSettingEdit.cgi?type=AuditLog, Page View
 <14>Aug 14 2019 14:02:38 platformSettingEdit.cgi: siem-management: admin@10.0.255.31, Devices > Platform Settings > Local System Configuration, Save Local System Configuration
-<14.2>Aug 14 2019 14:02:38 platformSettingEdit.cgi: siem-management: admin@10.0.255.31, Devices > Platform Settings > Audit Log Settings >  Modified: Send Audit Log to Syslog enabled > Disabled
+<14>Aug 14 2019 14:02:38 platformSettingEdit.cgi: siem-management: admin@10.0.255.31, Devices > Platform Settings > Audit Log Settings >  Modified: Send Audit Log to Syslog enabled > Disabled

(cherry picked from commit ea65707)

Co-authored-by: Dan Kortschak <90160302+efd6@users.noreply.github.com>
efd6 added a commit that referenced this pull request Jun 23, 2022
… handling (#32062)

* x-filebeat/modules/cisco: fix syslog priority handling (#32025)

With this change one of the existing test cases fails because of a non-
standard syslog pri value.

 <14.2>Aug 14 2019 14:02:38 platformSettingEdit.cgi: siem-management: admin@10.0.255.31, Devices > Platform Settings > Audit Log Settings >  Modified: Send Audit Log to Syslog enabled > Disabled

This format does not appear anywhere that I can find in the Cisco
documentation and I cannot find another similar example.

The original grok pattern definition was

 <%{NONNEGINT:syslog.facility:int}(?:.%{NONNEGINT:syslog.priority:int})?>

which gives a result for this input that does not make sense since pri > facility
according to the specs. Perhaps facility.severity? But this is inconsistent with
the remaining log lines and also with the semantics of the log line here. The
alternative interpretations are:

- 14.2 → facility 14 severity 2 → "log alert", "Critical"
- 14 → facility 1 severity 6 → "user-level messages", "Informational"

Note also that the pattern definition uses an unescaped dot rather than
a dot literal which is surprising.

My suspicion is that it is a corrupted input and I have edited it to prevent
the failure. Since the input is line terminated by "\00x0a\00x00" it shows
us as a binary file in the diff, so the change is shown here:

diff --git a/x-pack/filebeat/module/cisco/ftd/test/firepower-management.log2 b/x-pack/filebeat/module/cisco/ftd/test/firepower-management.log2
index cf204804bb..e362bb7f2d 100644
--- a/x-pack/filebeat/module/cisco/ftd/test/firepower-management.log
+++ b/x-pack/filebeat/module/cisco/ftd/test/firepower-management.log
@@ -31,4 +31,4 @@
 <14>Aug 14 2019 14:02:19 mojo_server.pl: siem-management: admin@10.0.255.31, System > Configuration > Configuration, Page View
 <14>Aug 14 2019 14:02:31 platformSettingEdit.cgi: siem-management: admin@10.0.255.31, System > Configuration > Configuration > /platinum/platformSettingEdit.cgi?type=AuditLog, Page View
 <14>Aug 14 2019 14:02:38 platformSettingEdit.cgi: siem-management: admin@10.0.255.31, Devices > Platform Settings > Local System Configuration, Save Local System Configuration
-<14.2>Aug 14 2019 14:02:38 platformSettingEdit.cgi: siem-management: admin@10.0.255.31, Devices > Platform Settings > Audit Log Settings >  Modified: Send Audit Log to Syslog enabled > Disabled
+<14>Aug 14 2019 14:02:38 platformSettingEdit.cgi: siem-management: admin@10.0.255.31, Devices > Platform Settings > Audit Log Settings >  Modified: Send Audit Log to Syslog enabled > Disabled

(cherry picked from commit ea65707)

# Conflicts:
#	x-pack/filebeat/module/cisco/asa/test/sample.log-expected.json

* resolve conflicts

Co-authored-by: Dan Kortschak <90160302+efd6@users.noreply.github.com>
Co-authored-by: Dan Kortschak <dan.kortschak@elastic.co>
leweafan pushed a commit to leweafan/beats that referenced this pull request Apr 28, 2023
…riority handling (elastic#32062)

* x-filebeat/modules/cisco: fix syslog priority handling (elastic#32025)

With this change one of the existing test cases fails because of a non-
standard syslog pri value.

 <14.2>Aug 14 2019 14:02:38 platformSettingEdit.cgi: siem-management: admin@10.0.255.31, Devices > Platform Settings > Audit Log Settings >  Modified: Send Audit Log to Syslog enabled > Disabled

This format does not appear anywhere that I can find in the Cisco
documentation and I cannot find another similar example.

The original grok pattern definition was

 <%{NONNEGINT:syslog.facility:int}(?:.%{NONNEGINT:syslog.priority:int})?>

which gives a result for this input that does not make sense since pri > facility
according to the specs. Perhaps facility.severity? But this is inconsistent with
the remaining log lines and also with the semantics of the log line here. The
alternative interpretations are:

- 14.2 → facility 14 severity 2 → "log alert", "Critical"
- 14 → facility 1 severity 6 → "user-level messages", "Informational"

Note also that the pattern definition uses an unescaped dot rather than
a dot literal which is surprising.

My suspicion is that it is a corrupted input and I have edited it to prevent
the failure. Since the input is line terminated by "\00x0a\00x00" it shows
us as a binary file in the diff, so the change is shown here:

diff --git a/x-pack/filebeat/module/cisco/ftd/test/firepower-management.log2 b/x-pack/filebeat/module/cisco/ftd/test/firepower-management.log2
index cf204804bb..e362bb7f2d 100644
--- a/x-pack/filebeat/module/cisco/ftd/test/firepower-management.log
+++ b/x-pack/filebeat/module/cisco/ftd/test/firepower-management.log
@@ -31,4 +31,4 @@
 <14>Aug 14 2019 14:02:19 mojo_server.pl: siem-management: admin@10.0.255.31, System > Configuration > Configuration, Page View
 <14>Aug 14 2019 14:02:31 platformSettingEdit.cgi: siem-management: admin@10.0.255.31, System > Configuration > Configuration > /platinum/platformSettingEdit.cgi?type=AuditLog, Page View
 <14>Aug 14 2019 14:02:38 platformSettingEdit.cgi: siem-management: admin@10.0.255.31, Devices > Platform Settings > Local System Configuration, Save Local System Configuration
-<14.2>Aug 14 2019 14:02:38 platformSettingEdit.cgi: siem-management: admin@10.0.255.31, Devices > Platform Settings > Audit Log Settings >  Modified: Send Audit Log to Syslog enabled > Disabled
+<14>Aug 14 2019 14:02:38 platformSettingEdit.cgi: siem-management: admin@10.0.255.31, Devices > Platform Settings > Audit Log Settings >  Modified: Send Audit Log to Syslog enabled > Disabled

(cherry picked from commit ea65707)

# Conflicts:
#	x-pack/filebeat/module/cisco/asa/test/sample.log-expected.json

* resolve conflicts

Co-authored-by: Dan Kortschak <90160302+efd6@users.noreply.github.com>
Co-authored-by: Dan Kortschak <dan.kortschak@elastic.co>
chrisberkhout pushed a commit that referenced this pull request Jun 1, 2023
With this change one of the existing test cases fails because of a non-
standard syslog pri value.

 <14.2>Aug 14 2019 14:02:38 platformSettingEdit.cgi: siem-management: admin@10.0.255.31, Devices > Platform Settings > Audit Log Settings >  Modified: Send Audit Log to Syslog enabled > Disabled

This format does not appear anywhere that I can find in the Cisco
documentation and I cannot find another similar example.

The original grok pattern definition was

 <%{NONNEGINT:syslog.facility:int}(?:.%{NONNEGINT:syslog.priority:int})?>

which gives a result for this input that does not make sense since pri > facility
according to the specs. Perhaps facility.severity? But this is inconsistent with
the remaining log lines and also with the semantics of the log line here. The
alternative interpretations are:

- 14.2 → facility 14 severity 2 → "log alert", "Critical"
- 14 → facility 1 severity 6 → "user-level messages", "Informational"

Note also that the pattern definition uses an unescaped dot rather than
a dot literal which is surprising.

My suspicion is that it is a corrupted input and I have edited it to prevent
the failure. Since the input is line terminated by "\00x0a\00x00" it shows
us as a binary file in the diff, so the change is shown here:

diff --git a/x-pack/filebeat/module/cisco/ftd/test/firepower-management.log2 b/x-pack/filebeat/module/cisco/ftd/test/firepower-management.log2
index cf204804bb..e362bb7f2d 100644
--- a/x-pack/filebeat/module/cisco/ftd/test/firepower-management.log
+++ b/x-pack/filebeat/module/cisco/ftd/test/firepower-management.log
@@ -31,4 +31,4 @@
 <14>Aug 14 2019 14:02:19 mojo_server.pl: siem-management: admin@10.0.255.31, System > Configuration > Configuration, Page View
 <14>Aug 14 2019 14:02:31 platformSettingEdit.cgi: siem-management: admin@10.0.255.31, System > Configuration > Configuration > /platinum/platformSettingEdit.cgi?type=AuditLog, Page View
 <14>Aug 14 2019 14:02:38 platformSettingEdit.cgi: siem-management: admin@10.0.255.31, Devices > Platform Settings > Local System Configuration, Save Local System Configuration
-<14.2>Aug 14 2019 14:02:38 platformSettingEdit.cgi: siem-management: admin@10.0.255.31, Devices > Platform Settings > Audit Log Settings >  Modified: Send Audit Log to Syslog enabled > Disabled
+<14>Aug 14 2019 14:02:38 platformSettingEdit.cgi: siem-management: admin@10.0.255.31, Devices > Platform Settings > Audit Log Settings >  Modified: Send Audit Log to Syslog enabled > Disabled
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

8.4-candidate backport-7.17 Automated backport to the 7.17 branch with mergify backport-v8.3.0 Automated backport with mergify bug

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants