Wrong format of the syslog messages

[thl-cmk]

Your syslog message looks at the moment like this

<133>{"time": "2022-01-01 19:35:25+0100", "hostname": "checkmk", "path": "/usr/bin/pycharm/pycharm-community-2021.1.3/lib/log4j.jar", "entry": "", "product": "Log4j 1", "version": "1.2.17.2", "cve": "CVE-2021-4104", "status": "MITIGATED", "fixed": false}

Acording to RFC5424 the header (the stuff before the STRUCTURED-DATA/MESSAGE) should look like this

PRI VERSION SP TIMESTAMP SP HOSTNAME SP APP-NAME SP PROCID SP MSGID

followed by SP STRUCTURED-DATA [SP MSG] (structured data must always be encoded in UTF-8). If the MESSAGE is UTF-8 encoded it must start with the BOM (BOM = %xEF.BB.BF).

You are using only the PRI (Facility/Severity) field (<133>) and the Message. This causes problems with some syslog implementations as the expect the message to be like in the RFC.

If you want to skip optional fileds you can use the NILVALUE (-).

So your syslog message should look at least like this

<133> 1 - - - - - - {your message}

It would be nice to have the TIMESTAMP, HOSTNAME and APP-NAME populated like this (skipped PROCID, MSGID and STRUCTURED-DATA)

<133> 1 2022-01-01T23:20:50.52Z CHECKMK LOG4J-SCANNER - - - {your message}

[xeraph]

@thl-cmk Would you comment some implementations which cannot parse current syslog? AFAIK, most log management or siem products have regex or grok configurations for syslog parsing. I agree with your opinion that it's worth complying with the RFC, but most security devices use pri header only. I saw 100+ log formats over 15 years as a SIEM developer..

[thl-cmk]

@xeraph At the moment I am only awere of one specific syslog implementation, it's CHECKMK, the monitoring system where I am integrating your scanner with. There is some automatic stuff going on begore one can access the massage (for futher filtering) and in this process the message is broken. Looks like this

A second implementation, that expects an RFC conform syslog message is filebeat for Elasticsearch or Logstash. This also expects rfc3164 or rfc5424 messages.

https://www.elastic.co/guide/en/beats/filebeat/current/filebeat-input-syslog.html

[xeraph]

This is a real world.. Even though they say it conforms to RFC5424, it cannot recognize BOM.

[xeraph]

@thl-cmk Would you test v2.7.0? Use --rfc5424 option to follow rfc5424. It would be great if checkmk screenshot is attached.

[thl-cmk]

@xeraph Great work :-) I certainly will test it.

I was in the process of closing this, as after more digging the issue with the syslog implementaion of CHECKMK is not only because of the non rfc5424 formating. Guess I have to check this with the tribe29 guys anyway :-(

and now I will test all the new options :-)

[thl-cmk]

I think ther is still a little issue with the rfc5424 formating, At the moment
facillity/severity is still ok.
version > now ok
timestamp > now ok
hostname > now ok
app name > not ok
process-id > not ok
message id > not ok

I guess the app name sould be LOPGRESSO LOG4J2-SCAN DETECT but because of the whitespaces it gets split into
app name > LOPGRESSO
process-id > LOG4J2-SCAN
message id > DETECT

so this needs to be changed into LOPGRESSO_LOG4J2-SCAN_DETECT or something else without withe spaces, followed by - - for process-id and message-id

[xeraph]

That is exactly what I intended :)
message id can be DETECT or ERROR

[thl-cmk]

Ok, and process-id should be LOG4J2-SCAN?
When do I get the ERROR?

At the moment the Skipping broken jar messages are all gone in syslog, is this also intended?

[xeraph]

Ok, and process-id should be LOG4J2-SCAN? When do I get the ERROR?

Yes. Process name LOG4J2-SCAN is intended.

RFC5424 6.2.6

The PROCID field is often used to provide the process name or process ID associated with a syslog system.

At the moment the Skipping broken jar messages are all gone in syslog, is this also intended?

If --syslog-level debug is specified, such message can be sent.

CVE-2021-44228-Scanner/src/main/java/com/logpresso/scanner/LogGenerator.java

Lines 144 to 157 in 8ca2110

	private void sendErrorSyslog(ReportEntry entry) {
	if (config.getSyslogLevel() != SyslogLevel.DEBUG)
	return;
	try {
	String msg = getErrorLog(entry);
	byte[] syslog = formatSyslog("<135>", "ERROR", msg);
	sendSyslogPacket(socket, config.getUdpSyslogAddr(), syslog);
	} catch (Throwable t) {
	System.out.println("Error: Cannot send syslog to " + config.getUdpSyslogAddr() + " - " + t.getMessage());
	if (config.isDebug())
	t.printStackTrace();
	}
	}

[thl-cmk]

CHECKMKs syslog implementation looks fine now ;-) THX

so I need only discuss the non rfc5424 message handling with tribe29

[xeraph]

Oops.. I found typo from attached image.. I will fix it later..

[thl-cmk]

If --syslog-level debug is specified, such message can be sent.

this is specified but i recive no debug messages :-(
Here the output from the scanner
`
thl-cmk@surfbox-ii:~$ /usr/lib/check_mk_agent/bin/log4j2-scan --syslog-level debug --syslog-udp checkmk --rfc5424 --scan-logback --scan-log4j1 --scan-zip --no-symlink --silent /
Logpresso CVE-2021-44228 Vulnerability Scanner 2.7.0 (2022-01-02)
Scanning directory: / (without /dev, /run, /dev/shm, /run/lock, /sys/fs/cgroup, /proc/sys/fs/binfmt_misc, /run/user/1000, /mnt/build, /mnt/cmk16nb)
Skipping broken jar file /home/thl/.cache/JetBrains/PyCharmCE2020.3/plugins/ideolog.zip ('Unexpected record signature: 0X1614')
Skipping broken jar file /home/thl/.cache/JetBrains/PyCharmCE2021.1/plugins/PowerShell.zip ('Unexpected record signature: 0X1754')
Skipping broken jar file /home/thl/Nextcloud/threema/threema-backup_RC3TCX3J_1506148683151_1.zip ('unsupported feature encryption used in entry settings')
Skipping broken jar file /home/thl/Nextcloud/threema/threema-backup_RC3TCX3J_1529166203088_1.zip ('unsupported feature encryption used in entry settings')
Skipping broken jar file /home/thl/Nextcloud/threema/threema-backup_RC3TCX3J_1529154326207_1.zip ('unsupported feature encryption used in entry settings')
[?] Found CVE-2021-4104 (log4j 1.2) vulnerability in /usr/bin/pycharm/pycharm-community-2021.1.3/lib/log4j.jar, log4j 1.2.17.2 (mitigated)
Skipping broken jar file /usr/share/java/libbase-1.1-SNAPSHOT.jar ('Unexpected record signature: 0XA0403')

Scanned 68698 directories and 389080 files
Found 0 vulnerable files
Found 0 potentially vulnerable files
Found 1 mitigated files
Completed in 127.75 seconds
`

and the corosponding output from the tcpdump on the syslog side
`
thl-cmk@checkmk:~$ sudo tcpdump -nnvti any udp port 514
IP (tos 0x0, ttl 64, id 48578, offset 0, flags [DF], proto UDP (17), length 357)
192.168.10.133.33895 > 192.168.10.13.514: SYSLOG, length: 329
Facility local0 (16), Severity notice (5)
Msg: 1 2022-01-02T15:36:17+0100 surfbox-ii LOPGRESSO LOG4J2-SCAN DETECT - \0xef\0xbb\0xbf{"time": "2022-01-02 15:36:17+0100", "hostname": "surfbox-ii", "path": "/usr/bin/pycharm/pycharm-community-2021.1.3/lib/log4j.jar", "entry": "", "product": "Log4j 1", "version": "1.2.17.2", "cve": "CVE-2021-4104", "status": "MITIGATED", "fixed": false}

`

only the MITIGATED messge is send, the Skipping broken jar file are missing

[xeraph]

Oh.. callback invocation is missing. I will fix it in 30min..

[thl-cmk]

I will fix it in 30min..

very nice working with you 👍

[thl-cmk]

works again. Perfect!

[xeraph]

@thl-cmk Would you test v2.7.1?

[xeraph]

@thl-cmk You were ahead.. good!

[thl-cmk]

@xeraph I think this can be cosed :-) (your typo is also gone)