ProxySG Function - Include Proxy Syslog Format in Notes

[dnfalk]

In the parser found https://raw.githubusercontent.com/chicduong/Azure-Sentinel/acn_cd/Parsers/SymantecProxySG/SymantecProxySG.txt , please provide the Proxy SG string required to be set within the parser.txt file. This will greatly help with onboarding usage of the function.

[shainw]

@chicduong - can you address this?

dnfalk could you elaborate? When ingesting through Syslog, the Proxy SG raw log string will appear in the SyslogMessage column. Usage Instruction #2, the user will have to update the Computer Name and Facility to identify the log string. Let me know if i"m missing the point here.

[dnfalk]

@chicduong I was hoping just to make things more clear in the format that should be set on the Proxy SG side in order to make the parser work as expected. I have tried the “main” format and it didn’t work as expected. For instance, just providing the actual format of the proxy SG log format to be set would be really helpful. I did change the computer. Name and facility etc. You mentioned in the parser.txt sample...

// LOG SAMPLES:
// This parser assumes the raw log are formatted as follows:
//
// 2020-05-01 23:07:37 628 10.0.0.0 user67 - - OBSERVED "Web Ads/Analytics" - 200 TCP_TUNNELED CONNECT - tcp gmtdmp.mookie1.com 443 / - - "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.129 Safari/537.36" 10.10.10.10 1601 1586 - "none" "none" 3128 Ambiguous%20-%20Special%20Use 2

I was hoping to just include the exact format to be set to produce that output.

[dnfalk]

The “raw syslog” concept I understand but you still have to define a format To send.

dnfalk are you able to identify the version number for the log format? The parser is expecting the standard log format and anything custom. Could you also provide a sample of the standard log record you're getting?

[dnfalk]

@chicduong I am using the built in format called "bcreportermain_v1" with the following format:

date time time-taken c-ip cs-username cs-auth-group s-supplier-name s-supplier-ip s-supplier-country s-supplier-failures x-exception-id sc-filter-result cs-categories cs(Referer) sc-status s-action cs-method rs(Content-Type) cs-uri-scheme cs-host cs-uri-port cs-uri-path cs-uri-query cs-uri-extension cs(User-Agent) s-ip sc-bytes cs-bytes x-virus-id x-bluecoat-application-name x-bluecoat-application-operation x-bluecoat-application-groups cs-threat-risk x-bluecoat-transaction-uuid x-icap-reqmod-header(X-ICAP-Metadata) x-icap-respmod-header(X-ICAP-Metadata)

I am not sure what you are referring to with the version number of the log format? The proxy sg device is running 6.7.4.8.

Example syslog message output is:

14:11 5 10.33.94.69 - - 10.33.243.178 10.33.243.178 Unavailable - - OBSERVED "Technology/Internet" - 200 TCP_ACCELERATED CONNECT - tcp www.mwjcomputing.com 443 / - - "Mozilla/5.0 (Windows NT 10.0 WOW64 Trident/7.0 Touch rv:11.0) like Gecko" 10.33.243.178 39 238 - "unavailable" "unavailable" "unavailable" unavailable a549a6c8012ceae9-00000000035ba116-000000005ef0bcb3 - -#15#0122020-06-22 14:14:11 40 10.33.94.69 - - 10.33.243.178 10.33.243.178 Unavailable - - OBSERVED "Technology/Internet" - 200 TCP_ACCELERATED CONNECT - tcp www.mwjcomputing.com 443 / - - "Mozilla/5.0 (Windows NT 10.0 WOW64 Trident/7.0 Touch rv:11.0) like Gecko" 10.33.243.178 39 238 - "unavailable" "unavailable" "unavailable" unavailable a549a6c8012ceae9-00000000035ba117-000000005ef0bcb3 - -#15#0122020-06-22 14:14:11 16 10.33.94.69 - - 10.33.243.178 10.33.243.178 Unavailable - - OBSERVED "Search Engines/Portals" - 200 TCP_ACCELERATED CONNECT - tcp bat.bing.com 443 / - - "Mozilla/5.0 (Windows NT 10.0 WOW64 Trident/7.0 Touch rv:11.0) like Gecko" 10.33.243.178 39 222 - "unavailable" "unavailable" "unavailable" unavailable a549a6c8012ceae9-00000000035ba118-000000005ef0bcb3 - -#15#0122020-06-22 14:14:11 18 10.33.94.69 - - 10.33.243.178 10.33.243.178 Unavailable - - OBSERVED "Netskope_Categories Technology/Internet File Storage/Sharing" - 200 TCP_ACCELERATED CONNECT - tcp gist.github.com 443 / - - "Mozilla/5.0 (Windows NT 10.0 WOW64 Trident/7.0 Touch rv:11.0) like Gecko" 10.33.243.178 39 228 - "unavailable" "unavailable" "unavailable" unavailable a549a6c8012ceae9-00000000035ba119-000000005ef0bcb3 - -#15#0122020-06-22 14:14:11 15 10.33.94.69 - - 10.33.243.178 10.33.243.178 Unavailable - - OBSERVED "Netskope_Categories Technology/Internet File Storage/Sharing" - 200 TCP_ACCELERATED CONNECT - tcp gist.github.com 443 / - - "Mozilla/5.0 (Windows NT 10.0 WOW64 Trident/7.0 Touch rv:11.0) like Gecko" 10.33.243.178 39 228 - "unavailable" "unavailable" "unavailable" unavailable a549a6c8012ceae9-00000000035

dnfalk this looks to be the same format as what the parser is written, is the log sample provided from Azure Sentinel? Could you provide what is shown in the SyslogMessage field?

[dnfalk]

@chicduong Yes that is the message from the syslog message field in Azure Sentinel.

dnfalk is this a single event in Sentinel? It looks to be 5 log records.

[dnfalk]

@chicduong I am seeing this come in on a single line. Please see the attached screenshots for configuration on the Proxy SG side...

dnfalk what other format types are available other than bcreportermain_v1?

[dnfalk]

@chicduong please find above.

dnfalk could you try to set it to the "main" log format?

[dnfalk]

This generated 2 syslog messages.... It appears to be being truncated perhaps?

28:22 19 10.33.94.69 200 TCP_ACCELERATED 39 230 CONNECT tcp beap-bc.yahoo.com 443 / - - - 10.33.243.178 - - "Mozilla/5.0 (Windows NT 10.0 Win64 x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.106 Safari/537.36" OBSERVED "Search Engines/Portals" - 10.33.243.178 a549a6c8012ceae9-00000000035bbba0-000000005ef0f846 - -#15#0122020-06-22 18:28:22 118 10.33.94.69 301 TCP_MISS 583 453 GET http mwjcomputing.com 80 / - - - mwjcomputing.com text/html %20charset=iso-8859-1 - "Mozilla/5.0 (Windows NT 10.0 Win64 x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.106 Safari/537.36" OBSERVED "Technology/Internet" - 10.33.243.178 a549a6c8012ceae9-00000000035bbba2-000000005ef0f846 - -#15#0122020-06-22 18:28:22 154 10.33.91.101 0 - 0 0 unknown ssl wstunnel10-1.rightscale.com 443 / - - - - - - - OBSERVED "Netskope_Categories Technology/Internet" - 10.33.243.178 a549a6c8012ceae9-00000000035bbba3-000000005ef0f846 - -#15#0122020-06-22 18:28:22 5 10.33.94.69 200 TCP_ACCELERATED 39 228 CONNECT tcp mwjcomputing.com 443 / - - - 10.33.243.178 - - "Mozilla/5.0 (Windows NT 10.0 Win64 x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.106 Safari/537.36" OBSERVED "Technology/Internet" - 10.33.243.178 a549a6c8012ceae9-00000000035bbba4-000000005ef0f846 - -#15#0122020-06-22 18:28:23 69 10.33.94.69 200 TCP_ACCELERATED 39 236 CONNECT tcp www.mwjcomputing.com 443 / - - - 10.33.243.178 - - "Mozilla/5.0 (Windows NT 10.0 Win64 x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.106 Safari/537.36" OBSERVED "Technology/Internet" - 10.33.243.178 a549a6c8012ceae9-00000000035bbba5-000000005ef0f847 - -#15#0122020-06-22 18:28:23 8 10.33.94.69 200 TCP_ACCELERATED 39 242 CONNECT tcp web.vortex.data.msn.com 443 / - - - 10.33.243.178 - - "Mozilla/5.0 (Windows NT 10.0 Win64 x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.106 Safari/537.36" OBSERVED "Search Engines/Portals" - 10.33.243.178 a549a6c8012ceae9-00000000035bbba6-000000005ef0f847 - -#15#0122020-06-22 18:28:2

28:23 4 10.33.94.69 200 TCP_ACCELERATED 39 214 CONNECT tcp c0.wp.com 443 / - - - 10.33.243.178 - - "Mozilla/5.0 (Windows NT 10.0 Win64 x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.106 Safari/537.36" OBSERVED "VCCAllowedSites Content Delivery Networks" - 10.33.243.178 a549a6c8012ceae9-00000000035bbba8-000000005ef0f847 - -#15#0122020-06-22 18:28:23 6 10.33.94.69 200 TCP_ACCELERATED 39 214 CONNECT tcp c0.wp.com 443 / - - - 10.33.243.178 - - "Mozilla/5.0 (Windows NT 10.0 Win64 x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.106 Safari/537.36" OBSERVED "VCCAllowedSites Content Delivery Networks" - 10.33.243.178 a549a6c8012ceae9-00000000035bbba9-000000005ef0f847 - -#15#0122020-06-22 18:28:23 2 10.33.94.69 200 TCP_ACCELERATED 39 236 CONNECT tcp www.mwjcomputing.com 443 / - - - 10.33.243.178 - - "Mozilla/5.0 (Windows NT 10.0 Win64 x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.106 Safari/537.36" OBSERVED "Technology/Internet" - 10.33.243.178 a549a6c8012ceae9-00000000035bbbaa-000000005ef0f847 - -#15#0122020-06-22 18:28:23 6 10.33.94.69 200 TCP_ACCELERATED 39 236 CONNECT tcp www.mwjcomputing.com 443 / - - - 10.33.243.178 - - "Mozilla/5.0 (Windows NT 10.0 Win64 x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.106 Safari/537.36" OBSERVED "Technology/Internet" - 10.33.243.178 a549a6c8012ceae9-00000000035bbbab-000000005ef0f847 - -#15#0122020-06-22 18:28:23 3 10.33.94.69 200 TCP_ACCELERATED 39 214 CONNECT tcp c0.wp.com 443 / - - - 10.33.243.178 - - "Mozilla/5.0 (Windows NT 10.0 Win64 x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.106 Safari/537.36" OBSERVED "VCCAllowedSites Content Delivery Networks" - 10.33.243.178 a549a6c8012ceae9-00000000035bbbac-000000005ef0f847 - -#15#0122020-06-22 18:28:23 10 10.33.94.69 200 TCP_ACCELERATED 39 214 CONNECT tcp c0.wp.com 443 / - - - 10.33.243.178 - - "Mozilla/5.0 (Windows NT 10.0 Win64 x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.106 Safari/537.36" OBSERVED "VC

dnfalk this seems to be an ingestion issue. The parser accommodates a single log entry.
Ensure these steps were followed: https://knowledge.broadcom.com/external/article/166529/sending-access-logs-to-a-syslog-server.html

Also, I see another issue with the log entry, it looks like the log entries are starting after the first colon, cutting off the date and part of the time prefix (notice how the log starts at 28:23, the minute values first). This is due to syslog RFC 3164. This is an issue Microsoft is currently working to resolve.