Debugging syslog table configuration

[zwass]

This issue attempts to help debug configuration errors with the Linux syslog table.

osquery reads syslog logs from rsyslogd via a named pipe. This pipe, and rsyslogd must be configured properly in order to query syslog.

Debugging Suggestions

1. Ensure that you are using the --enable_syslog flag (as well as --disable_events=false when using with osqueryi).
2. Turn on verbose logging (verbose configuration flag).
   • Is osquery able to create/open the pipe file? If so, skip to step 4.
3. If osquery cannot create/open the pipe:
   • Does /var/osquery/ (or the directory you used for syslog_pipe_path) exist? Is it read/writeable by the osquery process?
   • Does /var/osquery/syslog_pipe (or the path you used for syslog_pipe_path) exist? Is it readable by the osquery process?
4. If osquery does not report an error opening the pipe, but no logs are in the syslog table:

   • Check the permissions of the pipe file:

     • If rsyslogd runs as user syslog (default Ubuntu setup):

     $ ls -l /var/osquery/syslog_pipe
     pr--rw---- 1 root syslog 0 Apr  4 18:50 /var/osquery/syslog_pipe
     

     • If rsyslogd runs as user root:

     $ ls -l /var/osquery/syslog_pipe
     pr--rw---- 1 root root 0 Apr  4 18:50 /var/osquery/syslog_pipe
     

   • Make sure that rsyslogd is able to write to the pipe.

     1. Kill all osquery processes.
     2. In a separate shell: cat /var/osquery/syslog_pipe (or the path you used for syslog_pipe_path).
     3. Try sending a log using the logger utility in your shell: logger "test log foo". Did the output show up in the shell running cat?

   • If the above steps fail:

     1. Check that rsyslogd is properly configured to write to the pipe. See the configuration docs.
     2. Restart rsyslogd: sudo service rsyslog restart.
     3. Does rsyslogd report any errors in /var/log/syslog?

Other considerations

• Is SELinux turned on? Are SELinux policies preventing rsyslogd or osquery from accessing the pipe file?

[my-janala]

I am struggling to fix syslog table, still nothing is being populated on syslog table. The log shows

I0719 15:32:07.685186 5266 syslog.cpp:87] Successfully opened pipe for syslog ingestion

[jacknagz]

Does rsyslogd report any errors in /etc/log/sysog?

shouldn't that be /var/log/syslog

[zwass]

@jacknagz Yes, will correct above.

[zwass]

@my-janala Come on over to the osquery Slack if you would like more help debugging. You can catch my attention by mentioning me with @zwass in there.

[Dwijad]

@zwass The steps for debugging syslog table is nice one. I am also facing the same issue but with syslog-ng. In my case syslogs do logged in the file /var/log/osquery/osqueryd.results.log if i use system() in syslog-ng source but not pipe as a source.
.......................
pipe("/var/osquery/syslog_pipe");
unix-stream("/dev/log");
.......................

Obviously syslogs does not appear in the table syslog. The name pipe has been created and syslog-ng is running as root:adm. Although i have checked with root:root as well.

ls -l /var/osquery/syslog_pipe
pr--rw---- 1 root adm 0 Apr 14 15:41 /var/osquery/syslog_pipe

There is no error in /var/log/syslog as well. I am using Ubuntu 16.10 and not firewall is running.

[zwass]

@DwijadasDey I don't understand the issue you are experiencing. Please join the osquery Slack if you would like more help debugging. You can catch my attention by mentioning me with @zwass in there.

[zwass]

If anyone is interested in in using the syslog table with syslog-ng, please take a look at this discussion for more information.