Logstash init script does not respect the logstash user groups (FIX included)

[idanov]

I have experienced the issue that logstash (when started as a service) does not read log files unless they are world readable or owned by the logstash user itself. I have added the logstash user to the user group of the files (in my case apache user group) and that did not work. Then I found a solution to the problem here:
http://serverfault.com/questions/632079/logstash-not-reading-logs-unless-world-readable

[jordansissel]

You can also work around this by using setfacl to grant logstash permission to read files.

FYI, the init script is going to be going away on platforms that do not need it (Ubuntu 8.04 and beyond, CentOS/RHEL 6 and beyond, etc).

This bug will still be present on an Upstart systems (CentOS/RHEL 6, Ubuntu, etc) because Upstart doesn't support supplemental groups.

[magnusbaeck]

@jordansissel – how does the setfacl workaround respond to log rotations? Is it enough to adjust the ACL on e.g. /var/log to be more permissive and then rely on inheritance for newly created files?

FWIW, I've patched our Upstart script to start Logstash via sudo to work around this Upstart bug. Because the different versions of Upstart found in Ubuntu 10.04 and 12.04 have different forking behavior I have to maintain different versions of those files.

What Logstash could do to improve this situation is to be more verbose when permission problems prevent files from being read. It's not clear to me why --verbose is needed to get that rather crucial information. (I haven't checked if this has changed since 1.4.2.)

[jordansissel]

@magnusbaeck no idea, actually. I agree permission problems need to be surfaced. Previously I May ahve logged warnings about it, but I think users complaining about the noise level caused me to quiet their complates by moving it to an info log (verbose) instead of a warning. Can't win, I suppose :P

We should move permissions problems to warning.

[magnusbaeck]

Cool, I'll have a look and see if I can prepare a PR.

[spuder]

possibly related?

#1398

[Crote]

Please note that this bug, in combination with logstash-plugins/logstash-input-file#46, results in some incredibly hard-to-debug situations.

[suyograo]

Fixed in #1398