Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

td-agent v3.2.1 does not close log files on rotation #2200

Closed
zerkms opened this issue Dec 2, 2018 · 2 comments · Fixed by #3782
Closed

td-agent v3.2.1 does not close log files on rotation #2200

zerkms opened this issue Dec 2, 2018 · 2 comments · Fixed by #3782
Labels
bug Something isn't working v1

Comments

@zerkms
Copy link

zerkms commented Dec 2, 2018

Check CONTRIBUTING guideline first and here is the list to help us investigate the problem.

  • fluentd or td-agent version: 3.2.1-0, installed through .deb files from https://www.fluentd.org/download
  • Environment information, e.g. OS: ubuntu bionic 18.04
  • Your configuration: irrelevant
  • Your problem explanation. If you have an error logs, write it together.

When logrotate rotates the logs using the configuration shipped and installed by the .deb file:

/var/log/td-agent/td-agent.log {
  daily
  rotate 30
  compress
  delaycompress
  notifempty
  create 640 td-agent td-agent
  sharedscripts
  postrotate
    pid=/var/run/td-agent/td-agent.pid
    if [ -s "$pid" ]
    then
      kill -USR1 "$(cat $pid)"
    fi
  endscript
}
$ cat /var/run/td-agent/td-agent.pid
387

I have found that the "main" fluentd process does not close the file and holds it til the process restart.

Here is the corresponding lsof.

  1. Before rotation:
fluentd 387 td-agent    7w      REG             253,62  5296042    526173 /var/log/td-agent/td-agent.log.1
fluentd 387 td-agent    9w      REG             253,62     4381    528081 /var/log/td-agent/td-agent.log
ruby    392 td-agent    7w      REG             253,62     4381    528081 /var/log/td-agent/td-agent.log
  1. After rotation:
fluentd 387 td-agent    7w      REG             253,62  5296042    526173 /var/log/td-agent/td-agent.log.1 (deleted)
fluentd 387 td-agent    9w      REG             253,62     4381    530471 /var/log/td-agent/td-agent.log
ruby    392 td-agent    7w      REG             253,62     4381    530471 /var/log/td-agent/td-agent.log
  1. After yet another rotation
fluentd 387 td-agent    7w      REG             253,62  5296042    526173 /var/log/td-agent/td-agent.log.1 (deleted)
fluentd 387 td-agent    9w      REG             253,62       66    528081 /var/log/td-agent/td-agent.log
ruby    392 td-agent    7w      REG             253,62       66    528081 /var/log/td-agent/td-agent.log
@repeatedly repeatedly added bug Something isn't working v1 labels Dec 4, 2018
@repeatedly
Copy link
Member

I confirmed this problem happens with fluentd v1.
On my test, next rotation doesn't have this problem so initial logger setup may have a problem.

@ghanane
Copy link

ghanane commented Jun 4, 2021

We are facing the same issue and it is not fixed after the first rotation.

ashie added a commit that referenced this issue Jun 13, 2022
* Enable to set interval of restarting workers
  See #3749 and #3768
* Fix a bug that fluentd doesn't release its own log file even after
  rotated by external tools
  Fix #2200 and #2436

Signed-off-by: Takuro Ashie <[email protected]>
ashie added a commit that referenced this issue Jun 13, 2022
* Enable to set interval of restarting workers
  See #3749 and #3768
* Fix a bug that fluentd doesn't release its own log file even after
  rotated by external tools
  Fix #2200 and #2426

Signed-off-by: Takuro Ashie <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working v1
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants