[CBR-211] log rotation checks for size and age of files #3507
Conversation
| scribeSettings :: KC.ScribeSettings | ||
| scribeSettings = KC.ScribeSettings bufferSize | ||
| where | ||
| bufferSize = 5000 -- size of the queue (in log items) |
There was a problem hiding this comment.
we can set the size of katip's queue here.
| let le1 = updateEnv le0 getCurrentTime | ||
| -- use 'getCurrentTime' to get a more precise timestamp | ||
| -- as katip uses per default some internal buffered time variable | ||
| timer <- mkAutoUpdate defaultUpdateSettings { updateAction = getCurrentTime, updateFreq = 10000 } |
There was a problem hiding this comment.
the time is fetched from the OS at most 100 times a second.
| mkFileDescription bp fp = | ||
| -- if fp contains a filename in a directory path | ||
| -- move this path to the prefix and only keep the name | ||
| let (extbp, fname) = splitFileName fp |
There was a problem hiding this comment.
some log-config files contain as filename a path like this: pub/node.pub
in that case, we add the directory path to the prefix dir and keep the filename apart.
| trp <- initializeRotator rot fdesc | ||
| scribestate <- newMVar trp -- triple of (handle), (bytes remaining), (rotate time) | ||
| -- sporadically remove old log files - every 10 seconds | ||
| cleanup <- mkAutoUpdate defaultUpdateSettings { updateAction = cleanupRotator rot fdesc, updateFreq = 10000000 } |
There was a problem hiding this comment.
cleanup of old files which outnumber the limit set in the configuration file are deleted from the filestystem, at most every 10 seconds. (updateFreq is actually the sleep time).
| bracket_ (takeMVar locklocal) (putMVar locklocal ()) $ | ||
| T.hPutStrLn h $ encodeToLazyText $ itemJson v item | ||
| pure $ Scribe logger (hFlush h) | ||
| modifyMVar_ scribestate $ \(hdl, bytes, rottime) -> do |
There was a problem hiding this comment.
this part is based on the ingenious proposal by Neil. Thanks!
| mkFileScribeH :: Handle -> Bool -> Severity -> Verbosity -> IO Scribe | ||
| mkFileScribeH h colorize s v = do | ||
| hSetBuffering h LineBuffering | ||
| locklocal <- newMVar () |
| -- | global lock for file Scribes | ||
| {-# NOINLINE lock #-} | ||
| lock :: MVar () | ||
| lock = unsafePerformIO $ newMVar () |
There was a problem hiding this comment.
no more global lock.
CodiePP
force-pushed
the
adiemand/CBR-211/log-rotation
branch
2 times, most recently
from
c55269a to
e0fc654
Compare
|
The build failures here are due to a bad Guthub merge which has now been fixed. If you rebase against |
Signed-off-by: Alexander Diemand <[email protected]>
erikd
force-pushed
the
adiemand/CBR-211/log-rotation
branch
from
e0fc654 to
f15d462
Compare
|
All AppVeyor builds are failing because a file download is failing. |
Description
log rotation for 'katip'
logs/node.log-20180829201517--log-config) defines rotation parameters:** write to new file if max size is reached (currently 5 MB)
** write to new file if current log file was created more than ''t'' (default: 24) hours ago
** remove oldest files and only keep ''n'' files (currently 20)
(since IO is quite expensive, the removal of files from the filesystem is only done at most every 10 seconds)
Linked issue
CBR-211
CBR-97 (user story)
Type of change
Developer checklist
Testing checklist
QA Steps
the file
util/Pos/Util/Log/Rotator.hscontains some commands at the end which can be entered inGHCifor testing.