Truncated output from file handler

[ptaoussanis]

Have had a report that the file handler may produce truncated output.

My hunch is that the :needs-stopping? dispatch option isn't being enabled by default for this handler (but should be).

TODO:

• Check that :needs-stopping? is enabled by default for all relevant handlers
• Confirm that this addresses the issue with truncated output

[olavfosse]

Here's my MVE:

mve.tar.gz

[olavfosse]

For passer-bys: archiving can be circumvented by setting :max-file-size to a very high integer

[ptaoussanis]

For passer-bys: circumvented can be disabled by settings :max-file-size to a very high integer

That's helpful to know. The default max is meant to be 4MB, so could be there's something faulty with the logic there.

[olavfosse]

Oops, I accidentally replaced the wrong word 😅.. I meant to say "archiving can be circumvented by setting..."

FTR: on my machine telemere.log does fill up to 4mb before being archived. Nothing wrong in that regard.

Sorry for misleading you!

[ptaoussanis]

Update: just took a look at the example project and confirmed the problem 👍

Looks like it's actually the gzip code that's faulty, and incorrectly truncating input to exactly 4096 bytes. Should be a 1-line fix.

In the meantime, you can disable the gzip option in your example to see the correct behaviour:

(require '[taoensso.telemere :as tel])
(tel/add-handler! :durable
  (tel/handler:file
    {:output-fn (tel/pr-signal-fn)
     :max-file-size (* 1024 1024 4)
     :gzip-archives? false})
  {:needs-stopping? true})

(dotimes [n 1e6] (tel/log! n))

Thanks again for the report and example Olav!

[ptaoussanis]

Closing, this is now fixed on dev branch 👍