Winston Transport File - zippedArchive option doesn't work correctly

[KingRial]

If you use the following transport file:

new (winston.transports.File)({
          level: 'silly',
          filename: './test.log',
          maxsize: 500000,
          maxFiles: 4,
          format: format.combine(
            format.splat(),
            format.label({ label: 'test' }),
            format.timestamp(),
            format.prettyPrint(),
            format.printf( log => {
              return '[ ' + log.timestamp + ' ][ ' + log.level + ' ][ '+ log.label + ' ] ' + log.message;
            })
          ), tailable: true,
          zippedArchive: true,
          exitOnError: false
        }) );

The resulting "test.log" is zipped.

This is wrong since the documentation tells that:

zippedArchive: If true, all log files but the current one will be zipped.

[ChristiaanWillemsen]

I have the exact same issue with 3.0.0-rc4. and even the zipped file is useless, it is not appended when you log, and I cannot read it with gzcat.

[wjford]

I have the same issue with 3.0.0.

    new winston.transports.File({
      level: 'info',
      filename: LOGS_PATH,
      handleExceptions: true,
      maxsize: MAX_LOG_SIZE,
      timestamp: true,
      maxFiles: 5,
      eol: '\r\n',
      prettyPrint: true,
      zippedArchive: true
    })

This results in the latest file being gzipped. In 2.4.2, all but the latest file were zipped.

[sanderboom]

In my case the logfiles are not zipped at all, although .gz has been added.

λ file somelog.log.gz
somelog.log.gz: ASCII text

Gzipped would look like:

λ gzip package.json
λ file package.json.gz
package.json.gz: gzip compressed data, was "package.json", last modified: Thu Aug 23 16:41:38 2018, from Unix, original size 2354

[paolotremadio]

I use this package in all my projects: https://github.com/paolotremadio/homeautomation-winston-logger

All the files emitted are .gz but they are still in plain text (so they are not actually compressed). Can someone tell me what I'm doing wrong?

[masoudltf]

i have same issue, only ".gz" appends to the filename and not correctly gzipped.

[masoudltf]

can anyone help us?

[KingRial]

@masoudltf: it's odd, it should zip all the logs (even if the first one should be not zipped).
Could you share the code ?

[paolotremadio]

Here’s an example: https://github.com/paolotremadio/homeautomation-winston-logger/blob/master/index.js

[masoudltf]

@KingRial
thanks for reply.
this is my code:

const defaultLogFormat = [
    winston.format.timestamp(),
    winston.format.align(),
    winston.format.printf(info => {
        return '[${info.level}] [${info.timestamp}] ${info.message}';
    })
];

new winston.transports.File({
        format: winston.format.combine(
            ...defaultLogFormat
        ),
        filename: 'path-to-filename',
        maxsize: 100000000,
        zippedArchive: true
})

[masoudltf]

Here’s an example: https://github.com/paolotremadio/homeautomation-winston-logger/blob/master/index.js

@paolotremadio :
According to above code in my previous comment, i don't think this code is wrong.

[KingRial]

@masoudltf: you are right, the latest winston (3.1.0) doesn't zip any file but only adds the .gz extension.
Infact, it seems there is no method which inflates the logs, only the Zlib instance

[masoudltf]

bug report needed?

[DABH]

Sorry this issue went unattended. Looking at https://github.com/winstonjs/winston/blob/master/lib/winston/transports/file.js#L543 it looks like the File transport returns either a gzip stream or a regular stream. So if zippedArchive is true then your log should be getting zipped. There is also some code https://github.com/winstonjs/winston/blob/master/lib/winston/transports/file.js#L663 that looks related to adding .gz extensions to old files when rotation happens. But indeed I don't see something that indicates all but the latest log will be zipped -- looks more like all or none get zipped depending on zippedArchive. Also, there is no method to inflate logs, that would presumably be the user's job if they cared to view those logs later.

If you can reproduce issues using master, please let me know what code example you're using, and what your desired behavior is. We'll try to make everyone happy on this issue :)

[glemco]

What it seems is happening [in _createStream method in the File class] is that it tries to open a gzip stream but it's not actually piping anything into it, instead the source readable stream is directly piped into the file (as plain text).
While rotating (tailable option) the file is simply renamed as <logname>.gz but without any compression in it.
It looks like this feature is an hybrid between how it used to work in version 2.x (the file is compressed on rotation) and a new implementation in which all the log files are created as already compressed (hence they all should have the gz extension, also the first one).
This behaviour (if that's what the developer wants) can be achieved doing something like source.pipe(gzip).pipe(dest) once the destination file is opened, instead of just source.pipe(dest), leaving the gzipped stream alone [still in _createStream method].

[shayantabatabaee]

@DABH
I have the same issue too, I also upgraded my Winston version to latest version (3.2.1) but still no success. when the zippedArchive is true , the logger only rename the extension of file so the zip file is corrupted.

[paolotremadio]

Is anyone working on a patch? Othwrwise I can give it a try

[glemco]

I tried by just using the gzip stream, but it was giving me problems on the most recent log (the one that's currently written to, with tailable option). It was taking too much time so we rather decided to stay on 2.x . If you can make a pull request fixing it, that would be great

[smitalm]

+1
zippedArchive option on winston.transports.File is broken - resulting files have .gz suffix, but are corrupted.

winston version 3.2.1

[raapperez]

Any update on this? Anyone knows an older version that it works?

[paolotremadio]

2.4.4 does zip the rotated files.

[smitalm]

In our case, gzip functionality was crucial, so we decided to migrate our logging to https://log4js-node.github.io/log4js-node/ and everything worked flawlessly

[mattvasc]

@DABH
I have the same issue too, I also upgraded my Winston version to latest version (3.2.1) but still no success. when the zippedArchive is true , the logger only rename the extension of file so the zip file is corrupted.

Still happening on 3.3.3 😔

[max0x7ba]

The problem persists in version 3.8.1.

[Lilleri]

still persist in 3.8.2

[Pulkit0729]

Hi @DABH , I would like to give it a try. Please assign it to me.

[DABH]

Done — thank you so much for being willing to take this on!!

[Pulkit0729]

Hey @DABH , can you specify what is the expeected ideal behaviour for zipped archive option? Also was there a previous version, when it was working as expected?

[Pulkit0729]

Hi @DABH , I have a question that when should the log file be zipped? Is it after the file size is equal to maxsize and new file is created. If yes, in that case, we will start logging initially with no zip files?

[asnawisaharuddin]

Is there a workaround for this?

the logger only rename the extension of the file to .gz

Im using v3.10

[DABH]

@Pulkit0729 Yeah I think you rotate when the uncompressed file size reaches the max size, and then you compress the log file that you just finished writing to. And initially you would have no zip files. But if you have file rotation on, then with zipping set to true, you would end up with multiple zip files

[DABH]

@Pulkit0729 I think the last time this worked was probably Winston 2.x, it’s one of the features we sadly didn’t have test cases for and wasn’t carefully accounted for or tested when writing 3.x

[Pulkit0729]

Hi @DABH , I have added a PR #2337 for this. Please review it.

[DABH]

Closed via #2337