Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Gzip decoder too strict #5761

Closed
peterkooijmans opened this issue Mar 17, 2016 · 4 comments
Closed

Gzip decoder too strict #5761

peterkooijmans opened this issue Mar 17, 2016 · 4 comments
Labels
question Issues that look for answers. zlib Issues and PRs related to the zlib subsystem.

Comments

@peterkooijmans
Copy link

  • Version: 5.9.0
  • Platform: Windows 7 64 bit
  • Subsystem: zlib

While talking to a web server on an embedded device, asking for gzip content encoding I get responses that are considered OK by all major browsers (Chrome, Firefox, IE) and also cURL. However, the Node zlibmodule considers the gzip stream incomplete and throws an error. The error returned by the zlib module reads: Error: unexpected end of file at Zlib._handle.onerror (zlib.js:363:17). Inspecting the gzip response it seems to be missing a footer detailing the CRC and the length of the compressed data.

A small program that reproduces this with an actual response from the device in it:

var buffer = require("buffer");
var stream = require("stream");
var fs = require("fs");
var zlib = require("zlib");

var s = new stream.Readable();
var b = new buffer.Buffer([
    0x1F, 0x8B, 0x08, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x0B, 0xB2, 0xB1,
    0xAF, 0xC8, 0xCD, 0x51, 0x28, 0x4B, 0x2D, 0x2A, 0xCE, 0xCC, 0xCF, 0xB3,
    0x55, 0x32, 0xD4, 0x33, 0x50, 0x52, 0x48, 0xCD, 0x4B, 0xCE, 0x4F, 0xC9,
    0xCC, 0x4B, 0xB7, 0x55, 0x2A, 0x2D, 0x49, 0xD3, 0xB5, 0x50, 0x52, 0xB0,
    0xB7, 0xE3, 0xE5, 0xB2, 0x49, 0xCC, 0x49, 0x2C, 0xCA, 0x2D, 0x56, 0x48,
    0x4E, 0x4C, 0xCE, 0x48, 0xCD, 0x4C, 0xB1, 0x55, 0xB2, 0x54, 0x52, 0x50,
    0xD0, 0x07, 0x4A, 0x00, 0x00, 0x00, 0x00, 0xFF, 0xFF
]);

s.push(b);
s.push(null);
s.pipe(zlib.createGunzip()).pipe(process.stdout);

Would it be possible to make the Node gzip decoder more lenient and behave more like all major browsers and cURL do?

@jhamhader
Copy link
Contributor

This behavior was added in #2595. You can see the answer to a similar issue, #4030.

@mscdex mscdex added the zlib Issues and PRs related to the zlib subsystem. label Mar 17, 2016
@Fishrock123 Fishrock123 added the question Issues that look for answers. label Mar 17, 2016
@evanlucas
Copy link
Contributor

@peterkooijmans does this same issue occur on node v5.8.0?

@peterkooijmans
Copy link
Author

@evanlucas node v5.8.0 has the exact same behavior.

@Fishrock123
Copy link
Contributor

Looks like it has been this was since v5. @peterkooijmans could you take a look at #4030 for a potential solution?

addaleax added a commit to addaleax/node that referenced this issue Apr 7, 2016
Up to now, `Z_FINISH` was always the flushing flag that was used
for the last chunk of input data. This patch makes this choice
configurable so that advanced users can perform e.g. decompression of
partial data using `Z_SYNC_FLUSH`, if that suits their needs.

Add tests to make sure that an error is thrown upon encountering
invalid `flush` or `finishFlush` flags.

Fixes: nodejs#5761
MylesBorins pushed a commit that referenced this issue Apr 20, 2016
Up to now, `Z_FINISH` was always the flushing flag that was used
for the last chunk of input data. This patch makes this choice
configurable so that advanced users can perform e.g. decompression of
partial data using `Z_SYNC_FLUSH`, if that suits their needs.

Add tests to make sure that an error is thrown upon encountering
invalid `flush` or `finishFlush` flags.

Fixes: #5761
PR-URL: #6069
Reviewed-By: James M Snell <[email protected]>
MylesBorins pushed a commit that referenced this issue Apr 21, 2016
Up to now, `Z_FINISH` was always the flushing flag that was used
for the last chunk of input data. This patch makes this choice
configurable so that advanced users can perform e.g. decompression of
partial data using `Z_SYNC_FLUSH`, if that suits their needs.

Add tests to make sure that an error is thrown upon encountering
invalid `flush` or `finishFlush` flags.

Fixes: #5761
PR-URL: #6069
Reviewed-By: James M Snell <[email protected]>
joelostrowski pushed a commit to joelostrowski/node that referenced this issue Apr 25, 2016
Up to now, `Z_FINISH` was always the flushing flag that was used
for the last chunk of input data. This patch makes this choice
configurable so that advanced users can perform e.g. decompression of
partial data using `Z_SYNC_FLUSH`, if that suits their needs.

Add tests to make sure that an error is thrown upon encountering
invalid `flush` or `finishFlush` flags.

Fixes: nodejs#5761
PR-URL: nodejs#6069
Reviewed-By: James M Snell <[email protected]>
jasnell pushed a commit that referenced this issue Apr 26, 2016
Up to now, `Z_FINISH` was always the flushing flag that was used
for the last chunk of input data. This patch makes this choice
configurable so that advanced users can perform e.g. decompression of
partial data using `Z_SYNC_FLUSH`, if that suits their needs.

Add tests to make sure that an error is thrown upon encountering
invalid `flush` or `finishFlush` flags.

Fixes: #5761
PR-URL: #6069
Reviewed-By: James M Snell <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Issues that look for answers. zlib Issues and PRs related to the zlib subsystem.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants