Downloading file fails when running on Node version 20.11.0 'Iron' (LTS)

[nadavzipo]

Description

After upgrading Node to version 20.11.0, running await bucket.file(fileLink).download(); throw the following error -

 error: TypeError: Cannot read properties of null (reading 'length')
      at getStateLength (/Users/nadavnagel/Documents/dev/zipoapps/integrations-service/node_modules/stream-shift/index.js:16:28)
      at shift (/Users/nadavnagel/Documents/dev/zipoapps/integrations-service/node_modules/stream-shift/index.js:6:99)
      at Duplexify._forward (/Users/nadavnagel/Documents/dev/zipoapps/integrations-service/node_modules/duplexify/index.js:170:35)
      at PassThrough.onreadable (/Users/nadavnagel/Documents/dev/zipoapps/integrations-service/node_modules/duplexify/index.js:136:10)
      at PassThrough.emit (node:events:518:28)
      at emitReadable_ (node:internal/streams/readable:832:12)
      at process.processTicksAndRejections (node:internal/process/task_queues:81:21),

The error isn't caught by the function try/catch and only by .on('uncaughtException', handler.

The error also happens when trying to create readable stream.

I think the issue happens because of node change of handling streams - nodejs/node#50341

Environment details

• OS:
• Node.js version: 20.11.0
• npm version:
• @google-cloud/storage version: 7.7.0

Steps to reproduce

1. Upgrade to node version 20.11.0
2. Try to execute await bucket.file(fileLink).download(); on any file.

Making sure to follow these steps will guarantee the quickest resolution possible.

Thanks!

[LisanDia]

I was experimenting the same issue but with a docker image.

Specifiying the version of my nodejs in my Dockerfile did the job for me:

FROM node:20.10-buster-slim

The version I'm using of "@google-cloud/storage" is "^7.3.2"

[wvanderdeijl]

We experience these errors as well since upgrading from Node.js v20.10 to v20.11. It seems this is due to a change to private properties in nodejs/node#50341
This in turn throws an error in the stream-split library which has been fixed in v1.0.2: mafintosh/stream-shift#6
We still had v1.0.1 in our package-lock file. Upgrading that to 1.0.2 fixed the issue.

[Muratcol]

We faced the same problem. After a normal version update, the Download function stopped, and all our systems using download crashed. We switched back to an old revision.

It needs to be fixed as soon as possible, we can't update the API anymore.

[ddelgrosso1]

Hi all, I just want to reconfirm that this issue is arising when updating to Node v20.11? Has any other version been problematic?

[nadavzipo]

Hi all, I just want to reconfirm that this issue is arising when updating to Node v20.11? Has any other version been problematic?

Yes. Now v20.11.0 is the lts version so a lot of people will start using it.
I see there is a another issue opens on that as well for newer nodejs versions - #2368

[ddelgrosso1]

It looks like the suggestion from @wvanderdeijl (#2391 (comment)) is likely the fix. It appears that 1.0.2 of stream-shift was released within the last week and was addressing this problem.

[nadavzipo]

It looks like the suggestion from @wvanderdeijl (#2391 (comment)) is likely the fix. It appears that 1.0.2 of stream-shift was released within the last week and was addressing this problem.

I asked the maintainer to push stream-shift fix to duplexify (same maintaner) - mafintosh/stream-shift#10 (comment)
After it is pushed please bump it here as well

[ddelgrosso1]

@nadavzipo I'll keep an eye on duplexify to see when it gets updated and bump here accordingly.

[danielduhh]

Thanks for the investigation!
Please upgrade to stream-shift 1.0.2 (a dependency of duplexify in this repo). This will fix the error. We have plans to move away from duplexify altogether -- you can track that here: #2041