Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ERR_BUFFER_OUT_OF_BOUNDS error with Google Vision package on Node 22.7.0 #54518

Closed
Ecostack opened this issue Aug 23, 2024 · 15 comments
Closed
Labels
buffer Issues and PRs related to the buffer subsystem. confirmed-bug Issues with confirmed bugs. regression Issues related to regressions. v22.x v22.x Issues that can be reproduced on v22.x or PRs targeting the v22.x-staging branch.

Comments

@Ecostack
Copy link

Version

22.7.0

Platform

Darwin Sebastian-MacBook-Pro.local 23.5.0 Darwin Kernel Version 23.5.0: Wed May  1 20:12:58 PDT 2024; root:xnu-10063.121.3~5/RELEASE_ARM64_T6000 arm64

Subsystem

No response

What steps will reproduce the bug?

Create a file called test.js with the following content.
Add the @google-cloud/vision package to your package.json and run npm install.

const vision = require('@google-cloud/vision');

// Set up your Google Cloud Vision client
const client = new vision.ImageAnnotatorClient({
    keyFilename: 'XXX.json'
});

Start via node test.js

How often does it reproduce? Is there a required condition?

On each NodeJS start

What is the expected behavior? Why is that the expected behavior?

It should not crash.

The same code starts with version 22.5.0, here is the output:

node test.js                                                                                                                                                                       
(node:34273) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
(Use `node --trace-deprecation ...` to show where the warning was created)

What do you see instead?

node:internal/buffer:1066
      throw new ERR_BUFFER_OUT_OF_BOUNDS('length');
      ^

RangeError [ERR_BUFFER_OUT_OF_BOUNDS]: "length" is outside of buffer bounds
    at proto.utf8Write (node:internal/buffer:1066:13)
    at Op.writeStringBuffer [as fn] (/Users/sebastianscheibe/Code/XXX/node_modules/protobufjs/src/writer_buffer.js:61:13)
    at BufferWriter.finish (/Users/sebastianscheibe/Code/XXX/node_modules/protobufjs/src/writer.js:453:14)
    at /Users/sebastianscheibe/Code/XXX/node_modules/@grpc/proto-loader/build/src/index.js:177:109
    at Array.map (<anonymous>)
    at createPackageDefinition (/Users/sebastianscheibe/Code/XXX/node_modules/@grpc/proto-loader/build/src/index.js:177:39)
    at Object.fromJSON (/Users/sebastianscheibe/Code/XXX/node_modules/@grpc/proto-loader/build/src/index.js:230:12)
    at GrpcClient.loadProtoJSON (/Users/sebastianscheibe/Code/XXX/node_modules/google-gax/build/src/grpc.js:228:51)
    at new ImageAnnotatorClient (/Users/sebastianscheibe/Code/XXX/node_modules/@google-cloud/vision/build/src/v1/image_annotator_client.js:148:38)
    at Object.<anonymous> (/Users/sebastianscheibe/Code/XXX/test.js:5:16) {
  code: 'ERR_BUFFER_OUT_OF_BOUNDS'
}

Node.js v22.7.0

Additional information

No response

@RedYetiDev
Copy link
Member

Hi! Can you make a reproduction without using any dependencies?

@RedYetiDev RedYetiDev added the buffer Issues and PRs related to the buffer subsystem. label Aug 23, 2024
@Ecostack
Copy link
Author

Here is an example using purely Buffer.

let example = new Buffer("1234567890")
example.utf8Write("abc", 1)

Using 22.7.0

node:internal/buffer:1066
      throw new ERR_BUFFER_OUT_OF_BOUNDS('length');
      ^

RangeError [ERR_BUFFER_OUT_OF_BOUNDS]: "length" is outside of buffer bounds
    at proto.utf8Write (node:internal/buffer:1066:13)
    at Object.<anonymous> (/Users/sebastianscheibe/Code/XXX/test_github.js:4:9)
    at Module._compile (node:internal/modules/cjs/loader:1546:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1691:10)
    at Module.load (node:internal/modules/cjs/loader:1317:32)
    at Module._load (node:internal/modules/cjs/loader:1127:12)
    at TracingChannel.traceSync (node:diagnostics_channel:315:14)
    at wrapModuleLoad (node:internal/modules/cjs/loader:217:24)
    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:166:5)
    at node:internal/main/run_main_module:30:49 {
  code: 'ERR_BUFFER_OUT_OF_BOUNDS'
}

Node.js v22.7.0

Using 22.5

 node test_github.js                                                                                                                                                                   
(node:45481) [DEP0005] DeprecationWarning: Buffer() is deprecated due to security and usability issues. Please use the Buffer.alloc(), Buffer.allocUnsafe(), or Buffer.from() methods instead.
(Use `node --trace-deprecation ...` to show where the warning was created)

@Ecostack
Copy link
Author

There seems to be a change in the latest version which introduces this check regarding the offset.

22.5:

proto.utf8Write = utf8Write;

22.7:

if (length < 0 || length > this.byteLength - offset) {

@RedYetiDev RedYetiDev added regression Issues related to regressions. v22.x v22.x Issues that can be reproduced on v22.x or PRs targeting the v22.x-staging branch. labels Aug 23, 2024
@RedYetiDev
Copy link
Member

Thanks! @nodejs/buffer PTAL

@ericrange
Copy link

ericrange commented Aug 23, 2024

i dont know if my issue has something to do with this, but since 22.7 the encoding of buffers wont work anymore.

MongoError: text contains invalid UTF-8

even expressjs delivers garbage.

yesterday, i build my app on node:22-alpine and all works fine.
since: https://hub.docker.com/layers/library/node/22-alpine/images/sha256-3dbc5d17cf89e1d8ae6f4a8a562f8a5b5df52b4c7060bfb359de86de6a3ecc3c

all is broken. nothing changed in the code base!

btw. "node:20-alpine" works as expected (like node:22-alpine yesterday)

@RedYetiDev
Copy link
Member

RedYetiDev commented Aug 23, 2024

I was able to reproduce using both your snippet, and the slightly modified one below:

Buffer.from("abc").utf8Write("abc", 1)
$ node repro.js
node:internal/buffer:1066
      throw new ERR_BUFFER_OUT_OF_BOUNDS('length');
      ^

RangeError [ERR_BUFFER_OUT_OF_BOUNDS]: "length" is outside of buffer bounds
    at proto.utf8Write (node:internal/buffer:1066:13)
    at Object.<anonymous> (/repro.js:1:20)
    at Module._compile (node:internal/modules/cjs/loader:1546:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1691:10)
    at Module.load (node:internal/modules/cjs/loader:1317:32)
    at Module._load (node:internal/modules/cjs/loader:1127:12)
    at TracingChannel.traceSync (node:diagnostics_channel:315:14)
    at wrapModuleLoad (node:internal/modules/cjs/loader:217:24)
    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:166:5)
    at node:internal/main/run_main_module:30:49 {
  code: 'ERR_BUFFER_OUT_OF_BOUNDS'
}

Node.js v22.7.0

@andreialecu
Copy link

Leaving this here as it may help someone, this also happens with OpenTelemetry:

Error: PeriodicExportingMetricReader: metrics export failed (error RangeError [ERR_BUFFER_OUT_OF_BOUNDS]: "length" is outside of buffer bounds)
at doExport (/app/node_modules/@opentelemetry/sdk-metrics/src/export/PeriodicExportingMetricReader.ts:133:15) |  

@RedYetiDev
Copy link
Member

Fixed by #54524

@elliottAtTreatment
Copy link

We are getting the same thing with Firestore queries when we use the same version. We are downgrading node

@quinnspraut
Copy link

quinnspraut commented Aug 23, 2024

In case helpful - still seeing the RangeError [ERR_BUFFER_OUT_OF_BOUNDS]: "length" is outside of buffer bounds error when using GCP Secret Manager - temporarily resolved by reverting to 22.5.0

const { SecretManagerServiceClient } = require('@google-cloud/secret-manager');
...
const [response] = await (new SecretManagerServiceClient()).accessSecretVersion({
    name: secretPath,
});

@RafaelGSS
Copy link
Member

It should be fixed in next week's release (22.8.0) which includes: #54524 as a patch. I will take care of it.

Robot-Inventor added a commit to Robot-Inventor/Robot-Inventor.github.io that referenced this issue Aug 24, 2024
This reverts commit c597d78.

Node.js v22.7.0 has a bug that causes `ERR_BUFFER_OUT_OF_BOUNDS` error.
See nodejs/node#54518 for more details.
@nicolasnoble
Copy link
Contributor

Did we just witness an example of Hyrum's law?

@ericrange
Copy link

i dont know if my issue has something to do with this, but since 22.7 the encoding of buffers wont work anymore.

MongoError: text contains invalid UTF-8

even expressjs delivers garbage.

yesterday, i build my app on node:22-alpine and all works fine. since: https://hub.docker.com/layers/library/node/22-alpine/images/sha256-3dbc5d17cf89e1d8ae6f4a8a562f8a5b5df52b4c7060bfb359de86de6a3ecc3c

all is broken. nothing changed in the code base!

btw. "node:20-alpine" works as expected (like node:22-alpine yesterday)

#54543

watson added a commit to DataDog/dd-trace-js that referenced this issue Aug 27, 2024
This is a temprary fix until v22.8.0 is released due to a bug in
v22.7.0: nodejs/node#54518
watson added a commit to DataDog/dd-trace-js that referenced this issue Aug 27, 2024
This is a temporary fix until v22.8.0 is released due to a bug in
v22.7.0: nodejs/node#54518
basti1302 added a commit to dash0hq/opentelemetry-js-distribution that referenced this issue Aug 28, 2024
basti1302 added a commit to dash0hq/opentelemetry-js-distribution that referenced this issue Aug 28, 2024
bengl pushed a commit to DataDog/dd-trace-js that referenced this issue Aug 29, 2024
This is a temporary fix until v22.8.0 is released due to a bug in
v22.7.0: nodejs/node#54518
bengl pushed a commit to DataDog/dd-trace-js that referenced this issue Aug 29, 2024
This is a temporary fix until v22.8.0 is released due to a bug in
v22.7.0: nodejs/node#54518
peterMuriuki added a commit to onaio/fhir-web that referenced this issue Aug 30, 2024
bengl pushed a commit to DataDog/dd-trace-js that referenced this issue Aug 30, 2024
This is a temporary fix until v22.8.0 is released due to a bug in
v22.7.0: nodejs/node#54518
bengl pushed a commit to DataDog/dd-trace-js that referenced this issue Aug 30, 2024
This is a temporary fix until v22.8.0 is released due to a bug in
v22.7.0: nodejs/node#54518
RafaelGSS pushed a commit that referenced this issue Aug 30, 2024
Fixes: #54523
Fixes: #54518
PR-URL: #54524
Reviewed-By: Ethan Arrowood <[email protected]>
Reviewed-By: Benjamin Gruenbaum <[email protected]>
Reviewed-By: Ruben Bridgewater <[email protected]>
Reviewed-By: Rafael Gonzaga <[email protected]>
Reviewed-By: Anna Henningsen <[email protected]>
Reviewed-By: Matteo Collina <[email protected]>
RafaelGSS pushed a commit that referenced this issue Aug 30, 2024
Fixes: #54523
Fixes: #54518
PR-URL: #54524
Reviewed-By: Ethan Arrowood <[email protected]>
Reviewed-By: Benjamin Gruenbaum <[email protected]>
Reviewed-By: Ruben Bridgewater <[email protected]>
Reviewed-By: Rafael Gonzaga <[email protected]>
Reviewed-By: Anna Henningsen <[email protected]>
Reviewed-By: Matteo Collina <[email protected]>
JulianVennen added a commit to aternosorg/modbot that referenced this issue Sep 3, 2024
Node 22.7.0 has a bug that causes crashes for google cloud packages: nodejs/node#54518 (comment)
basti1302 added a commit to dash0hq/opentelemetry-js-distribution that referenced this issue Sep 4, 2024
Now that Node.js 22.8 is out, which fixes
nodejs/node#54518, we can unpin Node.js 22
again. This commit reverts #80345a1cd11a13a7fa9e7db8230244ed56c78034.
basti1302 added a commit to dash0hq/opentelemetry-js-distribution that referenced this issue Sep 17, 2024
Now that Node.js 22.8 is out, which fixes
nodejs/node#54518, we can unpin Node.js 22
again. This commit reverts #80345a1cd11a13a7fa9e7db8230244ed56c78034.
louwers pushed a commit to louwers/node that referenced this issue Nov 2, 2024
Fixes: nodejs#54523
Fixes: nodejs#54518
PR-URL: nodejs#54524
Reviewed-By: Ethan Arrowood <[email protected]>
Reviewed-By: Benjamin Gruenbaum <[email protected]>
Reviewed-By: Ruben Bridgewater <[email protected]>
Reviewed-By: Rafael Gonzaga <[email protected]>
Reviewed-By: Anna Henningsen <[email protected]>
Reviewed-By: Matteo Collina <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
buffer Issues and PRs related to the buffer subsystem. confirmed-bug Issues with confirmed bugs. regression Issues related to regressions. v22.x v22.x Issues that can be reproduced on v22.x or PRs targeting the v22.x-staging branch.
Projects
None yet
Development

No branches or pull requests

9 participants