-
Notifications
You must be signed in to change notification settings - Fork 29.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
StringDecoder undocumented size limit #35676
Comments
The size limit is exactly 0x1fffffe8 (536870888) characters. That's the maximum length a JS string can have in the V8 engine. The method correctly throws an error if the decoder is setup with |
Well, I don't know why there's no exception propagating to JS... Relevant lines in the code: Lines 265 to 268 in d146b25
Lines 208 to 211 in d146b25
Lines 32 to 38 in d146b25
/cc @addaleax |
It's like |
Yeah, it does – that’s a long-standing TODO in the V8 source… we should probably throw one ourselves :/ |
I see... Working on it! |
String::NewFromUtf8 doesn't generate an exception in V8 when the string is too long but is guaranteed to return an empty MaybeLocal only in that case. Generate a Node.js exception when it happens. Fixes: nodejs#35676
PR at #36661 |
String::NewFromUtf8 doesn't generate an exception in V8 when the string is too long but is guaranteed to return an empty MaybeLocal only in that case. Generate a Node.js exception when it happens. Fixes: #35676 PR-URL: #36661 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Gireesh Punathil <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: James M Snell <[email protected]>
String::NewFromUtf8 doesn't generate an exception in V8 when the string is too long but is guaranteed to return an empty MaybeLocal only in that case. Generate a Node.js exception when it happens. Fixes: #35676 PR-URL: #36661 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Gireesh Punathil <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: James M Snell <[email protected]>
String::NewFromUtf8 doesn't generate an exception in V8 when the string is too long but is guaranteed to return an empty MaybeLocal only in that case. Generate a Node.js exception when it happens. Fixes: #35676 PR-URL: #36661 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Gireesh Punathil <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: James M Snell <[email protected]>
String::NewFromUtf8 doesn't generate an exception in V8 when the string is too long but is guaranteed to return an empty MaybeLocal only in that case. Generate a Node.js exception when it happens. Fixes: #35676 PR-URL: #36661 Reviewed-By: Anna Henningsen <[email protected]> Reviewed-By: Gireesh Punathil <[email protected]> Reviewed-By: Rich Trott <[email protected]> Reviewed-By: Colin Ihrig <[email protected]> Reviewed-By: James M Snell <[email protected]>
What steps will reproduce the bug?
How often does it reproduce? Is there a required condition?
The above causes the issue 100% reliably for me.
What is the expected behavior?
Either the decoded buffer or an exception explaining that the input buffer is too long.
What do you see instead?
So sd.write(large buffer) returns undefined
The reason I'm including stream as an affected subsystem is this line which originally led me to this issue: https://github.com/nodejs/readable-stream/blob/040b813e60ecf0d68ac1461a3fc3157ea5785950/lib/_stream_readable.js#L277
So what happens is that if you have a stream (a pipe used for IPC in my case) and the other side sends an large chunk of data, the recipient throws an exception before any application code is invoked that could handle it gracefully.
Additional information
I don't know exactly how large the buffer has to be. In my application (electron 8.5.2 with node 12.13.0) this started happening at around 600MB, when reproducing in node on the command line I had to increase the buffer to 1GB.
Just to clarify: I understand that there will be a limit on how large these buffers can get, I just wish it would report an error that my application code can handle because right now I don't see how I could write robust client code that can deal with the other side sending crap.
The text was updated successfully, but these errors were encountered: