Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

querystring: use String.prototype.split's limit #2288

Closed
wants to merge 1 commit into from

Conversation

manvalls
Copy link
Contributor

@manvalls manvalls commented Aug 2, 2015

There's no need to add extra logic for it, String.prototype.split already has a limit argument. By using this argument we avoid keeping the whole array in memory, and V8 doesn't have to process the entire string.

@thefourtheye thefourtheye added the querystring Issues and PRs related to the built-in querystring module. label Aug 2, 2015
@targos
Copy link
Member

targos commented Aug 3, 2015

The changes LGTM.
I'm not sure about the style. Should the if...else be in 4 lines or is it fine like that ?

CI: https://jenkins-iojs.nodesource.com/job/node-test-commit/90/

@thefourtheye
Copy link
Contributor

Split is not in Array but in String

@manvalls manvalls changed the title querystring: use Array.prototype.split's limit querystring: use String.prototype.split's limit Aug 3, 2015
@manvalls
Copy link
Contributor Author

manvalls commented Aug 3, 2015

@targos ok, fixed

@thefourtheye that's some serious lapse on my side :(

@mscdex mscdex changed the title querystring: use String.prototype.split's limit querystring: use Array.prototype.split's limit Aug 3, 2015
@Fishrock123 Fishrock123 changed the title querystring: use Array.prototype.split's limit querystring: use String.prototype.split's limit Aug 11, 2015
@@ -209,18 +209,20 @@ QueryString.parse = QueryString.decode = function(qs, sep, eq, options) {
}

var regexp = /\+/g;
qs = qs.split(sep);

var maxKeys = 1000;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unrelated, but what's with this really arbitrary value?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's in the docs, changing it would mean a major version upgrade. I don't know the reasoning behind it, I guess 1000 is a safe but large enough number of key value pairs.

@jasnell
Copy link
Member

jasnell commented Nov 16, 2015

@manvalls ... can you take a look at @ronkorving's question here: #2288 (comment)

@thefourtheye @targos ... looks like the conversation on this stalled out. Any further thoughts?

@jasnell jasnell added the stalled Issues and PRs that are stalled. label Nov 16, 2015
@targos
Copy link
Member

targos commented Nov 16, 2015

I forgot about it, sorry ! This still LGTM.
We will just have to fix the commit message (to String.prototype.split) if we land it.

@jasnell
Copy link
Member

jasnell commented Nov 16, 2015

@manvalls ... could you squash the commits and fix up the commit message?

@thefourtheye
Copy link
Contributor

LGTM

@manvalls
Copy link
Contributor Author

@jasnell done

@targos
Copy link
Member

targos commented Nov 17, 2015

@targos
Copy link
Member

targos commented Jan 31, 2016

@manvalls I'd like to land this. Could you rebase on master?

There's no need to add extra logic for it, `String.prototype.split`
already has a `limit` argument. By using this argument we avoid keeping
the whole array in memory, and V8 doesn't have to process the entire
string.
@manvalls
Copy link
Contributor Author

@targos done

@targos
Copy link
Member

targos commented Jan 31, 2016

Thanks a lot.
CI: https://ci.nodejs.org/job/node-test-commit/2005/

targos pushed a commit that referenced this pull request Jan 31, 2016
There's no need to add extra logic for it, `String.prototype.split`
already has a `limit` argument. By using this argument we avoid keeping
the whole array in memory, and V8 doesn't have to process the entire
string.

PR-URL: #2288
Reviewed-By: Sakthipriyan Vairamani <[email protected]>
Reviewed-By: Michaël Zasso <[email protected]>
@targos
Copy link
Member

targos commented Jan 31, 2016

Landed in 27def4f

@targos targos closed this Jan 31, 2016
@jasnell jasnell added lts-watch-v4.x and removed stalled Issues and PRs that are stalled. labels Feb 1, 2016
@MylesBorins
Copy link
Contributor

@targos it looks like this commit is breaking body-parser. specifically when maxKeys is set to Infinity

@MylesBorins MylesBorins added the semver-major PRs that contain breaking changes and should be released in the next major version. label Feb 3, 2016
@MylesBorins
Copy link
Contributor

There is a very subtle change in behavior introduced with this patch.

String.prototype.split(value, Infinity) will always return an empty array.

The spec documents that the second argument must be an integer. Infinity is being interpreted as 0... instead of MAX_INT (perhaps this should be a fix upstream to v8).

In the past if querystring.parse was given Infinity, the split operation was unaffected.

body-parser is depending on this functionality, at the very least in their tests. We should likely find out how much querystring.parse is being called with infinity to see how much this will break.

I'm not 100% that the changes present here justify the edge case

@MylesBorins
Copy link
Contributor

Here is a gist for a test if you want to experiment
https://gist.github.com/9b27b9320820d7bcdcfd

@MylesBorins
Copy link
Contributor

It is worth mentioning that the documentation for querystring.parse does not mention anything about maxKeys not supporting Infinity

@rvagg
Copy link
Member

rvagg commented Feb 4, 2016

@nodejs/ctc please tune briefly in to this, interesting result from smoke testing work that @thealphanerd has been doing, see nodejs/citgm#74

For the sake of being conservative about breaking changes, particularly when we have examples of people using it in the wild (and if there's one, there's bound to be more, particularly on a prominent project that people use as a reference), I'd +1 a special case here for checking Infinity. However, it does look like an happy accident that Infinity is being interpreted as 0.

@MylesBorins
Copy link
Contributor

@rvagg I have submitted a PR to check for Infinity

#5066

@MylesBorins MylesBorins removed lts-watch-v4.x semver-major PRs that contain breaking changes and should be released in the next major version. labels Feb 5, 2016
@MylesBorins
Copy link
Contributor

fix for this landed in 878bcd4

rvagg pushed a commit that referenced this pull request Feb 8, 2016
There's no need to add extra logic for it, `String.prototype.split`
already has a `limit` argument. By using this argument we avoid keeping
the whole array in memory, and V8 doesn't have to process the entire
string.

PR-URL: #2288
Reviewed-By: Sakthipriyan Vairamani <[email protected]>
Reviewed-By: Michaël Zasso <[email protected]>
scovetta pushed a commit to scovetta/node that referenced this pull request Apr 2, 2016
There's no need to add extra logic for it, `String.prototype.split`
already has a `limit` argument. By using this argument we avoid keeping
the whole array in memory, and V8 doesn't have to process the entire
string.

PR-URL: nodejs#2288
Reviewed-By: Sakthipriyan Vairamani <[email protected]>
Reviewed-By: Michaël Zasso <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
querystring Issues and PRs related to the built-in querystring module.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants