fix improper character escaping

[norwood]

this will switch express-state to use json2.

pertinent lines in json2: https://github.com/douglascrockford/JSON-js/blob/master/json2.js#L351

[ericf]

Could you explain in more detail why the built-in JSON.stringify is not sufficient? I want to understand an example of where someone would hit the escaping difference. Also, is there a related v8 bug or discussion for this? Thanks.

[norwood]

we had user data where someone had used http://www.fileformat.info/info/unicode/char/2028/index.htm which is a line terminator, which is not a problem in json, but is in js. when the browser tries to parse the js it reads it as an unescaped newline in the middle of the json string.

relevant blog post: http://timelessrepo.com/json-isnt-a-javascript-subset
here is a relevant stackoverflow article: http://stackoverflow.com/questions/2965293/javascript-parse-error-on-u2028-unicode-character
here is the ecmascript standards portion: http://www.ecma-international.org/ecma-262/5.1/#sec-7.3

[ericf]

@norwood thanks for providing this info! I'll take a look and all this and evaluate whether we should simply escape the white space chars, or go the full route of using JSON2.

[ericf]

@norwood I created #22 as an alternate fix for this issue, instead of adding JSON2 as a dependency it instead escapes the line terminator characters that are invalid in JavaScript.

[ericf]

@mathiasbynens your feedback on this issue and #22 would be much appreciated. Thanks!

[mathiasbynens]

Yep, U+2028 and U+2029 are different in JavaScript vs. JSON. From a quick glance, the patch in #22 seems to account for those perfectly.

Also note that you may want to use escape sequences for lone surrogates in JSON-formatted data. If raw lone surrogate ‘symbols’ are used in strings, JSON parsing/serialization still works fine, but it may cause issues when the serialized JSON data is then passed to a UTF-8 decoder.

The second paragraph of the jsesc README mentions these problems: https://github.com/mathiasbynens/jsesc#readme

To work around both of them, you could use jsesc with its json option enabled: http://mths.be/jsesc#json

[ericf]

@mathiasbynens thanks for your help!

[ericf]

@norwood I've merged #22 (the alternate implantation that doesn't introduce a new dependency) and released v1.1.2: https://github.com/yahoo/express-state/releases/tag/v1.1.2

[norwood]

awesome. thanks for the quick turnaround

[mathiasbynens]

So you won’t be taking care of lone surrogates? Just wondering what the rationale is there.

[ericf]

I will address those in another PR. I ran into issues with jsesc because of how this package needs to serialize values and revive those in the browser. I need to dig into it more, but didn't want that to hold this up.