NJSON - next-json

Next JavaScript Object Notation

Why this package?

Because JSON is awesome, but...

JSON is awesome mainly for two reasons:

1. it offers an easy way to serialize and deserialize complex data;
2. a valid JSON encoded string can be pasted in a JavaScript source file, a really awesome feature while developing / debugging.

... but it has some limitations:

• doesn't support undefined values,
• doesn't support BigInt numbers,
• doesn't support many other features...

This package is intended to offer something as great as JSON... trying to add something more.

NJSON Features

• ☑ extends JSON
• ☑ supports C style comments
• ☑ supports escaped new line in strings
• ☑ supports trailing commas
• ☑ supports circular and repeated references
• ☑ supports undefined
• ☑ supports -0, NaN and Infinity
• ☑ supports BigInt
• ☑ supports Date
• ☑ supports Error (with exception)
• ☑ supports Map
• ☑ supports RegExp
• ☑ supports Set
• ☑ supports TypedArrays
• ☑ supports URL

NJSON extends JSON

This doesn't mean it's 100% compliant: due its higher number of supported features the result string of the serialization through NJSON.stringify may differs from the result of the serialization through JSON.stringify.

On the other hand, the result of the deserialization of a valid JSON encoded string through NJSON.parse will produce a value deep equal to the value produced by JSON.parse and the reviver function will be called the same amount of times, with the same parameters and in the same order.

Note: the reviver function still not implements the newly added context argument.

Taken the result of a JSON.parse call (i.e. a value which contains only valid JSON values), if serialized through JSON.stringify or NJSON.stringify produces two equal strings and the replacer function will be called the same amount of times, with the same parameters and in the same order.

NJSON parser

NJSON offers its own parser which means it doesn't use eval with its related security hole.

Even if the NJSON serialized string is JavaScript compliant, NJSON.parse is not able to parse any JavaScript code, but only the subset produced by NJSON.stringify (otherwise it would have been another eval implementation).

Not supported by design

NJSON do not supports some Objects by design; when one of them is encountered during the serialization process NJSON tries to act as JSON does. Nonetheless they take part in the repeated references algorithm anyway. Follow the details.

ArrayBuffer

ArrayBuffers can't be manipulated by JavaScript design: they are serialized as empty objects as JSON does.

Function

NJSON is designed to serialize / deserialize complex data to be shared between different systems, possibly written with other languages than JavaScript (once implementations in other languages will be written). Even if JavaScript can see a function as a piece of data, it is actually code, not data. More than this, for other languages, may be a complex problem execute JavaScript functions.

Last but not least, allowing the deserialization of a function would open once again the security hole implied by the use of eval, and one of the reasons why NJSON was born, is exactly to avoid that security hole.

Symbol

A Symbol is something strictly bound to the JavaScript execution environment which instantiate it: sharing it between distinct systems is something almost meaningless.

TypedArray

Note: except for Int8Array, Uint8Array and Uint8ClampedArray, TypedArrays are platform dependant: they are supported, but trying to transfer one of them between different architectures may be source of unexpected problems.

The Error exception

Error's are special objects. By specifications the properties cause, message, name and stack are not enumerable, NJSON serializes them as any other property. This, plus the nature of the stack property, originates the Error exception to the rule that an NJSON encoded string produces exactly the same value if parsed or evaluated.

• cause:

  • through NJSON.parse the result is a not enumerable property;
  • through eval the result may be an enumerable or a not enumerable property depending on the running JavaScript engine;

• stack:

  • if absent:

    • through NJSON.parse the result is a not enumerable property with value a pseudo-stack;
    • through eval the result is the standard stack property for the running JavaScript engine;

  • if present:

    • through NJSON.parse the result is a not enumerable property;
    • through eval the result may be an enumerable or a not enumerable property depending on the running JavaScript engine;

The only option in my mind to avoid this exception is the use of Object.defineProperties, but it would increase both the complexity of the parser and the size of the produced serialized string. Maybe in the future... configurable through an option... if this can't be really tolerated.

Installation

With npm:

npm install --save next-json

Usage

JavaScript

import { NJSON } from "next-json";

const serialized = NJSON.stringify({ some: "value" });
const deserialized = NJSON.parse(serialized);

TypeScript

import { NJSON, NjsonParseOptions, NjsonStringifyOptions } from "next-json";

const serialized = NJSON.stringify({ some: "value" });
const deserialized = NJSON.parse<{ some: string }>(serialized);

Example

const obj = { test: Infinity };
const set = new Set();
const arr = [NaN, obj, set];

set.add(obj);
set.add(arr);
arr.push(arr);

console.log(NJSON.stringify(arr));
// ((A,B)=>{B.push(A,new Set([A,B]),B);return B})({"test":Infinity},[NaN])

Polyfill

Server side

import express from "express";
import { expressNJSON } from "next-json";

const app = express();

app.use(expressNJSON()); // install the polyfill
app.all("/mirror", (req, res) => res.njson(req.body)); // there is an 'n' more than usual
app.listen(3000);

Client side

import { NJSON, fetchNJSON } from "next-json";

fetchNJSON(); // install the polyfill

const payload = { infinity: Infinity };
payload.circular = payload;

const response = await fetch("http://localhost:3000/mirror", {
  body: NJSON.stringify(payload), // there is an 'N' more than usual
  headers: { "Content-Type": "application/njson" }, // there is an 'n' more than usual
  method: "POST"
});
const body = await response.njson(); // there is an 'n' more than usual

Here payload deep equals payload.circular, which deep equals body, which deep equals body.circular, which deep equals req.body in server side, which deep equals req.body.circular in server side! 🎉

MIME type

The MIME type for NJSON format is: application/njson .

API

NJSON.parse(text[, reviver])

Just for compatibility with JSON.parse. Alias for:

NJSON.parse(text, { reviver });

NJSON.parse(text[, options])

Converts a Next JavaScript Object Notation (NJSON) string into an object.

• text: <string> The text to deserialize.
• options: <NjsonParseOptions> Deserialization options.
• Returns: <unknown> The value result of the deserialization of the NJSON encoded text.

NJSON.stringify(value[, replacer[, space]])

Just for compatibility with JSON.stringify. Alias for:

NJSON.stringify(value, { replacer, space });

NJSON.stringify(value[, options])

Converts a JavaScript value to a Next JavaScript Object Notation (NJSON) string.

• value: <unknown> The value to serialize.
• options: <NjsonStringifyOptions> Serialization options.
• Returns: <string> The NJSON encoded serialized form of value.

interface NjsonParseOptions

• numberKey: <boolean> Alters the type of the key argument for reviver. Default: false.
• reviver: <Function> Alters the behavior of the deserialization process. Default: null.

NjsonParseOptions.numberKey

If true, the reviver function, for Array elements, will be called with the key argument in a Number form.

NjsonParseOptions.reviver

As the reviver parameter of JSON.parse. See also replacer / reviver for NJSON specific details.

Note: the reviver function still not implements the newly added context argument.

interface NjsonStringifyOptions

• date: <string> Specifies Dates conversion method. Default: "time".
• numberKey: <boolean> Alters the type of the key argument for replacer. Default: false.
• omitStack: <boolean> Specifies if to stringify stack for Errors. Default: false.
• replacer: <Function> | <Array> | null Alters the behavior of the serialization process. Default: null.
• sortKeys: <boolean> Specifies whether to sort Object keys. Default: false.
• space: <number> | <string> | null Specifies the indentation. Default: null.
• stringLength: <number> | null Makes Strings to be treated as references. Default: null.
• undef: <boolean> Specifies the undefined behavior. Default: true.

NjsonStringifyOptions.date

Specifies the method of Date objects used to serialize them. Follows the list of the allowed values and the relative method used.

• "iso": Date.toISOString()
• "string": Date.toString()
• "time": Date.getTime() - the default
• "utc": Date.toUTCString()

NjsonStringifyOptions.numberKey

If true, the replacer function, for Array elements, will be called with the key argument in a Number form.

NjsonStringifyOptions.omitStack

For default NJSON.stringify serializes the stack property for Errors. If set to true, the property is omitted from the serialized representation.

NjsonStringifyOptions.replacer

As the replacer parameter of JSON.serialize. See also replacer / reviver for NJSON specific details.

NjsonStringifyOptions.sortKeys

For default NJSON stringifies (and replaces as well) Object keys in the order they appear in the Object itself. If set to true, Object keys are sorted alphabetically before both the processes. This can be useful to compare two references: using this option, the stringified representation of two deep equal references are two equal strings.

NjsonStringifyOptions.space

As the space parameter of JSON.serialize.

NjsonStringifyOptions.stringLength

If specified, Strings which length is greater or equal to the specified value take part in the repeated references algorithm.

NjsonStringifyOptions.undef

For default NJSON.stringify serializes undefined values as well. If set to false, undefined values are treated as JSON.stringify does.

expressNJSON([options])

An Express middleware which works as NJSON body parser and installs the Express.Response.njson method.

• options: <NjsonStringifyOptions> NJSON Express middleware options.
• Returns: Express middleware The NJSON Express middleware.

interface ExpressNjsonOptions

• parse: <NjsonParseOptions> The options passed to NJSON.parse by the middleware to parse the request body.
• stringify: <NjsonStringifyOptions> The default options passed to NJSON.stringify by Express.Response.njson to serialize the response body.

ExpressNjsonOptions.parse

The options passed to NJSON.parse by the middleware to parse the request body.

ExpressNjsonOptions.stringify

The default options passed to NJSON.stringify by Express.Response.njson to serialize the response.

Express.Response.njson(body[, options])

Encodes the body in NJSON format and sends it; sets the proper Content-Type header as well. Installed by expressNJSON.

• body: <unknown> The body to be sent serialized.
• options: <NjsonStringifyOptions> The options passed to NJSON.stringify to serialize the response body; overrides the default options.stringify passed to expressNJSON.

fetchNJSON([options])

Installs the Response.njson method.

• options: <NjsonParseOptions> The default options passed to NJSON.parse by Response.njson to parse the response body.

Response.njson([options])

Parses the response body with NJSON.parse. Installed by fetchNJSON.

• options: <NjsonParseOptions> The options passed to NJSON.parse to parse the response body; overrides the default options passed to fetchNJSON.
• Returns: <unknown> The body parsed with NJSON.parse.

replacer / reviver

Even if Date, RegExp, TypedArrays and URL are Objects, they are treated as native values i.e. replacer and reviver will be never called with one of them as this context.

Array

For Arrays the key argument is a positive integer, but in a String form for JSON compatibility. This can be altered (i.e. in a Number form) through the numberKey option.

Map

Map's keys can be Functions and Symbols; for Maps the key argument is a positive integer in a Number form and the value argument is the entry in the form [mapKey, mapValue]. This gives a way to replace/revive keys which can't be serialized. If replacer or reviver do not return a two elements array, the value is omitted.

Set

For Sets the key argument is a positive integer and it is passed in a Number form.

TypedArray

Unlike JSON, NJSON does not call replacer and reviver for each element. Except for Int8Array, Uint8Array and Uint8ClampedArray, TypedArrays are platform dependant: trying to transfer one of them between different architectures may be source of unexpected problems.

circular / repeated references

Regardless of whether they are omitted, serialized as native values or not, every Objects (but null), Functions and Symbols take part in the repeated references algorithm; long Strings can take part as well (refer to NjsonStringifyOptions.stringLength for details).
When a repeated reference is encountered, replacer and reviver are called against the reference, but it is not called recursively against its properties. If a property of a repeated reference is changed, the same change has effect in all other occurrences of the same reference.
Circular references are simply special cases of repeated references.

Compatibility

Requires Node.js v14.

Exception: fetchNJSON requires Node.js v18.

The package is tested under all Node.js versions currently supported accordingly to Node.js Release.

TypeScript

TypeScript types are distributed with the package itself.

License

MIT License

Bugs

Do not hesitate to report any bug or inconsistency @github.

Donating

If you find useful this package, please consider the opportunity to donate on one of following cryptos:

ADA: DdzFFzCqrhsxfKAujiyG5cv3Bz7wt5uahr9k3hEa8M6LSYQMu9bqc25mG72CBZS3vJEWtWj9PKDUVtfBFcq5hAtDYsZxfG55sCcBeHM9

BTC: 3BqXRqgCU2CWEoZUgrjU3b6VTR26Hee5gq

ETH: 0x8039fD67b895fAA1F3e0cF539b8F0290EDe1C042

See also

Other projects which aim to solve similar problems:

• arson by Ben Newman
• devalue by Rich Harris
• jsesc by Mathias Bynens
• lave by Jed Schmidt
• oson by Steffen Trog
• serialize-javascript by Eric Ferraiuolo
• superjson by Blitz
• tosource by Marcello Bastéa-Forte