Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Please provide a json basic type #1897

Closed
NoelAbrahams opened this issue Feb 2, 2015 · 87 comments
Closed

Please provide a json basic type #1897

NoelAbrahams opened this issue Feb 2, 2015 · 87 comments
Labels
Declined The issue was declined as something which matches the TypeScript vision Suggestion An idea for TypeScript

Comments

@NoelAbrahams
Copy link

Hi,

JSON is an important data exchange format. At present there is no explicit way to annotate an object as pure JSON. We were expecting union types to solve this problem via the following:

    interface Json {
        [x: string]: string|number|boolean|Date|Json|JsonArray;
    }
    interface JsonArray extends Array<string|number|boolean|Date|Json|JsonArray> { }

There are currently a number of problems with this.

class Bass {

    f(): Id{ return undefined;}
}

 // Error: missing index signature
class Foo extends Bass {
    f() { return { id: 10 }} 
}
    interface Foo extends Json {

        foo: { val: string }; // Error: Not assignable

    }
  • Other problems:

    // Error: Missing index signature
    var result: Id[] = 'a|b'.split('|').map(item => {
          return { id: 0 };
    });

The first two problems look likely to be resolved, but not the last two.

This motivates the introduction of a natively supported json type that conforms to the JSON spec.

It should be possible to derive a custom JSON object from the json type with non of the problems above. Furthermore, since JSON is a data exchange format it should be possible to mark fields as nullable on types deriving from json (which would trigger a type guard).

I am of course aware that this is all rather vague!

@RyanCavanaugh RyanCavanaugh added Suggestion An idea for TypeScript Needs More Info The issue still hasn't been fully clarified Needs Proposal This issue needs a plan that clarifies the finer details of how it could be implemented. labels Feb 2, 2015
@RyanCavanaugh
Copy link
Member

What kind help could the compiler provide with this?

In the case where you're receiving JSON, the fact that you know everything is a string, number, boolean, array, null, or object isn't terribly useful? You could prevent yourself from passing a JSON object into an function expecting a Function, but in all other cases it's within spitting distance of any.

In the case where you're emitting JSON, the type system doesn't really encapsulate all the aspects of JSON that are important. JSON.stringify skips properties from the prototype and properties of type Function, both of which are going to be common when doing class-based programs. I think people wouldn't like it if JSON.stringify(myClass) were an error, but now we're really back in the case where it's practically any (minus calling JSON.stringify(someFunc) which is hopefully rare and a quick error to spot). And obviously we have no way to warn you about circular data structures that will cause an error.

@DanielRosenwasser
Copy link
Member

For the record:

@NoelAbrahams
Copy link
Author

NoelAbrahams commented Feb 3, 2015

The discussion is with respect to the case where both the client and server are written in JavaScript. For this case the JSON is shared and can be quite large and complex.

There are use-cases even where Ajax is not involved where data transfer only works when the payload is string. An example of this is the HTML5 dataTransfer object. For this case having the json type ensures the payload is correct on both ends and that JSON.parse is not going to fail.

In the case where you're receiving JSON, the fact that you know everything is a string, number, boolean, array, null, or object isn't terribly useful

True if we were simply looking at how the received JSON is subsequently used. As I mentioned above, the json annotation helps to ensure invalid data structures are not created in the first place, e.g.

interface Foo extends json {
 image: File; // error
}

In the case where you're emitting JSON, the type system doesn't really encapsulate all the aspects of JSON that are important. JSON.stringify skips properties from the prototype and properties of type Function, both of which are going to be common when doing class-based programs.

The point is classes encapsulate behaviour. JSON represents data. People who .stringify classes are a special breed - rather like the dodo 😄.

Edit: Ignore the following in the light of strict-null-checks
I am also very interested in exploring what (if anything) can be done to describe nullability in JSON types. Pretty much all the null reference problems in my experience occur around the use of JSON. Since null is explicitly a JSON value, perhaps it should be an error to access a property on a json-derived type without a type guard:

interface Foo extends json {
  bar: string;
}

var x: Foo;
x.bar.toString(); // Error
x.bar && x.bar.toString(); // okay

@NoelAbrahams
Copy link
Author

@jbondc, I'm not sure that I understand. How does that help to provide compile-time safety for JSON?

@NoelAbrahams
Copy link
Author

Seems like not-null types

Yes, but shouldn't break existing code ;) I'm trying to explore the implications of defining a json type, same as any or int. As I noted above, this type would be an ideal candidate to have default nullable properties, firstly because that's part of the JSON spec and secondly, since we use JSON to transfer data, that's where a lot of the null reference problems occur.

JSON is a structured object, so an interface should be able to describe with 100% fidelity the data it contains.

Yes, but with int for example the following is an error:

var x: int = "10";

But not so for interfaces:

interface Foo extends json {
   html: HtmlElement; // We would like this to be an error
}

@AlicanC
Copy link

AlicanC commented Jun 22, 2015

Since when JSONs are objects?

type Json = string;

@NoelAbrahams
Copy link
Author

@AlicanC,

JSON is string over the wire, but when one calls JSON.parse it becomes an object. This object is a subset of regular JavaScript and that is what we'd like to model.

@AlicanC
Copy link

AlicanC commented Jun 23, 2015

Yes, the JSON becomes an object and it's not JSON anymore. JSON is what you give to JSON.parse(), not what you get from it. What you get is a value. Maybe you guys should rename your types to JsonValue.

@NoelAbrahams
Copy link
Author

Maybe you guys should rename your types to JsonValue.

Can you elaborate? Which guys and what types?

@Elephant-Vessel
Copy link

Elephant-Vessel commented Jan 8, 2017

Seems like the problems that you described above are resolved now? Trying out this approach and it looks like it's working just fine.

Also: I think this has deeper utility that just representing some arbitrary format like JSON. It enables us to more clearly distinguish data from operations on data. That's a quite fundamental piece of our domain as software developers, I don't think that would be a totally useless concept to have available and tangible in code, even if OOP still might be the norm.

@electricessence
Copy link

electricessence commented May 18, 2017

https://github.com/electricessence/TypeScript.NET/blob/master/source/JSON.d.ts

This is what I have, but I'm not sure if it's adding any value really. I still have to constantly add <type> constraints everywhere to make sure it's correct.

I'm wondering if it's simply more effective to do run-time validation than be concerned with compile time constraints.

I think part of the challenge is that a JSON blob can consist of either a map or an array and in those cases the indexers are different.

So for example, I've had to do use <T extends JsonMap | JsonArray> for the expected output parameter.
https://github.com/electricessence/TypeScript.NET/blob/master/_utility/file-promise.ts#L59-L80

And therefore still have to pass one or the other in order for it to work as show in the first link.

So again, I wish there were more examples of where typing JSON helps.

@JasonKleban
Copy link

JasonKleban commented May 21, 2017

This type would be useful for data that must be able to roundtrip through JSON serialization and deserialization, such as data to be stored or otherwise faithfully reproduce across a remoting boundary.

It seems to me that this simple definition of Json works to provide some guidance for some library that works with arbitrary user data in that way, but there are a few problems. I don't know if there is already a solution to them, or if not, what the solution ought to be. I'm just saying it would be useful. Not sure why the type system needs an explicit extend JsonMap here to recognize the compatibility. I don't think there's any way to disallow classes from extending Json.

export interface JsonMap { [member: string]: string | number | boolean | null | JsonArray | JsonMap };
export interface JsonArray extends Array<string | number | boolean | null | JsonArray | JsonMap> {}
export type Json = JsonMap | JsonArray | string | number | boolean | null;

interface Document extends JsonMap {
    one: string;
    two: boolean;
    3.141592: "pi" | boolean;
}

function clone<T extends Json>(data : T) { return <T>(JSON.parse(JSON.stringify(data))); }

var a : Json = "Hello"; clone(a).toLowerCase();     // 👍
var b : Json = 42; clone(b).toExponential();        // 👍
var c : Json = true; { let t = clone(c); }          // 👍
var d : Json = null; { let t = clone(d); }          // 👍
var e : Json = [1, 2, ""]; clone(e).length;         // 👍
var f : Json = {}; {let t = clone(f); }             // 👍 f : JsonMap

var g : Json = { a: "Hello" }; clone(g).a;          // 👍 g .. l : JsonMap, not especially useful as written
var h : Json = { b: 42 };
var i : Json = { c: true };
var j : Json = { d: null };
var k : Json = { e: [1, 2, ""] };
var l : Json = { e: { 5.4: "foo", mixed: "key types" } };

var m : Json = () => "baz";                         // ERROR 👍
var n : JsonMap = { a : "bar", fn: () => "baz" };   // ERROR 👍
var o : JsonMap = [{ a : "bar", fn: () => "baz" }]; // ERROR 👍

{
    let p : Document = { one : "foo", two : false, 3.141592: "pi" };
    let t = clone(p); // t : Document 👍
    t.one; // : string 👍
    t.two; // : boolean 👍
    let tt = t[3.141592]; // tt : "pi" | boolean 👍
    let tu = t[3.1415]; // tu : string | number | boolean | null | JsonArray | JsonMap
}

var q = () => "baz"; clone(q);                      // ERROR 👍
var r = { a : "bar", fn: () => "baz" }; clone(r);   // ERROR 👍
var s = [{ a : "bar", fn: () => "baz" }]; clone(s); // ERROR 👍

class MyClass implements JsonMap { [key : string] : string }  
clone(new MyClass()) instanceof MyClass; // 😱 false

// Structural typing isn't enough for otherwise-compatible interfaces
interface OtherLibDocument {
    one: string;
    two: boolean;
    3.141592: "pi" | boolean;
}

{
    let p2 : OtherLibDocument = { one : "foo", two : false, 3.141592: "pi" };
    let t = clone(p2); // 😱 Property 'includes' is missing in type 'OtherLibDocument'
}

interface OtherLibDocument2 {
    one: string;
    two: boolean;
    3.141592: "pi" | boolean;
}

// Definition-Merge other libraries' compatible interfaces
interface OtherLibDocument2 extends JsonMap { }

{
    let p3 : OtherLibDocument2 = { one : "foo", two : false, 3.141592: "pi" };
    let t = clone(p3); // t : Document 👍
    t.one; // : string 👍
    t.two; // : boolean 👍
    let tt = t[3.141592]; // tt : "pi" | boolean 👍
    let tu = t[3.1415]; // tu : string | number | boolean | null | JsonArray | JsonMap
}

@aluanhaddad
Copy link
Contributor

aluanhaddad commented May 21, 2017

@NoelAbrahams

The point is classes encapsulate behaviour. JSON represents data. People who .stringify classes are a special breed - rather like the dodo 😄.

I think you are on to something here 😆 but, judging by the number of Angular users on Stack Overflow who expect the simple use of TypeScript to automagically transform JSON.parse into Newtonsoft.Json.JsonConvert.DeserializeObject, they have no idea they are destined for extinction 😝.

@niedzielski
Copy link

Another "me too." Here's what we ended up with:

export type JSONPrimitive = string | number | boolean | null;
export type JSONValue = JSONPrimitive | JSONObject | JSONArray;
export type JSONObject = { [member: string]: JSONValue };
export interface JSONArray extends Array<JSONValue> {}

Unfortunately, usage on arbitrary types requires a type assertion like unmarshal(jsonType as {}).

@ghost
Copy link

ghost commented Oct 23, 2017

Looks like not too late to the party. Google search didn't take long to hit on this issue & what others suggest. What I came up with is basically same as @niedzielski. Despite caveats, it's simple enough to be useful and sits a comfortable distance from settling for just any.

In difference to @niedzielski's version, preferring Json to all-caps JSON prefix and thinking JsonMap instead of JsonObject (as others also have above) as feel "Map" is less overloaded than "Object".

Also going with AnyJson rather than JsonValue.

End result is

type AnyJson =  boolean | number | string | null | JsonArray | JsonMap;
interface JsonMap {  [key: string]: AnyJson; }
interface JsonArray extends Array<AnyJson> {}

Ambivalent about whether or not to separate out JsonPrimitive.

@bitjson
Copy link

bitjson commented Jun 30, 2018

For those finding this issue after the release of TypeScript 2.9, TypeScript now has Support for well-typed JSON imports.

TypeScript is now able to import JSON files as input files when using the node strategy for moduleResolution. This means you can use json files as part of their project, and they’ll be well-typed!

These JSON files will also carry over to your output directory so that things “just work” at runtime.

@nevir
Copy link

nevir commented Sep 24, 2018

This would still be quite useful. For example, I would like to describe a RPC interface via typescript, but assert that inputs/outputs are JSON-serializable. Unfortunately, even with the JSON* interfaces described above, that doesn't seem feasible w/ current typescript:

export type JSONPrimitive = string | number | boolean | null;
export type JSONValue = JSONPrimitive | JSONObject | JSONArray;
export type JSONObject = { [member: string]: JSONValue };
export interface JSONArray extends Array<JSONValue> {}

export interface ServiceDeclaration {
  [key: string]: (params?: JSONObject) => Promise<JSONValue>;
}

// Expected: No errors.
interface MyService extends ServiceDeclaration {
  // Error: Property 'doThing' of type '(params?: { id: string; } | undefined) => Promise<string>' is not assignable to string index type '(params?: JSONObject | undefined) => Promise<JSONValue>'.
  doThing(params?: { id: string }): Promise<string>;
}

One trick that gets me closer is to have template types that ask for keys explicitly, to drop the index type (e.g. the same as typescript’s built in Record type):

export type IsJSONObject<TKeys extends string = string> = { [Key in TKeys]: JSONValue };

interface Foo {
  id: string;
}

function doThing(params: JSONObject) {}
function doThing2<T>(params: IsJSONObject<keyof T>) {}

const foo: Foo = null as any;
// Error: Argument of type 'Foo' is not assignable to parameter of type 'JSONObject'.
//   Index signature is missing in type 'Foo'.
doThing(foo);
// No error!
doThing2(foo)

I'm not sure how to express that in terms of ServiceDeclaration, though…

@toolness
Copy link

Just wanted to mention that this type would also be very useful for me in light of the TS 3.1 breaking change "narrowing functions now intersects {}, Object, and unconstrained generic type parameters". My existing code essentially has a type that could be either something JSON-serializable or a function; before TS 3.1, the former was blissfully expressed as an unconstrained generic, but now I need to figure out how to narrow its type, and doing that would be much easier if a JSON type existed. (On the other hand, I might be going about solving this problem in entirely the wrong way.)

@felixfbecker
Copy link
Contributor

My use case is strictly typing cloneable values that can be passed to WebWorkers through postMessage(). If an object contains a method for example, it will through a runtime error when trying to send it to the Weorker. This should be possible to catch at compile time.

@soanvig
Copy link

soanvig commented Feb 3, 2023

Just bumping this up, providing my case for support for this.

I can have a function, that receives something which is serialized with JSON.stringify under the hood. I don't want function's user to KNOW that out of thin air.

It's really easy to make a mistake for example providing data that will be unexpectedly stringified (objects like Date), and leading to incorrect JSON.parse behavior.

Given in this discussion JsonValue type works OK, but fails for { [x: string]: JsonValue }. It doesn't work if the object is strict interface. Interface doesn't have index type.

@isoroka-plana
Copy link

isoroka-plana commented Sep 28, 2023

I wanted to create a function that would allow downloading some data as JSON for debugging purposes. For now, I am forced to use unknown, any or put together my own type, but really what I need is any JSON-serializable data just as some of the comments have suggested.

An example of how I would like types on my function to behave:

openDebugInfoDownloadDialog('message', { key: 'value' }); // valid

openDebugInfoDownloadDialod('message', {date: new Date(), func: () => {} }) // invalid

@DanielRosenwasser DanielRosenwasser added Declined The issue was declined as something which matches the TypeScript vision and removed Needs More Info The issue still hasn't been fully clarified Needs Proposal This issue needs a plan that clarifies the finer details of how it could be implemented. labels Mar 21, 2024
@DanielRosenwasser
Copy link
Member

TypeScript has come a long way since this issue was filed. I think it is sufficiently expressive enough to describe a json-able type, and there's a bunch of examples above of how one might do that. The one I defined in #57034 was this:

export type Json =
    | string
    | number
    | boolean
    | null
    | Json[]
    | { [key: string]: Json }
    ;

I don't think it warrants adding to lib.d.ts. Generally we've avoided adding new global types unless they are used somehow by the type-checker itself (or by some built-in functions), and this would not be. So defining it locally or in a library is fine for now. So I'm going to close this issue.

@DanielRosenwasser DanielRosenwasser closed this as not planned Won't fix, can't repro, duplicate, stale Mar 21, 2024
@nabe1653
Copy link

nabe1653 commented Mar 21, 2024

It's sad ending. Still I think built-in functions JSON.parse should return Json at least.

@kachkaev
Copy link

kachkaev commented Mar 22, 2024

I use JsonArray / JsonObject / JsonValue from type-fest. These types do job perfectly!

Using some Json type from TS core would be a bit more convenient for end users in short term, but I understand adding them today means more legacy to manage in the future.

@kachkaev
Copy link

kachkaev commented Mar 22, 2024

One thing that’d be great to see in the core is unknown instead of any in JSON.parse / stringify:

TypeScript/src/lib/es5.d.ts

Lines 1137 to 1159 in b75261d

interface JSON {
/**
* Converts a JavaScript Object Notation (JSON) string into an object.
* @param text A valid JSON string.
* @param reviver A function that transforms the results. This function is called for each member of the object.
* If a member contains nested objects, the nested objects are transformed before the parent object is.
*/
parse(text: string, reviver?: (this: any, key: string, value: any) => any): any;
/**
* Converts a JavaScript value to a JavaScript Object Notation (JSON) string.
* @param value A JavaScript value, usually an object or array, to be converted.
* @param replacer A function that transforms the results.
* @param space Adds indentation, white space, and line break characters to the return-value JSON text to make it easier to read.
*/
stringify(value: any, replacer?: (this: any, key: string, value: any) => any, space?: string | number): string;
/**
* Converts a JavaScript value to a JavaScript Object Notation (JSON) string.
* @param value A JavaScript value, usually an object or array, to be converted.
* @param replacer An array of strings and numbers that acts as an approved list for selecting the object properties that will be stringified.
* @param space Adds indentation, white space, and line break characters to the return-value JSON text to make it easier to read.
*/
stringify(value: any, replacer?: (number | string)[] | null, space?: string | number): string;
}

I was caught up by any a couple of times before.

Curious to know if a switch to unknown is possible in some next major version. Or are ESLint rules like @typescript-eslint/no-unsafe-argument or @typescript-eslint/no-unsafe-assignment the right thing to use here?

@yawaramin
Copy link

Wait, why would adding a JSON type for the return type of JSON.parse be a maintenance burden? JSON is not exactly a fast-changing standard, right? 🤔

@milesj
Copy link

milesj commented Mar 22, 2024

JSON.parse should always be any/unknown because JSON data is just that, unknown. Even if you are parsing a JSON file that you know the structure of, or receiving an HTTP response you know the structure of, the JS/TS interpreter does not, so having a type that is string | null | array | object | etc is entirely useless because you can't actually access any of the data in the result without type narrowing.

You'll still need to do typeof checks, Array.isArray checks, and every other kind of check for it to actually be accurate and type-safe. You're better of using an explicit type, like interface UserResponse {...} and casting it, or a robust solution like zod, joi, etc. But again, a JSON type is kind of useless.

At minimum, I agree with @kachkaev that the type should be unknown over any because any is just bad.

@Peeja
Copy link
Contributor

Peeja commented Mar 22, 2024

It's not useless if you're going to pass it to something that takes JSON objects.

For instance, as described above, a toJSON() function should return a JSON object. It would be valuable to type it as such, to ask the implementation to prove that it returns something JSON-y. The result of a JSON.parse() should be a valid return value for a toJSON() method.

@reverofevil
Copy link

But again, a JSON type is kind of useless

Those typeof checks you mentioned are harder to do on any or unknown. For a Json you know for a fact there would be no symbol, function, undefined, or bigint.

Also try JSON.stringify(undefined) and see what the resulting string is. How would you use any or unknown argument type to avoid that?

@milesj
Copy link

milesj commented Mar 22, 2024

Those typeof checks you mentioned are harder to do on any or unknown. For a Json you know for a fact there would be no symbol, function, undefined, or bigint.

In almost all cases you would be checking for number/string/object/null/etc explicitly, so those other branches can be ignored. And again, the fact that this would require so much type narrowing anyways just isn't worth the effort. Either trust the source outright, or use zod/etc.

Also try JSON.stringify(undefined) and see what the resulting string is. How would you use any or unknown argument type to avoid that?

Pretty sure this could just be solved with {} or {} | null.

@reverofevil
Copy link

reverofevil commented Mar 22, 2024

In almost all cases you would be checking for

What about exhaustive checks?

or use zod/etc

How would you implement zod? What if you cannot tolerate extra 57KiB in your bundle?

Pretty sure this could just be solved with {} or {} | null.

JSON.stringify(() => {})

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Declined The issue was declined as something which matches the TypeScript vision Suggestion An idea for TypeScript
Projects
None yet
Development

No branches or pull requests