DRAFT: Returning TypedDict for dataclasses.asdict

[syastrov]

I wanted to implement dataclasses.asdict returning TypedDicts. I ran into issues supporting the recursion that it does. Does anyone have suggestions as to the problems I commented in the tests? Thanks

Edit: Link to comment: https://github.com/python/mypy/pull/8339/files#diff-5b6d315f9321d8b1943195ed39e18f14R1084

Edit: Relates to #5152

[intgr]

Maybe this would be expanding the scope too much, but there have been many times I've wanted something similar. If you could implement this as a generic type, such as DictOf[SomeClass], this opens up potential new use cases, for example:

1. There are many serialization libraries that work on type info besides asdict, and could be supported with this. Pydantic and djangorestframework-dataclasses are ones that I'm personally familiar with.
2. For any class A, the attribute A().__dict__ could be inspected to have type DictOf[A], except properties and class variables. Maybe a future PR could implement NonTotalDicfOf[A, total=False] (similar to TypedDict(total=False))
3. There are times when you want to handle a subset of an object's fields as a dict. One example is Django's Model.objects.create_or_update(field=value, defaults={'field2': value}). The defaults argument could be annotated as NonTotalDictOf[Model]

Something like TypeScript's mapped types would be even better but that's rocket science compared to this.

[syastrov]

@intgr It's out of scope for this PR, but I have to agree with you and I've also really wanted to make it easier to type these kinds of things. It deserves an issue where it can be discussed and maybe on https://github.com/python/typing/ instead?

The problem I encountered is basically that at runtime isinstance succeeds for the newly asdicted type (e.g. if it's a NamedTuple), yet the fields don't have the same types (because the dataclasses were converted to TypedDicts) as the original NamedTuple.

[JukkaL]

You should be able to support list subclasses by using map_instance_to_supertype.

For named tuples and tuple subclasses, you can perhaps use the tuple_type attribute of TypeInfo.

Recursion is currently not possible, since general recursive types aren't supported. One idea would be to detect recursive references and replace them with Any, and to perhaps generate an error about recursive types not being supported (which could be ignored using # type: ignore if the user is fine with this).

[syastrov]

Hi @JukkaL. Thanks for the pointers.
I did manage to support subclasses of list/dict (hopefully correctly) using map_instance_to_supertype: so only the TypeVars used by the supertype (list or dict) are transformed according to asdict logic.

There is a problem I ran into with this, though. The type vars on the list/dict can be constrained or have variance in the subclass such that, once the type var is converted from a dataclass to a TypedDict, the resultant type violates the constraints/variance of the original TypeVar. I am not sure how I can get mypy to throw errors in that situation (or if we should just fallback to Any instead?). Currently, it just allows it:
0111e36#diff-5b6d315f9321d8b1943195ed39e18f14R1181

I am still not sure how to support subclasses of NamedTuple. It seems that I need to produce a new TypeInfo (somehow) which is essentially a totally new subclass of NamedTuple where the items are (recursively) transformed according to the asdict logic? I am not sure which APIs I could use to achieve this. I would be afraid I would have to duplicate a lot of the code in semanal_namedtuple.py.

I also managed to implement detection of recursion, where an error is thrown and it falls back to producing Dict[str, Any] for the first detected recursive reference.

TODO before ready for review:

• Update documentation.
• Throw an error when transforming type vars to TypedDicts if they violate constraints/variance after transformation?
• Support subclasses of Tuple (should be straightforward).
• Support subclasses of NamedTuple (hopefully doable depending on if I can figure out how to do it)
• Cleanup / type-check / reformat code

[syastrov]

Going to make a new PR