Disallow generics in mypy typing check #4052
Closed
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Aim: fulfill task
disallow_any_generics
in #4044.This PR is far from complete. There are over 100 warnings, and I ran into several issues. We should decide on those first before going through all issues.
LazyFrame
has due to Preserve DataFrame type after LazyFrame roundtrips #2862 and related PR's become a GenericType, to allow for nice typing synergy where one can subclassDataFrame
and get theLazyFrame
typed version as well. This is implemented (sorry for non-precise language here) by carrying around the type ofDataFrame
thatLazyFrame
refers to, or in other words,LazyFrame
wraps around theDataFrame
. This does mean that the specific DataFrame is a type input into LazyFrame, and mypy does not accept the default value: Unable to specify a default value for a generic parameter python/mypy#3737 So the solution for now is for each type annotation in the polars code base that involvesLazyFrame
to writeLazyFrame[Any]
I would think that we could also useLazyFrame[DF]
instead, but ran into issues with that. Open to suggestionsnumpy has been strongly improving their typing, and where we currently use
np.ndarray
as the type annotation, it seems we should move towardsnumpy.typing.NDArray
(https://numpy.org/doc/stable/reference/typing.html#numpy.typing.NDArray) However, this was only introduced in numpy 1.21, released in June 2021. We have always been very reluctant to impose version constraints on dependencies (as our usage of numpy is mostly high level). If we go down this path, we will have to. Otherwise, what I have done so far in some of the PR, is to simply add ignores.Most of the other issues are us not typing built-in generics such as
list
,tuple
,dict
,Callable
,Sequence
etc, those should be easy to fix.