[ty] Support typevar-specialized dynamic types in generic type aliases#21730
Merged
[ty] Support typevar-specialized dynamic types in generic type aliases#21730
Conversation
Diagnostic diff on typing conformance testsChanges were detected when running ty on typing conformance tests--- old-output.txt 2025-12-03 08:27:30.384262987 +0000
+++ new-output.txt 2025-12-03 08:27:34.022271510 +0000
@@ -4,25 +4,21 @@
_directives_deprecated_library.py:41:25: error[invalid-return-type] Function always implicitly returns `None`, which is not assignable to return type `int | float`
_directives_deprecated_library.py:45:24: error[invalid-return-type] Function always implicitly returns `None`, which is not assignable to return type `str`
aliases_explicit.py:41:24: error[invalid-type-form] List literals are not allowed in this context in a type expression: Did you mean `tuple[str, str]`?
-aliases_explicit.py:41:36: error[invalid-type-arguments] Too many type arguments: expected 1, got 2
-aliases_explicit.py:57:5: error[type-assertion-failure] Type `(int, str, str, /) -> None` does not match asserted type `(...) -> Unknown`
+aliases_explicit.py:57:5: error[type-assertion-failure] Type `(int, str, str, /) -> None` does not match asserted type `Unknown`
aliases_explicit.py:60:5: error[type-assertion-failure] Type `(...) -> None` does not match asserted type `@Todo(Callable[..] specialized with ParamSpec)`
aliases_explicit.py:67:24: error[invalid-type-arguments] Too many type arguments: expected 0, got 1
aliases_explicit.py:68:24: error[invalid-type-arguments] Too many type arguments: expected 0, got 1
aliases_explicit.py:69:29: error[invalid-type-arguments] Too many type arguments: expected 1, got 2
aliases_explicit.py:70:29: error[invalid-type-arguments] Too many type arguments: expected 1, got 2
-aliases_explicit.py:71:29: error[invalid-type-arguments] Too many type arguments: expected 1, got 2
aliases_explicit.py:101:6: error[call-non-callable] Object of type `UnionType` is not callable
aliases_explicit.py:102:20: error[invalid-type-arguments] Too many type arguments: expected 0, got 1
aliases_implicit.py:54:24: error[invalid-type-form] List literals are not allowed in this context in a type expression: Did you mean `tuple[str, str]`?
-aliases_implicit.py:54:36: error[invalid-type-arguments] Too many type arguments: expected 1, got 2
-aliases_implicit.py:68:5: error[type-assertion-failure] Type `(int, str, str, /) -> None` does not match asserted type `(...) -> Unknown`
+aliases_implicit.py:68:5: error[type-assertion-failure] Type `(int, str, str, /) -> None` does not match asserted type `Unknown`
aliases_implicit.py:72:5: error[type-assertion-failure] Type `(...) -> None` does not match asserted type `@Todo(Callable[..] specialized with ParamSpec)`
aliases_implicit.py:76:24: error[invalid-type-arguments] Too many type arguments: expected 0, got 1
aliases_implicit.py:77:24: error[invalid-type-arguments] Too many type arguments: expected 0, got 1
aliases_implicit.py:78:29: error[invalid-type-arguments] Too many type arguments: expected 1, got 2
aliases_implicit.py:79:29: error[invalid-type-arguments] Too many type arguments: expected 1, got 2
-aliases_implicit.py:80:29: error[invalid-type-arguments] Too many type arguments: expected 1, got 2
aliases_implicit.py:81:25: error[invalid-type-arguments] Type `str` is not assignable to upper bound `int | float` of type variable `TFloat@GoodTypeAlias12`
aliases_implicit.py:107:9: error[invalid-type-form] Variable of type `list[Unknown | <class 'int'> | <class 'str'>]` is not allowed in a type expression
aliases_implicit.py:108:9: error[invalid-type-form] Variable of type `tuple[tuple[<class 'int'>, <class 'str'>]]` is not allowed in a type expression
@@ -141,10 +137,6 @@
callables_annotation.py:58:5: error[invalid-type-form] Special form `typing.Callable` expected exactly two arguments (parameter types and return type)
callables_annotation.py:58:14: error[invalid-type-form] The first argument to `Callable` must be either a list of types, ParamSpec, Concatenate, or `...`
callables_annotation.py:157:20: error[invalid-assignment] Object of type `Proto7` is not assignable to `Proto6`
-callables_annotation.py:172:19: error[invalid-type-arguments] Too many type arguments: expected 0, got 1
-callables_annotation.py:175:19: error[invalid-type-arguments] Too many type arguments: expected 0, got 1
-callables_annotation.py:188:25: error[invalid-type-arguments] Too many type arguments: expected 0, got 1
-callables_annotation.py:189:25: error[invalid-type-arguments] Too many type arguments: expected 0, got 1
callables_kwargs.py:24:5: error[type-assertion-failure] Type `int` does not match asserted type `@Todo(`Unpack[]` special form)`
callables_kwargs.py:32:9: error[type-assertion-failure] Type `str` does not match asserted type `@Todo(`Unpack[]` special form)`
callables_kwargs.py:35:5: error[type-assertion-failure] Type `str` does not match asserted type `@Todo(`Unpack[]` special form)`
@@ -1028,4 +1020,4 @@
typeddicts_usage.py:28:17: error[missing-typed-dict-key] Missing required key 'name' in TypedDict `Movie` constructor
typeddicts_usage.py:28:18: error[invalid-key] Unknown key "title" for TypedDict `Movie`: Unknown key "title"
typeddicts_usage.py:40:24: error[invalid-type-form] The special form `typing.TypedDict` is not allowed in type expressions
-Found 1030 diagnostics
+Found 1022 diagnostics
|
|
sharkdp
force-pushed
the
david/fix-specialized-unknown
branch
from
d9f5c84 to
55e61e7
Compare
|
| Lint rule | Added | Removed | Changed |
|---|---|---|---|
invalid-type-arguments |
0 | 416 | 19 |
type-assertion-failure |
0 | 0 | 358 |
invalid-context-manager |
0 | 0 | 36 |
invalid-argument-type |
2 | 4 | 0 |
possibly-missing-attribute |
0 | 0 | 3 |
unsupported-base |
0 | 3 | 0 |
unsupported-operator |
2 | 0 | 1 |
no-matching-overload |
0 | 2 | 0 |
invalid-return-type |
0 | 0 | 1 |
unused-ignore-comment |
1 | 0 | 0 |
| Total | 5 | 425 | 418 |
sharkdp
force-pushed
the
david/fix-specialized-unknown
branch
from
55e61e7 to
7cab1a0
Compare
sharkdp
changed the title
sharkdp
force-pushed
the
david/fix-specialized-unknown
branch
from
7cab1a0 to
ea286c2
Compare
sharkdp
force-pushed
the
david/fix-specialized-unknown
branch
from
ea286c2 to
7072614
Compare
sharkdp
requested review from
AlexWaygood,
carljm and
dcreager
as code owners
Contributor
|
The conformance suite results here look fine. Most of the changes are around |
carljm
reviewed
Dec 3, 2025
Comment on lines
+220
to
+238
| # TODO: should be (list[int], /) -> int | ||
| reveal_type(c) # revealed: (Unknown, /) -> int | ||
|
|
||
| K = TypeVar("K") | ||
| V = TypeVar("V") | ||
|
|
||
| MyDict = TypeAliasType("MyDict", dict[K, V], type_params=(K, V)) | ||
|
|
||
| MyAlias6 = Callable[[MyDict[K, V]], int] | ||
|
|
||
| def _(c: MyAlias6[str, bytes]): | ||
| # TODO: should be (dict[str, bytes], /) -> int | ||
| reveal_type(c) # revealed: (Unknown, /) -> int | ||
|
|
||
| ListOrDict: TypeAlias = MyList[T] | dict[str, T] | ||
|
|
||
| def _(x: ListOrDict[int]): | ||
| # TODO: should be list[int] | dict[str, int] | ||
| reveal_type(x) # revealed: Unknown | dict[str, int] |
Contributor
There was a problem hiding this comment.
What's our barrier to getting these three TODOs right? I guess it's just that we currently don't support generics at all on "manual" PEP 695 type aliases?
Contributor
Author
There was a problem hiding this comment.
Yes, exactly. I considered this to be a (maybe lower-priority?) task separate from the generic implicit/PEP613 type alias work. I opened astral-sh/ty#1737.
| ); | ||
| let generic_context = | ||
| GenericContext::from_typevar_instances(self.db(), variables); | ||
| Type::Dynamic(DynamicType::UnknownGeneric(generic_context)) |
Contributor
There was a problem hiding this comment.
There should be a TODO here, yeah?
And we should probably file a follow-up issue for supporting generics in "manual" PEP 695 type aliases?
Contributor
Author
There was a problem hiding this comment.
There should be a TODO here, yeah?
Yes, I think in my mind, the whole DynamicType::UnknownGeneric variant was a sort of "Todo" marker, but I added an explicit TODO comment now 👍
And we should probably file a follow-up issue for supporting generics in "manual" PEP 695 type aliases?
| #[derive(Debug, Copy, Clone, Hash, PartialEq, Eq, get_size2::GetSize)] | ||
| pub enum SuperOwnerKind<'db> { | ||
| Dynamic(DynamicType), | ||
| Dynamic(DynamicType<'db>), |
Contributor
There was a problem hiding this comment.
Heh, this lifetime was just removed a week or so ago...
sharkdp
force-pushed
the
david/fix-specialized-unknown
branch
from
7072614 to
3e2ef9b
Compare
Contributor
Author
Thank you for checking, I was so focused on the ecosystem impact (which was erratically changing with every push), I forgot to take a look at the conformance results. |
dcreager
added a commit
that referenced
this pull request
Dec 3, 2025
* origin/main: [ty] Improve `@override`, `@final` and Liskov checks in cases where there are multiple reachable definitions (#21767) [ty] Extend `invalid-explicit-override` to also cover properties decorated with `@override` that do not override anything (#21756) [ty] Enable LRU collection for parsed module (#21749) [ty] Support typevar-specialized dynamic types in generic type aliases (#21730) Add token based `parenthesized_ranges` implementation (#21738) [ty] Default-specialization of generic type aliases (#21765) [ty] Suppress false positives when `dataclasses.dataclass(...)(cls)` is called imperatively (#21729) [syntax-error] Default type parameter followed by non-default type parameter (#21657) new module for parsing ranged suppressions (#21441) [ty] `type[T]` is assignable to an inferable typevar (#21766) Fix syntax error false positives for `await` outside functions (#21763) [ty] Improve diagnostics for unsupported comparison operations (#21737) Move `Token`, `TokenKind` and `Tokens` to `ruff-python-ast` (#21760) [ty] Don't confuse multiple occurrences of `typing.Self` when binding bound methods (#21754) Use our org-wide Renovate preset (#21759) Delete `my-script.py` (#21751) [ty] Move `all_members`, and related types/routines, out of `ide_support.rs` (#21695)
dcreager
added a commit
that referenced
this pull request
Dec 3, 2025
* origin/main: [ty] Reachability constraints: minor documentation fixes (#21774) [ty] Fix non-determinism in `ConstraintSet.specialize_constrained` (#21744) [ty] Improve `@override`, `@final` and Liskov checks in cases where there are multiple reachable definitions (#21767) [ty] Extend `invalid-explicit-override` to also cover properties decorated with `@override` that do not override anything (#21756) [ty] Enable LRU collection for parsed module (#21749) [ty] Support typevar-specialized dynamic types in generic type aliases (#21730) Add token based `parenthesized_ranges` implementation (#21738) [ty] Default-specialization of generic type aliases (#21765) [ty] Suppress false positives when `dataclasses.dataclass(...)(cls)` is called imperatively (#21729) [syntax-error] Default type parameter followed by non-default type parameter (#21657)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
For a type alias like the one below, where
UnknownClassis something with a dynamic type, we previously lost track of the fact that this dynamic type was explicitly specialized with a type variable. If that alias is then later explicitly specialized itself (MyAlias[int]), we would miscount the number of legacy type variables and emit ainvalid-type-argumentsdiagnostic (playground).The solution implemented here is not pretty, but we can hopefully get rid of it via astral-sh/ty#1711. Also, once we properly support
ParamSpecandConcatenate, we should be able to remove some of this code.This addresses many of the
invalid-type-argumentsfalse-positives in astral-sh/ty#1685. With this change, there are still some diagnostics of this type left. Instead of implementing even more (rather sophisticated) workarounds for these cases as well, it might be much easier to wait for fullParamSpec/Concatenatesupport and then try again.A disadvantage of this implementation is that we lose track of some
@Todotypes and replace them withUnknown. We could spend more effort and try to preserve them, but I'm unsure if this is the best use of our time right now.Test Plan
New Markdown tests.