[ty] Infer typevar specializations for `Callable` types #21551

dcreager · 2025-11-26T23:29:35Z

This TODO is not also removed because we end up inferring this constraint set when comparing head to Callable[[A], B]:

(B@invoke ≤ T@head) ∧ (list[T@head] ≤ A@invoke)

We then try to remove T@head from the constraint set by calculating

∃T@head ⋅ (B@invoke ≤ T@head) ∧ (list[T@head] ≤ A@invoke)

We should be able to pick T@head = B@invoke and simplify that to

(B@invoke = *) ∧ (list[B@invoke] ≤ A@invoke)

which I think would then be enough to propagate through the return type to discharge this TODO. I think this would require adding more derived facts to the sequent map.

carljm · 2025-12-12T02:44:06Z

Hmm, why do we lose the parameters here?

This is an example where we're now collectively taking into account all of the constraints across the entire call site. In this case, we're correctly flagging invalid-argument-type, because change_return_type expects a callable that returns an int, which neither overload of str_str does. Before we went ahead and inferred a specialization of P from the callable parameter, even though it was invalid. But now, since the argument is invalid, we don't infer anything for P, and fall back on its Unknown equivalent (...)

dcreager · 2025-12-10T00:58:24Z

This is an error condition anyway, and I find it better to infer Unknown here than to let the typevar leak through when we were supposed to have solved it.

dcreager · 2025-11-26T23:30:15Z

This is because we're now using specialize_recursive instead of specialize_partial.

-Original file line number
+Diff line change
@@ Expand Up / @@ -194,7 +194,7 @@ static SYMPY: Benchmark = Benchmark::new( @@
             max_dep_date: "2025-06-17",
             python_version: PythonVersion::PY312,
         },
-,
+,
     );
     static TANJUN: Benchmark = Benchmark::new(
@@ Expand Down Expand Up / @@ -223,7 +223,7 @@ static STATIC_FRAME: Benchmark = Benchmark::new( @@
             max_dep_date: "2025-08-09",
             python_version: PythonVersion::PY311,
         },
-,
+,
     );
     #[track_caller]
@@ Expand Down @@

-Original file line number
+Diff line change
@@ Expand Up / @@ -194,7 +194,7 @@ reveal_type(B().name_does_not_matter()) # revealed: B @@
     reveal_type(B().positional_only(1))  # revealed: B
     reveal_type(B().keyword_only(x=1))  # revealed: B
     # TODO: This should deally be `B`
-    reveal_type(B().decorated_method())  # revealed: Unknown
+    reveal_type(B().decorated_method())  # revealed: Self@decorated_method
     reveal_type(B().a_property)  # revealed: B
@@ Expand Down @@

-Original file line number
+Diff line change
@@ Expand Up / @@ -43,9 +43,7 @@ async def main(): @@
         loop = asyncio.get_event_loop()
         with concurrent.futures.ThreadPoolExecutor() as pool:
             result = await loop.run_in_executor(pool, blocking_function)
-            # TODO: should be `int`
-            reveal_type(result)  # revealed: Unknown
+            reveal_type(result)  # revealed: int
     ```
     ### `asyncio.Task`
@@ Expand Down @@

-Original file line number
+Diff line change
@@ Expand Up / @@ -82,8 +82,7 @@ def get_default() -> str: @@
     reveal_type(field(default=1))  # revealed: dataclasses.Field[Literal[1]]
     reveal_type(field(default=None))  # revealed: dataclasses.Field[None]
-    # TODO: this could ideally be `dataclasses.Field[str]` with a better generics solver
-    reveal_type(field(default_factory=get_default))  # revealed: dataclasses.Field[Unknown]
+    reveal_type(field(default_factory=get_default))  # revealed: dataclasses.Field[str]
     ```
     ## dataclass_transform field_specifiers
@@ Expand Down @@

-Original file line number
+Diff line change
@@ Expand Up / @@ -144,11 +144,10 @@ from functools import cache @@
     def f(x: int) -> int:
         return x**2
-    # TODO: Should be `_lru_cache_wrapper[int]`
-    reveal_type(f)  # revealed: _lru_cache_wrapper[Unknown]
-    # TODO: Should be `int`
-    reveal_type(f(1))  # revealed: Unknown
+    # revealed: _lru_cache_wrapper[int]
+    reveal_type(f)
+    # revealed: int
+    reveal_type(f(1))
     ```
     ## Lambdas as decorators
@@ Expand Down @@

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[ty] Infer typevar specializations for `Callable` types #21551

Uh oh!

Diff view

Diff view

There are no files selected for viewing

dcreager Nov 26, 2025

Uh oh!

Uh oh!

carljm Dec 12, 2025

Uh oh!

dcreager Dec 13, 2025

Uh oh!

dcreager Dec 10, 2025

Uh oh!

dcreager Nov 26, 2025

Uh oh!

Uh oh!

-Original file line number
+Diff line change
@@ Expand Up @@
     from typing_extensions import deprecated
     @deprecated("use OtherClass")
-    def myfunc(): ...
+    def myfunc(x: int): ...
-    myfunc()  # error: [deprecated] "use OtherClass"
+    myfunc(1)  # error: [deprecated] "use OtherClass"
     ```
     ```py
@@ Expand Down @@

-Original file line number
+Diff line change
@@ Expand Up / @@ -555,8 +555,7 @@ def identity(x: T) -> T: @@
     def head(xs: list[T]) -> T:
         return xs[0]
-    # TODO: this should be `Literal[1]`
-    reveal_type(invoke(identity, 1))  # revealed: Unknown
+    reveal_type(invoke(identity, 1))  # revealed: Literal[1]
     # TODO: this should be `Unknown | int`
     reveal_type(invoke(head, [1, 2, 3]))  # revealed: Unknown
@@ Expand Down @@

-Original file line number
+Diff line change
@@ Expand Up / @@ -518,8 +518,7 @@ V = TypeVar("V", default="V") @@
     class D(Generic[V]):
         x: V
-    # TODO: we shouldn't leak a typevar like this in type inference
-    reveal_type(D().x)  # revealed: V@D
+    reveal_type(D().x)  # revealed: Unknown
     ```
     ## Regression
@@ Expand Down @@

-Original file line number
+Diff line change
@@ Expand Up / @@ -493,8 +493,7 @@ def identity[T](x: T) -> T: @@
     def head[T](xs: list[T]) -> T:
         return xs[0]
-    # TODO: this should be `Literal[1]`
-    reveal_type(invoke(identity, 1))  # revealed: Unknown
+    reveal_type(invoke(identity, 1))  # revealed: Literal[1]
     # TODO: this should be `Unknown | int`
     reveal_type(invoke(head, [1, 2, 3]))  # revealed: Unknown
@@ Expand Down Expand Up / @@ -736,3 +735,159 @@ def f[T](x: T, y: Not[T]) -> T: @@
         y = x  # error: [invalid-assignment]
         return x
     ```
+    ## `Callable` parameters
+    We can recurse into the parameters and return values of `Callable` parameters to infer
+    specializations of a generic function.
+    ```py
+    from typing import Any, Callable, NoReturn, overload, Self
+    from ty_extensions import generic_context, into_callable
+    def accepts_callable[**P, R](callable: Callable[P, R]) -> Callable[P, R]:
+        return callable
+    def returns_int() -> int:
+        raise NotImplementedError
+    # revealed: () -> int
+    reveal_type(into_callable(returns_int))
+    # revealed: () -> int
+    reveal_type(accepts_callable(returns_int))
+    # revealed: int
+    reveal_type(accepts_callable(returns_int)())
+    class ClassWithoutConstructor: ...
+    # revealed: () -> ClassWithoutConstructor
+    reveal_type(into_callable(ClassWithoutConstructor))
+    # revealed: () -> ClassWithoutConstructor
+    reveal_type(accepts_callable(ClassWithoutConstructor))
+    # revealed: ClassWithoutConstructor
+    reveal_type(accepts_callable(ClassWithoutConstructor)())
+    class ClassWithNew:
+        def __new__(cls, *args, **kwargs) -> Self:
+            raise NotImplementedError
+    # revealed: (...) -> ClassWithNew
+    reveal_type(into_callable(ClassWithNew))
+    # revealed: (...) -> ClassWithNew
+    reveal_type(accepts_callable(ClassWithNew))
+    # revealed: ClassWithNew
+    reveal_type(accepts_callable(ClassWithNew)())
+    class ClassWithInit:
+        def __init__(self) -> None: ...
+    # revealed: () -> ClassWithInit
+    reveal_type(into_callable(ClassWithInit))
+    # revealed: () -> ClassWithInit
+    reveal_type(accepts_callable(ClassWithInit))
+    # revealed: ClassWithInit
+    reveal_type(accepts_callable(ClassWithInit)())
+    class ClassWithNewAndInit:
+        def __new__(cls, *args, **kwargs) -> Self:
+            raise NotImplementedError
+        def __init__(self, x: int) -> None: ...
+    # TODO: We do not currently solve a common behavioral supertype for the two solutions of P.
+    # revealed: ((...) -> ClassWithNewAndInit) | ((x: int) -> ClassWithNewAndInit)
+    reveal_type(into_callable(ClassWithNewAndInit))
+    # TODO: revealed: ((...) -> ClassWithNewAndInit) | ((x: int) -> ClassWithNewAndInit)
+    # revealed: (...) -> ClassWithNewAndInit
+    reveal_type(accepts_callable(ClassWithNewAndInit))
+    # revealed: ClassWithNewAndInit
+    reveal_type(accepts_callable(ClassWithNewAndInit)())
+    class Meta(type):
+        def __call__(cls, *args: Any, **kwargs: Any) -> NoReturn:
+            raise NotImplementedError
+    class ClassWithNoReturnMetatype(metaclass=Meta):
+        def __new__(cls, *args: Any, **kwargs: Any) -> Self:
+            raise NotImplementedError
+    # TODO: The return types here are wrong, because we end up creating a constraint (Never ≤ R), which
+    # we confuse with "R has no lower bound".
+    # revealed: (...) -> Never
+    reveal_type(into_callable(ClassWithNoReturnMetatype))
+    # TODO: revealed: (...) -> Never
+    # revealed: (...) -> Unknown
+    reveal_type(accepts_callable(ClassWithNoReturnMetatype))
+    # TODO: revealed: Never
+    # revealed: Unknown
+    reveal_type(accepts_callable(ClassWithNoReturnMetatype)())
+    class Proxy: ...
+    class ClassWithIgnoredInit:
+        def __new__(cls) -> Proxy:
+            return Proxy()
+        def __init__(self, x: int) -> None: ...
+    # revealed: () -> Proxy
+    reveal_type(into_callable(ClassWithIgnoredInit))
+    # revealed: () -> Proxy
+    reveal_type(accepts_callable(ClassWithIgnoredInit))
+    # revealed: Proxy
+    reveal_type(accepts_callable(ClassWithIgnoredInit)())
+    class ClassWithOverloadedInit[T]:
+        t: T  # invariant
+        @overload
+        def __init__(self: "ClassWithOverloadedInit[int]", x: int) -> None: ...
+        @overload
+        def __init__(self: "ClassWithOverloadedInit[str]", x: str) -> None: ...
+        def __init__(self, x: int | str) -> None: ...
+    # TODO: The old solver cannot handle this overloaded constructor. The ideal solution is that we
+    # would solve **P once, and map it to the entire overloaded signature of the constructor. This
+    # mapping would have to include the return types, since there are different return types for each
+    # overload. We would then also have to determine that R must be equal to the return type of **P's
+    # solution.
+    # revealed: Overload[(x: int) -> ClassWithOverloadedInit[int], (x: str) -> ClassWithOverloadedInit[str]]
+    reveal_type(into_callable(ClassWithOverloadedInit))
+    # TODO: revealed: Overload[(x: int) -> ClassWithOverloadedInit[int], (x: str) -> ClassWithOverloadedInit[str]]
+    # revealed: Overload[(x: int) -> ClassWithOverloadedInit[int] | ClassWithOverloadedInit[str], (x: str) -> ClassWithOverloadedInit[int] | ClassWithOverloadedInit[str]]
+    reveal_type(accepts_callable(ClassWithOverloadedInit))
+    # TODO: revealed: ClassWithOverloadedInit[int]
+    # revealed: ClassWithOverloadedInit[int] | ClassWithOverloadedInit[str]
+    reveal_type(accepts_callable(ClassWithOverloadedInit)(0))
+    # TODO: revealed: ClassWithOverloadedInit[str]
+    # revealed: ClassWithOverloadedInit[int] | ClassWithOverloadedInit[str]
+    reveal_type(accepts_callable(ClassWithOverloadedInit)(""))
+    class GenericClass[T]:
+        t: T  # invariant
+        def __new__(cls, x: list[T], y: list[T]) -> Self:
+            raise NotImplementedError
+    def _(x: list[str]):
+        # TODO: This fails because we are not propagating GenericClass's generic context into the
+        # Callable that we create for it.
+        # revealed: (x: list[T@GenericClass], y: list[T@GenericClass]) -> GenericClass[T@GenericClass]
+        reveal_type(into_callable(GenericClass))
+        # revealed: ty_extensions.GenericContext[T@GenericClass]
+        reveal_type(generic_context(into_callable(GenericClass)))
+        # revealed: (x: list[T@GenericClass], y: list[T@GenericClass]) -> GenericClass[T@GenericClass]
+        reveal_type(accepts_callable(GenericClass))
+        # TODO: revealed: ty_extensions.GenericContext[T@GenericClass]
+        # revealed: None
+        reveal_type(generic_context(accepts_callable(GenericClass)))
+        # TODO: revealed: GenericClass[str]
+        # TODO: no errors
+        # revealed: GenericClass[T@GenericClass]
+        # error: [invalid-argument-type]
+        # error: [invalid-argument-type]
+        reveal_type(accepts_callable(GenericClass)(x, x))
+    ```

-Original file line number
+Diff line change
@@ Expand Up / @@ -503,7 +503,8 @@ class C[**P]: @@
         def __init__(self, f: Callable[P, int]) -> None:
             self.f = f
-    def f(x: int, y: str) -> bool:
+    # Note that the return type must match exactly, since C is invariant on the return type of C.f.
+    def f(x: int, y: str) -> int:
         return True
     c = C(f)
@@ Expand Down Expand Up @@
     reveal_type(foo.method(1, "a"))  # revealed: str
     ```
+    ### Gradual types propagate through `ParamSpec` inference
+    ```py
+    from typing import Callable
+    def callable_identity[**P, R](func: Callable[P, R]) -> Callable[P, R]:
+        return func
+    @callable_identity
+    def f(env: dict) -> None:
+        pass
+    # revealed: (env: dict[Unknown, Unknown]) -> None
+    reveal_type(f)
+    ```
     ### Overloads
     `overloaded.pyi`:
@@ Expand Down Expand Up @@
     reveal_type(change_return_type(int_str))  # revealed: Overload[(x: int) -> str, (x: str) -> str]
     # error: [invalid-argument-type]
-    reveal_type(change_return_type(str_str))  # revealed: Overload[(x: int) -> str, (x: str) -> str]
+    reveal_type(change_return_type(str_str))  # revealed: (...) -> str
     # TODO: Both of these shouldn't raise an error
     # error: [invalid-argument-type]
@@ Expand Down @@

-Original file line number
+Diff line change
@@ Expand Up / @@ -883,7 +883,7 @@ reveal_type(C[int]().y) # revealed: int @@
     class D[T = T]:
         x: T
-    reveal_type(D().x)  # revealed: T@D
+    reveal_type(D().x)  # revealed: Unknown
     ```
     [pep 695]: https://peps.python.org/pep-0695/

-Original file line number
+Diff line change
@@ Expand Up / @@ -426,7 +426,8 @@ from ty_extensions import ConstraintSet, generic_context @@
     def mentions[T, U]():
         # (T@mentions ≤ int) ∧ (U@mentions = list[T@mentions])
         constraints = ConstraintSet.range(Never, T, int) & ConstraintSet.range(list[T], U, list[T])
-        # revealed: ty_extensions.Specialization[T@mentions = int, U@mentions = list[int]]
+        # TODO: revealed: ty_extensions.Specialization[T@mentions = int, U@mentions = list[int]]
+        # revealed: ty_extensions.Specialization[T@mentions = int, U@mentions = Unknown]
         reveal_type(generic_context(mentions).specialize_constrained(constraints))
     ```
@@ Expand Down @@

-Original file line number
+Diff line change
@@ Expand Up / @@ -304,7 +304,7 @@ x11: list[Literal[1] | Literal[2] | Literal[3]] = [1, 2, 3] @@
     reveal_type(x11)  # revealed: list[Literal[1, 2, 3]]
     x12: Y[Y[Literal[1]]] = [[1]]
-    reveal_type(x12)  # revealed: list[Y[Literal[1]]]
+    reveal_type(x12)  # revealed: list[list[Literal[1]]]
     x13: list[tuple[Literal[1], Literal[2], Literal[3]]] = [(1, 2, 3)]
     reveal_type(x13)  # revealed: list[tuple[Literal[1], Literal[2], Literal[3]]]
@@ Expand Down @@

-Original file line number
+Diff line change
@@ Expand Up @@
 | from typing_extensions import deprecated
 |
 | @deprecated("use OtherClass")
-| def myfunc(): ...
+| def myfunc(x: int): ...
 |
-| myfunc()  # error: [deprecated] "use OtherClass"
+| myfunc(1)  # error: [deprecated] "use OtherClass"
 | from typing_extensions import deprecated
 |
 | @deprecated("use BetterClass")
@@ Expand All @@
     warning[deprecated]: The function `myfunc` is deprecated
      --> src/mdtest_snippet.py:6:1
       |
-| def myfunc(): ...
+| def myfunc(x: int): ...
 |
-| myfunc()  # error: [deprecated] "use OtherClass"
+| myfunc(1)  # error: [deprecated] "use OtherClass"
       | ^^^^^^ use OtherClass
 | from typing_extensions import deprecated
       |
@@ Expand Down @@

[ty] Infer typevar specializations for Callable types #21551

Uh oh!

[ty] Infer typevar specializations for Callable types #21551

Uh oh!

Uh oh!

Diff view

Diff view

There are no files selected for viewing

dcreager Nov 26, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

carljm Dec 12, 2025

Choose a reason for hiding this comment

Uh oh!

dcreager Dec 13, 2025

Choose a reason for hiding this comment

Uh oh!

dcreager Dec 10, 2025

Choose a reason for hiding this comment

Uh oh!

dcreager Nov 26, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

[ty] Infer typevar specializations for `Callable` types #21551

[ty] Infer typevar specializations for `Callable` types #21551