Adding a limited form of parametrized Self

[dvarrazzo]

This question comes from psycopg/psycopg#308 and from the outdated #1197, and is more focused on only one of the aspects previously asked about. (The other is asked about in #1556).

I understand from PEP 673 that Self is doesn't accept a parameter:

Note that we reject using Self with type arguments, such as Self[int]. This is because it creates ambiguity about the type of the self parameter and introduces unnecessary complexity: [...]

Follows a pretty contrived example of an ambiguous situation. However, if the context is not ambiguous, I think that there are valid use cases of a parametric self. One example is with a generic class, where there is a meaningful default for the type. For instance, in psycopg, the connect() factory function creates an object that return tuples, as per DBAPI spec, but it can take an optional row_factory parameter to specify which Python objects to return the data as. A simplified version is:

from typing import Any, Callable, Generic, Sequence, TypeAlias, TypeVar, overload
from typing_extensions import Self


TupleRow: TypeAlias = tuple[Any, ...]

def tuple_row(values: Sequence[Any]) -> TupleRow:
    raise NotImplementedError


DictRow: TypeAlias = dict[str, Any]

def dict_row(values: Sequence[Any]) -> DictRow:
    raise NotImplementedError


Row = TypeVar("Row", covariant=True)
RowFactory: TypeAlias = Callable[[Sequence[Any]], Row]

class Connection(Generic[Row]):
    @overload
    @classmethod
    def connect(cls, *, row_factory: RowFactory[Row]) -> Self:
        ...

    @overload
    @classmethod
    def connect(cls) -> Self:
        ...

    @classmethod
    def connect(cls, *, row_factory: RowFactory[Row] | None = None) -> Self:
        raise NotImplementedError


c = Connection.connect()
reveal_type(c)
reveal_type(Connection.connect())
reveal_type(Connection.connect(row_factory=tuple_row))
reveal_type(Connection.connect(row_factory=dict_row))

Running mypy 1.8:

testtype-overload.py:36: error: Need type annotation for "c"  [var-annotated]
testtype-overload.py:37: note: Revealed type is "testtype-overload.Connection[Any]"
testtype-overload.py:38: note: Revealed type is "testtype-overload.Connection[Never]"
testtype-overload.py:39: note: Revealed type is "testtype-overload.Connection[builtins.tuple[Any, ...]]"
testtype-overload.py:40: note: Revealed type is "testtype-overload.Connection[builtins.dict[builtins.str, Any]]"

The desired outcome is that connect() would instead return a Connection[TupleRow], but the info is clearly not there. What I would change here is that I should be able to define:

    @overload
    @classmethod
    def connect(cls) -> Self[TupleRow]:
        ...

for the overload variant without a default, and the parameters passed to Self would indicate the parameters for the generic types.

Wouldn't this be a reasonable use for Self parametrization?

Note:

• I know that we can tell people to use Connection[TupleRow].connect(), but this is non backward-compatible and I may as well force people to use Connection.connect(row_factory=tuple_row).
• I know that we can return Connection[TupleRow], this is what we currently do, but this is the defeat of Self, and the main shortcoming is that it becomes horrible when the object must be subclassed, because complex signatures must be repeated just to change the return type annotations. Example: none of this should be necessary.

[erictraut]

I think I understand your use case, but a parameterized Self is probably not the right answer. The Self type literally means "the current type of self". You can think of it as a type variable that has an upper bound of the current (specialized) class. Subclasses don't necessarily have the same type parameters as their parent class. Consider the following:

class Parent[T]:
    def method(self) -> Self:
        return self

class Child(Parent[int]):
    pass

Child().method()

In this example, Child is not generic and has no type parameter. What would Self[<SomeType>] mean in this context? If Self happens to refer to Child, what would Self[<SomeType>] mean? I suppose it might make sense if you could somehow constrain all child classes to use the same type parameterization as their parents, but no such mechanism currently exists.

It would also require that there's a way to construct a new Child with new type parameters, which implies that the parent would need to know the signature of the child's constructor. Currently, the type system exempts constructors from being consistent across a type hierarchy, so there's no way for a type checker to enforce that the constructor in a child class is the same as the constructor of its parent.

I think the use case you're describing would require a generalized higher-kinded type (HKT) mechanism. This has been discussed previously, but discussions haven't progressed much. HKT's would add quite a bit of complexity. They're not supported in most languages that support generics.

[Viicos]

This would also resolve my use case described here: https://discuss.python.org/t/41483, and similar to @dvarrazzo, my only alternative is to redefine the overloads in each subclass.

In my opinion @erictraut raised valid concerns; I think this feature would only make sense in a very specific case:

• parametrized Self is used in a class method, such as Connection.connect or __new__.
• It should be parametrized with a type variable (and should probably be the same type var used by the class definition), not a specific type (as it would conflict with parametrized subclasses, such as Child)

Code sample in pyright playground

from typing import Self, reveal_type


class Parent[T]:

    # Valid
    @classmethod
    def method(cls, a: type[T]) -> Self[T]:
        ...

    # Invalid
    @classmethod
    def method2(cls) -> Self[str]:
        ...

class Child(Parent[int]):
    pass

reveal_type(Parent.method(int))  # pyright seems to handle it already, although probably not on purpose

reveal_type(Child.method(int))  # If another type is being used (e.g. `str`), the type checker will already complain as the inferred signature of `method` is method(a: type[int]) -> Child

But even then, I'm pretty sure I'm missing a lot of cases that could lead to undefined behavior (e.g. as said above when a subclass has an extra type var).

[dvarrazzo]

It seems that the right solution is to use the generic default as per PEP 696. It seems to work, both in the synthetic example provided in #1556 and in psycopg.