Skip to content

[ty] Retain the function-like-ness of Callable types when binding self#21614

Merged
AlexWaygood merged 2 commits intomainfrom
alex/callable-bind-self
Nov 24, 2025
Merged

[ty] Retain the function-like-ness of Callable types when binding self#21614
AlexWaygood merged 2 commits intomainfrom
alex/callable-bind-self

Conversation

@AlexWaygood
Copy link
Member

@AlexWaygood AlexWaygood commented Nov 24, 2025

Summary

For something like this:

from typing import Callable

def my_lossy_decorator(fn: Callable[..., int]) -> Callable[..., int]:
    return fn

class MyClass:
    @my_lossy_decorator
    def method(self) -> int:
        return 42

we will currently infer the type of MyClass.method as a function-like Callable, but we will infer the type of MyClass().method as a Callable that is not function-like. That's because a CallableType currently "forgets" whether it was function-like or not during the bound_self transformation:

pub(crate) fn bind_self(self, db: &'db dyn Db) -> CallableType<'db> {
CallableType::new(db, self.signatures(db).bind_self(db, None), false)
}

This seems incorrect, and it's quite different to what we do when binding the self parameter of FunctionLiteral types: BoundMethod types are all seen as subtypes of function-like Callable supertypes -- here's BoundMethodType::into_callable_type:

#[salsa::tracked(cycle_initial=into_callable_type_cycle_initial, heap_size=ruff_memory_usage::heap_size)]
pub(crate) fn into_callable_type(self, db: &'db dyn Db) -> CallableType<'db> {
let function = self.function(db);
let self_instance = self.typing_self_type(db);
CallableType::new(
db,
CallableSignature::from_overloads(
function
.signature(db)
.overloads
.iter()
.map(|signature| signature.bind_self(db, Some(self_instance))),
),
true,
)
}

The bug here is also causing lots of false positives in the ecosystem report on #21611: a decorated method on a subclass is currently not seen as validly overriding an undecorated method with the same signature on a superclass, because the undecorated superclass method is seen as function-like after binding self whereas the decorated subclass method is not.

Fixing the bug required adding a new API in protocol_class.rs, because it turns out that for our purposes in protocol subtyping/assignability, we really do want a callable type to forget its function-like-ness when binding self.

I initially tried out this change without changing anything in protocol_class.rs. However, it resulted in many ecosystem false positives and new false positives on the typing conformance test suite. This is because it would mean that no protocol with a __call__ method would ever be seen as a subtype of a Callable type, since the __call__ method on the protocol would be seen as being function-like whereas the Callable type would not be seen as function-like.

Test Plan

Added an mdtest that fails on main

@AlexWaygood AlexWaygood added ty Multi-file analysis & type inference ecosystem-analyzer labels Nov 24, 2025
@astral-sh-bot
Copy link

astral-sh-bot bot commented Nov 24, 2025

Diagnostic diff on typing conformance tests

No changes detected when running ty on typing conformance tests ✅

@astral-sh-bot
Copy link

astral-sh-bot bot commented Nov 24, 2025

mypy_primer results

Changes were detected when running on open source projects
antidote (https://github.com/Finistere/antidote)
- src/antidote/core/_raw/wrapper.py:102:13: error[unresolved-attribute] Object of type `(...) -> object` has no attribute `__get__`
- src/antidote/core/_raw/wrapper.py:136:13: error[unresolved-attribute] Object of type `(...) -> Awaitable[object]` has no attribute `__get__`
+ src/antidote/core/_raw/wrapper.py:102:13: warning[possibly-missing-attribute] Attribute `__get__` may be missing on object of type `((...) -> object) | ((...) -> object)`
+ src/antidote/core/_raw/wrapper.py:136:13: warning[possibly-missing-attribute] Attribute `__get__` may be missing on object of type `((...) -> Awaitable[object]) | ((...) -> Awaitable[object])`

dd-trace-py (https://github.com/DataDog/dd-trace-py)
- tests/debugging/probe/test_remoteconfig.py:69:9: error[invalid-assignment] Object of type `(...) -> Iterable[Probe]` is not assignable to attribute `get_probes` of type `(...) -> Iterable[Probe]`
- Found 8243 diagnostics
+ Found 8242 diagnostics

egglog-python (https://github.com/egraphs-good/egglog-python)
- python/egglog/runtime.py:745:9: error[invalid-assignment] Object of type `(() -> Declarations) | Unknown` is not assignable to attribute `__egg_decls_thunk__` of type `() -> Declarations`
- Found 1493 diagnostics
+ Found 1492 diagnostics

No memory usage changes detected ✅

@astral-sh-bot
Copy link

astral-sh-bot bot commented Nov 24, 2025

ecosystem-analyzer results

Lint rule Added Removed Changed
invalid-assignment 0 2 0
possibly-missing-attribute 2 0 0
unresolved-attribute 0 2 0
Total 2 4 0

Full report with detailed diff (timing results)

@AlexWaygood
Copy link
Member Author

The ecosystem diff shows two false-positive diagnostics going away. It also shows two pre-existing diagnostics becoming slightly more confusing due to the fact that we don't have a great way of distinguishing between function-like callables and non-function-like callables in our Type display yet.

@AlexWaygood AlexWaygood marked this pull request as ready for review November 24, 2025 15:56
@AlexWaygood AlexWaygood force-pushed the alex/callable-bind-self branch from 8023aaa to 70c526e Compare November 24, 2025 20:04
@AlexWaygood AlexWaygood marked this pull request as draft November 24, 2025 20:10
@AlexWaygood AlexWaygood force-pushed the alex/callable-bind-self branch from 70c526e to e204348 Compare November 24, 2025 20:13
@AlexWaygood AlexWaygood marked this pull request as ready for review November 24, 2025 20:13
Copy link
Contributor

@sharkdp sharkdp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

@AlexWaygood AlexWaygood merged commit bab688b into main Nov 24, 2025
41 checks passed
@AlexWaygood AlexWaygood deleted the alex/callable-bind-self branch November 24, 2025 21:14
carljm added a commit to mtshiba/ruff that referenced this pull request Nov 25, 2025
* main:
  [ty] Extend Liskov checks to also cover classmethods and staticmethods (astral-sh#21598)
  Dogfood ty on the `scripts` directory (astral-sh#21617)
  [ty] support generic aliases in `type[...]`, like `type[C[int]]` (astral-sh#21552)
  [ty] Retain the function-like-ness of `Callable` types when binding `self` (astral-sh#21614)
  [ty] Distinguish "unconstrained" from "constrained to any type" (astral-sh#21539)
  Disable ty workspace diagnostics for VSCode users (astral-sh#21620)
  [ty] Double click to insert inlay hint (astral-sh#21600)
  [ty] Switch the error code from `unresolved-attribute` to `possibly-missing-attribute` for submodules that may not be available (astral-sh#21618)
  [ty] Substitute for `typing.Self` when checking protocol members (astral-sh#21569)
  [ty] Don't suggest things that aren't subclasses of `BaseException` after `raise`
  [ty] Add hint about resolved Python version when a user attempts to import a member added on a newer version (astral-sh#21615)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ecosystem-analyzer ty Multi-file analysis & type inference

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants