Skip to content

[ty] Improve @override, @final and Liskov checks in cases where there are multiple reachable definitions#21767

Merged
AlexWaygood merged 5 commits intomainfrom
alex/first-declaration
Dec 3, 2025
Merged

[ty] Improve @override, @final and Liskov checks in cases where there are multiple reachable definitions#21767
AlexWaygood merged 5 commits intomainfrom
alex/first-declaration

Conversation

@AlexWaygood
Copy link
Member

@AlexWaygood AlexWaygood commented Dec 2, 2025

Summary

(Stacked on top of #21756; review that first.)

Fixes astral-sh/ty#1677. Currently we only keeping track of a Definition if there was only one single reachable definition for a Place. Now, we always keep track of the first Definition, even if there are multiple reachable definitions for that Place.

Test Plan

mdtests

@AlexWaygood AlexWaygood added the ty Multi-file analysis & type inference label Dec 2, 2025
@AlexWaygood AlexWaygood force-pushed the alex/first-declaration branch from 972c956 to a6650e9 Compare December 2, 2025 23:52
@astral-sh-bot
Copy link

astral-sh-bot bot commented Dec 2, 2025

Diagnostic diff on typing conformance tests

No changes detected when running ty on typing conformance tests ✅

@astral-sh-bot
Copy link

astral-sh-bot bot commented Dec 2, 2025

mypy_primer results

Changes were detected when running on open source projects
beartype (https://github.com/beartype/beartype)
- beartype/claw/_package/clawpkgtrie.py:66:29: warning[unsupported-base] Unsupported class base with type `<class 'dict[str, PackagesTrieBlacklist]'> | <class 'dict[str, Divergent]'>`
- beartype/claw/_package/clawpkgtrie.py:247:29: warning[unsupported-base] Unsupported class base with type `<class 'dict[str, PackagesTrieWhitelist]'> | <class 'dict[str, Divergent]'>`
- Found 494 diagnostics
+ Found 492 diagnostics

dulwich (https://github.com/dulwich/dulwich)
- dulwich/porcelain.py:481:71: warning[unused-ignore-comment] Unused blanket `type: ignore` directive
- dulwich/porcelain.py:485:76: warning[unused-ignore-comment] Unused blanket `type: ignore` directive
- Found 228 diagnostics
+ Found 226 diagnostics

mypy (https://github.com/python/mypy)
+ mypy/typeshed/stdlib/_frozen_importlib.pyi:79:13: error[invalid-method-override] Invalid override of method `create_module`: Definition is incompatible with `Loader.create_module`
+ mypy/typeshed/stdlib/_frozen_importlib.pyi:118:13: error[invalid-method-override] Invalid override of method `create_module`: Definition is incompatible with `Loader.create_module`
+ mypy/typeshed/stdlib/array.pyi:65:13: error[invalid-method-override] Invalid override of method `index`: Definition is incompatible with `Sequence.index`
+ mypy/typeshed/stdlib/asyncio/base_events.pyi:325:19: error[invalid-method-override] Invalid override of method `create_server`: Definition is incompatible with `AbstractEventLoop.create_server`
+ mypy/typeshed/stdlib/asyncio/base_events.pyi:325:19: error[invalid-method-override] Invalid override of method `create_server`: Definition is incompatible with `AbstractEventLoop.create_server`
+ mypy/typeshed/stdlib/fractions.pyi:125:13: error[invalid-method-override] Invalid override of method `__pow__`: Definition is incompatible with `Complex.__pow__`
+ mypy/typeshed/stdlib/fractions.pyi:125:13: error[invalid-method-override] Invalid override of method `__pow__`: Definition is incompatible with `Complex.__pow__`
+ mypy/typeshed/stdlib/fractions.pyi:135:13: error[invalid-method-override] Invalid override of method `__rpow__`: Definition is incompatible with `Complex.__rpow__`
+ mypy/typeshed/stdlib/fractions.pyi:135:13: error[invalid-method-override] Invalid override of method `__rpow__`: Definition is incompatible with `Complex.__rpow__`
- mypy/typeshed/stdlib/pydoc.pyi:166:26: warning[unused-ignore-comment] Unused blanket `type: ignore` directive
- mypy/typeshed/stdlib/pydoc.pyi:176:132: warning[unused-ignore-comment] Unused blanket `type: ignore` directive
- mypy/typeshed/stdlib/pydoc.pyi:177:128: warning[unused-ignore-comment] Unused blanket `type: ignore` directive
- mypy/typeshed/stdlib/pydoc.pyi:229:131: warning[unused-ignore-comment] Unused blanket `type: ignore` directive
- mypy/typeshed/stdlib/pydoc.pyi:230:107: warning[unused-ignore-comment] Unused blanket `type: ignore` directive
- mypy/typeshed/stdlib/pydoc.pyi:231:132: warning[unused-ignore-comment] Unused blanket `type: ignore` directive
- mypy/typeshed/stdlib/pydoc.pyi:232:128: warning[unused-ignore-comment] Unused blanket `type: ignore` directive
- mypy/typeshed/stdlib/pydoc.pyi:233:24: warning[unused-ignore-comment] Unused blanket `type: ignore` directive
- mypy/typeshed/stdlib/tempfile.pyi:379:67: warning[unused-ignore-comment] Unused blanket `type: ignore` directive

scikit-build-core (https://github.com/scikit-build/scikit-build-core)
+ src/scikit_build_core/_logging.py:153:13: warning[unsupported-base] Unsupported class base with type `<class 'Mapping[str, Style]'> | <class 'Mapping[str, Divergent]'>`
- Found 41 diagnostics
+ Found 42 diagnostics

pandas-stubs (https://github.com/pandas-dev/pandas-stubs)
- pandas-stubs/_typing.pyi:1209:16: warning[unused-ignore-comment] Unused blanket `type: ignore` directive
- Found 5512 diagnostics
+ Found 5511 diagnostics

No memory usage changes detected ✅

@AlexWaygood AlexWaygood changed the title [ty] Improve @override and @final checks in cases where there are multiple reachable definitions [ty] Improve @override, @final and Liskov checks in cases where there are multiple reachable definitions Dec 2, 2025
@astral-sh-bot
Copy link

astral-sh-bot bot commented Dec 3, 2025

ecosystem-analyzer results

Lint rule Added Removed Changed
unused-ignore-comment 1 11 0
invalid-method-override 9 0 0
invalid-argument-type 2 0 0
Total 12 11 0

Full report with detailed diff (timing results)

@AlexWaygood
Copy link
Member Author

Okay, the weird new Liskov violations on mypy's vendored version of typeshed are because e.g. ty thinks that the ModuleSpec definition in mypy's vendored version of stdlib/importlib/_frozen_importlib.pyi is a different class to the ModuleSpec definition in ty's vendored version of stdlib/importlib/_frozen_importlib.pyi. So "obviously" a class that overrides the a method that takes one ModuleSpec with a method that takes another ModuleSpec must violate the Liskov Subsitution Principle.

@AlexWaygood AlexWaygood marked this pull request as ready for review December 3, 2025 00:44
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@BurntSushi -- both of the tests being changed in this file have FIXME comments above them. But I'm not sure if the changes this PR makes to the tests fixes the problems, or makes the problems worse 😆 I'm not totally sure I understand what these tests are meant to be asserting

Copy link
Member Author

@AlexWaygood AlexWaygood Dec 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LMK if you consider this a regression and I can fix it in a followup. It shouldn't be too hard to track whether a member has multiple definitions and propagate that information upwards, if that's what the autocomplete machinery wants to know

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the idea here was just trying to test what happens when we try to insert an import for a symbol that is already imported, but whose imports are conditional. I think in this case we don't want to add any new imports. But we weren't doing that before either. So I think this is fine for now.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cool, thank you!

Copy link
Contributor

@carljm carljm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

def method3(self) -> None: ... # error: [override-of-final-method]
def method4(self) -> None: ... # error: [override-of-final-method]

# TODO: we should emit Liskov violations here too:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems like we do? At least at the first definition...

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no, we emit a diagnostic stating that an @final method has been overridden (which is not to my mind a Liskov violation), but we are silent about the fact that the type has also been incompatibly overridden (which is, to my mind, a Liskov violation)

@AlexWaygood AlexWaygood force-pushed the alex/first-declaration branch from 87d8593 to d2e8da5 Compare December 3, 2025 11:28
@AlexWaygood AlexWaygood enabled auto-merge (squash) December 3, 2025 12:16
@AlexWaygood AlexWaygood disabled auto-merge December 3, 2025 12:20
@AlexWaygood AlexWaygood enabled auto-merge (squash) December 3, 2025 12:48
@AlexWaygood AlexWaygood merged commit cd079bd into main Dec 3, 2025
40 checks passed
@AlexWaygood AlexWaygood deleted the alex/first-declaration branch December 3, 2025 12:51
dcreager added a commit that referenced this pull request Dec 3, 2025
* origin/main:
  [ty] Improve `@override`, `@final` and Liskov checks in cases where there are multiple reachable definitions (#21767)
  [ty] Extend `invalid-explicit-override` to also cover properties decorated with `@override` that do not override anything (#21756)
  [ty] Enable LRU collection for parsed module (#21749)
  [ty] Support typevar-specialized dynamic types in generic type aliases (#21730)
  Add token based `parenthesized_ranges` implementation (#21738)
  [ty] Default-specialization of generic type aliases (#21765)
  [ty] Suppress false positives when `dataclasses.dataclass(...)(cls)` is called imperatively (#21729)
  [syntax-error] Default type parameter followed by non-default type parameter (#21657)
  new module for parsing ranged suppressions (#21441)
  [ty] `type[T]` is assignable to an inferable typevar (#21766)
  Fix syntax error false positives for `await` outside functions (#21763)
  [ty] Improve diagnostics for unsupported comparison operations (#21737)
  Move `Token`, `TokenKind` and `Tokens` to `ruff-python-ast` (#21760)
  [ty] Don't confuse multiple occurrences of `typing.Self` when binding bound methods (#21754)
  Use our org-wide Renovate preset (#21759)
  Delete `my-script.py` (#21751)
  [ty] Move `all_members`, and related types/routines, out of `ide_support.rs` (#21695)
dcreager added a commit that referenced this pull request Dec 3, 2025
* origin/main:
  [ty] Reachability constraints: minor documentation fixes (#21774)
  [ty] Fix non-determinism in `ConstraintSet.specialize_constrained` (#21744)
  [ty] Improve `@override`, `@final` and Liskov checks in cases where there are multiple reachable definitions (#21767)
  [ty] Extend `invalid-explicit-override` to also cover properties decorated with `@override` that do not override anything (#21756)
  [ty] Enable LRU collection for parsed module (#21749)
  [ty] Support typevar-specialized dynamic types in generic type aliases (#21730)
  Add token based `parenthesized_ranges` implementation (#21738)
  [ty] Default-specialization of generic type aliases (#21765)
  [ty] Suppress false positives when `dataclasses.dataclass(...)(cls)` is called imperatively (#21729)
  [syntax-error] Default type parameter followed by non-default type parameter (#21657)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ecosystem-analyzer ty Multi-file analysis & type inference

Projects

None yet

Development

Successfully merging this pull request may close these issues.

@final and @override diagnostics are not emitted if there are multiple reachable definitions

3 participants