[ty] Fix bug where ty would think all types had an __mro__ attribute#20995
[ty] Fix bug where ty would think all types had an __mro__ attribute#20995AlexWaygood merged 3 commits intomainfrom
__mro__ attribute#20995Conversation
CodSpeed Performance ReportMerging #20995 will improve performances by 6.23%Comparing Summary
Benchmarks breakdown
Footnotes
|
|
It seems like this could have been fixed by moving that special handling to Special-casing things in type inference (instead of core
They probably don't need it, but I found it kind of nice? |
Diagnostic diff on typing conformance testsChanges were detected when running ty on typing conformance tests--- old-output.txt 2025-10-27 11:15:58.398773069 +0000
+++ new-output.txt 2025-10-27 11:16:01.520786835 +0000
@@ -867,11 +867,9 @@
specialtypes_type.py:56:7: error[invalid-argument-type] Argument to function `func4` is incorrect: Expected `type[BasicUser] | type[ProUser]`, found `<class 'TeamUser'>`
specialtypes_type.py:76:17: error[invalid-type-form] type[...] must have exactly one type argument
specialtypes_type.py:84:5: error[type-assertion-failure] Argument does not have asserted type `type[Any]`
-specialtypes_type.py:98:5: error[type-assertion-failure] Argument does not have asserted type `tuple[type, ...]`
specialtypes_type.py:99:17: error[unresolved-attribute] Object of type `type` has no attribute `unknown`
specialtypes_type.py:100:17: error[unresolved-attribute] Object of type `type` has no attribute `unknown`
specialtypes_type.py:102:5: error[type-assertion-failure] Argument does not have asserted type `tuple[type, ...]`
-specialtypes_type.py:106:5: error[type-assertion-failure] Argument does not have asserted type `tuple[type, ...]`
specialtypes_type.py:107:17: error[unresolved-attribute] Object of type `type` has no attribute `unknown`
specialtypes_type.py:108:17: error[unresolved-attribute] Object of type `type` has no attribute `unknown`
specialtypes_type.py:110:5: error[type-assertion-failure] Argument does not have asserted type `tuple[type, ...]`
@@ -948,5 +946,5 @@
typeddicts_usage.py:28:17: error[missing-typed-dict-key] Missing required key 'name' in TypedDict `Movie` constructor
typeddicts_usage.py:28:18: error[invalid-key] Invalid key access on TypedDict `Movie`: Unknown key "title"
typeddicts_usage.py:40:24: error[invalid-type-form] The special form `typing.TypedDict` is not allowed in type expressions. Did you mean to use a concrete TypedDict or `collections.abc.Mapping[str, object]` instead?
-Found 950 diagnostics
+Found 948 diagnostics
WARN A fatal error occurred while checking some files. Not all project files were analyzed. See the diagnostics list above for details. |
+1 |
In general, I absolutely agree. In this specific case, though, I worry that ty's inferred MRO for a class is often pretty different to a class's actual MRO at runtime, because of various fictions typeshed would (for very good reasons) have us believe. For example, we infer the MRO of >>> tuple.__mro__
(<class 'tuple'>, <class 'object'>)Not only is typeshed's MRO for Honestly, after writing this all out, I'm not even sure whether we should special-case inference of the |
I got the same feeling when reading your comment and was glad to find that paragraph at the end 👍 👍 |
8774382 to
0c1b397
Compare
AlexWaygood
left a comment
There was a problem hiding this comment.
Most of the changes to the tests are now a result of switching over to the custom reveal_mro assertions, so I'm annotating the changes that show how the semantics are improved as a result of this PR:
| if hasattr(DoesNotExist, "__mro__"): | ||
| # TODO: this should be `Unknown & <Protocol with members '__mro__'>` or similar | ||
| # (The second part of the intersection is incorrectly simplified to `object` due to https://github.com/astral-sh/ty/issues/986) | ||
| reveal_type(DoesNotExist) # revealed: Unknown | ||
| reveal_type(DoesNotExist) # revealed: Unknown & <Protocol with members '__mro__'> |
There was a problem hiding this comment.
this demonstrates the improved semantics from this PR
| # TODO: this should cause us to emit a diagnostic and reveal `Unknown` (function objects don't have an `__mro__` attribute), | ||
| # but the fact that we don't isn't actually a `NamedTuple` bug (https://github.com/astral-sh/ty/issues/986) | ||
| reveal_type(typing.NamedTuple.__mro__) # revealed: tuple[<class 'FunctionType'>, <class 'object'>] | ||
| # error: [unresolved-attribute] | ||
| reveal_type(typing.NamedTuple.__mro__) # revealed: Unknown |
There was a problem hiding this comment.
this demonstrates the improved semantics from this PR
| scope-simple-long-identifier,main.py,0,1 | ||
| tstring-completions,main.py,0,1 | ||
| ty-extensions-lower-stdlib,main.py,0,7 | ||
| ty-extensions-lower-stdlib,main.py,0,8 |
There was a problem hiding this comment.
this change is because there is now another ty_extensions function (reveal_mro) that currently has a higher rank than typing.reveal_type in autocomplete suggestions, but would have a lower rank in an ideal world:
|
The beartype primer hit is clearly a false negative going away. The sympy primer hits are a bit more complicated, but also look like true positives to me. The issue is that at line 177, we infer the type of The conformance suite diff is also good: two false positives going away. This PR should be ready for another round of review now. |
sharkdp
left a comment
There was a problem hiding this comment.
Thank you! This looks great now.
0c1b397 to
7c5ad22
Compare
* origin/main: Respect `--output-format` with `--watch` (#21097) [`pyflakes`] Revert to stable behavior if imports for module lie in alternate branches for `F401` (#20878) Fix finding keyword range for clause header after statement ending with semicolon (#21067) [ty] Fix bug where ty would think all types had an `__mro__` attribute (#20995) Restore `indent.py` (#21094) [`flake8-django`] Apply `DJ001` to annotated fields (#20907) Clearer error message when `line-length` goes beyond threshold (#21072) Update upload and download artifacts github actions (#21083) Update dependency mdformat-mkdocs to v4.4.2 (#21088) Update cargo-bins/cargo-binstall action to v1.15.9 (#21086) Update Rust crate clap to v4.5.50 (#21090) Update Rust crate get-size2 to v0.7.1 (#21091) Update Rust crate bstr to v1.12.1 (#21089) Add missing docstring sections to the numpy list (#20931) [`pydoclint`] Fix false positive on explicit exception re-raising (`DOC501`, `DOC502`) (#21011) [ty] Use constructor parameter types as type context (#21054)
Summary
This PR is best reviewed with the "hide whitespace changes" option enabled on GitHub.
This PR fixes a bug where ty would consider all types as having an
__mro__attribute, when in reality this is only true for instances oftype. The bug is fixed by lifting our special-casing for the__mro__attribute out ofType::member()and intoTypeInferenceBuilder::infer_attribute_load(). This means that we now only infer precise types for the__mro__attribute for literal attribute accesses, whereas we previously also inferred this precise type for implicit attribute accesses. I think that's okay, however: we only really need the precise inference of__mro__for our internal tests. Callingiter_mro()at a higher level (rather than in the guts ofType::member()) also appears to have the effect that we properly propagate the specialisation of a generic class down through the entire MRO when inferring the type ofsomeclass.__mro__, resulting in more intuitive types being revealed in ourmro.mdtest.The type displayed in the tooltip for some autocomplete suggestions also becomes less precise, but that also seems fine. I don't think users of autocompletion need a type that's so precise there; the less precise type seems like it's probably less noisy.
Fixes astral-sh/ty#986
Test Plan
Existing mdtests updated and TODOs removed