[ty] Infer slightly more precise types for comprehensions#20111
Merged
AlexWaygood merged 2 commits intomainfrom Aug 27, 2025
Merged
[ty] Infer slightly more precise types for comprehensions#20111AlexWaygood merged 2 commits intomainfrom
AlexWaygood merged 2 commits intomainfrom
Conversation
Contributor
Diagnostic diff on typing conformance testsChanges were detected when running ty on typing conformance tests--- old-output.txt 2025-08-27 10:42:18.799623048 +0000
+++ new-output.txt 2025-08-27 10:42:21.354647826 +0000
@@ -35,9 +35,10 @@
aliases_implicit.py:70:5: error[type-assertion-failure] Argument does not have asserted type `int | str | None | list[list[int]]`
aliases_implicit.py:71:5: error[type-assertion-failure] Argument does not have asserted type `list[bool]`
aliases_implicit.py:72:5: error[type-assertion-failure] Argument does not have asserted type `(...) -> None`
-aliases_implicit.py:107:9: error[invalid-type-form] Variable of type `list[Unknown]` is not allowed in a type expression
+aliases_implicit.py:107:9: error[invalid-type-form] Variable of type `list[@Todo(list literal element type)]` is not allowed in a type expression
aliases_implicit.py:108:9: error[invalid-type-form] Variable of type `tuple[tuple[<class 'int'>, <class 'str'>]]` is not allowed in a type expression
-aliases_implicit.py:110:9: error[invalid-type-form] Variable of type `dict[Unknown, Unknown]` is not allowed in a type expression
+aliases_implicit.py:109:9: error[invalid-type-form] Variable of type `list[@Todo(list comprehension element type)]` is not allowed in a type expression
+aliases_implicit.py:110:9: error[invalid-type-form] Variable of type `dict[@Todo(dict literal key type), @Todo(dict literal value type)]` is not allowed in a type expression
aliases_implicit.py:114:9: error[invalid-type-form] Variable of type `Literal[3]` is not allowed in a type expression
aliases_implicit.py:115:10: error[invalid-type-form] Variable of type `Literal[True]` is not allowed in a type expression
aliases_implicit.py:116:10: error[invalid-type-form] Variable of type `Literal[1]` is not allowed in a type expression
@@ -137,7 +138,7 @@
classes_classvar.py:38:11: error[invalid-type-form] Type qualifier `typing.ClassVar` expected exactly 1 argument, got 2
classes_classvar.py:39:14: error[invalid-type-form] Int literals are not allowed in this context in a type expression
classes_classvar.py:40:14: error[unresolved-reference] Name `var` used when not defined
-classes_classvar.py:52:5: error[invalid-assignment] Object of type `dict[Unknown, Unknown]` is not assignable to `list[str]`
+classes_classvar.py:52:5: error[invalid-assignment] Object of type `dict[@Todo(dict literal key type), @Todo(dict literal value type)]` is not assignable to `list[str]`
classes_classvar.py:55:17: error[invalid-type-form] Type qualifier `typing.ClassVar` is not allowed in type expressions (only in annotation expressions)
classes_classvar.py:69:23: error[invalid-type-form] `ClassVar` is not allowed in function parameter annotations
classes_classvar.py:70:12: error[invalid-type-form] `ClassVar` annotations are only allowed in class-body scopes
@@ -852,11 +853,11 @@
typeddicts_operations.py:62:1: error[unresolved-attribute] Type `MovieOptional` has no attribute `clear`
typeddicts_readonly_inheritance.py:65:1: error[missing-typed-dict-key] Missing required key 'name' in TypedDict `RequiredName` constructor
typeddicts_type_consistency.py:69:21: error[invalid-key] Invalid key access on TypedDict `A3`: Unknown key "y"
-typeddicts_type_consistency.py:101:1: error[invalid-assignment] Object of type `Unknown | None` is not assignable to `str`
+typeddicts_type_consistency.py:101:1: error[invalid-assignment] Object of type `@Todo(dict literal value type) | None` is not assignable to `str`
typeddicts_usage.py:23:7: error[invalid-key] Invalid key access on TypedDict `Movie`: Unknown key "director"
typeddicts_usage.py:24:17: error[invalid-assignment] Invalid assignment to key "year" with declared type `int` on TypedDict `Movie`: value of type `Literal["1982"]`
typeddicts_usage.py:28:1: error[missing-typed-dict-key] Missing required key 'name' in TypedDict `Movie` constructor
typeddicts_usage.py:28:18: error[invalid-key] Invalid key access on TypedDict `Movie`: Unknown key "title"
typeddicts_usage.py:40:24: error[invalid-type-form] The special form `typing.TypedDict` is not allowed in type expressions. Did you mean to use a concrete TypedDict or `collections.abc.Mapping[str, object]` instead?
-Found 859 diagnostics
+Found 860 diagnostics
WARN A fatal error occurred while checking some files. Not all project files were analyzed. See the diagnostics list above for details. |
Contributor
|
Member
Author
|
The conformance test suite diff is good -- we're now correctly emitting an error on this line |
AlexWaygood
commented
Aug 27, 2025
| b: tuple[int] = ("foo",) | ||
|
|
||
| # error: [invalid-assignment] "Object of type `tuple[list[Unknown], Literal["foo"]]` is not assignable to `tuple[str | int, str]`" | ||
| # error: [invalid-assignment] |
Member
Author
There was a problem hiding this comment.
asserting the full error message here is now slightly more complicated because Todo types are rendered differently in release builds, and our mdtest regex that usually deals with that didn't seem able to handle a Todo type nested inside two pairs of square brackets. It didn't seem essential for us to assert the full error message here, so I just simplified the test
Comment on lines
-36
to
-41
| def _(): | ||
| assert_never([]) # error: [type-assertion-failure] | ||
|
|
||
| def _(): | ||
| assert_never({}) # error: [type-assertion-failure] | ||
|
|
Member
Author
There was a problem hiding this comment.
<!-- snapshot-diagnostics --> is enabled in this section, and it gets complicated to have snapshots for diagnostics that talk about Todo types because of the fact that Todo types are rendered differently in release builds. It seemed like we already had good test coverage of assert_never() diagnostics in this section even without these lines, so I just removed them
Contributor
|
| Lint rule | Added | Removed | Changed |
|---|---|---|---|
invalid-argument-type |
11 | 2 | 273 |
unsupported-operator |
5 | 0 | 63 |
invalid-assignment |
10 | 0 | 55 |
possibly-unbound-attribute |
31 | 0 | 33 |
invalid-return-type |
5 | 0 | 54 |
not-iterable |
1 | 0 | 16 |
possibly-unbound-implicit-call |
0 | 0 | 11 |
non-subscriptable |
6 | 0 | 2 |
type-assertion-failure |
0 | 5 | 0 |
unresolved-attribute |
1 | 0 | 1 |
unused-ignore-comment |
0 | 2 | 0 |
call-non-callable |
0 | 0 | 1 |
invalid-exception-caught |
0 | 0 | 1 |
no-matching-overload |
1 | 0 | 0 |
| Total | 71 | 9 | 510 |
Member
Author
|
I haven't done a detailed analysis of the primer report -- I can do if we want, but from a brief skim through it looks like basically what we'd expect. We're getting more diagnostics because we now understand that dict comprehensions create dicts and list comprehensions create lists, and dicts and lists (unlike |
carljm
added a commit
to leandrobbraga/ruff
that referenced
this pull request
Aug 27, 2025
* main: [`ruff`] Preserve relative whitespace in multi-line expressions (`RUF033`) (astral-sh#19647) [ty] Optimize TDD atom ordering (astral-sh#20098) [`airflow`] Extend `AIR311` and `AIR312` rules (astral-sh#20082) [ty] Preserve qualifiers when accessing attributes on unions/intersections (astral-sh#20114) [ty] Fix the inferred interface of specialized generic protocols (astral-sh#19866) [ty] Infer slightly more precise types for comprehensions (astral-sh#20111) [ty] Add more tests for protocols (astral-sh#20095) [ty] don't eagerly unpack aliases in user-authored unions (astral-sh#20055) [`flake8-use-pathlib`] Update links to the table showing the correspondence between `os` and `pathlib` (astral-sh#20103) [`flake8-use-pathlib`] Make `PTH100` fix unsafe because it can change behavior (astral-sh#20100) [`flake8-use-pathlib`] Delete unused `Rule::OsSymlink` enabled check (astral-sh#20099) [ty] Add search paths info to unresolved import diagnostics (astral-sh#20040) [`flake8-logging-format`] Add auto-fix for f-string logging calls (`G004`) (astral-sh#19303) Add a `ScopeKind` for the `__class__` cell (astral-sh#20048) Fix incorrect D413 links in docstrings convention FAQ (astral-sh#20089) [ty] Refactor inlay hints structure to use separate parts (astral-sh#20052)
second-ed
pushed a commit
to second-ed/ruff
that referenced
this pull request
Sep 9, 2025
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
We currently infer the expression
[1, 2]aslist[Unknown], but we infer the expression[x for x in range(42)]as@Todo. This seems inconsistent, and the second one seems needlessly imprecise. This PR changes things so that they're both inferred aslist[@Todo], and makes similar changes to the inferred types for dicts, sets and generator expressions.Obviously we hope to address these TODOs pretty soon, but there's still some discussions to be had internally about the best way to go about that. This PR makes our inference more precise now, and (by using
Todotypes instead ofUnknown) makes it clearer to users when our remaining lack of precision is due to a known limitation in our current model.Test Plan
mdtests