[ty] Infer yield expression#23796
Conversation
Typing conformance results improved 🎉The percentage of diagnostics emitted that were expected errors increased from 85.29% to 85.38%. The percentage of expected errors that received a diagnostic increased from 78.13% to 78.70%. The number of fully passing files held steady at 64/132. SummaryHow are test cases classified?Each test case represents one expected error annotation or a group of annotations sharing a tag. Counts are per test case, not per diagnostic — multiple diagnostics on the same line count as one. Required annotations (
Test file breakdown1 file altered
True positives added (6)6 diagnostics
|
|
Memory usage reportSummary
Significant changesClick to expand detailed breakdownprefect
sphinx
flake8
trio
|
7130978 to
893d465
Compare
ef14f4b to
1debc29
Compare
|
| Lint rule | Added | Removed | Changed |
|---|---|---|---|
invalid-yield |
79 | 0 | 0 |
invalid-await |
0 | 40 | 0 |
unused-type-ignore-comment |
0 | 10 | 0 |
invalid-assignment |
0 | 0 | 2 |
invalid-argument-type |
1 | 0 | 0 |
invalid-return-type |
0 | 1 | 0 |
| Total | 80 | 51 | 2 |
Changes in flaky projects detected. Raw diff output excludes flaky projects; see the HTML report for details.
Raw diff (91 changes)
Expression (https://github.com/cognitedata/Expression)
+ README.py:358:20 [error] [invalid-yield] Send type `None` does not match annotated send type `int`
+ README.py:382:20 [error] [invalid-yield] Send type `None` does not match annotated send type `int`
+ README.py:385:20 [error] [invalid-yield] Send type `None` does not match annotated send type `int`
+ README.py:455:20 [error] [invalid-yield] Send type `None` does not match annotated send type `int`
+ README.py:456:20 [error] [invalid-yield] Send type `None` does not match annotated send type `int`
+ tests/test_option_builder.py:43:29 [error] [invalid-yield] Send type `None` does not match annotated send type `int`
+ tests/test_option_builder.py:107:29 [error] [invalid-yield] Send type `None` does not match annotated send type `int`
+ tests/test_option_builder.py:123:29 [error] [invalid-yield] Send type `None` does not match annotated send type `int`
+ tests/test_option_builder.py:136:29 [error] [invalid-yield] Send type `None` does not match annotated send type `int`
+ tests/test_option_builder.py:159:29 [error] [invalid-yield] Send type `None` does not match annotated send type `int`
+ tests/test_option_builder.py:218:9 [error] [invalid-yield] Yield type `None` does not match annotated yield type `Option[int]`
+ tests/test_option_builder.py:269:32 [error] [invalid-yield] Send type `None` does not match annotated send type `str`
+ tests/test_option_builder.py:285:24 [error] [invalid-yield] Send type `None` does not match annotated send type `int`
+ tests/test_option_builder.py:288:24 [error] [invalid-yield] Send type `None` does not match annotated send type `int`
+ tests/test_result_builder.py:42:29 [error] [invalid-yield] Send type `None` does not match annotated send type `int`
+ tests/test_result_builder.py:106:29 [error] [invalid-yield] Send type `None` does not match annotated send type `int`
+ tests/test_result_builder.py:122:29 [error] [invalid-yield] Send type `None` does not match annotated send type `int`
+ tests/test_result_builder.py:139:29 [error] [invalid-yield] Send type `None` does not match annotated send type `int`
+ tests/test_result_builder.py:162:29 [error] [invalid-yield] Send type `None` does not match annotated send type `int`
+ tests/test_result_builder.py:221:9 [error] [invalid-yield] Yield type `None` does not match annotated yield type `Result[int, str]`
+ tests/test_result_builder.py:277:32 [error] [invalid-yield] Send type `None` does not match annotated send type `str`
+ tests/test_result_builder.py:293:24 [error] [invalid-yield] Send type `None` does not match annotated send type `int`
+ tests/test_result_builder.py:296:24 [error] [invalid-yield] Send type `None` does not match annotated send type `int`
+ tests/test_seq_builder.py:186:9 [error] [invalid-yield] Yield type `None` does not match annotated yield type `Iterable[int]`
+ expression/core/option.py:308:15 [error] [invalid-yield] Yield type `None` does not match annotated yield type `_TSourceOut@Option`
+ expression/core/result.py:267:15 [error] [invalid-yield] Yield type `None` does not match annotated yield type `_TSourceOut@Result`
+ tests/test_catch.py:72:24 [error] [invalid-yield] Send type `None` does not match annotated send type `int`
+ tests/test_catch.py:83:24 [error] [invalid-yield] Send type `None` does not match annotated send type `int`
+ tests/test_catch.py:99:24 [error] [invalid-yield] Send type `None` does not match annotated send type `str`
PyGithub (https://github.com/PyGithub/PyGithub)
- github/PaginatedList.py:125:47 [warning] [unused-type-ignore-comment] Unused blanket `type: ignore` directive
Tanjun (https://github.com/FasterSpeeding/Tanjun)
- tanjun/clients.py:2720:13 [error] [invalid-assignment] Invalid subscript assignment with key of type `str | Path` and value of type `@Todo` on object of type `dict[Path, ModuleType]`
+ tanjun/clients.py:2720:13 [error] [invalid-assignment] Invalid subscript assignment with key of type `str | Path` and value of type `ModuleType` on object of type `dict[Path, ModuleType]`
- tanjun/clients.py:2720:13 [error] [invalid-assignment] Invalid subscript assignment with key of type `str | Path` and value of type `@Todo` on object of type `dict[str, ModuleType]`
+ tanjun/clients.py:2720:13 [error] [invalid-assignment] Invalid subscript assignment with key of type `str | Path` and value of type `ModuleType` on object of type `dict[str, ModuleType]`
aiohttp (https://github.com/aio-libs/aiohttp)
- aiohttp/cookiejar.py:592:25 [warning] [unused-type-ignore-comment] Unused blanket `type: ignore` directive
artigraph (https://github.com/artigraph/artigraph)
+ src/arti/internal/mappings.py:240:28 [error] [invalid-yield] Yield type `tuple[str, object]` does not match annotated yield type `tuple[str, V@TypedBox]`
bidict (https://github.com/jab/bidict)
+ bidict/_iter.py:26:20 [error] [invalid-yield] Yield type `tuple[object, object]` does not match annotated yield type `tuple[KT@iteritems, VT@iteritems]`
+ bidict/_iter.py:28:20 [error] [invalid-yield] Yield type `tuple[object, object]` does not match annotated yield type `tuple[KT@iteritems, VT@iteritems]`
black (https://github.com/psf/black)
+ src/black/trans.py:502:19 [error] [invalid-yield] Yield type `Ok[object]` does not match annotated yield type `Ok[Line] | Err[CannotTransform]`
boostedblob (https://github.com/hauntsaninja/boostedblob)
+ boostedblob/boost.py:602:19 [error] [invalid-yield] Yield type `object` does not match annotated yield type `T@iter_underlying`
core (https://github.com/home-assistant/core)
+ homeassistant/components/backup/manager.py:1840:35 [error] [invalid-yield] Yield type `(Unknown & ~AlwaysFalsy) | (str & ~AlwaysFalsy) | (bytes & ~AlwaysFalsy)` does not match annotated yield type `bytes`
+ homeassistant/components/backup/manager.py:1990:27 [error] [invalid-yield] Yield type `(Unknown & ~AlwaysFalsy) | (str & ~AlwaysFalsy) | (bytes & ~AlwaysFalsy)` does not match annotated yield type `bytes`
cwltool (https://github.com/common-workflow-language/cwltool)
+ cwltool/process.py:315:23 [error] [invalid-yield] Yield type `(MutableMapping[str, None | int | str | ... omitted 3 union elements] & Top[dict[Unknown, Unknown]]) | (MutableSequence[MutableMapping[str, None | int | str | ... omitted 3 union elements]] & Top[dict[Unknown, Unknown]])` does not match annotated yield type `MutableMapping[str, None | int | str | ... omitted 3 union elements]`
dd-trace-py (https://github.com/DataDog/dd-trace-py)
+ tests/debugging/mocking.py:199:15 [error] [invalid-yield] Yield type `Debugger` does not match annotated yield type `TestDebugger`
+ tests/debugging/mocking.py:234:15 [error] [invalid-yield] Yield type `Unknown | SignalUploader | None` does not match annotated yield type `MockSignalUploader`
discord.py (https://github.com/Rapptz/discord.py)
+ discord/abc.py:1821:23 [error] [invalid-yield] Yield type `Message` does not match annotated yield type `PinnedMessage`
+ discord/ext/commands/cog.py:461:32 [error] [invalid-yield] Yield type `Command[Never, (...), Any]` does not match annotated yield type `Command[Self@walk_commands, (...), Any]`
+ discord/ext/commands/core.py:1455:28 [error] [invalid-yield] Yield type `Command[Never, (...), Any]` does not match annotated yield type `Command[CogT@GroupMixin, (...), Any]`
+ discord/utils.py:1046:19 [error] [invalid-yield] Yield type `list[object]` does not match annotated yield type `list[T@_chunk]`
dulwich (https://github.com/dulwich/dulwich)
+ dulwich/porcelain/__init__.py:5047:23 [error] [invalid-yield] Yield type `str | bytes | PathLike[str]` does not match annotated yield type `str`
httpx-caching (https://github.com/johtso/httpx-caching)
- httpx_caching/_policy.py:216:76 [warning] [unused-type-ignore-comment] Unused blanket `type: ignore` directive
- httpx_caching/_policy.py:233:60 [warning] [unused-type-ignore-comment] Unused blanket `type: ignore` directive
+ httpx_caching/_policy.py:126:46 [error] [invalid-yield] Send type `tuple[Response, dict[Unknown, Unknown]]` does not match annotated send type `Response`
ibis (https://github.com/ibis-project/ibis)
+ ibis/expr/rewrites.py:183:23 [error] [invalid-yield] Yield type `ibis.expr.types.generic.Value` does not match annotated yield type `ibis.expr.operations.core.Value[Unknown, Any]`
+ ibis/util.py:703:15 [error] [invalid-yield] Yield type `type[T@get_subclasses]` does not match annotated yield type `type[S@get_subclasses]`
kopf (https://github.com/nolar/kopf)
+ kopf/_cogs/structs/dicts.py:261:28 [error] [invalid-yield] Send type `Never` does not match annotated send type `None`
openlibrary (https://github.com/internetarchive/openlibrary)
+ openlibrary/plugins/openlibrary/lists.py:125:23 [error] [invalid-yield] Yield type `dict[str, dict[str, ThingReferenceDict | str | AnnotatedSeedDict]]` does not match annotated yield type `AnnotatedSeedDict`
+ openlibrary/plugins/openlibrary/lists.py:125:41 [error] [invalid-argument-type] Invalid argument to key "key" with declared type `str` on TypedDict `ThingReferenceDict`: value of type `(ThingReferenceDict & ~Top[dict[Unknown, Unknown]]) | str | (AnnotatedSeedDict & ~Top[dict[Unknown, Unknown]])`
pandas (https://github.com/pandas-dev/pandas)
- pandas/io/formats/format.py:1068:20 [warning] [unused-type-ignore-comment] Unused blanket `type: ignore` directive
pegen (https://github.com/we-like-parsers/pegen)
+ src/pegen/grammar.py:399:13 [error] [invalid-yield] Yield type `None` does not match annotated yield type `tuple[str, str]`
psycopg (https://github.com/psycopg/psycopg)
+ psycopg/psycopg/_compat.py:47:13 [error] [invalid-yield] Yield type `None` does not match annotated yield type `str | Interpolation`
pwndbg (https://github.com/pwndbg/pwndbg)
+ pwndbg/aglib/macho.py:406:19 [error] [invalid-yield] Yield type `bytearray` does not match annotated yield type `bytes`
+ pwndbg/aglib/macho.py:578:19 [error] [invalid-yield] Yield type `tuple[bytearray, int]` does not match annotated yield type `tuple[bytes, int]`
+ pwndbg/commands/rop.py:197:15 [error] [invalid-yield] Yield type `tuple[str | None, None]` does not match annotated yield type `tuple[str, Page | None]`
pylint (https://github.com/pycqa/pylint)
+ pylint/checkers/utils.py:2055:28 [error] [invalid-yield] Send type `Never` does not match annotated send type `None`
+ pylint/checkers/utils.py:2067:28 [error] [invalid-yield] Send type `Never` does not match annotated send type `None`
+ pylint/checkers/utils.py:2070:28 [error] [invalid-yield] Send type `Never` does not match annotated send type `None`
rotki (https://github.com/rotki/rotki)
- rotkehlchen/chain/aggregator.py:1282:55 [warning] [unused-type-ignore-comment] Unused blanket `type: ignore` directive
schemathesis (https://github.com/schemathesis/schemathesis)
- src/schemathesis/engine/core.py:134:50 [warning] [unused-type-ignore-comment] Unused blanket `type: ignore` directive
scipy (https://github.com/scipy/scipy)
- subprojects/array_api_extra/tests/conftest.py:124:32 [warning] [unused-type-ignore-comment] Unused blanket `type: ignore` directive
scrapy (https://github.com/scrapy/scrapy)
+ scrapy/utils/asyncgen.py:19:19 [error] [invalid-yield] Yield type `object` does not match annotated yield type `_T@as_async_generator`
starlette (https://github.com/encode/starlette)
+ tests/test_routing.py:697:9 [error] [invalid-yield] Yield type `None` does not match annotated yield type `Never`
static-frame (https://github.com/static-frame/static-frame)
- static_frame/test/integration/test_field_stats.py:61:78 [warning] [unused-type-ignore-comment] Unused blanket `type: ignore` directive
+ static_frame/core/bus_mapping.py:45:20 [error] [invalid-yield] Yield type `Unknown | tuple[Hashable, @Todo]` does not match annotated yield type `tuple[TVKeys@BusMappingItemsView, @Todo]`
+ static_frame/core/memory_measure.py:241:23 [error] [invalid-yield] Yield type `tuple[str, list[Unknown]]` does not match annotated yield type `tuple[tuple[str, ...], list[int]]`
- static_frame/core/quilt.py:859:45 [warning] [unused-type-ignore-comment] Unused blanket `type: ignore` directive
+ static_frame/core/type_blocks.py:882:19 [error] [invalid-yield] Yield type `tuple[None | Unknown, int | Unknown]` does not match annotated yield type `tuple[dtype[Any], int]`
+ static_frame/core/type_blocks.py:1286:24 [error] [invalid-yield] Yield type `tuple[Hashable, ndarray[Any, Any], TypeBlocks | ndarray[Any, Any]]` does not match annotated yield type `tuple[ndarray[Any, Any], ndarray[Any, Any] | slice[Any, Any, Any], TypeBlocks]`
+ static_frame/core/type_blocks.py:1307:24 [error] [invalid-yield] Yield type `tuple[Hashable, slice[Any, Any, Any], TypeBlocks | ndarray[Any, Any]]` does not match annotated yield type `tuple[Hashable, ndarray[Any, Any] | slice[Any, Any, Any], ndarray[Any, Any]]`
+ static_frame/core/type_blocks.py:1316:24 [error] [invalid-yield] Yield type `tuple[Hashable, ndarray[Any, Any], TypeBlocks | ndarray[Any, Any]]` does not match annotated yield type `tuple[Hashable, ndarray[Any, Any] | slice[Any, Any, Any], ndarray[Any, Any]]`
+ static_frame/core/type_blocks.py:2927:27 [error] [invalid-yield] Yield type `Sequence[Any] | @Todo` does not match annotated yield type `tuple[Any, ...]`
+ static_frame/core/type_blocks.py:2930:27 [error] [invalid-yield] Yield type `Sequence[Any] | @Todo` does not match annotated yield type `tuple[Any, ...]`
+ static_frame/core/type_blocks.py:2941:23 [error] [invalid-yield] Yield type `Sequence[Any] | @Todo` does not match annotated yield type `tuple[Any, ...]`
+ static_frame/core/type_blocks.py:1284:24 [error] [invalid-yield] Yield type `tuple[Hashable, slice[Any, Any, Any], TypeBlocks | ndarray[Any, Any]]` does not match annotated yield type `tuple[ndarray[Any, Any], ndarray[Any, Any] | slice[Any, Any, Any], TypeBlocks]`
+ static_frame/core/type_blocks.py:2989:23 [error] [invalid-yield] Yield type `Sequence[Any] | @Todo` does not match annotated yield type `tuple[Any, ...]`
+ static_frame/test/unit/test_frame.py:6641:23 [error] [invalid-yield] Yield type `tuple[int, ndarray[tuple[Any, ...], dtype[Any]]]` does not match annotated yield type `tuple[int, tuple[int, int]]`
+ static_frame/test/unit/test_store_zip.py:310:24 [error] [invalid-yield] Yield type `tuple[Hashable, Frame]` does not match annotated yield type `tuple[str, Frame]`
vision (https://github.com/pytorch/vision)
+ test/datasets_utils.py:385:19 [error] [invalid-yield] Yield type `tuple[Unknown, Unknown | None]` does not match annotated yield type `tuple[VisionDataset, dict[str, Any]]`
werkzeug (https://github.com/pallets/werkzeug)
+ src/werkzeug/datastructures/structures.py:33:20 [error] [invalid-yield] Yield type `tuple[object, object]` does not match annotated yield type `tuple[K@iter_multi_items, V@iter_multi_items]`
+ src/werkzeug/datastructures/structures.py:38:27 [error] [invalid-yield] Yield type `tuple[object, object]` does not match annotated yield type `tuple[K@iter_multi_items, V@iter_multi_items]`
+ src/werkzeug/datastructures/structures.py:40:23 [error] [invalid-yield] Yield type `tuple[object, ~Top[list[Unknown]] & ~tuple[object, ...] & ~Top[set[Unknown]]]` does not match annotated yield type `tuple[K@iter_multi_items, V@iter_multi_items]`
+ src/werkzeug/test.py:167:20 [error] [invalid-yield] Send type `Never` does not match annotated send type `None`
xarray (https://github.com/pydata/xarray)
+ xarray/core/dataset.py:2675:23 [error] [invalid-yield] Yield type `tuple[Hashable, _arrayfunction[Any, Any] | _arrayapi[Any, Any] | ndarray[tuple[Any, ...], dtype[Unknown]] | Unknown]` does not match annotated yield type `tuple[Hashable, int | slice[Any, Any, Any] | ndarray[tuple[Any, ...], dtype[Any]] | Variable]`f07b1f1 to
72a0a9e
Compare
|
I just rebased the branch as I was going to work on the remaining failing yield test case in conformance tests(I move that in a separate PR to not change the ecosystem results) for return type checking. |
sharkdp
left a comment
There was a problem hiding this comment.
Thank you for your contribution and sorry for not getting back to you earlier. It looks good from a first scan, I plan to fully review it tomorrow.
Just one question upfront.
sharkdp
left a comment
There was a problem hiding this comment.
This looks great. I'm pushing a change that should remove some of the ecosystem false positives.
f6040b1 to
1c2c6bd
Compare
1c2c6bd to
497a0a7
Compare
Summary
Infer yield and yield from expression using function annotation.
Part of astral-sh/ty#1718
Remaining part is checking return types, I'm planning to do that in a separate PR.
All the remaining conformance tests are because of the return type checking.
Notes
Mixing Generator and Async Generator is allowed.
I was not sure if there's valid use case for this or not.
But I found this code and mypy allows this. So I'm not doing any checks for this case. Just the normal logic that unions the types.
Pyright rejects this.
Test Plan
I checked ecosystem results locally. There are new diagnostics added because we are now verifying that type of yielded value can be assigned annotated function type.
I can revert this if you think it would be too noisy to have it right now.
Added todo case for when return type does not match: