Skip to content

[ty] Preserve pure negation types in descriptor protocol#22907

Merged
charliermarsh merged 3 commits intomainfrom
charlie/negation
Jan 29, 2026
Merged

[ty] Preserve pure negation types in descriptor protocol#22907
charliermarsh merged 3 commits intomainfrom
charlie/negation

Conversation

@charliermarsh
Copy link
Member

Summary

When accessing an attribute with a pure negation type (an intersection with only negative contributions, like ~AlwaysFalsy), the descriptor protocol would incorrectly return object instead of preserving the negation type, because map_with_boundness iterates over positive_elements_or_object, which falls back to object when there are no positive elements.

@charliermarsh charliermarsh added the ty Multi-file analysis & type inference label Jan 28, 2026
@astral-sh-bot
Copy link

astral-sh-bot bot commented Jan 28, 2026

Typing conformance results

No changes detected ✅

@astral-sh-bot
Copy link

astral-sh-bot bot commented Jan 28, 2026

mypy_primer results

Changes were detected when running on open source projects
prefect (https://github.com/PrefectHQ/prefect)
+ src/integrations/prefect-dbt/prefect_dbt/core/settings.py:94:28: error[invalid-assignment] Object of type `dict[Any, Any] | int | dict[str, Any] | ... omitted 4 union elements` is not assignable to `dict[str, Any]`
+ src/integrations/prefect-dbt/prefect_dbt/core/settings.py:99:28: error[invalid-assignment] Object of type `int | dict[Any, Any] | float | ... omitted 3 union elements` is not assignable to `dict[str, Any]`
+ src/prefect/cli/deploy/_core.py:86:21: error[invalid-assignment] Object of type `dict[Any, Any] | int | dict[str, Any] | ... omitted 4 union elements` is not assignable to `dict[str, Any]`
+ src/prefect/cli/deploy/_core.py:87:21: error[invalid-assignment] Object of type `int | dict[Any, Any] | float | ... omitted 3 union elements` is not assignable to `dict[str, Any]`
+ src/prefect/deployments/steps/core.py:137:38: error[invalid-argument-type] Argument is incorrect: Argument type `dict[Any, Any] | int | dict[str, Any] | ... omitted 4 union elements` does not satisfy constraints (`str`, `int`, `int | float`, `bool`, `dict[Any, Any]`, `list[Any]`, `None`) of type variable `T`
- src/prefect/utilities/templating.py:320:13: error[invalid-assignment] Invalid subscript assignment with key of type `object` and value of type `Unknown | dict[str, Any]` on object of type `dict[str, Any]`
+ src/prefect/utilities/templating.py:320:13: error[invalid-assignment] Invalid subscript assignment with key of type `object` and value of type `Unknown | int | dict[str, Any] | ... omitted 4 union elements` on object of type `dict[str, Any]`
- src/prefect/utilities/templating.py:323:16: error[invalid-return-type] Return type does not match returned value: expected `T@resolve_block_document_references | dict[str, Any]`, found `list[Unknown | dict[str, Any]]`
+ src/prefect/utilities/templating.py:323:16: error[invalid-return-type] Return type does not match returned value: expected `T@resolve_block_document_references | dict[str, Any]`, found `list[Unknown | int | dict[str, Any] | ... omitted 4 union elements]`
- src/prefect/utilities/templating.py:437:16: error[invalid-return-type] Return type does not match returned value: expected `T@resolve_variables`, found `dict[object, Unknown]`
+ src/prefect/utilities/templating.py:437:16: error[invalid-return-type] Return type does not match returned value: expected `T@resolve_variables`, found `dict[object, Unknown | int | float | ... omitted 4 union elements]`
- src/prefect/utilities/templating.py:442:16: error[invalid-return-type] Return type does not match returned value: expected `T@resolve_variables`, found `list[Unknown]`
+ src/prefect/utilities/templating.py:442:16: error[invalid-return-type] Return type does not match returned value: expected `T@resolve_variables`, found `list[Unknown | int | float | ... omitted 4 union elements]`
- src/prefect/workers/base.py:232:13: error[invalid-argument-type] Argument is incorrect: Argument type `str | dict[str, Any]` does not satisfy constraints (`str`, `int`, `int | float`, `bool`, `dict[Any, Any]`, `list[Any]`, `None`) of type variable `T`
+ src/prefect/workers/base.py:232:13: error[invalid-argument-type] Argument is incorrect: Argument type `str | int | dict[str, Any] | ... omitted 3 union elements` does not satisfy constraints (`str`, `int`, `int | float`, `bool`, `dict[Any, Any]`, `list[Any]`, `None`) of type variable `T`
+ src/prefect/workers/base.py:234:20: error[invalid-argument-type] Argument expression after ** must be a mapping type: Found `int | Unknown | float | ... omitted 4 union elements`
- Found 5368 diagnostics
+ Found 5374 diagnostics

rotki (https://github.com/rotki/rotki)
- rotkehlchen/chain/decoding/tools.py:96:44: warning[unused-type-ignore-comment] Unused blanket `type: ignore` directive
- rotkehlchen/chain/decoding/tools.py:99:13: error[invalid-argument-type] Argument to function `decode_transfer_direction` is incorrect: Expected `Sequence[A@BaseDecoderTools]`, found `Unknown | tuple[BTCAddress, ...] | tuple[ChecksumAddress, ...] | tuple[SubstrateAddress, ...] | tuple[SolanaAddress, ...]`
- rotkehlchen/chain/decoding/tools.py:100:62: warning[unused-type-ignore-comment] Unused blanket `type: ignore` directive
+ rotkehlchen/chain/decoding/tools.py:97:13: error[invalid-argument-type] Argument to function `decode_transfer_direction` is incorrect: Expected `BTCAddress | ChecksumAddress | SubstrateAddress | SolanaAddress`, found `A@BaseDecoderTools`
+ rotkehlchen/chain/decoding/tools.py:98:13: error[invalid-argument-type] Argument to function `decode_transfer_direction` is incorrect: Expected `BTCAddress | ChecksumAddress | SubstrateAddress | SolanaAddress | None`, found `A@BaseDecoderTools | None`
- Found 2050 diagnostics
+ Found 2049 diagnostics

core (https://github.com/home-assistant/core)
- homeassistant/util/variance.py:47:12: error[invalid-return-type] Return type does not match returned value: expected `(**_P@ignore_variance) -> _R@ignore_variance`, found `_Wrapped[_P@ignore_variance, int | _R@ignore_variance | float | datetime, _P@ignore_variance, _R@ignore_variance | int | float | datetime]`
- Found 14503 diagnostics
+ Found 14502 diagnostics

No memory usage changes detected ✅

@charliermarsh charliermarsh marked this pull request as ready for review January 28, 2026 14:36
Copy link
Contributor

@carljm carljm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

Comment on lines 12711 to 12714
// For pure negation types (no positive elements), there's nothing to map.
if self.positive(db).is_empty() {
return Place::Undefined;
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this is right. We should implicitly include object in such an intersection, and we should still call the transform_fn on object.

Comment on lines 2815 to 2819
let place = if mapped.is_undefined() {
attribute.place
} else {
mapped
};
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we follow the suggestion below to still call the transform_fn on object when we have a negative-only intersection, then we would never get back Place::Undefined (because our transform_fn above always returns Place::Defined), and then we wouldn't need this new case either.

@charliermarsh
Copy link
Member Author

@carljm -- I think I understood the concerns but not necessarily what you viewed as the appropriate fix? I addressed the comments and added a block where we re-add the negations. Or are you saying that changing the behavior of map_with_boundness at all is incorrect, and it's correct that map_with_boundness returns object?

@carljm
Copy link
Contributor

carljm commented Jan 29, 2026

@charliermarsh Well, that makes sense, because looking at it again, I clearly didn't think it through all the way 😆 I correctly observed that the change originally made here wasn't correct for map_with_boundness, but if you just applied my comments directly, it would eliminate the fix here entirely. (And I was proposing a "fix" that isn't even necessary, because map_with_boundness already uses self.positive_elements_or_object).

are you saying that changing the behavior of map_with_boundness at all is incorrect, and it's correct that map_with_boundness returns object?

On a more careful second look: yes, exactly this. If we wanted map_with_boundness to be able to handle more precise transforms of negative elements, there would need to be some change to its API such that there's a separate transform_fn provided for negative elements, or the transform_fn is informed when it is handling a negative element. (Because if F is our mapping function, F(~T) is not necessarily equivalent to ~F(T) -- we can't just pass negative elements as if they were positive elements to our mapping function and then negate the result.) But I don't think we have any uses of map_with_boundness where that would actually be useful (including this one), because it's very rare that you know enough about a negative type to be able to transform it more precisely than treating it as object. (Even if we had that feature in map_with_boundness, we couldn't use it here, because we can't transform any individual negative element more precisely than as object. It's only if there are no elements in the intersection that are descriptors that we can safely preserve the negative elements of the original type.)

So I think the right fix here is to leave map_with_boundness alone, and add logic in the call-site to not even bother using map_with_boundness if we have an only-negative intersection. We know we can't consider any such type a descriptor, so we just leave the original type untouched.

@charliermarsh
Copy link
Member Author

Haha okay, this makes sense. Ironically, I started with what you described, then decided it was too special-case and I should pursue a deeper fix, and here we are :)

Copy link
Contributor

@carljm carljm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

orthogonal nit, but I don't think there's any reason we need to be renaming this here

Comment on lines 2806 to 2812
Place::Defined(DefinedPlace {
ty: Type::Intersection(intersection),
origin,
definedness: boundness,
widening,
})
.with_qualifiers(qualifiers),
.with_qualifiers(qualifiers)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is equivalent?

Suggested change
Place::Defined(DefinedPlace {
ty: Type::Intersection(intersection),
origin,
definedness: boundness,
widening,
})
.with_qualifiers(qualifiers),
.with_qualifiers(qualifiers)
attribute

@charliermarsh charliermarsh enabled auto-merge (squash) January 29, 2026 00:51
@charliermarsh charliermarsh merged commit 8657ef8 into main Jan 29, 2026
48 checks passed
@charliermarsh charliermarsh deleted the charlie/negation branch January 29, 2026 00:53
carljm added a commit that referenced this pull request Jan 30, 2026
* main: (76 commits)
  [ty] Improve the check for `NewType`s with generic bases (#22961)
  [ty] Ban legacy `TypeVar` bounds or constraints from containing type variables (#22949)
  Bump the typing conformance suite pin (#22960)
  [ty] Emit an error if a TypeVarTuple is used to subscript `Generic` or `Protocol` without being unpacked (#22952)
  [ty] Reduce false positives when subscripting classes generic over `TypeVarTuple`s (#22950)
  [ty] Detect invalid attempts to subclass `Protocol[]` and `Generic[]` simultaneously (#22948)
  Fix suppression indentation matching (#22903)
  Remove hidden `--output-format` warning (#22944)
  [ty] Validate signatures of dataclass `__post_init__` methods (#22730)
  [ty] extend special-cased `numbers` diagnostic to `invalid-argument-type` errors (#22938)
  [ty] Avoid false positive for `not-iterable` with no-positive intersection types (#22089)
  [ty] Preserve pure negation types in descriptor protocol (#22907)
  [ty] add special-case diagnostic for `numbers` module (#22931)
  [ty] Move the location of more `invalid-overload` diagnostics (#22933)
  [ty] Fix unary and comparison operators for TypeVars with union bounds (#22925)
  [ty] Rule Selection: ignore/warn/select all rules (unless subsequently overriden) (#22832)
  [ty] Fix TypedDict construction from existing TypedDict values (#22904)
  [ty] fix bug in string annotations and clean up diagnostics (#22913)
  [ty] Improve support for goto-type, goto-declaration, hover, and highlighting of string annotations (#22878)
  [ty] Rename old typing imports to new on `unresolved-reference`. (#22827)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ty Multi-file analysis & type inference

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants