Fix stack overflow with recursive generic protocols#21857
Closed
Fix stack overflow with recursive generic protocols#21857
Conversation
This fixes astral-sh/ty#1736 where recursive generic protocols with growing specializations caused a stack overflow. The issue occurred with protocols like: ```python class C[T](Protocol): a: 'C[set[T]]' ``` When checking `C[set[int]]` against `C[Unknown]`, member `a` requires checking `C[set[set[int]]]`, which requires `C[set[set[set[int]]]]`, etc. Each level has different type specializations, so the existing cycle detection (using full types as cache keys) didn't catch the infinite recursion. The fix introduces `TypeRelationKey`, an enum that can be either a full `Type` or a `ClassLiteral` (protocol class without specialization). For protocol-to-protocol comparisons, we use `ClassLiteral` keys, which detects when we're comparing the same protocol class regardless of specialization. When a cycle is detected, we return the fallback value (assume compatible) to safely terminate the recursion.
Diagnostic diff on typing conformance testsNo changes detected when running ty on typing conformance tests ✅ |
|
|
| Lint rule | Added | Removed | Changed |
|---|---|---|---|
invalid-argument-type |
3 | 5 | 4 |
unused-ignore-comment |
5 | 0 | 0 |
no-matching-overload |
0 | 4 | 0 |
possibly-missing-attribute |
0 | 0 | 4 |
unsupported-base |
2 | 0 | 0 |
invalid-assignment |
0 | 0 | 1 |
| Total | 10 | 9 | 9 |
Contributor
Author
|
Closing in favor of #21858 -- this has too much ecosystem impact. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This fixes astral-sh/ty#1736 where recursive generic protocols with growing specializations caused a stack overflow.
The issue occurred with protocols like:
When checking
C[set[int]]againstC[Unknown], memberarequires checkingC[set[set[int]]], which requiresC[set[set[set[int]]]], etc. Each level has different type specializations, so the existing cycle detection (using full types as cache keys) didn't catch the infinite recursion.The fix introduces
TypeRelationKey, an enum that can be either a fullTypeor aClassLiteral(protocol class without specialization). For protocol-to-protocol comparisons, we useClassLiteralkeys, which detects when we're comparing the same protocol class regardless of specialization. When a cycle is detected, we return the fallback value (assume compatible) to safely terminate the recursion.In theory this could mean some false positives in cycle detection, but I haven't been able to come up with a practical example where this would be a problem. We'll see how the ecosystem tests feel about it.
Test Plan
Added mdtest.