[ty] add TDD profiling infrastructure#23234
Draft
mtshiba wants to merge 13 commits intoastral-sh:mainfrom
Draft
Conversation
Replace sorted-list-based narrowing constraints (AND-only) with TDD (Ternary Decision Diagram) nodes that support AND, OR, and NOT. This changes the merge operation from intersection to OR, so narrowing from non-terminal branches is properly preserved after if/elif/else. Additionally, when a branch contains a NoReturn call (e.g. sys.exit()), gate the narrowing constraints by the ReturnsNever predicate. During the narrowing TDD walk, non-narrowing predicates are evaluated to determine reachability, so NoReturn branches contribute Never (absorbed by union) rather than polluting the merged type. Fixes astral-sh/ty#690 Fixes astral-sh/ty#685 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Record if-statement conditions as narrowing constraints for all places in scope, not just those directly narrowed by the condition. This ensures that unreachable branches (e.g., the else-branch of `if 1 + 1 == 2:`) contribute `Never` rather than diluting narrowing from reachable branches. Uses the same ScopedPredicateId for both specific-places and all-places recording, so TDD atoms are shared via hash-consing. For places already narrowed by the condition, AND(atom, atom) = atom — no duplication. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add tests for: - Always-true condition nested inside a narrowing branch - Match statement with terminal default branch (class patterns) - Match statement with reassignment and terminal default (issue astral-sh#1384) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The all-places narrowing (recording if-conditions in every variable's narrowing TDD) caused exponential evaluation time for long if/elif chains. Each non-narrowing ambiguous atom doubles the evaluation paths, so a 16-condition chain like black's `whitespace()` function caused 2^16 paths per variable, hanging ty. Cap the number of conditions per if/elif chain that get all-places narrowing at 4, bounding evaluation at ~16 paths while preserving the always-true condition behavior for common cases. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Revert the all-places narrowing propagation introduced in the previous two commits, as it causes exponential blowup on long if/elif chains (e.g. black/nodes.py with 16 conditions). Keep the test cases but update expectations to reflect the current behavior, with TODO comments for future improvement. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
mtshiba
added
internal
Typing conformance resultsNo changes detected ✅ |
Memory usage reportMemory usage unchanged ✅ |
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Stacked on #23109
While investigating the performance issue in #23109, I began to think it would be useful to be able to profile the information about the TDD (width, depth, etc.) that ty creates during type checking. Since the size of the TDD determines the worst-case computational complexity of narrowing/reachability analysis, care must be taken to ensure it does not increase unnecessarily. Conversely, reducing these numbers can be expected to improve memory footprint and performance.
In this PR, internal APIs for obtaining TDD statistics are implemented in
UseDefMap,ReachabilityConstraints, etc., and add an option to output the statistics viatracing.This is calculated and output only when ty is built with the
tdd-statsfeature flag enabled.Here is an example of the output:
Test Plan
N/A