Teach regex about more disjoint sets #117869
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
The regex optimizer tries to make loops atomic whenever it's safe to do so, as doing so removes possible backtracking. When it finds a set loop followed by another set, to be able to make the loop atomic, it needs to prove that the sets are disjoint, that there's nothing that can match both sets. It already does so for a wide variety of cases, but it's missing logic today that helps it to determine that for sets composed only of Unicode categories, e.g. \d, \w, p{C}, etc. This adds that in.
There was a problem hiding this comment.
Pull Request Overview
This PR enhances the regex optimizer to better identify disjoint sets when dealing with Unicode categories, enabling more atomic loops and reducing backtracking. The key improvement is adding logic to recognize when sets composed of Unicode categories (like \d, \w, \p{C}) don't overlap, allowing the optimizer to make loops atomic in more cases.
Key changes:
- Added Unicode category analysis to determine set disjointness
- Enhanced known distinct sets detection with more comprehensive pattern matching
- Added comprehensive test coverage for the new optimization scenarios
Reviewed Changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| RegexCharClass.cs | Added TryGetOnlyCategories logic to analyze Unicode category composition and enhanced KnownDistinctSets method |
| RegexReductionTests.cs | Added test cases for atomic loop optimization with Unicode category patterns |
...ibraries/System.Text.RegularExpressions/src/System/Text/RegularExpressions/RegexCharClass.cs
Show resolved
Hide resolved
...ibraries/System.Text.RegularExpressions/src/System/Text/RegularExpressions/RegexCharClass.cs
Show resolved
Hide resolved
|
Tagging subscribers to this area: @dotnet/area-system-text-regularexpressions |
This was referenced Jul 21, 2025
Open
|
/ba-g "due to space limitations" |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The regex optimizer tries to make loops atomic whenever it's safe to do so, as doing so removes possible backtracking. When it finds a set loop followed by another set, to be able to make the loop atomic, it needs to prove that the sets are disjoint, that there's nothing that can match both sets. It already does so for a wide variety of cases, but it's missing logic today that helps it to determine that for sets composed only of Unicode categories, e.g. \d, \w, p{C}, etc. This adds that in.