Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

QueryRewrite: convert correlated SelectMany to JOINs #15081

Closed
maumar opened this issue Mar 20, 2019 · 0 comments · Fixed by #17077
Closed

QueryRewrite: convert correlated SelectMany to JOINs #15081

maumar opened this issue Mar 20, 2019 · 0 comments · Fixed by #17077
Assignees
Labels
closed-fixed The issue has been fixed and is/will be included in the release indicated by the issue milestone. punted-for-3.0 type-enhancement
Milestone

Comments

@maumar
Copy link
Contributor

maumar commented Mar 20, 2019

query:

entities.SelectMany(l1 => l1.OneToMany_Required1)

expected:

SELECT [l1.OneToMany_Required1].[Id], [l1.OneToMany_Required1].[Date], [l1.OneToMany_Required1].[Level1_Optional_Id], [l1.OneToMany_Required1].[Level1_Required_Id], [l1.OneToMany_Required1].[Name], [l1.OneToMany_Required1].[OneToMany_Optional_Inverse2Id], [l1.OneToMany_Required1].[OneToMany_Optional_Self_Inverse2Id], [l1.OneToMany_Required1].[OneToMany_Required_Inverse2Id], [l1.OneToMany_Required1].[OneToMany_Required_Self_Inverse2Id], [l1.OneToMany_Required1].[OneToOne_Optional_PK_Inverse2Id], [l1.OneToMany_Required1].[OneToOne_Optional_Self2Id]
FROM [LevelOne] AS [l1]
INNER JOIN [LevelTwo] AS [l1.OneToMany_Required1] ON [l1].[Id] = [l1.OneToMany_Required1].[OneToMany_Required_Inverse2Id]

current:

SELECT [l].[Id], [l].[Date], [l].[Level1_Optional_Id], [l].[Level1_Required_Id], [l].[Name], [l].[OneToMany_Optional_Inverse2Id], [l].[OneToMany_Optional_Self_Inverse2Id], [l].[OneToMany_Required_Inverse2Id], [l].[OneToMany_Required_Self_Inverse2Id], [l].[OneToOne_Optional_PK_Inverse2Id], [l].[OneToOne_Optional_Self2Id]
FROM [LevelOne] AS [l1]
CROSS JOIN [LevelTwo] AS [l]
WHERE ([l1].[Id] = [l].[OneToMany_Required_Inverse2Id])
@maumar maumar self-assigned this Mar 20, 2019
@maumar maumar added this to the 3.0.0 milestone Mar 20, 2019
@maumar maumar changed the title QueryRewrite: convert correlated SelectMany to INNER JOIN QueryRewrite: convert correlated SelectMany to JOINs Mar 20, 2019
@ajcvickers ajcvickers modified the milestones: 3.0.0, Backlog Jun 28, 2019
@smitpatel smitpatel assigned smitpatel and unassigned smitpatel Aug 7, 2019
@smitpatel smitpatel modified the milestones: Backlog, 3.0.0 Aug 9, 2019
@smitpatel smitpatel added the closed-fixed The issue has been fixed and is/will be included in the release indicated by the issue milestone. label Aug 10, 2019
smitpatel added a commit that referenced this issue Aug 10, 2019
SelectMany in linq look something like following (second clause is considered the collection selector)
```C#
from c in cs
from o in c.Orders
select....
```
Following are the translations of SelectMany

Unrelated collection selector
```C#
from c in cs
from o in os
```
Generates CROSS JOIN

Correlated collection selector with correlation being a predicate
```C#
from c in cs
from o in os.Where(o => o.CustomerID == c.CustomerID)
```
Such predicate can be lifted and used in generating a join. So this query generates JOIN.
If collection selector ends with DefaultIfEmpty then it is LEFT JOIN.

Correlated collection selector with correlation not a predicate
```C#
from c in cs
from o in os.Select(o => c.City)
```
Since we cannot generate a join predicate here, this translates to CROSS APPLY.
If collection selector ends with DefaultIfEmpty then it is OUTER APPLY.

Add support for Cross Apply
Add Support for Outer Apply
Convert Cross Apply to Inner Join when possible
Convert Outer Apply to Left Join when possible
Add translation for DefaultIfEmpty
Add translation for both overloads of SelectMany
Handle DefaultIfEmpty & SelectMany without collectionSelector in Navigation Expansion
Ensure columns are in projection when generating join predicate from a correlated subquery

Currently there are no tests for Cross/Outer Apply.
Our earlier Cross Apply got converted to join. N+1 evaluation tests are disabled right now, which would generate Cross Apply.
We never supported Outer Apply in past.

Resolves #15711
Resolves #12567
Resolves #12572
Resolves #12872
Resolves #16330
Resolves #15081
Resolves #16989

Re-enable tests for #12449
smitpatel added a commit that referenced this issue Aug 10, 2019
SelectMany in linq look something like following (second clause is considered the collection selector)
```C#
from c in cs
from o in c.Orders
select....
```
Following are the translations of SelectMany

Unrelated collection selector
```C#
from c in cs
from o in os
```
Generates CROSS JOIN

Correlated collection selector with correlation being a predicate
```C#
from c in cs
from o in os.Where(o => o.CustomerID == c.CustomerID)
```
Such predicate can be lifted and used in generating a join. So this query generates JOIN.
If collection selector ends with DefaultIfEmpty then it is LEFT JOIN.

Correlated collection selector with correlation not a predicate
```C#
from c in cs
from o in os.Select(o => c.City)
```
Since we cannot generate a join predicate here, this translates to CROSS APPLY.
If collection selector ends with DefaultIfEmpty then it is OUTER APPLY.

- Add support for Cross Apply
- Add Support for Outer Apply
- Convert Cross Apply to Inner Join when possible
- Convert Outer Apply to Left Join when possible
- Add translation for DefaultIfEmpty
- Add translation for both overloads of SelectMany
- Handle DefaultIfEmpty & SelectMany without collectionSelector in Navigation Expansion
- Ensure columns are in projection when generating join predicate from a correlated subquery

Currently there are no tests for Cross/Outer Apply.
Our earlier Cross Apply got converted to join. N+1 evaluation tests are disabled right now, which would generate Cross Apply.
We never supported Outer Apply in past.

Resolves #15711
Resolves #12567
Resolves #12572
Resolves #12872
Resolves #16330
Resolves #15081
Resolves #16989

Re-enable tests for #12449
smitpatel added a commit that referenced this issue Aug 12, 2019
SelectMany in linq look something like following (second clause is considered the collection selector)
```C#
from c in cs
from o in c.Orders
select....
```
Following are the translations of SelectMany

Unrelated collection selector
```C#
from c in cs
from o in os
```
Generates CROSS JOIN

Correlated collection selector with correlation being a predicate
```C#
from c in cs
from o in os.Where(o => o.CustomerID == c.CustomerID)
```
Such predicate can be lifted and used in generating a join. So this query generates JOIN.
If collection selector ends with DefaultIfEmpty then it is LEFT JOIN.

Correlated collection selector with correlation not a predicate
```C#
from c in cs
from o in os.Select(o => c.City)
```
Since we cannot generate a join predicate here, this translates to CROSS APPLY.
If collection selector ends with DefaultIfEmpty then it is OUTER APPLY.

- Add support for Cross Apply
- Add Support for Outer Apply
- Convert Cross Apply to Inner Join when possible
- Convert Outer Apply to Left Join when possible
- Add translation for DefaultIfEmpty
- Add translation for both overloads of SelectMany
- Handle DefaultIfEmpty & SelectMany without collectionSelector in Navigation Expansion
- Ensure columns are in projection when generating join predicate from a correlated subquery

Currently there are no tests for Cross/Outer Apply.
Our earlier Cross Apply got converted to join. N+1 evaluation tests are disabled right now, which would generate Cross Apply.
We never supported Outer Apply in past.

Resolves #15711
Resolves #12567
Resolves #12572
Resolves #12872
Resolves #16330
Resolves #15081
Resolves #16989

Re-enable tests for #12449
smitpatel added a commit that referenced this issue Aug 13, 2019
SelectMany in linq look something like following (second clause is considered the collection selector)
```C#
from c in cs
from o in c.Orders
select....
```
Following are the translations of SelectMany

Unrelated collection selector
```C#
from c in cs
from o in os
```
Generates CROSS JOIN

Correlated collection selector with correlation being a predicate
```C#
from c in cs
from o in os.Where(o => o.CustomerID == c.CustomerID)
```
Such predicate can be lifted and used in generating a join. So this query generates JOIN.
If collection selector ends with DefaultIfEmpty then it is LEFT JOIN.

Correlated collection selector with correlation not a predicate
```C#
from c in cs
from o in os.Select(o => c.City)
```
Since we cannot generate a join predicate here, this translates to CROSS APPLY.
If collection selector ends with DefaultIfEmpty then it is OUTER APPLY.

- Add support for Cross Apply
- Add Support for Outer Apply
- Convert Cross Apply to Inner Join when possible
- Convert Outer Apply to Left Join when possible
- Add translation for DefaultIfEmpty
- Add translation for both overloads of SelectMany
- Handle DefaultIfEmpty & SelectMany without collectionSelector in Navigation Expansion
- Ensure columns are in projection when generating join predicate from a correlated subquery

Currently there are no tests for Cross/Outer Apply.
Our earlier Cross Apply got converted to join. N+1 evaluation tests are disabled right now, which would generate Cross Apply.
We never supported Outer Apply in past.

Resolves #15711
Resolves #12567
Resolves #12572
Resolves #12872
Resolves #16330
Resolves #15081
Resolves #16989

Re-enable tests for #12449
@ajcvickers ajcvickers modified the milestones: 3.0.0, 3.0.0-preview9 Aug 21, 2019
@ajcvickers ajcvickers modified the milestones: 3.0.0-preview9, 3.0.0 Nov 11, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
closed-fixed The issue has been fixed and is/will be included in the release indicated by the issue milestone. punted-for-3.0 type-enhancement
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants