Query: Add support for GroupBy patterns beyond aggregate

[smitpatel]

• Allow expanding navigations after GroupBy operator applied before reducing it to non-grouping Fixes Query: Expand navigations after GroupBy operator is applied #22609
• Translate FirstOrDefault over grouping element Fixes Translate GroupBy followed by FirstOrDefault over group #12088
• Add ability to select N element over grouping element Fixes Support ability to select top N of each group #13805

Overall approach:
A grouping element (the range variable you get after applying GroupBy operator) is of type IGrouping<TKey, TElement> which implements IEnumerable<TElement>. Hence we treat this enumerable as if it is queryable during nav expansion phase. During translation phase we inject ShapedQueryExpression in place of the grouping element which is being enumerated. What this allows us is to expand navigation just like any other query root and translate a subquery similar to other subqueries to facilitate reusing same code for the tasks.
During translation phase in relational layer, since aggregate operation can be lifted into projection for SelectExpression containing SQL GROUP BY. This code path works in 2 ways, when translating we try to combine predicate/distinct into the aggregate operation (so in future when we support custom aggregate operators, we don't have to understand the shape of it to modify it later. When adding this scalar subquery to SelectExpression, we try to pattern match it to see if we can lift it. Further during lifting, we also lift any additional joins in the subquery (which implies there were some joins expanded on grouping element before aggregate) including the navigation expanded from owned navigations. A pending TODO is to de-dupe navigation expanded. It is not straight forward since aliases of table would have changed when previous was lifted.
Given every enumerable grouping element act as query root, every time we replace it inside a lambda expression, we need to create a copy of the root. Navigation expansion and individual queryableMethodTranslatingEV does this. So each root act and translate independently from each other.

Bug fixes:

• Fix a bug in identifying single result in InMemory to convert it to enumerable
• Null out _groupingParameter in InMemoryQueryExpression once the projection to reduce it has been applied
• Throw better error message when translating Min/Max over an entity type for InMemory

[roji]

Was somewhat less bewildered than expected! Here are some nits, LGTM but will let @maumar do final approval.

[ghost]

Hello @smitpatel!

Because this pull request has the auto-merge label, I will be glad to assist with helping to merge this pull request once all check-in policies pass.

p.s. you can customize the way I help with merging this pull request, such as holding this pull request until a specific person approves. Simply @mention me (@msftbot) and give me an instruction to get started! Learn more here.

[ghost]

Apologies, while this PR appears ready to be merged, I've been configured to only merge when all checks have explicitly passed. The following integrations have not reported any progress on their checks and are blocking auto-merge:

1. Azure Pipelines

These integrations are possibly never going to report a check, and unblocking auto-merge likely requires a human being to update my configuration to exempt these integrations from requiring a passing check.

Give feedback on this

From the bot dev team

We've tried to tune the bot such that it posts a comment like this only when auto-merge is blocked for exceptional, non-intuitive reasons. When the bot's auto-merge capability is properly configured, auto-merge should operate as you would intuitively expect and you should not see any spurious comments.

Please reach out to us at [email protected] to provide feedback if you believe you're seeing this comment appear spuriously. Please note that we usually are unable to update your bot configuration on your team's behalf, but we're happy to help you identify your bot admin.

[ghost]

Apologies, while this PR appears ready to be merged, I've been configured to only merge when all checks have explicitly passed. The following integrations have not reported any progress on their checks and are blocking auto-merge:

1. Azure Pipelines

These integrations are possibly never going to report a check, and unblocking auto-merge likely requires a human being to update my configuration to exempt these integrations from requiring a passing check.

Give feedback on this

From the bot dev team

We've tried to tune the bot such that it posts a comment like this only when auto-merge is blocked for exceptional, non-intuitive reasons. When the bot's auto-merge capability is properly configured, auto-merge should operate as you would intuitively expect and you should not see any spurious comments.

Please reach out to us at [email protected] to provide feedback if you believe you're seeing this comment appear spuriously. Please note that we usually are unable to update your bot configuration on your team's behalf, but we're happy to help you identify your bot admin.