Add support for GROUP BY AUTO aggregation

[ebyhr]

Description

This syntax allows omitting column positions or names after GROUP BY.
For instance, SELECT name, count(1) FROM test GROUP BY AUTO will be translated to SELECT name, count(1) FROM test GROUP BY name

References in other database/query engines

• BigQuery https://cloud.google.com/bigquery/docs/reference/standard-sql/query-syntax#group_by_all
• Snowflake https://docs.snowflake.com/en/sql-reference/constructs/group-by#label-group-by-all-columns
• Databricks https://docs.databricks.com/sql/language-manual/sql-ref-syntax-qry-select-groupby.html
• ClickHouse https://clickhouse.com/docs/en/sql-reference/statements/select/group-by#group-by-all
• Databend https://databend.rs/doc/sql-commands/query-syntax/query-group-by#group-by-all-columns

Release notes

(x) Release notes are required, with the following suggested text:

# General
* Add support for `GROUP BY AUTO` aggregation that implicitly groups all non-aggregated columns. ({issue}`18390`)

[electrum]

This is great. I fully support adding this syntax.

[martint]

This conflicts with the standard syntax for GROUP BY:

 GROUP BY [ <set quantifier> ] <grouping element list>

where <set quantifier> can be ALL or DISTINCT and defaults to ALL if omitted. The quantifier affects the semantics for queries involving grouping sets. Overloading the meaning to indicate how the keys are selected instead of how the rows in the result are de-duplicated is confusing and error prone.

Before we could consider such syntax, we'd need to define the precise semantics and how it interacts and relates to the broader GROUP BY feature.

[martint]

In particular, here are some inconsistencies:

SELECT k, count(*) 
FROM (VALUES 1,2,3) t(k) 
GROUP BY ALL

produces a result, but

SELECT k, count(*) 
FROM (VALUES 1,2,3) t(k) 
GROUP BY

fails with 'k' must be an aggregate expression or appear in GROUP BY clause, even though ALL is implied if missing, per the SQL standard.

SELECT k, count(*) 
FROM (VALUES 1,2,3) t(k) 
GROUP BY DISTINCT

fails with groupingElements must not be empty when type is DISTINCT. Presumably, it should also work. But, what would the meaning of such a query?

Without precise semantics, it's hard to tell what's the behavior of these queries after this change:

SELECt k, count(*) 
FROM (VALUES 1,2,3) t(k) 
GROUP BY ALL k

SELECT k, v, count(*) 
FROM (VALUES (1,1),(2,2),(3,3)) t(k, v) 
GROUP BY ALL

SELECT k, v, count(*) 
FROM (VALUES (1,1),(2,2),(3,3)) t(k, v) 
GROUP BY ALL k

SELECT k, v, count(*) 
FROM (VALUES (1,1),(2,2),(3,3)) t(k, v) 
GROUP BY ALL k, v

SELECT count(*) 
FROM (VALUES 1,2,3) t(k) 
GROUP BY ALL ()

[ebyhr]

1. Since GROUP BY ALL without grouping element list isn't defined in standard, we don't need to default to ALL in a such case and the failure is expected for me.
2. I don't think it should work. GROUP BY DISTINCT without grouping element list was disallowed even before this change. Also, the standard also disallows it in my understanding. The unclear semantics you mentioned implies the current behavior of throwing an exception is better.
3. Only 2nd example should be affected in this PR because it's the only example which uses GROUP BY ALL without grouping element list. Other examples should keep the original behavior. This rule is clear to me.

[martint]

To be clear, the ALL and DISTINCT qualifiers control whether grouping sets that have the same combination of keys are deduped.

Therefore, the only way this syntax makes sense is if we give meaning to omitting the grouping set specification. Specifically, to be equivalent to having a single grouping set composed of all the expressions in the group by clause that don’t contain aggregations.

In that case, the qualifier is orthogonal to such feature. Allowing one but not the other looks arbitrary and introduces cognitive load for a user who has to understand that they are somehow connected even though intuitively they should not be.

Another aspect that complicates issues conceptually is that the GROUP BY operation occurs before the SELECT clause is computed, so it’s a chicken-and-egg problem to determine which columns are grouping keys and which ones are derived. Also, the GROUP BY clause operates on input columns (those coming from the FROM clause) not on those from the SELECT clause. The implication arrow goes the other way: an expression in the SELECT clause is valid if it’s functionally dependent on the input columns used for computing the grouping sets.

[ebyhr]

@martint Thank you for your detailed explanation. Can you suggest alternative syntax? Or we don't want to add this feature?

[ebyhr]

Updated the syntax to GROUP BY IMPLICIT.

[martint]

I just had another idea -- instead of IMPLICIT, we could use the term AUTO. It's shorter, easy to type, and conveys the intention clearly.

[ebyhr]

I prefer AUTO to IMPLICIT. Updated.

[ebyhr]

@martint Gentle reminder.

[ebyhr]

Addressed comments.

[martint]

Don't forget to squash the fixup commits.