Skip to content

Include on clauses in impossible queries, to avoid only_full_group_by in aggregations with joins#2433

Merged
sougou merged 2 commits intovitessio:masterfrom
HubSpot:impossible_with_on_clause
Jan 4, 2017
Merged

Include on clauses in impossible queries, to avoid only_full_group_by in aggregations with joins#2433
sougou merged 2 commits intovitessio:masterfrom
HubSpot:impossible_with_on_clause

Conversation

@bbeaudreault
Copy link
Copy Markdown
Contributor

We have some queries which have aggregations as well as a join, and the existing impossible query was causing only_full_group_by errors in MySQL. Here's an example query:

mysql> select incident.*, t.id, t.team, t.slack_room, count(ais_other.id) as suppressionCount from incidents as incident join teams as t join active_incident_suppressions as ais_filter left join active_incident_suppressions as ais_other on 1 != 1 where 1 != 1 group by incident.id;
ERROR 1055 (42000): Expression #16 of SELECT list is not in GROUP BY clause and contains nonaggregated column 'Otto.t.id' which is not functionally dependent on columns in GROUP BY clause; this is incompatible with sql_mode=only_full_group_by

The t.id needs to either be joined on, or added to the group by. In this case it makes sense to be joined on.

This PR adds the On clause when it exists, and falls back to 1 != 1 when it does not, for left and right joins. Updated all tests to account for new formatting.

I verified with EXPLAIN that the new queries still result in Impossible WHERE and so should be just as fast, but safer.

cc @sougou

@bbeaudreault bbeaudreault force-pushed the impossible_with_on_clause branch from a7b381c to 793d493 Compare January 4, 2017 01:01
@bbeaudreault
Copy link
Copy Markdown
Contributor Author

I missed a couple of tests in the endtoend package. Working on those now, but I don't anticipate much issue aside from updating them to use the new format

case *JoinTableExpr:
if node.Join == LeftJoinStr || node.Join == RightJoinStr {
node.Format(buf)
if node.On == nil && (node.Join == LeftJoinStr || node.Join == RightJoinStr) {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The nil check is not required, because the parser rules enforce it.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In other words, I think you can remove this entire case.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea, I just realized that :) Simplifying now

@bbeaudreault
Copy link
Copy Markdown
Contributor Author

End to end tests fixed, and removed the explicit handling of JoinTableExpr since we're now passing through the on clause

@sougou
Copy link
Copy Markdown
Contributor

sougou commented Jan 4, 2017

LGTM

Approved with PullApprove

@sougou sougou merged commit d6465c0 into vitessio:master Jan 4, 2017
@bbeaudreault bbeaudreault deleted the impossible_with_on_clause branch January 4, 2017 10:34
dispalt added a commit to dispalt/vitess that referenced this pull request Jan 9, 2017
* 'master' of https://github.com/youtube/vitess: (220 commits)
  vtgate/vindexes: Do not run testfiles.Locate() from init() (2nd time).
  Add comment to document when new fields are included in response
  publish site Fri Jan  6 08:02:30 PST 2017
  Updating doc.
  Optionally send additional C API fields through in responses
  Fixing explorer to handle empty files.
  MultiRow Insert Optimization (vitessio#2422)
  Now the local examples use 'zk2' topology.
  Adding Consul 0.7.2 to base docker image.
  etcd2topo: using lease ID in lock filenames.
  Implementing the Consul topology client.
  test: vtgate_buffer.py: Add second test for vtctl TabletExternallyReparented.
  test: vtgate_buffer.py: Stop the the two threads in case of a test error.
  test: vtgate_buffer.py: Move setup code from setUp to setUpModule.
  test: Add "demote_master_commands" to MysqlFlavor class.
  Include on clauses in impossible queries, to avoid only_full_group_by in aggregations with joins (vitessio#2433)
  Adds support for parsing queries with boolean value comparisons, i.e. true/false and =, !=, >, <, >=, <= (vitessio#2432)
  V3 Engine Code Refactor (vitessio#2423)
  Refactor impossible query generation, and support group by - (vitessio#2409)
  docker/test/run: Avoid using non-universal --tmpdir flag on mktemp (vitessio#2429)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants