Implement Join pushdown for JDBC connectors by findepi · Pull Request #6874 · trinodb/trino

findepi · 2021-02-11T21:02:01Z

For #6620

plugin/trino-base-jdbc/src/main/java/io/trino/plugin/jdbc/JdbcMetadata.java

plugin/trino-memsql/src/main/java/io/trino/plugin/memsql/MemSqlClient.java

kokosing

ack

plugin/trino-base-jdbc/src/main/java/io/trino/plugin/jdbc/JdbcMetadataConfig.java

kokosing · 2021-02-22T13:59:46Z

plugin/trino-mysql/src/main/java/io/trino/plugin/mysql/MySqlClient.java

Don't we need to extract this as method and reuse it? As:

Remote database can be case insensitive.

is a quite common case.

Need -- no.

Want -- i don't know.
I don't want to bloat BaseJdbcClient, but this can go somewhere else

...-postgresql/src/test/java/io/trino/plugin/postgresql/TestPostgreSqlIntegrationSmokeTest.java

kokosing · 2021-02-22T14:03:56Z

...-postgresql/src/test/java/io/trino/plugin/postgresql/TestPostgreSqlIntegrationSmokeTest.java

It looks like a feature is written in a generic way, while testing only happens for PostgreSQL. Can we somehow disable this in other connectors until the moment test coverage is added to them?

it's disabled by default, and i am still thinking how to apply the test to all connectors, sanely.

...-postgresql/src/test/java/io/trino/plugin/postgresql/TestPostgreSqlIntegrationSmokeTest.java

findepi · 2021-03-03T12:31:12Z

PTAL

findepi · 2021-03-03T12:33:00Z

not PTAL yet

findepi · 2021-03-03T22:40:49Z

May work now. PTAL

hashhar

Some comments. Didn't review any changes in planner or JdbcMetadata.

plugin/trino-base-jdbc/src/main/java/io/trino/plugin/jdbc/JdbcMetadataConfig.java

plugin/trino-base-jdbc/src/test/java/io/trino/plugin/jdbc/BaseJdbcConnectorTest.java

hashhar · 2021-03-04T06:09:20Z

plugin/trino-memsql/src/main/java/io/trino/plugin/memsql/MemSqlClient.java

static import for this and IS_DISTINCT_FROM.

I didn't want to static import these, because there can be multiple different ways of referring to joins and Client is not join-specific.
All am saying is that it was conscious, but still am open to suggestions

plugin/trino-base-jdbc/src/test/java/io/trino/plugin/jdbc/TestJdbcMetadataConfig.java

plugin/trino-base-jdbc/src/main/java/io/trino/plugin/jdbc/JdbcMetadata.java

findepi · 2021-03-04T10:33:38Z

I changed the // varchar inequality along with an equality, which constitutes an equi-condition and allows filter to remain as part of the Join test case so that it exploits case sensitivity issues, if connectors pushes down to much
it now uses nation and nation_lowercase tables for this.

findepi · 2021-03-04T21:06:30Z

@losipiuk @hashhar @kokosing PTAL

losipiuk · 2021-03-05T10:55:14Z

good to go

findepi · 2021-03-05T11:00:20Z

good to go

these are reviewed over here: #7145
and yes, i am gonna merge that

losipiuk

Nice!

kokosing

ack.

rsaw4 · 2021-07-06T05:14:17Z

sqlserver join pushdown across dbs blocked by #4690

cla-bot bot added the cla-signed label Feb 11, 2021

findepi mentioned this pull request Feb 11, 2021

Determine automatically if push join to table scan #6818

Closed

findepi force-pushed the findepi/join-jdbc branch 2 times, most recently from 15a633f to 90a814d Compare February 11, 2021 22:04

findepi force-pushed the findepi/join-jdbc branch 3 times, most recently from 6dcb5e7 to b912651 Compare February 19, 2021 16:49

findepi mentioned this pull request Feb 19, 2021

Validate no dangling columns in TableScanNode.enforcedConstraint #6963

Closed

findepi force-pushed the findepi/join-jdbc branch from 52e8f42 to 0cec252 Compare February 19, 2021 21:43

findepi requested review from kokosing and losipiuk February 19, 2021 21:44

findepi force-pushed the findepi/join-jdbc branch 2 times, most recently from 9309fe0 to 6f64b48 Compare February 20, 2021 17:23

findepi marked this pull request as ready for review February 20, 2021 23:23

losipiuk reviewed Feb 22, 2021

View reviewed changes

plugin/trino-base-jdbc/src/main/java/io/trino/plugin/jdbc/JdbcMetadata.java Outdated Show resolved Hide resolved

losipiuk reviewed Feb 22, 2021

View reviewed changes

plugin/trino-memsql/src/main/java/io/trino/plugin/memsql/MemSqlClient.java Outdated Show resolved Hide resolved

losipiuk approved these changes Feb 22, 2021

View reviewed changes

kokosing reviewed Feb 22, 2021

View reviewed changes

findepi force-pushed the findepi/join-jdbc branch 3 times, most recently from cc69438 to 0e3aa2c Compare March 3, 2021 10:53

findepi mentioned this pull request Mar 3, 2021

Introduce common connector test for JDBC connectors #7139

Merged

findepi force-pushed the findepi/join-jdbc branch from 0e3aa2c to 82975a1 Compare March 3, 2021 12:30

findepi force-pushed the findepi/join-jdbc branch 2 times, most recently from 9b632a2 to 6bfe97d Compare March 3, 2021 22:38

findepi mentioned this pull request Mar 3, 2021

Disable incorrect character pushdown in PostgreSQL #7145

Merged

findepi requested review from electrum and hashhar March 3, 2021 22:40

findepi requested review from kokosing and losipiuk March 3, 2021 22:40

findepi force-pushed the findepi/join-jdbc branch 2 times, most recently from 1cc7e39 to fcb1a5b Compare March 3, 2021 23:23

hashhar reviewed Mar 4, 2021

View reviewed changes

findepi force-pushed the findepi/join-jdbc branch 2 times, most recently from 5b692ff to 8b85913 Compare March 4, 2021 10:32

findepi force-pushed the findepi/join-jdbc branch from 8b85913 to 3167b44 Compare March 4, 2021 16:15

findepi mentioned this pull request Mar 4, 2021

Add reusable connector smoke test #7155

Merged

losipiuk approved these changes Mar 5, 2021

View reviewed changes

kokosing reviewed Mar 5, 2021

View reviewed changes

Implement Join pushdown for JDBC connectors

2dca5ff

findepi force-pushed the findepi/join-jdbc branch from 1d9ed57 to 2dca5ff Compare March 5, 2021 12:34

findepi added this to the 353 milestone Mar 5, 2021

findepi added the enhancement New feature or request label Mar 5, 2021

findepi merged commit d7c825c into trinodb:master Mar 5, 2021

findepi deleted the findepi/join-jdbc branch March 5, 2021 12:34

This was referenced Mar 5, 2021

Join pushdown #6620

Closed

Release notes for 353 #6858

Closed

FishermanZzhang mentioned this pull request Apr 2, 2021

Trino sql PushIntoTableScan cloudnativecube/octopus#56

Closed

vlad-lyutenko mentioned this pull request Oct 31, 2022

Add additional tests for outer join push downs #14841

Merged

Conversation

findepi commented Feb 11, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

kokosing left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

kokosing Feb 22, 2021

Choose a reason for hiding this comment

Uh oh!

findepi Mar 3, 2021

Choose a reason for hiding this comment

Uh oh!

Uh oh!

kokosing Feb 22, 2021

Choose a reason for hiding this comment

Uh oh!

findepi Mar 3, 2021

Choose a reason for hiding this comment

Uh oh!

Uh oh!

findepi commented Mar 3, 2021

Uh oh!

findepi commented Mar 3, 2021

Uh oh!

findepi commented Mar 3, 2021

Uh oh!

hashhar left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

hashhar Mar 4, 2021

Choose a reason for hiding this comment

Uh oh!

findepi Mar 4, 2021

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

findepi commented Mar 4, 2021

Uh oh!

findepi commented Mar 4, 2021

Uh oh!

losipiuk commented Mar 5, 2021

Uh oh!

findepi commented Mar 5, 2021

Uh oh!

losipiuk left a comment

Choose a reason for hiding this comment

Uh oh!

kokosing left a comment

Choose a reason for hiding this comment

Uh oh!

rsaw4 commented Jul 6, 2021

Uh oh!

Reviewers

Assignees

Labels

Milestone

Development

Uh oh!

5 participants

findepi commented Feb 11, 2021 •

edited

Loading