Handle NaN when pushing filter for JDBC connector#21923
Handle NaN when pushing filter for JDBC connector#21923Praveen2112 wants to merge 3 commits intotrinodb:masterfrom
Conversation
Praveen2112
force-pushed
the
praveen/nan_for_pg
branch
5 times, most recently
from
8e50017 to
ded8441
Compare
Praveen2112
force-pushed
the
praveen/nan_for_pg
branch
from
ded8441 to
4391b08
Compare
|
cc: @findepi I attempted a fix for it Please share your feedback for the same. |
plugin/trino-postgresql/src/main/java/io/trino/plugin/postgresql/PostgreSqlClientModule.java
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
I think we don't need this check of other operators as they work as expected - in case of Postgres NaN is placed about Infinity so we need to add them only for > or >= operator.
There was a problem hiding this comment.
this should be a code comment.
when you write it down say "in PostgreSQL ..." in a generic class, you'll realize that you're exploiting a (common) implementation detail.
- what does the spec say about double and NaN ordering? (i think the spec omits existence of NaNs, but i may be wrong)
- even if the spec said something definitive, implementation vary (for example, Trino and PostgreSQL have different behavior). Which means the behavior should be implemented in connector-specific manner.
There was a problem hiding this comment.
'NaN' as a varchar?
are you sure this is portable SQL?
There was a problem hiding this comment.
From the tests 'NaN' is not parsed as a Varchar but considered as a NaN representation.
There was a problem hiding this comment.
Since it is used along with double/real columns I think it is coerced automatically and the tests also proves the same.
There was a problem hiding this comment.
This may work for PostgreSQL and may also work for some other databases, but I doubt this to be a really portable behavior (for example, Trino does not support comparing double values with 'NaN' literal, does it?) and this class is meant to be portable.
plugin/trino-base-jdbc/src/test/java/io/trino/plugin/jdbc/TestJdbcConnectorTest.java
Outdated
Show resolved
Hide resolved
plugin/trino-base-jdbc/src/test/java/io/trino/plugin/jdbc/TestingH2JdbcClient.java
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
looks wrong. why is it OK to skip this test for mysql?
There was a problem hiding this comment.
braces around left < right are redundant
braces around whole replaced expression are also redundant
(braces are inserted automatically by rewrite engine)
There was a problem hiding this comment.
what about other comparison operators?
plugin/trino-sqlserver/src/test/java/io/trino/plugin/sqlserver/BaseSqlServerConnectorTest.java
Outdated
Show resolved
Hide resolved
Praveen2112
force-pushed
the
praveen/nan_for_pg
branch
8 times, most recently
from
1df9d6e to
0e5acfc
Compare
Praveen2112
changed the title
Praveen2112
force-pushed
the
praveen/nan_for_pg
branch
2 times, most recently
from
22a1cf4 to
86a9211
Compare
plugin/trino-ignite/src/main/java/io/trino/plugin/ignite/IgniteClient.java
Outdated
Show resolved
Hide resolved
plugin/trino-base-jdbc/src/main/java/io/trino/plugin/jdbc/NaNSpecificQueryBuilder.java
Outdated
Show resolved
Hide resolved
Praveen2112
force-pushed
the
praveen/nan_for_pg
branch
from
86a9211 to
d3db6cc
Compare
For equals and not equals rewrite we capture them as GenericRewrite expression and usage of RewriteComparison rule is redundant here.
Some JDBC based sources like PotgreSql, Oracle, Ignite etc treats NaN values as equal, and greater than all non-NaN values. This is different from Trino's behaviour where NaN values are not equal, and they are not greater than or lesser than non-NaN values which results in in-correct results when they are pushed down to the underlying datasource. So we push down additional condition to ensure that NaN values are not considered in comparison operations.
Praveen2112
force-pushed
the
praveen/nan_for_pg
branch
from
d3db6cc to
1caffaf
Compare
|
@findepi Thanks for the review. AC |
|
First two commits lgtm. You can merge them. |
|
This pull request has gone a while without any activity. Tagging the Trino developer relations team: @bitsondatadev @colebow @mosabua |
|
Closing this pull request, as it has been stale for six weeks. Feel free to re-open at any time. |
|
@Praveen2112 @findepi how do you want to proceed on this. Looks like at least part of the PR is approved .. maybe update it all or cherrypick some commits into a separate PR and merge that? |
|
This pull request has gone a while without any activity. Tagging the Trino developer relations team: @bitsondatadev @colebow @mosabua |
|
Closing this pull request, as it has been stale for six weeks. Feel free to re-open at any time. |
3 participants
Description
Fixes #21922
Some JDBC based sources like PotgreSql, Oracle, Ignite etc treats NaN values as equal,
and greater than all non-NaN values. This is different from Trino's behaviour where NaN values
are not equal, and they are not greater than or lesser than non-NaN values which results in in-correct
results when they are pushed down to the underlying datasource. So we push down additional condition
to ensure that NaN values are not considered in comparison operations.
Additional context and related issues
Release notes
( ) This is not user-visible or is docs only, and no release notes are required.
( ) Release notes are required. Please propose a release note for me.
( ) Release notes are required, with the following suggested text: