-
Notifications
You must be signed in to change notification settings - Fork 29k
[SPARK-28552][SQL] Case-insensitive database URLs in JdbcDialect #25287
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Closed
Closed
Changes from 24 commits
Commits
Show all changes
46 commits
Select commit
Hold shift + click to select a range
02f5cf9
jdbc dialect can match non-lowercase URL prefixes
fa6b8a4
jdbc dialect matching is not case sensitive
2d52578
Merge branch 'master' of https://github.com/apache/spark into sql_dia…
c89c859
jdbc dialect matching is not case sensitive
0db9f75
jdbc dialect matching is not case sensitive
7591113
jdbc dialect matching is not case sensitive
7fc974e
jdbc dialect matching is not case sensitive
3b7a689
jdbc dialect matching is not case sensitive
0334680
jdbc dialect matching is not case sensitive
530741c
Merge branch 'master' of https://github.com/apache/spark into sql_dia…
5913568
Merge branch 'master' of https://github.com/apache/spark into sql_dia…
93027ca
jdbc dialect matching is not case sensitive
404bb34
jdbc dialect matching is not case sensitive
c27dfff
jdbc dialect matching is not case sensitive
acb42cf
jdbc dialect matching is not case sensitive
5c495aa
jdbc dialect matching is not case sensitive
fbb5b4d
dialect
a16a0b6
Merge branch 'sql_dialect' of https://github.com/teeyog/spark into sq…
76e80e7
jdbc dialect matching is not case sensitive
626d8a4
fix
28af8f9
Merge remote-tracking branch 'upstream/master' into sql_dialect
baef0d3
[SPARK-29653][SQL] Fix MICROS_PER_MONTH in IntervalUtils
yaooqinn 6c55e42
[SPARK-29120][SQL][TESTS] Port create_view.sql
maropu 6718c49
[SPARK-29668][DOCS] Deprecate Python 3 prior to version 3.6
dongjoon-hyun e808616
[SPARK-29646][BUILD] Allow pyspark version name format `${versionNumb…
jiangxb1987 c582128
[SPARK-29666][BUILD] Fix the publish release failure under dry-run mode
jiangxb1987 d3ab1b8
Prepare Spark release v3.0.0-preview-rc2
jiangxb1987 b587956
Revert "Prepare Spark release v3.0.0-preview-rc2"
jiangxb1987 48c662f
[SPARK-29277][SQL] Add early DSv2 filter and projection pushdown
rdblue bcfe98a
[SPARK-29651][SQL] Fix parsing of interval seconds fraction
MaxGekk c9155ac
[SPARK-29126][PYSPARK][DOC] Pandas Cogroup udf usage guide
d80tb7 d98fed4
Merge remote-tracking branch 'upstream/master'
d8acf7f
Merge branch 'master' of https://github.com/apache/spark
844c0aa
Merge branch 'master' of https://github.com/apache/spark
a5148f0
Merge branch 'master' into sql_dialect
teeyog 14814ef
fix conflicts
3e58b81
fix conflict
ac70da4
fix conflict
e08c95f
Merge branch 'master' of https://github.com/apache/spark
1f8b514
update
68cdff8
update
3e1585b
update
2dba4b1
Merge branch 'master' of https://github.com/apache/spark
0f5f776
update
6f855b1
update
1883847
Merge branch 'sql_dialect' of https://github.com/teeyog/spark into sq…
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -58,7 +58,7 @@ class JDBCSuite extends QueryTest | |
| } | ||
|
|
||
| val testH2DialectTinyInt = new JdbcDialect { | ||
| override def canHandle(url: String): Boolean = url.startsWith("jdbc:h2") | ||
| override def canHandle(url: String) : Boolean = url.startsWith("jdbc:h2") | ||
| override def getCatalystType( | ||
| sqlType: Int, | ||
| typeName: String, | ||
|
|
@@ -1662,4 +1662,50 @@ class JDBCSuite extends QueryTest | |
| "Invalid value `test` for parameter `isolationLevel`. This can be " + | ||
| "`NONE`, `READ_UNCOMMITTED`, `READ_COMMITTED`, `REPEATABLE_READ` or `SERIALIZABLE`.")) | ||
| } | ||
|
|
||
| test("SPARK-28552: Check whether a dialect instance can be applied on the given jdbc url") { | ||
|
||
| var dialects = List[JdbcDialect]() | ||
|
|
||
| def registerDialect(dialect: JdbcDialect) : Unit = { | ||
| dialects = dialect :: dialects.filterNot(_ == dialect) | ||
| } | ||
|
|
||
| registerDialect(MySQLDialect) | ||
| registerDialect(PostgresDialect) | ||
| registerDialect(DB2Dialect) | ||
| registerDialect(MsSqlServerDialect) | ||
| registerDialect(DerbyDialect) | ||
| registerDialect(OracleDialect) | ||
| registerDialect(TeradataDialect) | ||
|
|
||
| def get(url: String): JdbcDialect = { | ||
| val matchingDialects = dialects.filter(_.canHandle(url)) | ||
| matchingDialects.length match { | ||
| case 0 => NoopDialect | ||
| case 1 => matchingDialects.head | ||
| case _ => new AggregatedDialect(matchingDialects) | ||
|
||
| } | ||
| } | ||
|
|
||
| assert(get("jdbc:mysql://localhost/db") == MySQLDialect) | ||
| assert(get("jdbc:MySQL://localhost/db") == MySQLDialect) | ||
|
|
||
| assert(get("jdbc:postgresql://localhost/db") == PostgresDialect) | ||
| assert(get("jdbc:postGresql://localhost/db") == PostgresDialect) | ||
|
|
||
| assert(get("jdbc:db2://localhost/db") == DB2Dialect) | ||
| assert(get("jdbc:DB2://localhost/db") == DB2Dialect) | ||
|
|
||
| assert(get("jdbc:sqlserver://localhost/db") == MsSqlServerDialect) | ||
| assert(get("jdbc:sqlServer://localhost/db") == MsSqlServerDialect) | ||
|
|
||
| assert(get("jdbc:derby://localhost/db") == DerbyDialect) | ||
| assert(get("jdbc:derBy://localhost/db") == DerbyDialect) | ||
|
|
||
| assert(get("jdbc:oracle://localhost/db") == OracleDialect) | ||
| assert(get("jdbc:Oracle://localhost/db") == OracleDialect) | ||
|
|
||
| assert(get("jdbc:teradata://localhost/db") == TeradataDialect) | ||
| assert(get("jdbc:Teradata://localhost/db") == TeradataDialect) | ||
| } | ||
| } | ||
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: revert this.