Core: Add view support for JDBC catalog#9487
Conversation
|
Few notes about this PR:
|
|
@ajantha-bhat @nk1506 if you guys want to take a look :) Thanks ! |
ajantha-bhat
left a comment
There was a problem hiding this comment.
Great work. Thanks for the contributions.
Some minor comments about public interfaces. Almost look good to me.
core/src/main/java/org/apache/iceberg/jdbc/JdbcViewOperations.java
Outdated
Show resolved
Hide resolved
core/src/main/java/org/apache/iceberg/jdbc/JdbcViewOperations.java
Outdated
Show resolved
Hide resolved
|
Tagging @amogh-jahagirdar, as I feel this PR can help in reducing the changes at Trino side for REST catalog view support (trinodb/trino#19818 (comment)) |
989b15a to
ac2d4c1
Compare
|
@ajantha-bhat I should have addressed your comments 😄 |
ajantha-bhat
left a comment
There was a problem hiding this comment.
Looks much cleaner now. Almost there.
core/src/test/java/org/apache/iceberg/jdbc/TestJdbcViewCatalog.java
Outdated
Show resolved
Hide resolved
core/src/test/java/org/apache/iceberg/jdbc/TestJdbcViewCatalog.java
Outdated
Show resolved
Hide resolved
core/src/test/java/org/apache/iceberg/jdbc/TestJdbcViewCatalog.java
Outdated
Show resolved
Hide resolved
core/src/test/java/org/apache/iceberg/jdbc/TestJdbcViewCatalog.java
Outdated
Show resolved
Hide resolved
core/src/test/java/org/apache/iceberg/jdbc/TestJdbcViewCatalog.java
Outdated
Show resolved
Hide resolved
nastra
left a comment
There was a problem hiding this comment.
overall this looks almost ready, I've added a few comments but nothing major
|
@nastra thanks ! I addressed your comments. The PR is ready for a new round 😄 Thanks again ! |
core/src/test/java/org/apache/iceberg/jdbc/TestJdbcCatalog.java
Outdated
Show resolved
Hide resolved
danielcweeks
left a comment
There was a problem hiding this comment.
This is looking much closer. The main issue I see is that a lot of the differentiation between the original and new schemas are referenced as "old" and "new", which is very contextual to this particular change.
I think we need to say the original schmea and related queries are V0 and this set of changes is V1 to enable future changes.
This would mean that we don't rely on checks like:
if (isNewSchema) ? newBehavior : oldBehavior
which will be difficult to maintain going forward.
Ideally, we would have somethimg more like:
switch(schemaVersion) {
case V0: ...
case V1: ...
}
core/src/main/java/org/apache/iceberg/jdbc/JdbcTableOperations.java
Outdated
Show resolved
Hide resolved
|
@danielcweeks Thanks ! Yeah, I assumed we won't have new schema update, but you are right, it's possible to have new updates in the future. So let me update the PR with schema versioning. Thanks ! |
|
@nastra @ajantha-bhat @danielcweeks @rdblue I updated the PR by introducing
|
core/src/main/java/org/apache/iceberg/jdbc/JdbcTableOperations.java
Outdated
Show resolved
Hide resolved
core/src/test/java/org/apache/iceberg/jdbc/TestJdbcCatalog.java
Outdated
Show resolved
Hide resolved
nastra
left a comment
There was a problem hiding this comment.
overall this LGTM and thanks @jbonofre for working on this. I'm ok using using if with V0/ V1 for now, but obviously that won't scale in the long run, so we would need some kind of wrapper class that contains all the SQLs for a particular schema version. We could then instantiate this schema class for the given schema version and make the code easier to read and maintain. However, I don't think that this is a 1.5.0 blocker, so it should be ok to ship this with the next release
|
@nastra yes, actually, I plan to improve Thanks ! |
|
Thanks for working on this @jbonofre. Thanks for reviewing it @danielcweeks, @nastra, @rdblue, @amogh-jahagirdar. I think we also need to keep the PR ready for Trino and pyiceberg. I will start working on 1.5.0 release and try to have an RC by Monday. |
This PR adds view support to the JDBC catalog:
JdbcCatalogextendsBaseMetastoreViewCatalogJdbcViewOperationsJdbcUtilrefactoring to deal with SQL statements common to table and viewTestJdbcViewCatalogby extendingViewCatalogTestsClose #8697