Flink: Fail if both Catalog type and catalog-impl are configured and add TestFlinkCatalogFactory tests #3308
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
rdblue
reviewed
Oct 18, 2021
| if (catalogImpl != null) { | ||
| String catalogType = properties.get(ICEBERG_CATALOG_TYPE); | ||
| Preconditions.checkArgument(catalogType == null, | ||
| "Cannot create catalog %s, both catalog-type and catalog-impl are set: catalog-type=%s, catalog-impl=%s", |
Contributor
There was a problem hiding this comment.
catalog-type isn't a property right? Shouldn't this by type=%s instead?
Contributor
There was a problem hiding this comment.
Nevermind, looks like Flink sets this differently.
Contributor
There was a problem hiding this comment.
Do we have recommendations on which values to set that we can provide?
Member
Author
There was a problem hiding this comment.
Do we have recommendations on which values to set that we can provide?
You mean only hive or hadoop? 🤔
rdblue
reviewed
Oct 18, 2021
kbendick
reviewed
Oct 19, 2021
flink/src/test/java/org/apache/iceberg/flink/TestFlinkCatalogFactory.java
Show resolved
Hide resolved
flink/src/test/java/org/apache/iceberg/flink/TestFlinkCatalogFactory.java
Outdated
Show resolved
Hide resolved
| if (catalogImpl != null) { | ||
| String catalogType = properties.get(ICEBERG_CATALOG_TYPE); | ||
| Preconditions.checkArgument(catalogType == null, | ||
| "Cannot create catalog %s, both catalog-type and catalog-impl are set: catalog-type=%s, catalog-impl=%s", |
Contributor
There was a problem hiding this comment.
Do we have recommendations on which values to set that we can provide?
omarsmak
force-pushed
the
flink-throw-exception
branch
from
5b913ea to
2e9cc9c
Compare
Member
Author
rdblue
reviewed
Oct 19, 2021
flink/src/main/java/org/apache/iceberg/flink/FlinkCatalogFactory.java
Outdated
Show resolved
Hide resolved
rdblue
reviewed
Oct 19, 2021
rdblue
reviewed
Oct 19, 2021
rdblue
reviewed
Oct 19, 2021
nastra
reviewed
Oct 19, 2021
flink/src/test/java/org/apache/iceberg/flink/TestFlinkCatalogFactory.java
Outdated
Show resolved
Hide resolved
flink/src/test/java/org/apache/iceberg/flink/TestFlinkCatalogFactory.java
Outdated
Show resolved
Hide resolved
flink/src/test/java/org/apache/iceberg/flink/TestFlinkCatalogFactory.java
Outdated
Show resolved
Hide resolved
…add TestFlinkCatalogFactory tests
omarsmak
force-pushed
the
flink-throw-exception
branch
from
2e9cc9c to
6024d55
Compare
Member
Author
nastra
approved these changes
Oct 19, 2021
rdblue
approved these changes
Oct 19, 2021
Contributor
|
Thanks, @omarsmak! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is related to #3162 where if the user tries to set both,
catalog-implandcatalog-type, it will fail with an exception.We have this behavior already in Spark and Hive integrations due to #3162. This PR brings the same behavior to Flink integration as well.
cc: @rdblue @nastra