Consolidate iceberg native-mode into the catalog type#17233
Consolidate iceberg native-mode into the catalog type#17233zhenxiao merged 1 commit intoprestodb:masterfrom
Conversation
There was a problem hiding this comment.
Nice catch! I refactored the if/else if statements to switch.
There was a problem hiding this comment.
might be out of scope of this PR, I'm thinking make this getIcebergTable logic to be a utility method or in polymorphism
There was a problem hiding this comment.
Agree! I created a utility function to get the iceberg table.
Actually there are some other places such as IcebergMetadata and IcebergNativeMetadata where we could do some optimization by creating new abstraction layers. As this PR just focuses on merging the native-mode config into the catalog type, I'll send some follow-up PRs for these new abstraction layers.
a003ade to
a956976
Compare
a956976 to
52aa772
Compare
zhenxiao
left a comment
There was a problem hiding this comment.
looks good @ChunxuTang could you please add release notes? This PR changes the Iceberg config, removes iceberg.native-mode, let's make corresponding doc changes
As when the iceberg catalog type is
HADOOP, the connector won't depend on the Hive metastore anymore, aka, in the native mode, we can safely remove the native mode from the iceberg connector config and only use the catalog type to determine whether it's in the native mode.If the catalog type is
HIVE, we use Hive metastore and it's in the non-native mode.If the catalog type is
HADOOP, we use HDFS to store both data and metadata and it's in the native mode.Test plan - Unit test and integration test