Core: HadoopTables support dynamic FileIO #4459
Conversation
|
cc @jackye1995 |
| this(new Configuration()); | ||
| } | ||
|
|
||
| public HadoopTables(FileIO fileIO) { |
There was a problem hiding this comment.
Could you please explain why do we need the other FileIO rather than initializing the HadoopFileIO by default ?
There was a problem hiding this comment.
Thanks for your review, My FileSystem which has some external function such as authentication is not a standard implementation. In other cases, Many Engines has owner FileSystem Manager so they don't need HadoopTables to initialize FileSystem.
There was a problem hiding this comment.
If the FileSystem is not a standard HDFS implementation, then I will think the table is not a stand hadoop table. So why not just define the customized catalog & table implementation ?
There was a problem hiding this comment.
In the one hand, The only difference is the FileIO. The constructor of HadoopTableOperations is protected and constructor args has FileIO, So the design philosophy of HadoopTableOperations is support Dynamic FileIO.
In other hand, like #1618
2 participants
HadoopTables support dynamic FileIO