Hudi connector #7
Conversation
vinothchandar
left a comment
vinothchandar
left a comment
There was a problem hiding this comment.
The fast path and the core logic looks good to me. want to do an interactive session to go over any simplifications, understand some threading model/design better. Great job!
There was a problem hiding this comment.
revert this and get rid of the whitespace changes?
plugin/trino-hudi/pom.xml
Outdated
There was a problem hiding this comment.
We need hudi-hive-sync? for querying?
There was a problem hiding this comment.
this wrapping. bit odd to me. but seems like the convention.
There was a problem hiding this comment.
rename hudiBaseFileFormat and lets use this terminology consistently everywhere? including HudiConfig
There was a problem hiding this comment.
how much of this class is code reused from HiveSplitBackgroundLoader
plugin/trino-hudi/pom.xml
Outdated
There was a problem hiding this comment.
can we do the same trimming around hbase/parquet-avro - we did for Presto, here as well?
There was a problem hiding this comment.
whats the life cycle of this object? Once per query?
There was a problem hiding this comment.
Modelled after an existing HiveSplit?
There was a problem hiding this comment.
Can you point me to where the file pruning happens i.e filtering out files that don't match the filter ?
codope
force-pushed
the
hudi-plugin-cleanup
branch
from
e86685b to
c988546
Compare
Rebase and resolve conflicts Use cached thread pool for split generation
codope
force-pushed
the
hudi-plugin-cleanup
branch
from
c988546 to
e440e49
Compare
Class view in RFC.