Conversation
electrum
force-pushed
the
hive-update
branch
2 times, most recently
from
ebcd5d2 to
5dc39aa
Compare
kokosing
left a comment
kokosing
left a comment
There was a problem hiding this comment.
Was the Hive update blocked by infra in FB?
There was a problem hiding this comment.
why did you use is_distinct_from and not is_equal_to? Is it due Maps need to be compared without regard to order.? Maps are equal if they have same entries and they are sorted in the same way?
There was a problem hiding this comment.
Equality for a map with null values will return null. I'm not sure if this is correct, but that's how it works today.
pom.xml
Outdated
There was a problem hiding this comment.
how is that possible that automation is green? Have you deployed SNAPSHOT?
There was a problem hiding this comment.
Yes, I deployed the snapshot so that I could test Travis before releasing it
There was a problem hiding this comment.
What is com.facebook.hive.orc.OrcConf? Why config no longer work?
There was a problem hiding this comment.
It is DWRF, Facebook's fork for ORC. The old HIVE_ORC_DEFAULT_COMPRESS doesn't exist in Hive 2.x. The config names could be set directly if I inlined the constants, but it's cleaner to use the OrcConf class which does exactly that.
|
There was no blocker, rather there was no reason to update. The primary reason to update now is to move to newer Parquet versions for Netflix Iceberg. I'm going to merge this tomorrow, then put up another one to move to Hive 3.1.1 which has the latest Parquet. |
|
What about updating hadoop? |
|
We need to wait for a new Hadoop release as all the recent ones are missing source jars due to HADOOP-15205 |
Maps need to be compared without regard to order.
Fix TestOrcBatchPageSourceMemoryTracking unit test failures Fork BloomFilter class from org.apache.hive.common.util.BloomFilter Fix presto-orc plugin unit test failures Cherry-pick of trinodb/trino#57 (trinodb/trino#57) and trinodb/trino#203 (trinodb/trino#203) Co-authored-by: David Phillips <david@acz.org>
Fix TestOrcBatchPageSourceMemoryTracking unit test failures Fork BloomFilter class from org.apache.hive.common.util.BloomFilter Fix presto-orc plugin unit test failures Cherry-pick of trinodb/trino#57 (trinodb/trino#57) and trinodb/trino#203 (trinodb/trino#203) Co-authored-by: David Phillips <david@acz.org>
…emas (trinodb#57) Co-authored-by: Vamsi <vamsi@onehouse.ai>
3 participants
Depends on trinodb/trino-hive-apache#1