Support adding fields to nested Parquet structs#40

Open

billonahill wants to merge 134 commits intotwitter-forks:twitter-masterfrom

billonahill:billg/IQ-201_support_adding_to_structs

Commits on Sep 25, 2015

Ignore if the partitions have no corresponfing hdfs valid path.
Sailesh Mittal
committed

Commits on Oct 1, 2015

Append nulls for missing values in Parquet.
Sailesh Mittal
committed
Merge pull request #1 from twitter-forks/twtr-filenotfound-bypass
saileshmittal
committed
Merge pull request #2 from twitter-forks/appendNullsForMissingParquetColumns
saileshmittal
committed
[maven-release-plugin] prepare for tw-0.1 release.
Sailesh Mittal
committed
[maven-release-plugin] prepare for release 0.119-tw-0.1.
Sailesh Mittal
committed

Commits on Oct 2, 2015

Merge branch 'master' into twitter-master
Sailesh Mittal
committed
Compare types ignoring cases. This is not complete solution and should be reverted once #2863 is resolved.
Sailesh Mittal
committed
Merge branch 'master' into twitter-master
Sailesh Mittal
committed
Merge branch 'twitter-master' into case-insensitive-types
Sailesh Mittal
committed
Merge pull request #3 from twitter-forks/case-insensitive-types
saileshmittal
committed

Commits on Oct 9, 2015

Revert "Reduce allocation of hadoop configuration objects"
Sailesh Mittal
committed
Merge branch 'master' into twitter-master
Sailesh Mittal
committed

Commits on Oct 13, 2015

Bump up zookeeper version to 3.4.6
Sailesh Mittal
committed
Read metastore info from zookeeper.
Sailesh Mittal
committed

Commits on Oct 14, 2015

Add some tests and fix injection issue.
Sailesh Mittal
committed
Remove Created by comments added by Idea.
Sailesh Mittal
committed
Merge pull request #5 from twitter-forks/zookeeperBasedMetastore
saileshmittal
committed

Commits on Oct 21, 2015

Merge branch 'master' into twitter-master
Sailesh Mittal
committed

Commits on Oct 22, 2015

Fix some warnings after merging latest master.
Sailesh Mittal
committed

Commits on Oct 23, 2015

Create ParquetHiveRecordCursor as user, so that hdfs reads happen as that user.
Sailesh Mittal
committed

Commits on Oct 26, 2015

Propagate user session information throughout hive metastore.
Sailesh Mittal
committed
Revert 972d5d166bc8174c9cef50b1066f6efc9a07a02b
Sailesh Mittal
committed
Load splits as user so that hdfs reads happen as that user.
Sailesh Mittal
committed
UserBasedHiveMetastore to access metastore as user issuing the query.
Sailesh Mittal
committed
Merge pull request #6 from twitter-forks/run-queries-as-user
saileshmittal
committed
Import missed import.
Sailesh Mittal
committed
Use standard charset for UTF-8 instead of string.
Sailesh Mittal
committed

Commits on Nov 3, 2015

Commits on Nov 23, 2015

Revert "Propagate user session information throughout hive metastore."
Sailesh Mittal
committed
Revert "Load splits as user so that hdfs reads happen as that user."
Sailesh Mittal
committed
Revert "UserBasedHiveMetastore to access metastore as user issuing the query."
Sailesh Mittal
committed
Merge pull request #8 from twitter-forks/revert-user-based-metastore
saileshmittal
committed
Merge tag '0.126' into twitter-master
Sailesh Mittal
committed
Prepare for release 0.126-tw-0.7.
Sailesh Mittal
committed

Commits on Dec 8, 2015

HiveSplitManager to start HiveSplitLoader as session user
Bill Graham
committed
fixing indentation
Bill Graham
committed
fixing indentation
Bill Graham
committed
Merge pull request #9 from billonahill/billg/IQ-23_runAsUser
saileshmittal
committed

Commits on Dec 9, 2015

Updating version from to 0.126-tw-0.8
Bill Graham
committed

Commits on Dec 17, 2015

Revert runAsUser in preparation for 0.130 upgrade
Bill Graham
committed
Changing pom version to 0.130 to avoid merge conflicts when merging in 0.130
Bill Graham
committed
Merge 0.130 from master
Bill Graham
committed
Prepare for release 0.130-tw-0.9
Bill Graham
committed
Merge pull request #10 from billonahill/billg/revert_run_as
saileshmittal
committed
Remove empty line to fix stylecheck error
Bill Graham
committed
Merge pull request #11 from billonahill/upgrade_to_130
Bill Graham
committed

Commits on Dec 23, 2015

Allow Hive queries to read as the querying user
Bill Graham
committed
add links to jiras
Bill Graham
committed
Clean up comments and constant order
Bill Graham
committed

Commits on Jan 4, 2016

Merge pull request #12 from billonahill/billg/IQ-39_hive_run_as_user
Bill Graham
committed
Upgrading to 0.130-tw-0.10
Bill Graham
committed
Updating unit tests for ReadAsQueryUser flag
Bill Graham
committed
Merge pull request #13 from billonahill/billg/IQ-39_hive_run_as_user_tests
Bill Graham
committed

Commits on Jan 6, 2016

Avoid NPE when ZK changed state is related to connection change.
Sailesh Mittal
committed
Merge pull request #14 from twitter-forks/IQ-96_zk_metastore_NPE
Bill Graham
committed

Commits on Jan 12, 2016

Cache the UGI object per user in BackgroundHiveSplitLoader
Bill Graham
committed
Merge pull request #15 from billonahill/billg/IQ-93_hive_ugi_leak
Bill Graham
committed
Upgrading version to 0.130-tw-0.11
Bill Graham
committed
Merge pull request #16 from billonahill/upgrade_to_130_11
Bill Graham
committed

Commits on Jan 13, 2016

Change version to 0.132.
Sailesh Mittal
committed
Merge tag '0.132' into upgrade_to_0.132_12
Sailesh Mittal
committed
Change version to 0.132-tw-0.12.
Sailesh Mittal
committed

Commits on Jan 14, 2016

Merge pull request #17 from saileshmittal/upgrade_to_0.132_12
Bill Graham
committed

Commits on Jan 15, 2016

HivePageSourceProvider to support query as user from workers
Bill Graham
committed
Merge pull request #18 from billonahill/billg/IQ-122_presto_tasks_as_user
Bill Graham
committed
Upgrading presto to 0.132-tw-13
Bill Graham
committed
Merge pull request #19 from billonahill/upgrade_to_132_13
saileshmittal
committed

Commits on Jan 31, 2016

Revert PR#3, case insensitive types, since it is now supported.
Sailesh Mittal
committed

Commits on Feb 3, 2016

Update to use UGI.createProxyUser instead of createRemoteUser
Bill Graham
committed
Merge pull request #21 from billonahill/billg/IQ-85_fix_run_as
Bill Graham
committed
Upgrade to 0.132-tw-0.14
Bill Graham
committed
Merge pull request #22 from billonahill/upgrade_to_132_14
Bill Graham
committed

Commits on Feb 4, 2016

Whitelist zeppelin temporarily
Bill Graham
committed
Merge pull request #23 from billonahill/billg/IQ-152_zeppelin_whitelist
Bill Graham
committed
Upgrade to 0.132-tw-0.15
Bill Graham
committed
Merge pull request #24 from billonahill/upgrade_to_132_15
Bill Graham
committed

Commits on Feb 8, 2016

Merge pull request #20 from saileshmittal/revert-pr-3
Bill Graham
committed

Commits on Feb 11, 2016

Rolling back Zeppelin auth hack, add some logging
Bill Graham
committed
Merge pull request #25 from billonahill/billg/IQ-152_roll_back_zeppelin_hack
Bill Graham
committed
Upgrading to 0.132-tw-0.16
Bill Graham
committed
Merge pull request #26 from billonahill/upgrade_to_132_16
Bill Graham
committed

Commits on Feb 24, 2016

Change version to 0.139 to not conflict with merge
Bill Graham
committed
merge from 0.139
Bill Graham
committed
upgrade pom version to 0.139-tw-0.17
Bill Graham
committed
Merge pull request #27 from billonahill/upgrade_to_139_17
Bill Graham
committed
Adding query logging to presto queries
Bill Graham
committed

Commits on Mar 1, 2016

Look up parquet columns by name case-insensitive
Bill Graham
committed

Commits on Mar 2, 2016

Merge pull request #31 from billonahill/billg/case_insensitive_parquet_column_match
Bill Graham
committed
Upgrading to 0.139-tw-0.19
Bill Graham
committed
Handle hive keywords when doing a name-based parquet field lookup
Bill Graham
committed
Merge branch 'billg/keyword_aware_parquet_column_match' into upgrade_to_139_19
Bill Graham
committed
Refactor to make diff cleaner
Bill Graham
committed
Merge branch 'billg/keyword_aware_parquet_column_match' into upgrade_to_139_19
Bill Graham
committed
Merge pull request #32 from billonahill/billg/keyword_aware_parquet_column_match
Bill Graham
committed
Merge branch 'twitter-master' into upgrade_to_139_19
Bill Graham
committed
Merge pull request #33 from billonahill/upgrade_to_139_19
Bill Graham
committed

Commits on Mar 11, 2016

Upgrade to 0.141.
Sailesh Mittal
committed
Merge tag '0.141' into upgrade_to_141_20
Sailesh Mittal
committed
Upgrade to 0.141-tw-0.20.
Sailesh Mittal
committed
Merge pull request #34 from saileshmittal/upgrade_to_141_20
Bill Graham
committed

Commits on Mar 24, 2016

Use modules and query events for logging
Bill Graham
committed
Add javadocs
Bill Graham
committed
Add javadocs
Bill Graham
committed
fix imports
Bill Graham
committed
add splits, rows and bytes
Bill Graham
committed
Change to use QueryComplete
Bill Graham
committed
Merge pull request #36 from billonahill/billg/IQ-192_refactor_logging
Bill Graham
committed

Commits on Mar 29, 2016

Add event process and handlers and scribe query completion events
Bill Graham
committed
code clean-up
Bill Graham
committed
Fix package leakage
Bill Graham
committed
Move twitter classes to a twitter package
Bill Graham
committed
Default package is presto and thrift is custom
Bill Graham
committed

Commits on Mar 30, 2016

Merge pull request #38 from billonahill/billg/twitter_specific_cleanup
Bill Graham
committed
Removing test from scribe category
Bill Graham
committed
Merge pull request #37 from billonahill/billg/scribe_query_complete_events
Bill Graham
committed
change version to avoid merge conflicts
Bill Graham
committed
Merge from 0.143
Bill Graham
committed
Update version to 0.143-tw-21
Bill Graham
committed
Fixed to not use illegal import
Bill Graham
committed
Reduce logging of unsupported HttpRequest events
Bill Graham
committed
Merge pull request #39 from billonahill/upgrade_to_143_21
Bill Graham
committed
Fix version from 0.143-tw-21 to 0.143-tw-0.21
Bill Graham
committed

Commits on Mar 31, 2016

Fix name of log category
Bill Graham
committed

Commits on Apr 4, 2016

Support adding fields to nested Parquet structs
Bill Graham
committed

Commits on Apr 5, 2016

Fix table partition typo
Bill Graham
committed
Fix table partition typo
Bill Graham
committed
Refactored type comparison to use HiveType
Bill Graham
committed