ANALYZE command fails with Hive Server #39
Comments
gabbasb
added a commit
to gabbasb/hdfs_fdw
that referenced
this issue
Aug 22, 2017
Problem Statement: The HDFS_FDW was inferring the authentication type from the options of the CREATE USER MAPPING command If it found that the username is empty, it concluded NOSASL otherwise it concluded LDAP. In case of NOSASL since the username was empty while connecting to the hive server the FDW was specifying an arbitrary username. While this technique worked for the rest of the cases, it failed in case of ANALYZE. ANALYZE needed an OS user that had write access to the folder /tmp/hadoop-yarn Solution: Add another option in CREATE SERVER command, called the auth_type. This option, which can be omitted, can have the following values: NOSASL & LDAP When specified a non empty username must be specified while creating the user mapping. When omitted the FDW tries to infer the auth_type from the options of the user mapping, like it used to do. Hence the FDW is backwards compatible and existing test cases do not need any modification. If auth_type is specified and a valid username is specified that has write access to the folder /tmp/hadoop-yarn, then ANALYZE works fine. For Example: postgres=# CREATE EXTENSION hdfs_fdw; CREATE EXTENSION postgres=# CREATE SERVER hdfs_svr FOREIGN DATA WRAPPER hdfs_fdw OPTIONS (host '127.0.0.1',port '10000',client_type 'hiveserver2', auth_type 'NOSASL'); CREATE SERVER postgres=# CREATE USER MAPPING FOR abbasbutt server hdfs_svr OPTIONS (username 'abbasbutt', password ''); CREATE USER MAPPING postgres=# CREATE FOREIGN TABLE fnt( a int, name varchar(255)) SERVER hdfs_svr OPTIONS (dbname 'testdb', table_name 'names_tab'); CREATE FOREIGN TABLE postgres=# SELECT * FROM fnt; a | name ---+------- 1 | abcd 2 | pqrs 3 | wxyz 4 | a_b_c 5 | p_q_r | 1 | abcd 2 | pqrs 3 | wxyz 4 | a_b_c 5 | p_q_r | (12 rows) postgres=# ANALYZE fnt; ANALYZE
ibrarahmad
added a commit
that referenced
this issue
Aug 22, 2017
Issue - (#39) - Added new server parameter auth_type.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
ANALYZE jobhist;
ERROR: failed to fetch execute query: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
ANALYZE emp(empno);
ERROR: failed to fetch execute query: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
VACUUM ANALYZE emp;
The text was updated successfully, but these errors were encountered: