Skip to content

Conversation

@xiaozhch5
Copy link
Contributor

Tips

What is the purpose of the pull request

Solve the problem of "Caused by: java.lang.ClassNotFoundException: org.apache.hudi.org.apache.hadoop.hive.metastore.api.NoSuchObjectException
" in hive 3.

Brief change log

add dependency

        <dependency>
          <groupId>org.apache.hive</groupId>
          <artifactId>hive-standalone-metastore</artifactId>
          <version>${hive.version}</version>
          <scope>${flink.bundle.hive.scope}</scope>
        </dependency>

to hudi-flink-bundle module

Verify this pull request

(Please pick either of the following options)

This pull request is already covered by existing tests, such as (please describe tests).

I compile the source code of hudi with hive 3, and succeed in synchronizing the data to hive;

The test tutorial:

CREATE TABLE sourceT (
  uuid varchar(20),
  name varchar(10),
  age int,
  ts timestamp(3),
  `partition` varchar(20)
) WITH (
  'connector' = 'datagen',
  'rows-per-second' = '1'
);

create table t2(
  uuid varchar(20),
  name varchar(10),
  age int,
  ts timestamp(3),
  `partition` varchar(20)
)
with (
  'connector' = 'hudi',
  'path' = 'hdfs://host146:8020/hudi/t4', -- $HUDI_DEMO 替换成的绝对路径
  'table.type' = 'MERGE_ON_READ',
  'write.bucket_assign.tasks' = '2',
  'write.tasks' = '2',
  'hive_sync.enable' = 'true',
  'hive_sync.mode' = 'hms',
  'hive_sync.metastore.uris' = 'thrift://host145:9083' -- ip 替换成 HMS 的地址
);

insert into t2 select * from sourceT;

Test result:

image

Committer checklist

  • Has a corresponding JIRA in PR title & commit

  • [✔ ] Commit message is descriptive of the change

  • CI is green

  • Necessary doc changes done or have another open PR

  • For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.

@xiaozhch5 xiaozhch5 changed the title Add hive-standalone-metastore dependency to hudi-flink-bundle module [MINOR] Add hive-standalone-metastore dependency to hudi-flink-bundle module Feb 22, 2022
@hudi-bot
Copy link
Collaborator

CI report:

Bot commands @hudi-bot supports the following commands:
  • @hudi-bot run azure re-run the last Azure build

Copy link
Contributor

@danny0405 danny0405 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, thanks for the contribution ~

@danny0405 danny0405 merged commit 01cbdde into apache:master Feb 23, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants