Skip to content

[WIP] Support Hadoop 3.x Hive 3.x and Spark 3.2.x default #5402

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
rahil-c wants to merge 85 commits into from
Closed

[WIP] Support Hadoop 3.x Hive 3.x and Spark 3.2.x default #5402

rahil-c wants to merge 85 commits into from

Conversation

@rahil-c
Copy link
Collaborator

@rahil-c rahil-c commented Apr 22, 2022

Tips

What is the purpose of the pull request

(For example: This pull request adds quick-start document.)

Brief change log

(for example:)

  • Modify AnnotationLocation checkstyle rule in checkstyle.xml

Verify this pull request

(Please pick either of the following options)

This pull request is a trivial rework / code cleanup without any test coverage.

(or)

This pull request is already covered by existing tests, such as (please describe tests).

(or)

This change added tests and can be verified as follows:

(example:)

  • Added integration tests for end-to-end.
  • Added HoodieClientWriteTest to verify the change.
  • Manually verified the change by running a job locally.

Committer checklist

  • Has a corresponding JIRA in PR title & commit

  • Commit message is descriptive of the change

  • CI is green

  • Necessary doc changes done or have another open PR

  • For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.

@rahil-c rahil-c force-pushed the hdp3-hive3-spark3-branch branch from 57fea4b to 0111427 Compare April 23, 2022 01:13
@rahil-c
Copy link
Collaborator Author

rahil-c commented Apr 23, 2022

@hudi-bot run azure

@rahil-c rahil-c force-pushed the hdp3-hive3-spark3-branch branch 4 times, most recently from 8c6f6e1 to b97a98e Compare April 26, 2022 04:15
@rahil-c
Copy link
Collaborator Author

rahil-c commented Apr 26, 2022

@hudi-bot run azure

1 similar comment
@rahil-c
Copy link
Collaborator Author

rahil-c commented Apr 26, 2022

@hudi-bot run azure

This was referenced Apr 27, 2022
Closed
Closed
@yihua yihua self-assigned this Apr 27, 2022
@yihua yihua added status:in-progress Work in progress dependencies Dependency updates priority:blocker Production down; release blocker engine:spark Spark integration engine:hive Hive integration labels Apr 27, 2022
This was referenced Apr 27, 2022
Closed
Closed
@yihua
Copy link
Contributor

yihua commented May 6, 2022

@hudi-bot run azure

@cdmikechen
Copy link
Contributor

cdmikechen commented May 6, 2022 •
edited
Loading

@yihua @rahil-c
Hi~ I have a question:
As we know, from hive2 to hive3 is a major version change, which will involve the adjustment of many original methods and classes. Should we also split hive2 and hive3 (or the newest hive4) into different packages?

@rahil-c rahil-c force-pushed the hdp3-hive3-spark3-branch branch from f82fcab to 8c629d5 Compare May 7, 2022 23:28
@rahil-c
Copy link
Collaborator Author

rahil-c commented Jun 3, 2022

@hudi-bot run azure

@rahil-c
Copy link
Collaborator Author

rahil-c commented Jun 3, 2022

@hudi-bot run azure

@rahil-c rahil-c force-pushed the hdp3-hive3-spark3-branch branch from e941b4b to 53c5159 Compare June 3, 2022 21:36
@rahil-c
Copy link
Collaborator Author

rahil-c commented Jun 4, 2022

@hudi-bot run azure

@rahil-c
Copy link
Collaborator Author

rahil-c commented Jun 4, 2022

@hudi-bot run azure

@nsivabalan nsivabalan added priority:critical Production degraded; pipelines stalled and removed priority:blocker Production down; release blocker labels Jun 6, 2022
@rahil-c
Copy link
Collaborator Author

rahil-c commented Jun 7, 2022

@hudi-bot run azure

@rahil-c
Copy link
Collaborator Author

rahil-c commented Jun 7, 2022

@hudi-bot run azure

@rahil-c
Copy link
Collaborator Author

rahil-c commented Jun 7, 2022

@hudi-bot run azure

@rahil-c
Copy link
Collaborator Author

rahil-c commented Jun 7, 2022

@hudi-bot run azure

@rahil-c
Copy link
Collaborator Author

rahil-c commented Jun 7, 2022

@hudi-bot run azure

@rahil-c
Copy link
Collaborator Author

rahil-c commented Jun 7, 2022

@hudi-bot run azure

@rahil-c
Copy link
Collaborator Author

rahil-c commented Jun 7, 2022

@hudi-bot run azure

@rahil-c
Copy link
Collaborator Author

rahil-c commented Jun 8, 2022

@hudi-bot run azure

@rahil-c
Copy link
Collaborator Author

rahil-c commented Jun 8, 2022

@hudi-bot run azure

@rahil-c
Copy link
Collaborator Author

rahil-c commented Jun 8, 2022

@hudi-bot run azure

@rahil-c
Copy link
Collaborator Author

rahil-c commented Jun 8, 2022

@hudi-bot run azure

@rahil-c
Copy link
Collaborator Author

rahil-c commented Jun 9, 2022

@hudi-bot run azure

@hudi-bot
Copy link
Collaborator

hudi-bot commented Jun 9, 2022

CI report:

Bot commands @hudi-bot supports the following commands:
  • @hudi-bot run azure re-run the last Azure build

@rahil-c rahil-c closed this Jul 6, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@yihua yihua yihua left review comments

Assignees

@yihua yihua

@xushiyan xushiyan

Labels

dependencies Dependency updates engine:hive Hive integration engine:spark Spark integration priority:critical Production degraded; pipelines stalled status:in-progress Work in progress

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

8 participants

@rahil-c @yihua @cdmikechen @danny0405 @hudi-bot @nsivabalan @xushiyan @codope