[HUDI-2439] Replace RDD with HoodieData in HoodieSparkTable and commit executors #4856
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
xushiyan
changed the title
xushiyan
force-pushed
the
HUDI-2439-refactoring-spark-commit-executors
branch
4 times, most recently
from
8bdb930 to
58ac0ed
Compare
xushiyan
force-pushed
the
HUDI-2439-refactoring-spark-commit-executors
branch
2 times, most recently
from
f44ffdb to
146e710
Compare
xushiyan
changed the title
xushiyan
force-pushed
the
HUDI-2439-refactoring-spark-commit-executors
branch
3 times, most recently
from
0371840 to
55d3617
Compare
xushiyan
commented
Mar 2, 2022
...hudi-client-common/src/main/java/org/apache/hudi/table/action/commit/HoodieDeleteHelper.java
Show resolved
Hide resolved
xushiyan
force-pushed
the
HUDI-2439-refactoring-spark-commit-executors
branch
from
55d3617 to
801d0ae
Compare
xushiyan
force-pushed
the
HUDI-2439-refactoring-spark-commit-executors
branch
3 times, most recently
from
2aa57d9 to
8de07cc
Compare
yihua
reviewed
Mar 10, 2022
...hudi-client-common/src/main/java/org/apache/hudi/table/action/commit/HoodieDeleteHelper.java
Outdated
Show resolved
Hide resolved
...hudi-client-common/src/main/java/org/apache/hudi/table/action/commit/HoodieDeleteHelper.java
Outdated
Show resolved
Hide resolved
...hudi-client-common/src/main/java/org/apache/hudi/table/action/commit/HoodieDeleteHelper.java
Show resolved
Hide resolved
.../hudi-client-common/src/main/java/org/apache/hudi/table/action/commit/HoodieMergeHelper.java
Show resolved
Hide resolved
| import org.apache.hudi.table.HoodieTable; | ||
|
|
||
| import org.apache.spark.api.java.JavaRDD; | ||
| public class HoodieWriteHelper<T extends HoodieRecordPayload, R> extends BaseWriteHelper<T, HoodieData<HoodieRecord<T>>, |
Contributor
There was a problem hiding this comment.
Similar here for the write helper regarding Flink, Java, and BaseWriteHelper.
Member
Author
There was a problem hiding this comment.
...n/java/org/apache/hudi/client/clustering/run/strategy/MultipleSparkJobExecutionStrategy.java
Outdated
Show resolved
Hide resolved
.../java/org/apache/hudi/table/action/commit/SparkInsertOverwriteTableCommitActionExecutor.java
Show resolved
Hide resolved
...nt/hudi-spark-client/src/test/java/org/apache/hudi/table/action/commit/TestDeleteHelper.java
Show resolved
Hide resolved
hudi-common/src/main/java/org/apache/hudi/common/data/HoodieList.java
Outdated
Show resolved
Hide resolved
hudi-common/src/main/java/org/apache/hudi/common/data/HoodieMapPair.java
Show resolved
Hide resolved
Contributor
|
As long as we don't touch public-facing APIs marked by |
xushiyan
force-pushed
the
HUDI-2439-refactoring-spark-commit-executors
branch
4 times, most recently
from
ef731b3 to
1969d6f
Compare
Contributor
|
@xushiyan : can you tag w/ right priority |
5 tasks
- Adopt HoodieData - Move Spark independent DeleteHelper, WriteHelper to client-common
xushiyan
force-pushed
the
HUDI-2439-refactoring-spark-commit-executors
branch
from
1969d6f to
1e90c19
Compare
vingov
pushed a commit
to vingov/hudi
that referenced
this pull request
Apr 3, 2022
…t executors (apache#4856) - Adopt HoodieData in Spark action commit executors - Make Spark independent DeleteHelper, WriteHelper, MergeHelper in hudi-client-common - Make HoodieTable in WriteClient APIs have raw type to decouple with Client's generic types
stayrascal
pushed a commit
to stayrascal/hudi
that referenced
this pull request
Apr 12, 2022
…t executors (apache#4856) - Adopt HoodieData in Spark action commit executors - Make Spark independent DeleteHelper, WriteHelper, MergeHelper in hudi-client-common - Make HoodieTable in WriteClient APIs have raw type to decouple with Client's generic types
This was referenced Mar 12, 2023
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.