[Feature][Connector-V2] Optimize hudi sink #7662
Conversation
…efore fetch task start (apache#5008)
github-actions
bot
added
document
core
| | min_commits_to_keep | Int | no | 20 | | ||
| | max_commits_to_keep | Int | no | 30 | | ||
| | common-options | config | no | - | | ||
| Base configuration: |
There was a problem hiding this comment.
Please update zh docs
done.
| "sh", "-c", "cd /tmp" + " && tar -zxvf " + NAMESPACE_TAR); | ||
| try { | ||
| Process process = processBuilder.start(); | ||
| // 等待命令执行完成 |
There was a problem hiding this comment.
Please remove Chinese
done.
Hisoka-X
changed the title
| op_type="UPSERT" | ||
| table_dfs_path = "/tmp/hudi" | ||
| database = "st" | ||
| table_name = "st_test" | ||
| table_type="COPY_ON_WRITE" | ||
| record_key_fields="c_bigint" |
There was a problem hiding this comment.
Without new config key op_type, database, table_type, record_key_fields, can the config execute normally?
There was a problem hiding this comment.
Without new config key
op_type,database,table_type,record_key_fields, can the config execute normally?
Sorry, I did not consider that these item is empty, I have modified it.
|
Thanks @happyboy1024 |
|
I have find some problem when using hudi sink connector.
|
You can temporarily set |
Purpose of this pull request
This pr mainly focuses on some optimization of hudi sink. #7597
Moreover, fixed dependency problem with S3 as storage, and fixed multiple tables in spark and flink engines that did not execute savemode properly
Does this PR introduce any user-facing change?
How was this patch tested?
Check list
New License Guide
release-note.