Skip to content

Conversation

@flashJd
Copy link
Contributor

@flashJd flashJd commented Aug 18, 2022

Change Logs

when changelog disabled, flink will output pk columns only,
https://github.com/apache/hudi/blob/master/hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/table/format/mor/MergeOnReadInputFormat.java#L364
We need the preCombine and partition fields also, so pull this request.

Impact

Describe any public API or user-facing feature change or any performance impact.

Risk level: none | low | medium | high

Choose one. If medium or high, explain what verification was done to mitigate the risks.

Contributor's checklist

  • Read through contributor's guide
  • Change Logs and Impact were stated clearly
  • Adequate tests were added if applicable
  • CI passed

@flashJd flashJd force-pushed the output_preCombine&partition_fields_of_delete_records branch from 126dd3e to 8b3b995 Compare September 6, 2022 03:15
@hudi-bot
Copy link
Collaborator

hudi-bot commented Sep 6, 2022

CI report:

Bot commands @hudi-bot supports the following commands:
  • @hudi-bot run azure re-run the last Azure build

@danny0405
Copy link
Contributor

We need the preCombine and partition fields also, so pull this request.

Can you explain why we need this then, do you want to write to another hudi table using these records ?

@flashJd
Copy link
Contributor Author

flashJd commented Sep 7, 2022

We need the preCombine and partition fields also, so pull this request.

Can you explain why we need this then, do you want to write to another hudi table using these records ?

@danny0405 we want to write this delete records to a warehouse or database, like apache phoenix, and want the precombine&partition field

@nsivabalan nsivabalan added the release-0.12.2 Patches targetted for 0.12.2 label Dec 6, 2022
@github-actions github-actions bot added the size:M PR with lines of changes in (100, 300] label Feb 26, 2024
Copy link
Contributor

@yihua yihua left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@danny0405 is this already fixed on Flink?

@danny0405
Copy link
Contributor

We need more discussion on this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

engine:flink Flink integration priority:high Significant impact; potential bugs release-0.12.2 Patches targetted for 0.12.2 size:M PR with lines of changes in (100, 300]

Projects

Status: 🚧 Needs Repro
Status: ✅ Done

Development

Successfully merging this pull request may close these issues.

5 participants