Skip to content

Flink schema evolution support#7321

Closed
trushev wants to merge 1 commit intoapache:masterfrom
trushev:se-reopened
Closed

Flink schema evolution support#7321
trushev wants to merge 1 commit intoapache:masterfrom
trushev:se-reopened

Conversation

@trushev
Copy link
Copy Markdown
Contributor

@trushev trushev commented Nov 29, 2022

Change Logs

Azure doesn't run on the origin PR #5830

This PR adds support of reading by flink when comprehensive schema evolution(RFC-33) enabled and there are operations add column, rename column, change type of column, drop column.

Impact

user-facing feature change: comprehensive schema evolution in flink

Risk level medium

This change added tests and can be verified as follows:

  • Added unit test TestCastMap to verify that type conversion is correct
  • Added integration test ITTestSchemaEvolution to verify that table with added, renamed, casted, dropped columns is read as expected.

Documentation Update

There is schema evolution doc https://hudi.apache.org/docs/schema_evolution

Contributor's checklist

  • Read through contributor's guide
  • Change Logs and Impact were stated clearly
  • Adequate tests were added if applicable
  • CI passed

@trushev
Copy link
Copy Markdown
Contributor Author

trushev commented Nov 29, 2022

@hudi-bot run azure

@codope codope added area:schema Schema evolution and data types priority:high Significant impact; potential bugs engine:flink Flink integration labels Nov 29, 2022
@trushev trushev changed the title [HUDI-3981][RFC-33] Flink engine support for comprehensive schema evolution Flink schema evolution support Nov 30, 2022
@hudi-bot
Copy link
Copy Markdown
Collaborator

CI report:

Bot commands @hudi-bot supports the following commands:
  • @hudi-bot run azure re-run the last Azure build

@trushev
Copy link
Copy Markdown
Contributor Author

trushev commented Nov 30, 2022

Closing this PR because the origin one triggers azure now

@trushev trushev closed this Nov 30, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:schema Schema evolution and data types engine:flink Flink integration priority:high Significant impact; potential bugs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants