Skip to content

Conversation

@nsivabalan
Copy link
Contributor

@nsivabalan nsivabalan commented Feb 18, 2022

What is the purpose of the pull request

  • Found couple of issues while testing restore w/ metadata enabled (COW and MOR table).
  • avro schema for metadata payload did not have default set to null for colsStats and bloomFilterMetadata that got added recently. Was running into issues with avro deserialization.
  • While reading logs for metadata table, partition name was not set with MergedLogRecordScanner. Have created a follow ticket to chase all such code paths and fix.

Verify this pull request

  • Tested restore for COW and MOR table for few scenarios. restore succeeded and validated the table after restore to ensure table record count matched the restored commit.

Committer checklist

  • Has a corresponding JIRA in PR title & commit

  • Commit message is descriptive of the change

  • CI is green

  • Necessary doc changes done or have another open PR

  • For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.

@hudi-bot
Copy link
Collaborator

CI report:

Bot commands @hudi-bot supports the following commands:
  • @hudi-bot run azure re-run the last Azure build

@codope codope merged commit 17cb5cb into apache:master Feb 21, 2022
vingov pushed a commit to vingov/hudi that referenced this pull request Apr 3, 2022
* Fixing restore with metadata enabled

* Fixing test failures
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants