[HUDI-6389] Fix instant time check against the active timeline in meta sync

[yihua]

Change Logs

#7561 introduced a bug where in a case, the partition changes may be missed in the meta sync, e.g., the following active timeline:

ts3.rollback, ts50.commit, ts51.commit, ts52.commit, ...

and ts49.commit and ts48.commit are archived.

If ts47 is the last sync commit time, and ts48.commit or ts49.commit has partition changes, meta sync misses such changes.

This above issue is solved by a separate PR #8388 where the hive sync client returns the right timeline which gives write commits only for checking.

This PR makes sure the problematic API implementation (TimelineUtils.getCommitsTimelineAfter) is still fixed.

Existing tests are enhanced:

• TestTimelineUtils.testGetCommitsTimelineAfter: this test fails before this PR and passes after the PR.
• TestHiveSyncTool. testBasicSync: this test passes before and after this PR on master. Note that, without [HUDI-5816] List all partitions as the fallback mechanism in Hive and Glue Sync #8388, this test would fail without this PR.

Impact

Fixes a bug.

Risk level

low

Documentation Update

We need to update release notes on the regression.

Contributor's checklist

• Read through contributor's guide
• Change Logs and Impact were stated clearly
• Adequate tests were added if applicable
• CI passed

[hudi-bot]

CI report:

• 707d4cd Azure: SUCCESS

Bot commands

@hudi-bot supports the following commands:

• @hudi-bot run azure re-run the last Azure build