[HUDI-6228] Re-enable tests that were flaky before

[yihua]

Change Logs

We fixed a few issues around concurrency control that can lead to flaky tests (e.g., #8658). This PR re-enables disabled tests that are important.

Impact

Adds more test coverage.

Risk level

none

Documentation Update

N/A

Contributor's checklist

• Read through contributor's guide
• Change Logs and Impact were stated clearly
• Adequate tests were added if applicable
• CI passed

[yihua]

@hudi-bot run azure

[nsivabalan]

did you trigger azure CI for 5 times may be to ensure this is not flaky anymore ?

[yihua]

did you trigger azure CI for 5 times may be to ensure this is not flaky anymore ?

Yes, in the process of triggering more runs. Currently, the CI has a long backlog, so this will take some time. Will at least run 5 times to be certain before merging this PR.

[nsivabalan]

sg

[yihua]

@hudi-bot run azure

[yihua]

@hudi-bot run azure

[yihua]

@hudi-bot run azure

[hudi-bot]

CI report:

• d8d8926 Azure: SUCCESS Azure: SUCCESS Azure: SUCCESS Azure: FAILURE Azure: FAILURE

Bot commands

@hudi-bot supports the following commands:

• @hudi-bot run azure re-run the last Azure build

[yihua]

CI report:

• d8d8926 Azure: SUCCESS Azure: SUCCESS Azure: SUCCESS Azure: FAILURE Azure: FAILURE

Bot commands

The fourth run failed due to other flaky tests in hudi-common and memory issue of the Azure worker. The fifth run failed due to a Spark flaky test. Both are not due to the re-enabled tests added.