[HUDI-802] Fixing deletes for inserts in same batch in write path

[nsivabalan]

What is the purpose of the pull request

Fixing deletes for inserts in same write batch.

Brief change log

If same record is both inserted and deleted in same batch(possible when listening to events via deltastreamer), precombine will favor delete record. But then, in our OverwriteWithLatestAvroPayload, only combineAndGetUpdateValue checks for delete flag, where as getInsertValue does not. Hence the issue is that, for these cases, deleted records are still seen in read query.

Verify this pull request

This change added tests and can be verified as follows:

• TestHoodieClientOnCopyOnWriteStorage#testDeletesForInsertsInSameBatch

Committer checklist

• Has a corresponding JIRA in PR title & commit

• Commit message is descriptive of the change

• CI is green

• Necessary doc changes done or have another open PR

• For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.

[nsivabalan]

@vinothchandar : awaiting your review. if this is not doable (checking delete in getInsertValue()) for some reason, we might have to make some changes to #1819 as well. so would appreciate your review.

[vinothchandar]

@nsivabalan this does seem good to me. Can we add a Unit test specifically for OverwriteWithLatestAvroPayload which just tests these scenarios at the single class elvel?

[nsivabalan]

Added tests

[vinothchandar]

LGTM. please go ahead and merge once CI is happy

[vinothchandar]

[ERROR] Failures: 
2552[ERROR] org.apache.hudi.client.TestHoodieClientOnCopyOnWriteStorage.testUpsertsUpdatePartitionPathGlobalBloom(HoodieIndex$IndexType)
2553[INFO]   Run 1: PASS
2554[ERROR]   Run 2: TestHoodieClientOnCopyOnWriteStorage.testUpsertsUpdatePartitionPathGlobalBloom:448->testUpsertsUpdatePartitionPath:506 expected: <1> but was: <0>

@nsivabalan this was a recent test right? this seems to be failing here. may be run this until failiure locally and go from there?