Core: Checks for Equality Delete when Row Lineage is Enabled - Using Snapshot Summary

[RussellSpitzer]

No description provided.

[pvary]

@RussellSpitzer: Sorry to open this up again, but I having double thoughts about disallowing equality deletes with row lineage.

While it is true that equality delete is often used by the streaming writers to update a row without actually looking up the old version of the row, it is not true that the equality delete could not work with row lineage. The issue is not with the delete itself. The issue is with how it is used by the streaming ingests. The same bad behavior could be replicated by positional deletes, if some user would update records by deleting the previous row version , and inserting the new version of the row without adding the old row-id to it.

There are some perfectly legal use-cases for equality deletes which could work well with row lineage, like someone uses equality delete to remove GDPR related rows from the table.

WDYT?

[RussellSpitzer]

You'll probably want to bring it up with @rdblue , I think he was the main proponent for disallowing it. Personally I don't mind equality deletes existing in context with row lineage, although it does make things a bit sloppier. Some records will get proper updates and some will be treated as remove-add.

[github-actions]

This pull request has been marked as stale due to 30 days of inactivity. It will be closed in 1 week if no further activity occurs. If you think that’s incorrect or this pull request requires a review, please simply write any comment. If closed, you can revive the PR at any time and @mention a reviewer or discuss it on the [email protected] list. Thank you for your contributions.

[github-actions]

This pull request has been closed due to lack of activity. This is not a judgement on the merit of the PR in any way. It is just a way of keeping the PR queue manageable. If you think that is incorrect, or the pull request requires review, you can revive the PR at any time.