Spark 3.1: Use snapshot schema for time travel queries #3768
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Port of #3722 to v3.1.
Notes:
I took the diff from #3722, edited it to change v3.2 to v3.1 in the paths, and tried to
git apply 3722.diff. That did not apply, so I didgit am -3 < 3722.diff, which left conflicts in SparkTable.java and TestDeleteFrom.java.The conflicts in SparkTable.java were straightforward to resolve.
TestDeleteFromin v3.2 hastestDeleteFromUnpartitionedTableandtestDeleteFromPartitionedTablewhich are note present in v3.1; I had based the new test,testDeleteFromTableAtSnapshotontestDeleteFromUnpartitionedTable, and it is still usable because we don't actually delete anything, just check that an exception is thrown.Also, for the negative test cases added in
TestDeleteFromandTestUnpartitionedWrites, I had to exclude theSparkSessionCatalogconfiguration from the parameterized testing, as otherwise I getinstead of the
IllegalArgumentExceptionI was looking for.