0.12.1 Cherry-Picks: Part 2

[kbendick]

This PR is to get the following two PRs, slated for release in 0.12.1, to be able to merge.

Neither of them cherry-pick cleanly, though I do know the set of PRs that would be needed to do so (as mentioned in another PR).

• Core: Validate concurrently added delete files in RowDelta #3195
• Core: Validate concurrently added delete files in OverwriteFiles #3199

** Merge first - After a decision is made about #3264

[kbendick]

This should be included before #3428.

This also includes, but doesn't need #3264, just as #3428 does. I added it to try to make the test suite run faster yesterday as it was really slow (maybe just noisy neighbors on the runners). Can remove on rebase.

Lastly, this includes #3071, which wasn't marked for inclusion but made the whole process much simpler. If we don't want it, I can relatively easily remove it.

[kbendick]

The fact that both of these have failMissingDeletePaths makes me think something happened with the cherry-picks. Investigating further.

[kbendick]

Actually, this is the way both of these functions look in master - Is this intentional?:

iceberg/core/src/main/java/org/apache/iceberg/BaseOverwriteFiles.java

Lines 100 to 111 in fad77e8

	public OverwriteFiles validateNoConflictingData() {
	this.validateNewDataFiles = true;
	failMissingDeletePaths();
	return this;
	}
	@Override
	public OverwriteFiles validateNoConflictingDeletes() {
	this.validateNewDeleteFiles = true;
	failMissingDeletePaths();
	return this;
	}

Wondering if this is correct.

[kbendick]

Nevermind. This looks correct. Here's the actual function called:

iceberg/core/src/main/java/org/apache/iceberg/MergingSnapshotProducer.java

Lines 143 to 146 in fad77e8

	protected void failMissingDeletePaths() {
	filterManager.failMissingDeletePaths();
	deleteFilterManager.failMissingDeletePaths();
	}

[rdblue]

Yes, this is intentional. This was to preserve the old behavior.

[rdblue]

Looks great! I merged these into 0.12.x. Thanks, @kbendick!