[SPARK-31801][API][SHUFFLE][TESTS] Tests for registering map output metadata

[mccheah]

What changes were proposed in this pull request?

Builds on #28618 in providing tests via a mock implementation of the shuffle plugin system. Note that a proper review of this should be done via comparing to #28618. Check out mccheah/spark@register-map-output-metadata...mccheah:register-map-output-metadata-with-tests for a more illustrative diff, at least until the dependent PR merges into maste.

Requires #28618.

Why are the changes needed?

We aim to provide test coverage to ensure the plugin system works in a meaningful way. Additionally, it's important to provide a reference implementation for plugin developers to look at.

Does this PR introduce any user-facing change?

No - tests only

How was this patch tested?

The patch should be entirely tests.

[SparkQA]

Test build #124384 has finished for PR 28902 at commit 8dddd15.

• This patch fails build dependency tests.
• This patch merges cleanly.
• This patch adds the following public classes (experimental):
• class ShuffleStoragePluginSuite extends SortShuffleSuite
• case class MockAsyncBackupMapOutputMetadata(
• class MockAsyncBackupMapOutputWriter(
• class MockAsyncBackupShuffleDataIO(sparkConf: SparkConf) extends ShuffleDataIO
• class MockAsyncBackupShuffleDriverComponents(
• class MockAsyncBackupShuffleExecutorComponents(
• class MockAsyncBackupShuffleOutputTracker(
• class MockAsyncShuffleBlockBackupManager(

[gatorsmile]

cc @Ngone51 @jiangxb1987

[SparkQA]

Test build #126832 has finished for PR 28902 at commit 72dfb1e.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #126834 has finished for PR 28902 at commit 5b2373a.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #126835 has finished for PR 28902 at commit 3b485cb.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #126895 has finished for PR 28902 at commit 2c57a9a.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #126903 has finished for PR 28902 at commit e7a548a.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #126997 has finished for PR 28902 at commit 933f9c6.

• This patch fails to build.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #127001 has finished for PR 28902 at commit b0c7a86.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[SparkQA]

Test build #127011 has finished for PR 28902 at commit bc06c90.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds the following public classes (experimental):
• case class RegExpExtractAll(subject: Expression, regexp: Expression, idx: Expression)
• .doc(\"The name of a class that implements \" +
• trait CachedBatch
• trait CachedBatchSerializer extends Serializable
• trait SimpleMetricsCachedBatch extends CachedBatch
• abstract class SimpleMetricsCachedBatchSerializer extends CachedBatchSerializer with Logging
• case class ColumnarToRowExec(child: SparkPlan) extends ColumnarToRowTransition with CodegenSupport
• case class ApplyColumnarRulesAndInsertTransitions(
• class ColumnStatisticsSchema(a: Attribute) extends Serializable
• class PartitionStatistics(tableSchema: Seq[Attribute]) extends Serializable
• case class DefaultCachedBatch(numRows: Int, buffers: Array[Array[Byte]], stats: InternalRow)
• class DefaultCachedBatchSerializer extends SimpleMetricsCachedBatchSerializer
• trait HashJoin extends BaseJoinExec with CodegenSupport

[SparkQA]

Test build #127294 has finished for PR 28902 at commit c61d547.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[github-actions]

We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable.
If you'd like to revive this PR, please reopen it and ask a committer to remove the Stale tag!