[SPARK-29313][SQL] Fix failure on writing to noop in benchmarks

[MaxGekk]

What changes were proposed in this pull request?

In the PR, I propose to specify the save mode explicitly while writing to the noop datasource in benchmarks. I set Overwrite mode in the following benchmarks:

• JsonBenchmark
• CSVBenchmark
• UDFBenchmark
• MakeDateTimeBenchmark
• ExtractBenchmark
• DateTimeBenchmark
• NestedSchemaPruningBenchmark

Why are the changes needed?

Otherwise writing to noop fails with:

[error] Exception in thread "main" org.apache.spark.sql.AnalysisException: TableProvider implementation noop cannot be written with ErrorIfExists mode, please use Append or Overwrite modes instead.;
[error] 	at org.apache.spark.sql.DataFrameWriter.save(DataFrameWriter.scala:284)

most likely due to #25876

Does this PR introduce any user-facing change?

No

How was this patch tested?

I generated results of ExtractBenchmark via the command:

SPARK_GENERATE_BENCHMARK_FILES=1 build/sbt "sql/test:runMain org.apache.spark.sql.execution.benchmark.ExtractBenchmark"

[MaxGekk]

@cloud-fan @brkyvz @rdblue Please, take a look at the PR.

[srowen]

Seems OK to me. I wonder why noop fails in any mode? you could argue the mode is just irrelevant. In particular the 'output' never 'exists' in noop mode, conceptually.

[MaxGekk]

I agree that the save mode for noop doesn't matter but for default mode

spark/sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala

Line 844 in c8159c7

private var mode: SaveMode = SaveMode.ErrorIfExists

there is no case currently

spark/sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala

Lines 270 to 287 in c8159c7

	mode match {
	case SaveMode.Append =>
	runCommand(df.sparkSession, "save") {
	AppendData.byName(relation, df.logicalPlan, extraOptions.toMap)
	}
	case SaveMode.Overwrite if table.supportsAny(TRUNCATE, OVERWRITE_BY_FILTER) =>
	// truncate the table
	runCommand(df.sparkSession, "save") {
	OverwriteByExpression.byName(
	relation, df.logicalPlan, Literal(true), extraOptions.toMap)
	}
	case other =>
	throw new AnalysisException(s"TableProvider implementation $source cannot be " +
	s"written with $other mode, please use Append or Overwrite " +
	"modes instead.")
	}

. Maybe it should be implemented in more generic way.

[SparkQA]

Test build #111643 has finished for PR 25988 at commit ec104fa.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[dongjoon-hyun]

I'm not sure about this. For me, we need to fix the root cause inside DSv2.

[dongjoon-hyun]

@brkyvz and @cloud-fan . Is this change intentional?

[cloud-fan]

we are going to support all save modes in DataFrameWriter. For now this change is OK to me to unblock benchmark changes.

[dongjoon-hyun]

Thanks. Then, I'll merge this. This will unblock the benchmarks.

[dongjoon-hyun]

Merged to master.

[dongjoon-hyun]

Thank you, @MaxGekk , @cloud-fan , @srowen .

[rdblue]

Sorry I'm just now getting to look at this. @cloud-fan, any idea why v2 was used here for these sources? Built-in v2 implementations should be disabled, so it is concerning that they are inadvertently used, right?

[cloud-fan]

we only disable file source v2, but noop souce is not file source.