[SPARK-27197][SQL][TEST] Add ReadNestedSchemaTest for file-based data sources

[dongjoon-hyun]

What changes were proposed in this pull request?

The reader schema is said to be evolved (or projected) when it changed after the data is written by writers. Apache Spark file-based data sources have a test coverage for that; e.g. ReadSchemaSuite.scala. This PR aims to add a test coverage for nested columns by adding and hiding nested columns.

How was this patch tested?

Pass the Jenkins with newly added tests.

[dongjoon-hyun]

cc @cloud-fan , @gatorsmile , @maropu , @dbtsai , @viirya , @HyukjinKwon .

[dbtsai]

LGTM.

[SparkQA]

Test build #103653 has finished for PR 24139 at commit f903978.

• This patch fails due to an unknown error code, -9.
• This patch merges cleanly.
• This patch adds the following public classes (experimental):
• trait AddNestedColumnTest extends ReadSchemaTest
• trait HideNestedColumnTest extends ReadSchemaTest

[viirya]

retest this please.

[maropu]

It this pr related to #23964?

[SparkQA]

Test build #103661 has finished for PR 24139 at commit f903978.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds the following public classes (experimental):
• trait AddNestedColumnTest extends ReadSchemaTest
• trait HideNestedColumnTest extends ReadSchemaTest

[dongjoon-hyun]

Yes. @maropu . There was a question about the other data sources (like Avro/JSON) works with nested column pruning. This PR verifies that explicitly.

[dongjoon-hyun]

Thank you for review, @dbtsai , @viirya , @maropu . I'll update once more to address @maropu 's comment.

[SparkQA]

Test build #103676 has started for PR 24139 at commit 4f148b9.

[dongjoon-hyun]

Retest this please.

[dbtsai]

Thanks. Merged into master.

[HyukjinKwon]

@dbtsai, tests were running and the last comments addressed are quite code changes ...

[SparkQA]

Test build #103695 has finished for PR 24139 at commit 4f148b9.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[dongjoon-hyun]

It passes finally. Thank you all, @dbtsai , @viirya , @maropu , @HyukjinKwon .