[HUDI-3268] Fix NPE while reading table with Spark datasource

[yihua]

What is the purpose of the pull request

Existing SparkHoodieTableFileIndex requires checking against the key generator class name to generate the partition schema. This logic introduces NPE when the key generator class name is not present in hoodie.properties. This PR fixes the issue so that if the key generator class name is not present, default logic is used.

Stacktrace:

scala> val basePath = "/tmp/hoodie/hudi-test-topic"
basePath: String = /tmp/hoodie/hudi-test-topic


scala> val df = spark.read.format("hudi").load(basePath)
df: org.apache.spark.sql.DataFrame = [_hoodie_commit_time: string, _hoodie_commit_seqno: string ... 15 more fields]


scala> df.count
java.lang.NullPointerException
  at org.apache.hudi.SparkHoodieTableFileIndex._partitionSchemaFromProperties$lzycompute(SparkHoodieTableFileIndex.scala:83)
  at org.apache.hudi.SparkHoodieTableFileIndex._partitionSchemaFromProperties(SparkHoodieTableFileIndex.scala:77)
  at org.apache.hudi.SparkHoodieTableFileIndex.partitionSchema(SparkHoodieTableFileIndex.scala:118)
  at org.apache.hudi.SparkHoodieTableFileIndex.parsePartitionColumnValues(SparkHoodieTableFileIndex.scala:218)

Brief change log

• Changes the way of matching key generator class name in SparkHoodieTableFileIndex

Verify this pull request

Verified by loading a Hudi MOR table written by Kafka Connect Sink for Hudi, without having key generator class name in hoodie.properties, with Spark datasource and the table can be read successfully.

scala> val basePath = "/tmp/hoodie/hudi-test-topic"
basePath: String = /tmp/hoodie/hudi-test-topic

scala> val df = spark.read.format("hudi").load(basePath)
df: org.apache.spark.sql.DataFrame = [_hoodie_commit_time: string, _hoodie_commit_seqno: string ... 15 more fields]

scala> df.count
res0: Long = 1500       

Committer checklist

• Has a corresponding JIRA in PR title & commit

• Commit message is descriptive of the change

• CI is green

• Necessary doc changes done or have another open PR

• For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.

[hudi-bot]

CI report:

• de51b55 Azure: SUCCESS

Bot commands

@hudi-bot supports the following commands:

• @hudi-bot run azure re-run the last Azure build

[nsivabalan]

one nit question