[SPARK-37289][SQL] Remove partitionSchemaOption function
#34582
Conversation
What changes were proposed in this pull request? This PR remove the unnecessary function called partitionSchemaOption in HadoopFsRelation.scala. Why are the changes needed? The partitionSchemaOption is unnecessary in HadoopFsRelation, it will make more complicated logical in HadoopFsRelation,which we can simply use partitionSchema.isEmpty or partitionSchema.nonEmpty instead Does this PR introduce any user-facing change? No,only change the expression. How was this patch tested? Pass existed tests.
|
jenkins, test this please |
|
@tenglei please make the title more specific by pointing out the function name |
|
Kubernetes integration test starting |
|
Kubernetes integration test status failure |
|
Test build #145191 has finished for PR 34582 at commit
|
dongjoon-hyun
left a comment
dongjoon-hyun
left a comment
There was a problem hiding this comment.
This is not a removal of unnecessary functions technically.
Please be clear what you are doing.
|
Can one of the admins verify this patch? |
tenglei
changed the title
dongjoon-hyun
changed the title
partitionSchemaOption function
dongjoon-hyun
left a comment
dongjoon-hyun
left a comment
There was a problem hiding this comment.
+1, LGTM. Thank you for updating, @tenglei .
Merged to master.
|
@tenglei . I added you to the Apache Spark contributor group and assigned SPARK-37289 to you. |
Thanks for the assist, appreciate your time. I'm very happy to join the Spark community~ |
5 participants
What changes were proposed in this pull request?
This PR remove the unnecessary function called partitionSchemaOption in HadoopFsRelation.scala.
Why are the changes needed?
The partitionSchemaOption is unnecessary in HadoopFsRelation, it will make more complicated logical in HadoopFsRelation,which we can simply use partitionSchema.isEmpty or partitionSchema.nonEmpty instead
Does this PR introduce any user-facing change?
No
How was this patch tested?
Pass existed tests.