[SPARK-26820][SQL] Prints log messages when hints ignored in the analyzer #24055
Conversation
|
@gatorsmile Does this pr matches your intention in the Jira? |
|
Test build #103324 has finished for PR 24055 at commit
|
|
Retest this please |
|
Test build #103560 has finished for PR 24055 at commit
|
|
ping @gatorsmile @maryannxue |
| .stringConf | ||
| .transform(_.toUpperCase(Locale.ROOT)) | ||
| .checkValue(logLevel => Set("TRACE", "DEBUG", "INFO", "WARN", "ERROR").contains(logLevel), | ||
| "Invalid value for 'spark.sql.optimizer.planChangeLog.level'. Valid values are " + |
There was a problem hiding this comment.
spark.sql.optimizer.planChangeLog.level -> spark.sql.optimizer.planHintIgnoreLog.level?
|
Test build #103698 has finished for PR 24055 at commit
|
|
Retest this please |
|
Test build #103706 has finished for PR 24055 at commit
|
|
Retest this please |
|
Test build #103717 has finished for PR 24055 at commit
|
|
Test build #103714 has finished for PR 24055 at commit
|
| .stringConf | ||
| .createOptional | ||
|
|
||
| val PLAN_HINT_IGNORE_LOG_LEVEL = |
There was a problem hiding this comment.
Does this really worth a config?
There was a problem hiding this comment.
+1. I don't think we need to make it configurable. We can just log warning.
| Seq(errMsgRepa)) | ||
| } | ||
|
|
||
| private def withLogLevel(logLevel: String)(f: => Unit): String = { |
There was a problem hiding this comment.
Can we reuse SparkFunSuite.withLogAppender?
| } | ||
| } | ||
|
|
||
| private class IgnoredHintLogger { |
There was a problem hiding this comment.
Why do we need a new class here? Is there any reason why we can't use HintErrorLogger?
There was a problem hiding this comment.
Yea, HintErrorLogger can covert all the case in this pr, so I'll close this. Thanks (I didn't notice that the pr of HintErrorLogger had been merged after this pr made.
7 participants
What changes were proposed in this pull request?
This pr added code to log ignored hints in
RemoveAllHints.How was this patch tested?
Pass Jenkins.