[SPARK-32537][SQL][TEST] Add a CTEHintSuite for test coverage #29359
Conversation
|
Test build #127092 has finished for PR 29359 at commit
|
| import org.apache.log4j.Level | ||
|
|
||
| import org.apache.spark.sql.catalyst.plans.PlanTest | ||
| import org.apache.spark.sql.catalyst.plans.logical.{BROADCAST, HintInfo, Join, JoinHint, Repartition, RepartitionByExpression, ResolvedHint, SHUFFLE_HASH, SHUFFLE_MERGE, SHUFFLE_REPLICATE_NL} |
There was a problem hiding this comment.
nit: I think its okay to fold it: import org.apache.spark.sql.catalyst.plans.logical._
| import org.apache.spark.sql.execution.adaptive.AdaptiveSparkPlanHelper | ||
| import org.apache.spark.sql.test.SharedSparkSession | ||
|
|
||
| class CTEHintSuite extends PlanTest with SharedSparkSession with AdaptiveSparkPlanHelper { |
There was a problem hiding this comment.
I will remove it. I just copy it from JoinHintSuite :P
|
Test build #127134 has finished for PR 29359 at commit
|
|
Test build #127135 has finished for PR 29359 at commit
|
|
|
||
| class CTEHintSuite extends QueryTest with SharedSparkSession { | ||
|
|
||
| def verifyCoalesceHint(df: DataFrame): Unit = { |
There was a problem hiding this comment.
ResolveCoalesceHints accepts names "COALESCE", "REPARTITION", and "REPARTITION_BY_RANGE".
There was a problem hiding this comment.
Changed to def verifyCoalesceOrRepartitionHint
| val repartitions = plan collect { | ||
| case r: Repartition => r | ||
| case r: RepartitionByExpression => r | ||
| case _: ResolvedHint => fail("ResolvedHint should not appear after optimize.") |
There was a problem hiding this comment.
Is this error message correct even if the test is for an analyzed plan? https://github.com/apache/spark/pull/29359/files#diff-0d893e4fe9a621d3e084c2913c533e00R38
| sql(s""" | ||
| |WITH cte AS (SELECT /*+ REPARTITION(3) */ * FROM t) | ||
| |SELECT * FROM cte | ||
| """.stripMargin), |
There was a problem hiding this comment.
nit format:
checkAnswer(
sql(
s"""
|WITH cte AS (SELECT /*+ REPARTITION(3) */ * FROM t)
|SELECT * FROM cte
|""".stripMargin),
Row(1) :: Nil)
|
Test build #127176 has finished for PR 29359 at commit
|
|
retest this please |
|
Test build #127187 has finished for PR 29359 at commit
|
| } | ||
| } | ||
|
|
||
| test("SPARK-32237: Hint in CTE") { |
There was a problem hiding this comment.
Actually we can remove it now. It's already covered by Resolve coalesce hint in CTE.
|
Test build #127251 has finished for PR 29359 at commit
|
|
github action passed, merging to master, thanks! |
5 participants
What changes were proposed in this pull request?
Add a new test suite
CTEHintSuiteWhy are the changes needed?
This ticket is to address the below comments to help us understand the test coverage of SQL HINT for CTE.
#29062 (comment)
#29062 (comment)
Does this PR introduce any user-facing change?
No
How was this patch tested?
Add a test suite.