[SPARK-34696][SQL][TESTS] Fix CodegenInterpretedPlanTest to generate correct test cases #31766

dongjoon-hyun · 2021-03-06T20:49:57Z

What changes were proposed in this pull request?

SPARK-23596 added CodegenInterpretedPlanTest at Apache Spark 2.4.0 in a wrong way because withSQLConf depends on the execution time SQLConf.get instead of test function declaration time. So, the following code executes the test twice without controlling the CodegenObjectFactoryMode. This PR aims to fix it correct and introduce a new function testFallback.

trait CodegenInterpretedPlanTest extends PlanTest {

   override protected def test(
       testName: String,
       testTags: Tag*)(testFun: => Any)(implicit pos: source.Position): Unit = {
     val codegenMode = CodegenObjectFactoryMode.CODEGEN_ONLY.toString
     val interpretedMode = CodegenObjectFactoryMode.NO_CODEGEN.toString

     withSQLConf(SQLConf.CODEGEN_FACTORY_MODE.key -> codegenMode) {
       super.test(testName + " (codegen path)", testTags: _*)(testFun)(pos)
     }
     withSQLConf(SQLConf.CODEGEN_FACTORY_MODE.key -> interpretedMode) {
       super.test(testName + " (interpreted path)", testTags: _*)(testFun)(pos)
     }
   }
 }

Why are the changes needed?

We need to use like the following.

super.test(testName + " (codegen path)", testTags: _*)(
   withSQLConf(SQLConf.CODEGEN_FACTORY_MODE.key -> codegenMode) { testFun })(pos)
super.test(testName + " (interpreted path)", testTags: _*)(
   withSQLConf(SQLConf.CODEGEN_FACTORY_MODE.key -> interpretedMode) { testFun })(pos)

After we fix this behavior with the above code, several test cases including SPARK-34596 and SPARK-34607 fail because they didn't work at both CODEGEN and INTERPRETED mode. Those test cases only work at FALLBACK mode. So, inevitably, we need to introduce testFallback.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

Pass the CIs.

dongjoon-hyun · 2021-03-10T07:29:02Z

Hi, @rednaxelafx , @cloud-fan , @viirya , @kiszk , @maropu , @HyukjinKwon .

Here is a summary of the progress.

[SPARK-34660][TESTS][3.1] Don't use ParVector with withExistingConf which is not thread-safe #31775 (SPARK-34660 Don't use ParVector with withExistingConf which is not thread-safe) fixed the UT failure at branch-3.1 last weekend.
[SPARK-33498][SQL][TESTS][FOLLOWUP] Remove SQLConf.withExistingConf in CastSuite #31785 (SPARK-33498 Remove SQLConf.withExistingConf in CastSuite) removes the unneeded withExistingConf at master and will be backported to branch-3.1.

So, the above two were the answers why we hit the UT failure only at branch-3.1 while we have the patch at master/3.1/3.0.

Now, this PR aims to do only refactoring PlanTest and test cases to have a clear test design. As we discussed in the previous PR, we need this.

[SPARK-34596][SPARK-34607][SQL][FOLLOWUP][TESTS][3.1] Add PlanTest. testFallback and use it #31764 (comment) (@cloud-fan )

The current test framework assumes that codegen and non-codegen should have consistent behaviors, while this codegen bug breaks the assumption. The test case fails with codegen but passes with non-codegen.

+1 to add this for such test cases. One thing I'm curious about is why this test works in master...

[SPARK-34596][SPARK-34607][SQL][FOLLOWUP][TESTS][3.1] Add PlanTest. testFallback and use it #31764 (comment) (@maropu )

+1 to add it, too.

I updated this PR to the master and address the two comments.

So, could you review this again freshly? And, please let me know if I missed some of your comments, (especially @viirya 's one). I hope we can have this refactoring for all branches with SPARK-34596 and SPARK-34607.

sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/plans/PlanTest.scala

dongjoon-hyun · 2021-03-10T18:03:07Z

Since this is a long standing bug at SPARK-23596, re-ping @viirya , @HyukjinKwon , @hvanhovell , @kiszk .

…correct test cases

sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/plans/PlanTest.scala

SparkQA · 2021-03-10T19:34:35Z

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40526/

SparkQA · 2021-03-10T20:12:00Z

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40526/

SparkQA · 2021-03-10T20:28:42Z

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40528/

SparkQA · 2021-03-10T20:53:05Z

Test build #135942 has finished for PR 31766 at commit 0b2915d.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2021-03-10T21:13:50Z

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40528/

HyukjinKwon · 2021-03-11T00:58:40Z

retest this please

SparkQA · 2021-03-11T02:35:44Z

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40538/

SparkQA · 2021-03-11T03:10:03Z

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/40538/

cloud-fan · 2021-03-11T05:08:01Z

sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/plans/PlanTest.scala

    val interpretedMode = CodegenObjectFactoryMode.NO_CODEGEN.toString

-    withSQLConf(SQLConf.CODEGEN_FACTORY_MODE.key -> codegenMode) {
-      super.test(testName + " (codegen path)", testTags: _*)(testFun)(pos)


super.test sets the sql conf again?

withSQLConf depends on SQLConf.get. The existing code cannot guarantee the configuration setting of line 47 at super.test's execution time.

SQLConf is thread-local. Does super.test runs in another thread by the SBT test runner?

Here, we define these tests by using super.test instead of executing.

When the derived classes define test cases with this test, two test cases are created but withSQLConf is not inside the defined tests.

ah, now I got it! What a hidden bug 😂

SparkQA · 2021-03-11T05:32:05Z

Test build #135954 has finished for PR 31766 at commit 54636fc.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

cloud-fan · 2021-03-11T05:56:00Z

sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/encoders/ExpressionEncoderSuite.scala

    "seq of seq of string")

-  encodeDecodeTest(Array(31, -123, 4), "array of int")
+  encodeDecodeTest(Array(31, -123, 4), "array of int", useFallback = true)


do we know why this simply test has to use fallback test?

I also was surprised that we have several cases instead of the new two tests.
It seems to be broken at some commits during the time because we didn't notice it due to this CodegenInterpretedPlanTest test framework bug.
Actually, I decided to focus on the test suite first and makes the recovery the beyond of the scope of this PR.

We may had better file a new JIRA aiming the recovery of those test cases one by one if possible.

BTW, for this specific instance, the following was the error.

[info] - encode/decode for array of int: [I@74655d03 (interpreted path) *** FAILED *** (14 milliseconds) [info] Exception thrown while decoding [info] Converted: [0,1000000020,3,0,ffffff850000001f,4] [info] Schema: value#680 [info] root [info] -- value: array (nullable = true) [info] |-- element: integer (containsNull = false) [info] [info] [info] Encoder: [info] class[value[0]: array<int>] (ExpressionEncoderSuite.scala:578) [info] org.scalatest.exceptions.TestFailedException: [info] at org.scalatest.Assertions.newAssertionFailedException(Assertions.scala:472) [info] at org.scalatest.Assertions.newAssertionFailedException$(Assertions.scala:471) [info] at org.scalatest.funsuite.AnyFunSuite.newAssertionFailedException(AnyFunSuite.scala:1563) [info] at org.scalatest.Assertions.fail(Assertions.scala:949) [info] at org.scalatest.Assertions.fail$(Assertions.scala:945) [info] at org.scalatest.funsuite.AnyFunSuite.fail(AnyFunSuite.scala:1563) [info] at org.apache.spark.sql.catalyst.encoders.ExpressionEncoderSuite.$anonfun$encodeDecodeTest$1(ExpressionEncoderSuite.scala:578) [info] at org.apache.spark.sql.catalyst.encoders.ExpressionEncoderSuite.verifyNotLeakingReflectionObjects(ExpressionEncoderSuite.scala:656) [info] at org.apache.spark.sql.catalyst.encoders.ExpressionEncoderSuite.$anonfun$testAndVerifyNotLeakingReflectionObjects$2(ExpressionEncoderSuite.scala:669) [info] at org.apache.spark.sql.catalyst.plans.CodegenInterpretedPlanTest.$anonfun$test$4(PlanTest.scala:50) [info] at org.apache.spark.sql.catalyst.plans.SQLHelper.withSQLConf(SQLHelper.scala:54) [info] at org.apache.spark.sql.catalyst.plans.SQLHelper.withSQLConf$(SQLHelper.scala:38) [info] at org.apache.spark.sql.catalyst.encoders.ExpressionEncoderSuite.withSQLConf(ExpressionEncoderSuite.scala:118) [info] at org.apache.spark.sql.catalyst.plans.CodegenInterpretedPlanTest.$anonfun$test$3(PlanTest.scala:50) [info] at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23) [info] at org.scalatest.OutcomeOf.outcomeOf(OutcomeOf.scala:85) [info] at org.scalatest.OutcomeOf.outcomeOf$(OutcomeOf.scala:83) [info] at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104) [info] at org.scalatest.Transformer.apply(Transformer.scala:22) [info] at org.scalatest.Transformer.apply(Transformer.scala:20) [info] at org.scalatest.funsuite.AnyFunSuiteLike$$anon$1.apply(AnyFunSuiteLike.scala:190) [info] at org.apache.spark.SparkFunSuite.withFixture(SparkFunSuite.scala:178) [info] at org.scalatest.funsuite.AnyFunSuiteLike.invokeWithFixture$1(AnyFunSuiteLike.scala:188) [info] at org.scalatest.funsuite.AnyFunSuiteLike.$anonfun$runTest$1(AnyFunSuiteLike.scala:200) [info] at org.scalatest.SuperEngine.runTestImpl(Engine.scala:306) [info] at org.scalatest.funsuite.AnyFunSuiteLike.runTest(AnyFunSuiteLike.scala:200) [info] at org.scalatest.funsuite.AnyFunSuiteLike.runTest$(AnyFunSuiteLike.scala:182) [info] at org.apache.spark.SparkFunSuite.org$scalatest$BeforeAndAfterEach$$super$runTest(SparkFunSuite.scala:61) [info] at org.scalatest.BeforeAndAfterEach.runTest(BeforeAndAfterEach.scala:234) [info] at org.scalatest.BeforeAndAfterEach.runTest$(BeforeAndAfterEach.scala:227) [info] at org.apache.spark.SparkFunSuite.runTest(SparkFunSuite.scala:61) [info] at org.scalatest.funsuite.AnyFunSuiteLike.$anonfun$runTests$1(AnyFunSuiteLike.scala:233) [info] at org.scalatest.SuperEngine.$anonfun$runTestsInBranch$1(Engine.scala:413) [info] at scala.collection.immutable.List.foreach(List.scala:392) [info] at org.scalatest.SuperEngine.traverseSubNodes$1(Engine.scala:401) [info] at org.scalatest.SuperEngine.runTestsInBranch(Engine.scala:396) [info] at org.scalatest.SuperEngine.runTestsImpl(Engine.scala:475) [info] at org.scalatest.funsuite.AnyFunSuiteLike.runTests(AnyFunSuiteLike.scala:233) [info] at org.scalatest.funsuite.AnyFunSuiteLike.runTests$(AnyFunSuiteLike.scala:232) [info] at org.scalatest.funsuite.AnyFunSuite.runTests(AnyFunSuite.scala:1563) [info] at org.scalatest.Suite.run(Suite.scala:1112) [info] at org.scalatest.Suite.run$(Suite.scala:1094) [info] at org.scalatest.funsuite.AnyFunSuite.org$scalatest$funsuite$AnyFunSuiteLike$$super$run(AnyFunSuite.scala:1563) [info] at org.scalatest.funsuite.AnyFunSuiteLike.$anonfun$run$1(AnyFunSuiteLike.scala:237) [info] at org.scalatest.SuperEngine.runImpl(Engine.scala:535) [info] at org.scalatest.funsuite.AnyFunSuiteLike.run(AnyFunSuiteLike.scala:237) [info] at org.scalatest.funsuite.AnyFunSuiteLike.run$(AnyFunSuiteLike.scala:236) [info] at org.apache.spark.SparkFunSuite.org$scalatest$BeforeAndAfterAll$$super$run(SparkFunSuite.scala:61) [info] at org.scalatest.BeforeAndAfterAll.liftedTree1$1(BeforeAndAfterAll.scala:213) [info] at org.scalatest.BeforeAndAfterAll.run(BeforeAndAfterAll.scala:210) [info] at org.scalatest.BeforeAndAfterAll.run$(BeforeAndAfterAll.scala:208) [info] at org.apache.spark.SparkFunSuite.run(SparkFunSuite.scala:61) [info] at org.scalatest.tools.Framework.org$scalatest$tools$Framework$$runSuite(Framework.scala:318) [info] at org.scalatest.tools.Framework$ScalaTestTask.execute(Framework.scala:513) [info] at sbt.ForkMain$Run.lambda$runTest$1(ForkMain.java:413) [info] at java.util.concurrent.FutureTask.run(FutureTask.java:266) [info] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [info] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [info] at java.lang.Thread.run(Thread.java:748) [info] Cause: java.lang.RuntimeException: Error while decoding: java.lang.NoSuchMethodException: org.apache.spark.sql.catalyst.util.GenericArrayData.toIntArray() [info] mapobjects(lambdavariable(MapObject, IntegerType, false, -1), assertnotnull(lambdavariable(MapObject, IntegerType, false, -1)), input[0, array<int>, true], None).toIntArray [info] at org.apache.spark.sql.catalyst.encoders.ExpressionEncoder$Deserializer.apply(ExpressionEncoder.scala:186) [info] at org.apache.spark.sql.catalyst.encoders.ExpressionEncoderSuite.$anonfun$encodeDecodeTest$1(ExpressionEncoderSuite.scala:576) [info] at org.apache.spark.sql.catalyst.encoders.ExpressionEncoderSuite.verifyNotLeakingReflectionObjects(ExpressionEncoderSuite.scala:656) [info] at org.apache.spark.sql.catalyst.encoders.ExpressionEncoderSuite.$anonfun$testAndVerifyNotLeakingReflectionObjects$2(ExpressionEncoderSuite.scala:669)

This test suite fix aims to land to master/3.1/3.0/2.4.

sounds good, let's fix these bugs one by one later.

dongjoon-hyun · 2021-03-11T07:41:16Z

Thank you all!

…correct test cases ### What changes were proposed in this pull request? SPARK-23596 added `CodegenInterpretedPlanTest` at Apache Spark 2.4.0 in a wrong way because `withSQLConf` depends on the execution time `SQLConf.get` instead of `test` function declaration time. So, the following code executes the test twice without controlling the `CodegenObjectFactoryMode`. This PR aims to fix it correct and introduce a new function `testFallback`. ```scala trait CodegenInterpretedPlanTest extends PlanTest { override protected def test( testName: String, testTags: Tag*)(testFun: => Any)(implicit pos: source.Position): Unit = { val codegenMode = CodegenObjectFactoryMode.CODEGEN_ONLY.toString val interpretedMode = CodegenObjectFactoryMode.NO_CODEGEN.toString withSQLConf(SQLConf.CODEGEN_FACTORY_MODE.key -> codegenMode) { super.test(testName + " (codegen path)", testTags: _*)(testFun)(pos) } withSQLConf(SQLConf.CODEGEN_FACTORY_MODE.key -> interpretedMode) { super.test(testName + " (interpreted path)", testTags: _*)(testFun)(pos) } } } ``` ### Why are the changes needed? 1. We need to use like the following. ```scala super.test(testName + " (codegen path)", testTags: _*)( withSQLConf(SQLConf.CODEGEN_FACTORY_MODE.key -> codegenMode) { testFun })(pos) super.test(testName + " (interpreted path)", testTags: _*)( withSQLConf(SQLConf.CODEGEN_FACTORY_MODE.key -> interpretedMode) { testFun })(pos) ``` 2. After we fix this behavior with the above code, several test cases including SPARK-34596 and SPARK-34607 fail because they didn't work at both `CODEGEN` and `INTERPRETED` mode. Those test cases only work at `FALLBACK` mode. So, inevitably, we need to introduce `testFallback`. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Pass the CIs. Closes #31766 from dongjoon-hyun/SPARK-34596-SPARK-34607. Lead-authored-by: Dongjoon Hyun <[email protected]> Co-authored-by: Dongjoon Hyun <[email protected]> Signed-off-by: Dongjoon Hyun <[email protected]> (cherry picked from commit 5c4d8f9) Signed-off-by: Dongjoon Hyun <[email protected]>

…correct test cases SPARK-23596 added `CodegenInterpretedPlanTest` at Apache Spark 2.4.0 in a wrong way because `withSQLConf` depends on the execution time `SQLConf.get` instead of `test` function declaration time. So, the following code executes the test twice without controlling the `CodegenObjectFactoryMode`. This PR aims to fix it correct and introduce a new function `testFallback`. ```scala trait CodegenInterpretedPlanTest extends PlanTest { override protected def test( testName: String, testTags: Tag*)(testFun: => Any)(implicit pos: source.Position): Unit = { val codegenMode = CodegenObjectFactoryMode.CODEGEN_ONLY.toString val interpretedMode = CodegenObjectFactoryMode.NO_CODEGEN.toString withSQLConf(SQLConf.CODEGEN_FACTORY_MODE.key -> codegenMode) { super.test(testName + " (codegen path)", testTags: _*)(testFun)(pos) } withSQLConf(SQLConf.CODEGEN_FACTORY_MODE.key -> interpretedMode) { super.test(testName + " (interpreted path)", testTags: _*)(testFun)(pos) } } } ``` 1. We need to use like the following. ```scala super.test(testName + " (codegen path)", testTags: _*)( withSQLConf(SQLConf.CODEGEN_FACTORY_MODE.key -> codegenMode) { testFun })(pos) super.test(testName + " (interpreted path)", testTags: _*)( withSQLConf(SQLConf.CODEGEN_FACTORY_MODE.key -> interpretedMode) { testFun })(pos) ``` 2. After we fix this behavior with the above code, several test cases including SPARK-34596 and SPARK-34607 fail because they didn't work at both `CODEGEN` and `INTERPRETED` mode. Those test cases only work at `FALLBACK` mode. So, inevitably, we need to introduce `testFallback`. No. Pass the CIs. Closes #31766 from dongjoon-hyun/SPARK-34596-SPARK-34607. Lead-authored-by: Dongjoon Hyun <[email protected]> Co-authored-by: Dongjoon Hyun <[email protected]> Signed-off-by: Dongjoon Hyun <[email protected]> (cherry picked from commit 5c4d8f9) Signed-off-by: Dongjoon Hyun <[email protected]>

kiszk · 2021-03-11T12:08:05Z

Late LGTM, thank you!

dongjoon-hyun · 2021-03-11T16:52:06Z

Thank you, @kiszk .

maropu · 2021-03-12T01:04:20Z

late lgtm. Nice fix for making these hidden bugs clearer!

…correct test cases ### What changes were proposed in this pull request? SPARK-23596 added `CodegenInterpretedPlanTest` at Apache Spark 2.4.0 in a wrong way because `withSQLConf` depends on the execution time `SQLConf.get` instead of `test` function declaration time. So, the following code executes the test twice without controlling the `CodegenObjectFactoryMode`. This PR aims to fix it correct and introduce a new function `testFallback`. ```scala trait CodegenInterpretedPlanTest extends PlanTest { override protected def test( testName: String, testTags: Tag*)(testFun: => Any)(implicit pos: source.Position): Unit = { val codegenMode = CodegenObjectFactoryMode.CODEGEN_ONLY.toString val interpretedMode = CodegenObjectFactoryMode.NO_CODEGEN.toString withSQLConf(SQLConf.CODEGEN_FACTORY_MODE.key -> codegenMode) { super.test(testName + " (codegen path)", testTags: _*)(testFun)(pos) } withSQLConf(SQLConf.CODEGEN_FACTORY_MODE.key -> interpretedMode) { super.test(testName + " (interpreted path)", testTags: _*)(testFun)(pos) } } } ``` ### Why are the changes needed? 1. We need to use like the following. ```scala super.test(testName + " (codegen path)", testTags: _*)( withSQLConf(SQLConf.CODEGEN_FACTORY_MODE.key -> codegenMode) { testFun })(pos) super.test(testName + " (interpreted path)", testTags: _*)( withSQLConf(SQLConf.CODEGEN_FACTORY_MODE.key -> interpretedMode) { testFun })(pos) ``` 2. After we fix this behavior with the above code, several test cases including SPARK-34596 and SPARK-34607 fail because they didn't work at both `CODEGEN` and `INTERPRETED` mode. Those test cases only work at `FALLBACK` mode. So, inevitably, we need to introduce `testFallback`. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Pass the CIs. Closes apache#31766 from dongjoon-hyun/SPARK-34596-SPARK-34607. Lead-authored-by: Dongjoon Hyun <[email protected]> Co-authored-by: Dongjoon Hyun <[email protected]> Signed-off-by: Dongjoon Hyun <[email protected]> (cherry picked from commit 5c4d8f9) Signed-off-by: Dongjoon Hyun <[email protected]>

github-actions bot added the SQL label Mar 6, 2021

This was referenced Mar 6, 2021

[SPARK-34596][SPARK-34607][SQL][FOLLOWUP][TESTS][3.1] Add PlanTest. testFallback and use it #31764

Closed

[SPARK-34596][SQL] Use Utils.getSimpleName to avoid hitting Malformed class name in NewInstance.doGenCode #31709

Closed

This comment has been minimized.

Sign in to view

dongjoon-hyun marked this pull request as draft March 8, 2021 07:35

dongjoon-hyun marked this pull request as ready for review March 10, 2021 07:29

cloud-fan reviewed Mar 10, 2021

View reviewed changes

sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/plans/PlanTest.scala Outdated Show resolved Hide resolved

This comment has been minimized.

Sign in to view

dongjoon-hyun changed the title ~~[SPARK-34596][SPARK-34607][SQL][FOLLOWUP][TESTS] Add PlanTest. testFallback and use it~~ [SPARK-34696][SQL][TESTS] Fix PlanTest to generate correct test cases Mar 10, 2021

dongjoon-hyun changed the title ~~[SPARK-34696][SQL][TESTS] Fix PlanTest to generate correct test cases~~ [SPARK-34696][SQL][TESTS] Fix CodegenInterpretedPlanTest to generate correct test cases Mar 10, 2021

[SPARK-34696][SQL][TESTS] Fix CodegenInterpretedPlanTest to generate …

0b2915d

…correct test cases

dongjoon-hyun commented Mar 10, 2021

View reviewed changes

sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/plans/PlanTest.scala Show resolved Hide resolved

use fallback for all failures

54636fc

dongjoon-hyun requested a review from kiszk March 10, 2021 19:15

dongjoon-hyun requested a review from hvanhovell March 10, 2021 20:05

dongjoon-hyun mentioned this pull request Mar 10, 2021

[SPARK-34607][SQL][2.4] Add Utils.isMemberClass to fix a malformed class name error on jdk8u #31747

Closed

cloud-fan reviewed Mar 11, 2021

View reviewed changes

cloud-fan approved these changes Mar 11, 2021

View reviewed changes

viirya approved these changes Mar 11, 2021

View reviewed changes

dongjoon-hyun closed this in 5c4d8f9 Mar 11, 2021

dongjoon-hyun deleted the SPARK-34596-SPARK-34607 branch March 11, 2021 16:51

dongjoon-hyun mentioned this pull request Mar 12, 2021

[SPARK-34724][SQL] Fix Interpreted evaluation by using getMethod instead of getDeclaredMethod #31816

Closed

eejbyfeldt mentioned this pull request Jun 8, 2021

[SPARK-20384][SQL] Support value class in schema of Dataset (without breaking existing current projection) #27153

Closed

[SPARK-34696][SQL][TESTS] Fix CodegenInterpretedPlanTest to generate correct test cases #31766

[SPARK-34696][SQL][TESTS] Fix CodegenInterpretedPlanTest to generate correct test cases #31766

Uh oh!

Conversation

dongjoon-hyun commented Mar 6, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What changes were proposed in this pull request?

Why are the changes needed?

Does this PR introduce any user-facing change?

How was this patch tested?

Uh oh!

This comment has been minimized.

dongjoon-hyun commented Mar 10, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

dongjoon-hyun commented Mar 10, 2021

Uh oh!

Uh oh!

SparkQA commented Mar 10, 2021

Uh oh!

SparkQA commented Mar 10, 2021

Uh oh!

SparkQA commented Mar 10, 2021

Uh oh!

SparkQA commented Mar 10, 2021

Uh oh!

SparkQA commented Mar 10, 2021

Uh oh!

HyukjinKwon commented Mar 11, 2021

Uh oh!

SparkQA commented Mar 11, 2021

Uh oh!

SparkQA commented Mar 11, 2021

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dongjoon-hyun Mar 11, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

SparkQA commented Mar 11, 2021

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dongjoon-hyun Mar 11, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dongjoon-hyun commented Mar 11, 2021

Uh oh!

kiszk commented Mar 11, 2021

Uh oh!

dongjoon-hyun commented Mar 11, 2021

Uh oh!

maropu commented Mar 12, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

dongjoon-hyun commented Mar 6, 2021 •

edited

Loading

dongjoon-hyun commented Mar 10, 2021 •

edited

Loading

dongjoon-hyun Mar 11, 2021 •

edited

Loading

dongjoon-hyun Mar 11, 2021 •

edited

Loading