Skip to content

Conversation

@wangyum
Copy link
Member

@wangyum wangyum commented May 19, 2019

What changes were proposed in this pull request?

This pr fix hadoop-3.2 test error:

- SPARK-27699 Converting disjunctions into ORC SearchArguments *** FAILED ***
  Expected "...SS_THAN_EQUALS a 10)[
  leaf-1 = (LESS_THAN a 1)
  ]expr = (or (not leaf...", but got "...SS_THAN_EQUALS a 10)[, leaf-1 = (LESS_THAN a 1), ]expr = (or (not leaf..." (HiveOrcFilterSuite.scala:445)

https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/105514/consoleFull

How was this patch tested?

N/A

Copy link
Member

@dongjoon-hyun dongjoon-hyun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, LGTM. (Pending Jenkins). Thank you, @wangyum !

@SparkQA
Copy link

SparkQA commented May 19, 2019

Test build #105520 has finished for PR 24639 at commit 5cc0ceb.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@dongjoon-hyun
Copy link
Member

Retest this please.

@SparkQA
Copy link

SparkQA commented May 19, 2019

Test build #105523 has finished for PR 24639 at commit 5cc0ceb.

  • This patch fails due to an unknown error code, -9.
  • This patch merges cleanly.
  • This patch adds no public classes.

@HyukjinKwon
Copy link
Member

retest this please

@SparkQA
Copy link

SparkQA commented May 19, 2019

Test build #105528 has finished for PR 24639 at commit 5cc0ceb.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@wangyum
Copy link
Member Author

wangyum commented May 19, 2019

@dongjoon-hyun @HyukjinKwon Could we merge this first. I don't know why it didn't skip the hive-thriftserver module. This module should be skipped since SPARK-27402. The hive-thriftserver module should be supported since SPARK-27749, but SPARK-27749 need this PR.

========================================================================
Running Spark unit tests
========================================================================
[info] Running Spark tests using Maven with these arguments:  -Phadoop-3.2 -Phive-thriftserver -Phive -Dtest.exclude.tags=org.apache.spark.tags.ExtendedYarnTest test --fail-at-end

...

[INFO] Reactor Summary for Spark Project Parent POM 3.0.0-SNAPSHOT:
[INFO] 
[INFO] Spark Project Parent POM ........................... SUCCESS [  3.144 s]
[INFO] Spark Project Tags ................................. SUCCESS [  2.686 s]
[INFO] Spark Project Sketch ............................... SUCCESS [ 14.802 s]
[INFO] Spark Project Local DB ............................. SUCCESS [  4.504 s]
[INFO] Spark Project Networking ........................... SUCCESS [ 49.985 s]
[INFO] Spark Project Shuffle Streaming Service ............ SUCCESS [ 10.712 s]
[INFO] Spark Project Unsafe ............................... SUCCESS [  2.982 s]
[INFO] Spark Project Launcher ............................. SUCCESS [  4.089 s]
[INFO] Spark Project Core ................................. SUCCESS [20:38 min]
[INFO] Spark Project ML Local Library ..................... SUCCESS [  5.313 s]
[INFO] Spark Project GraphX ............................... SUCCESS [01:38 min]
[INFO] Spark Project Streaming ............................ SUCCESS [05:47 min]
[INFO] Spark Project Catalyst ............................. SUCCESS [09:21 min]
[INFO] Spark Project SQL .................................. SUCCESS [  01:04 h]
[INFO] Spark Project ML Library ........................... SUCCESS [31:09 min]
[INFO] Spark Project Tools ................................ SUCCESS [  1.047 s]
[INFO] Spark Project Hive ................................. SUCCESS [  01:29 h]
[INFO] Spark Project REPL ................................. SUCCESS [01:39 min]
[INFO] Spark Project Hive Thrift Server ................... FAILURE [ 14.892 s]
[INFO] Spark Project Assembly ............................. SKIPPED
[INFO] Kafka 0.10+ Token Provider for Streaming ........... SUCCESS [ 10.002 s]
[INFO] Spark Integration for Kafka 0.10 ................... SUCCESS [01:31 min]
[INFO] Kafka 0.10+ Source for Structured Streaming ........ SUCCESS [13:33 min]
[INFO] Spark Project Examples ............................. SUCCESS [ 33.803 s]
[INFO] Spark Integration for Kafka 0.10 Assembly .......... SUCCESS [  1.255 s]
[INFO] Spark Avro ......................................... SUCCESS [01:20 min]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  04:02 h
[INFO] Finished at: 2019-05-18T22:50:32-07:00

@dongjoon-hyun
Copy link
Member

Sorry, @wangyum . We cannot merge like that.

@dongjoon-hyun
Copy link
Member

cc @gatorsmile since he worked with you at the previous commit about test-hadoop-3.2.

@wangyum
Copy link
Member Author

wangyum commented May 20, 2019

Test this change in another PR: #24628 (comment)

@dongjoon-hyun
Copy link
Member

@wangyum . I reopened https://issues.apache.org/jira/browse/SPARK-27402 . Please make an independent follow-up PR using ID SPARK-27402. That will give us traceability.

@HyukjinKwon
Copy link
Member

I made that #24644

@dongjoon-hyun
Copy link
Member

Since #24644 is merged, I'll retrigger this.

@dongjoon-hyun
Copy link
Member

Retest this please.

@dongjoon-hyun
Copy link
Member

dongjoon-hyun commented May 20, 2019

I checked the used profile. It looks correct.

[info] Building Spark using Maven with these arguments:
-Phadoop-3.2 -Pkubernetes -Pkinesis-asl -Pyarn -Pspark-ganglia-lgpl
-Phive -Pmesos clean package -DskipTests

@SparkQA
Copy link

SparkQA commented May 20, 2019

Test build #105573 has finished for PR 24639 at commit 5cc0ceb.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@dongjoon-hyun
Copy link
Member

Thank you, @wangyum and @HyukjinKwon . Merged to master.

@wangyum wangyum deleted the SPARK-27699 branch May 20, 2019 21:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants