Skip to content

Conversation

@HyukjinKwon
Copy link
Member

@HyukjinKwon HyukjinKwon commented Jun 23, 2020

What changes were proposed in this pull request?

R version 4.0.2 was released, see https://cran.r-project.org/doc/manuals/r-release/NEWS.html. This PR targets to upgrade R version in AppVeyor CI environment.

Why are the changes needed?

To test the latest R versions before the release, and see if there are any regressions.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

AppVeyor will test.

@HyukjinKwon HyukjinKwon changed the title [SPARK-32074][BUILD][R] Update AppVeyor R version to 3.6.2 [SPARK-32074][BUILD][R] Update AppVeyor R version to 4.0.2 Jun 23, 2020
@SparkQA
Copy link

SparkQA commented Jun 23, 2020

Test build #124422 has finished for PR 28909 at commit 281b9c1.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@HyukjinKwon HyukjinKwon changed the title [SPARK-32074][BUILD][R] Update AppVeyor R version to 4.0.2 [WIP][SPARK-32074][BUILD][R] Update AppVeyor R version to 4.0.2 Jun 23, 2020
@SparkQA
Copy link

SparkQA commented Jun 24, 2020

Test build #124453 has finished for PR 28909 at commit 4b0a539.

  • This patch fails some tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Jun 24, 2020

Test build #124455 has finished for PR 28909 at commit fa4729b.

  • This patch fails some tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Jun 24, 2020

Test build #124457 has finished for PR 28909 at commit 22f2481.

  • This patch fails some tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Jun 24, 2020

Test build #124458 has finished for PR 28909 at commit 1a6c3b0.

  • This patch fails build dependency tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Jun 24, 2020

Test build #124459 has finished for PR 28909 at commit 1d7cfaf.

  • This patch fails build dependency tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@HyukjinKwon HyukjinKwon changed the title [WIP][SPARK-32074][BUILD][R] Update AppVeyor R version to 4.0.2 [SPARK-32074][BUILD][R] Update AppVeyor R version to 4.0.2 Jun 24, 2020
@SparkQA
Copy link

SparkQA commented Jun 24, 2020

Test build #124462 has finished for PR 28909 at commit 5027b09.

  • This patch fails build dependency tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Jun 24, 2020

Test build #124449 has finished for PR 28909 at commit ed00363.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@HyukjinKwon
Copy link
Member Author

The tests passed:

Spark package found in SPARK_HOME: C:\projects\spark\bin\..
== testthat results  ===========================================================
[ OK: 13 | SKIPPED: 0 | WARNINGS: 0 | FAILED: 0 ]
v |  OK F W S | Context
v |  11       | binary functions [23.7 s]
v |   4       | functions on binary files [6.2 s]
v |   2       | broadcast variables [2.2 s]
v |   5       | functions in client.R
v |  46       | test functions in sparkR.R [17.2 s]
v |   2       | include R packages [0.2 s]
v |   2       | JVM API [0.2 s]
v |  75       | MLlib classification algorithms, except for tree-based algorithms [127.3 s]
v |  70       | MLlib clustering algorithms [47.9 s]
v |   6       | MLlib frequent pattern mining [9.1 s]
v |   8       | MLlib recommendation algorithms [13.2 s]
v | 136       | MLlib regression algorithms, except for tree-based algorithms [311.6 s]
v |   8       | MLlib statistics algorithms [3.8 s]
v |  94       | MLlib tree-based algorithms [118.1 s]
v |  29       | parallelize() and collect() [0.4 s]
v | 428       | basic RDD functions [337.5 s]
v |  39       | SerDe functionality [12.3 s]
v |  20       | partitionBy, groupByKey, reduceByKey etc. [44.7 s]
v |   4       | functions in sparkR.R
v | 1153     5 | SparkSQL functions [468.2 s]
--------------------------------------------------------------------------------
test_sparkSQL.R:307: skip: create DataFrame from RDD
Reason: Hive is not build with SparkSQL, skipped
test_sparkSQL.R:1341: skip: test HiveContext
Reason: Hive is not build with SparkSQL, skipped
test_sparkSQL.R:2813: skip: read/write ORC files
Reason: Hive is not build with SparkSQL, skipped
test_sparkSQL.R:2834: skip: read/write ORC files - compression option
Reason: Hive is not build with SparkSQL, skipped
test_sparkSQL.R:3727: skip: enableHiveSupport on SparkSession
Reason: Hive is not build with SparkSQL, skipped
--------------------------------------------------------------------------------
v |  16       | SparkSQL Arrow optimization [571.4 s]
v |   6       | test show SparkDataFrame when eager execution is enabled. [2.6 s]
v |  42       | Structured Streaming [109.2 s]
v |  16       | tests RDD function take() [0.6 s]
v |  14       | the textFile() function [17.9 s]
v |  45       | functions in utils.R [2.3 s]
v |   1       | Windows-specific tests [0.2 s]
== Results =====================================================================
Duration: 2248.2 s
OK:       2282
Failed:   0
Warnings: 0
Skipped:  5
Build success

The last commit only added some comments

@HyukjinKwon
Copy link
Member Author

cc @shivaram, @felixcheung and @dongjoon-hyun, I am going to merge this as it doesn't affect anything in other PR builders, and the tests passed in AppVeyor.

Now, SparkR tests on R 4.0.2 with Windows via AppVeyor, and on R 3.5.2 with Linux via Jenkins.

@HyukjinKwon
Copy link
Member Author

Merged to master.

@HyukjinKwon HyukjinKwon reopened this Jun 24, 2020
@SparkQA
Copy link

SparkQA commented Jun 24, 2020

Test build #124460 has finished for PR 28909 at commit 741b7a5.

  • This patch fails due to an unknown error code, -9.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Jun 24, 2020

Test build #124456 has finished for PR 28909 at commit 5b1929a.

  • This patch fails due to an unknown error code, -9.
  • This patch merges cleanly.
  • This patch adds no public classes.

@HyukjinKwon HyukjinKwon deleted the SPARK-32074 branch July 27, 2020 07:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants