Skip to content

Conversation

@srowen
Copy link
Member

@srowen srowen commented Apr 21, 2021

What changes were proposed in this pull request?

Use new Apache 'closer.lua' syntax to obtain Maven

Why are the changes needed?

The current closer.lua redirector, which redirects to download Maven from a local mirror, has a new syntax. build/mvn does not work properly otherwise now.

Does this PR introduce any user-facing change?

No.

How was this patch tested?

Manual testing.

@github-actions github-actions bot added the BUILD label Apr 21, 2021
@srowen
Copy link
Member Author

srowen commented Apr 21, 2021

Note we should backport this all the way to 2.4, as it would affect all builds

@srowen
Copy link
Member Author

srowen commented Apr 21, 2021

Wait a beat - https://issues.apache.org/jira/browse/INFRA-21767 says there is a simpler updated URL to use.

@srowen srowen changed the title [SPARK-35178][BUILD] Use Apache 'closer' service to obtain mirror host [SPARK-35178][BUILD] Use new Apache 'closer.lua' syntax to obtain Maven Apr 21, 2021
@SparkQA
Copy link

SparkQA commented Apr 21, 2021

Kubernetes integration test starting
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42272/

@SparkQA
Copy link

SparkQA commented Apr 21, 2021

Kubernetes integration test status failure
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42272/

@SparkQA
Copy link

SparkQA commented Apr 21, 2021

Kubernetes integration test unable to build dist.

exiting with code: 1
URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42275/

@SparkQA
Copy link

SparkQA commented Apr 21, 2021

Test build #137745 has finished for PR 32277 at commit 5169b16.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Apr 21, 2021

Test build #137748 has finished for PR 32277 at commit 9a79232.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

Copy link
Member

@dongjoon-hyun dongjoon-hyun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, LGTM. Thank you, @srowen .
I checked that this works correctly in your GitHub Action job (linter).
And, also checked manually.

$ build/mvn clean
exec: curl --silent --show-error -L https://downloads.lightbend.com/scala/2.12.10/scala-2.12.10.tgz
exec: curl --silent --show-error -L https://www.apache.org/dyn/closer.lua/maven/maven-3/3.6.3/binaries/apache-maven-3.6.3-bin.tar.gz?action=download
Using `mvn` from path: /Users/dongjoon/PRS/SPARK-PR-32277/build/apache-maven-3.6.3/bin/mvn
...

dongjoon-hyun pushed a commit that referenced this pull request Apr 22, 2021
### What changes were proposed in this pull request?

Use new Apache 'closer.lua' syntax to obtain Maven

### Why are the changes needed?

The current closer.lua redirector, which redirects to download Maven from a local mirror, has a new syntax. build/mvn does not work properly otherwise now.

### Does this PR introduce _any_ user-facing change?

No.

### How was this patch tested?

Manual testing.

Closes #32277 from srowen/SPARK-35178.

Authored-by: Sean Owen <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
(cherry picked from commit 6860efe)
Signed-off-by: Dongjoon Hyun <[email protected]>
dongjoon-hyun pushed a commit that referenced this pull request Apr 22, 2021
### What changes were proposed in this pull request?

Use new Apache 'closer.lua' syntax to obtain Maven

### Why are the changes needed?

The current closer.lua redirector, which redirects to download Maven from a local mirror, has a new syntax. build/mvn does not work properly otherwise now.

### Does this PR introduce _any_ user-facing change?

No.

### How was this patch tested?

Manual testing.

Closes #32277 from srowen/SPARK-35178.

Authored-by: Sean Owen <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
(cherry picked from commit 6860efe)
Signed-off-by: Dongjoon Hyun <[email protected]>
@dongjoon-hyun
Copy link
Member

Merged to master/3.1/3.0. For 2.4, could you make a backport PR?

@dongjoon-hyun
Copy link
Member

Oh, 3.1/3.0 has Zinc installation. I'll make a follow-up for 3.1/3.0.

@dongjoon-hyun
Copy link
Member

Here is the follow-up to update Zinc.

@srowen srowen deleted the SPARK-35178 branch June 1, 2021 13:48
flyrain pushed a commit to flyrain/spark that referenced this pull request Sep 21, 2021
### What changes were proposed in this pull request?

Use new Apache 'closer.lua' syntax to obtain Maven

### Why are the changes needed?

The current closer.lua redirector, which redirects to download Maven from a local mirror, has a new syntax. build/mvn does not work properly otherwise now.

### Does this PR introduce _any_ user-facing change?

No.

### How was this patch tested?

Manual testing.

Closes apache#32277 from srowen/SPARK-35178.

Authored-by: Sean Owen <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
(cherry picked from commit 6860efe)
Signed-off-by: Dongjoon Hyun <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants