Skip to content

[SPARK-27179][BUILD] Exclude javax.ws.rs:jsr311-api from hadoop-client#24114

Closed
wangyum wants to merge 1 commit intoapache:masterfrom
wangyum:SPARK-27179
Closed

[SPARK-27179][BUILD] Exclude javax.ws.rs:jsr311-api from hadoop-client#24114
wangyum wants to merge 1 commit intoapache:masterfrom
wangyum:SPARK-27179

Conversation

@wangyum
Copy link
Copy Markdown
Member

@wangyum wangyum commented Mar 16, 2019 •
edited
Loading

What changes were proposed in this pull request?

Since YARN-7113(Hadoop-3.1.0), hadoop-client add javax.ws.rs:jsr311-api to its dependency. This conflict with javax.ws.rs-api-2.0.1.jar.

build/sbt  "core/testOnly *.UISeleniumSuite *.HistoryServerSuite" -Phadoop-3.2
...
[info] <pre>    Server Error</pre></p><h3>Caused by:</h3><pre>java.lang.NoSuchMethodError: javax.ws.rs.core.Application.getProperties()Ljava/util/Map;
...

This pr exclude javax.ws.rs:jsr311-api from hadoop-client.

How was this patch tested?

manual tests:

build/sbt  "core/testOnly *.UISeleniumSuite *.HistoryServerSuite" -Phadoop-3.2

@SparkQA
Copy link
Copy Markdown

SparkQA commented Mar 16, 2019

Test build #103562 has finished for PR 24114 at commit 082c313.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@wangyum
Copy link
Copy Markdown
Member Author

wangyum commented Mar 16, 2019

retest this please

felixcheung
felixcheung reviewed Mar 16, 2019
View reviewed changes
Copy link
Copy Markdown
Member

@felixcheung felixcheung left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it looks like jenkins is only -Phadoop-2.7 ? is there a way to build -Phadoop-3.1?

@wangyum
Copy link
Copy Markdown
Member Author

wangyum commented Mar 16, 2019

We can build it. But it won't succeed because the built-in Hive is still 1.2.1.
#24045

srowen
srowen approved these changes Mar 17, 2019
View reviewed changes
Copy link
Copy Markdown
Member

@srowen srowen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The JSR 311 jar would be the old version of the same API right? this ought to be OK.

@wangyum
Copy link
Copy Markdown
Member Author

wangyum commented Mar 17, 2019

jsr311-api - This is the official spec jar for the JAX-RS 1.x line.
javax.ws.rs-api - This is the official spec jar for the JAX-RS 2.x line.

https://stackoverflow.com/questions/25264669/what-is-the-difference-between-rs-api-and-jsr311-api
https://stackoverflow.com/questions/32106428/jaxrs-api-vs-jsr311-api-vs-javax-ws-rs-api-vs-jersey-core-vs-jaxrs-ri

@SparkQA
Copy link
Copy Markdown

SparkQA commented Mar 17, 2019

Test build #103575 has finished for PR 24114 at commit 082c313.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@wangyum
Copy link
Copy Markdown
Member Author

wangyum commented Mar 17, 2019

retest this please

@SparkQA
Copy link
Copy Markdown

SparkQA commented Mar 17, 2019

Test build #103583 has finished for PR 24114 at commit 082c313.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@srowen
Copy link
Copy Markdown
Member

srowen commented Mar 18, 2019

This should be fine as these are API classes and 2.0 should be a superset of 1.0 anyway

@srowen
Copy link
Copy Markdown
Member

srowen commented Mar 19, 2019

Merged to master

@srowen srowen closed this in 6783831 Mar 19, 2019
@wangyum wangyum deleted the SPARK-27179 branch March 20, 2019 05:28
lwwmanning pushed a commit to palantir/spark that referenced this pull request Jun 19, 2020
@wangyum
[SPARK-27179][BUILD] Exclude javax.ws.rs:jsr311-api from hadoop-client
d00feec 
## What changes were proposed in this pull request?
Since [YARN-7113](https://issues.apache.org/jira/browse/YARN-7113)(Hadoop-3.1.0), `hadoop-client` add `javax.ws.rs:jsr311-api` to its dependency. This conflict with [javax.ws.rs-api-2.0.1.jar](https://github.com/apache/spark/blob/f26a1f3d3766207595af6cb26d62d54218f5ac1d/dev/deps/spark-deps-hadoop-3.1#L105).
```shell
build/sbt  "core/testOnly *.UISeleniumSuite *.HistoryServerSuite" -Phadoop-3.2
...
[info] <pre>    Server Error</pre></p><h3>Caused by:</h3><pre>java.lang.NoSuchMethodError: javax.ws.rs.core.Application.getProperties()Ljava/util/Map;
...
```

This pr exclude `javax.ws.rs:jsr311-api` from hadoop-client.

## How was this patch tested?

manual tests:
```shell
build/sbt  "core/testOnly *.UISeleniumSuite *.HistoryServerSuite" -Phadoop-3.2
```

Closes apache#24114 from wangyum/SPARK-27179.

Authored-by: Yuming Wang <yumwang@ebay.com>
Signed-off-by: Sean Owen <sean.owen@databricks.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@felixcheung felixcheung felixcheung left review comments

@srowen srowen srowen approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@wangyum @SparkQA @srowen @felixcheung