Skip to content

Conversation

@markgrover
Copy link
Member

Upgrade Jetty so it can work with Hadoop 3 (alpha 2 release, in particular).
Without this change, because of incompatibily between Jetty versions,
Spark fails to compile when built against Hadoop 3

How was this patch tested?

Unit tests being run.

Upgrade Jetty so it can work with Hadoop 3 (alpha 2 release, in particular).
Without this change, because of incompatibily between Jetty versions,
Spark fails to compile when built against Hadoop 3
@markgrover
Copy link
Member Author

I am running unit tests locally and while the run hasn't finished, it's looking pretty good so far. I think it'd be good to get Jenkins to run the tests here anyways so putting a PR sooner than later.

<parquet.version>1.8.2</parquet.version>
<hive.parquet.version>1.6.0</hive.parquet.version>
<jetty.version>9.2.16.v20160414</jetty.version>
<jetty.version>9.3.11.v20160721</jetty.version>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR title says one thing but your code says another.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, thanks, there was some confusion about this. I have fixed the JIRA title and the PR title to match up with the actual code - which uses the exact same version as Apache Hadoop (9.3.11.v20160721).

@srowen
Copy link
Member

srowen commented Apr 27, 2017

CC @zsxwing as it touches Jetty.
This is probably OK, even though it will impact the classpath for user apps and Jetty could have some non-trivial changes. I agree there's good reason to do this though.

@vanzin
Copy link
Contributor

vanzin commented Apr 27, 2017

it will impact the classpath for user apps

Jetty is shaded in Spark.

@zsxwing
Copy link
Member

zsxwing commented Apr 27, 2017

cc @yhuai

@markgrover markgrover changed the title [SPARK-20514][CORE] Upgrade Jetty to 9.3.13.v20161014 [SPARK-20514][CORE] Upgrade Jetty to 9.3.11.v20160721 Apr 27, 2017
@SparkQA
Copy link

SparkQA commented Apr 28, 2017

Test build #76244 has finished for PR 17790 at commit ecfb8e3.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@vanzin
Copy link
Contributor

vanzin commented Apr 28, 2017

LGTM.

@markgrover
Copy link
Member Author

Thanks for the review!

@vanzin
Copy link
Contributor

vanzin commented Apr 28, 2017

If there are no more concerns I'll push this in the PM.

@zsxwing
Copy link
Member

zsxwing commented Apr 28, 2017

LGTM.

@vanzin
Copy link
Contributor

vanzin commented Apr 28, 2017

Merging to master / 2.2.

asfgit pushed a commit that referenced this pull request Apr 28, 2017
Upgrade Jetty so it can work with Hadoop 3 (alpha 2 release, in particular).
Without this change, because of incompatibily between Jetty versions,
Spark fails to compile when built against Hadoop 3

## How was this patch tested?
Unit tests being run.

Author: Mark Grover <[email protected]>

Closes #17790 from markgrover/spark-20514.

(cherry picked from commit 5d71f3d)
Signed-off-by: Marcelo Vanzin <[email protected]>
@asfgit asfgit closed this in 5d71f3d Apr 28, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants