[SPARK-3797] Run external shuffle service in Yarn NM#3082

Closed

andrewor14 wants to merge 23 commits intoapache:masterfrom

andrewor14:yarn-shuffle-service

Commits on Nov 3, 2014

Initial skeleton for Yarn shuffle service
Andrew Or
committed
First cut integration of shuffle service with Yarn aux service
Andrew Or
committed
4 tabs -> 2 tabs
Andrew Or
committed

Commits on Nov 4, 2014

Use the shuffle service port specified through hadoop config
Andrew Or
committed
Connect to Yarn shuffle service only if it's enabled
Andrew Or
committed
Require external shuffle service for dynamic allocation
Andrew Or
committed
Include the Yarn shuffle service jar in the distribution
Andrew Or
committed
Add missing license header
Andrew Or
committed
Address a few minor comments
Andrew Or
committed
Fix tests
Andrew Or
committed
Fix build for Hadoop 1.x
Andrew Or
committed
Merge branch 'master' of github.com:apache/spark into yarn-shuffle-service
Andrew Or
committed
Do not make network-yarn an assembly module
Andrew Or
committed
Fix tests again
Andrew Or
committed
Address various feedback
Andrew Or
committed

Commits on Nov 5, 2014

Merge branch 'master' of github.com:apache/spark into yarn-shuffle-service
Andrew Or
committed
Add security to shuffle service (INCOMPLETE)
Andrew Or
committed
Add detailed java docs + reword a few comments
Andrew Or
committed
Try catch at the right places
Andrew Or
committed
Merge branch 'master' of github.com:apache/spark into yarn-shuffle-service
Andrew Or
committed
Remove unused provided dependencies
Andrew Or
committed
Minor wording suggestions
Andrew Or
committed
Merge branch 'master' of github.com:apache/spark into yarn-shuffle-service
Andrew Or
committed