Skip to content

[SPARK-6469] Improving documentation on YARN local directories usage #5165

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
preaudc wants to merge 12 commits into from
Closed

[SPARK-6469] Improving documentation on YARN local directories usage #5165

preaudc wants to merge 12 commits into from

Conversation

@preaudc
Copy link

@preaudc preaudc commented Mar 24, 2015

Clarify the local directories usage in YARN

Christophe Préaud added 10 commits October 20, 2014 11:58
Ensure that files are fetched atomically
8ea871f 
tempFile is created in the same directory than targetFile, so that the
move from tempFile to targetFile is always atomic
Merge remote-tracking branch 'upstream/master'
7456a33
Revert commit 8ea871f
c6a5590
Merge remote-tracking branch 'upstream/master'
54419ae
Ensure that files are fetched atomically
9ba89ca 
tempFile is created in the same directory than targetFile, so that the
move from tempFile to targetFile is always atomic
Merge remote-tracking branch 'upstream/master'
a49a2ce 
Conflicts:
	core/src/main/scala/org/apache/spark/util/Utils.scala
Merge remote-tracking branch 'upstream/master'
608dbfa
Clarify the local directories usage in YARN
876ae5e
Revert "Clarify the local directories usage in YARN"
436fb7d 
This reverts commit 876ae5e.
Clarify the local directories usage in YARN
eaaf519
@preaudc
Copy link
Author

preaudc commented Mar 24, 2015

I don't know if this is normal that all commits show up in the PR, I'm not yet familiar with this tool... Anyway, only the last one is relevant for this PR, and that's what can be seen in the "Files changed" tab.

@SparkQA
Copy link

SparkQA commented Mar 24, 2015

Test build #29079 has started for PR 5165 at commit eaaf519.

  • This patch merges cleanly.

srowen
srowen reviewed Mar 24, 2015
View reviewed changes
docs/running-on-yarn.md Outdated
Copy link
Member

@srowen srowen Mar 24, 2015

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nits: you can back-tick-quote code-like values like yarn-cluster. The last parenthetical can just be a sentence. Yeah it's normal to show all the commits you made to your branch. They will be squashed on merge. You can also squash and force-push your branch if desired, though it's not that important.

@SparkQA
Copy link

SparkQA commented Mar 24, 2015

Test build #29082 has started for PR 5165 at commit 6912b90.

  • This patch merges cleanly.

@SparkQA
Copy link

SparkQA commented Mar 24, 2015

Test build #29079 has finished for PR 5165 at commit eaaf519.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@AmplabJenkins
Copy link

AmplabJenkins commented Mar 24, 2015

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29079/
Test FAILed.

@SparkQA
Copy link

SparkQA commented Mar 24, 2015

Test build #29082 has finished for PR 5165 at commit 6912b90.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@AmplabJenkins
Copy link

AmplabJenkins commented Mar 24, 2015

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/29082/
Test PASSed.

@tgravescs
Copy link
Contributor

tgravescs commented Mar 24, 2015

Thanks for providing the pull request. The text here looks good. I'm +1. I'll let @srowen finish his review.

@sryza
Copy link
Contributor

sryza commented Mar 24, 2015

Mind changing the title to reflect that this is a documentation fix?

@preaudc preaudc changed the title [SPARK-6469] The YARN driver in yarn-client mode will not use the local directories configured for YARN [SPARK-6469] Improving documentation on YARN local directories usage Mar 24, 2015
@preaudc
Copy link
Author

preaudc commented Mar 24, 2015

Sure, is the new title ok?

@sryza
Copy link
Contributor

sryza commented Mar 24, 2015

New title looks great to me

@andrewor14
Copy link
Contributor

andrewor14 commented Mar 25, 2015

LGTM merging this into master 1.3 thanks.

@asfgit asfgit closed this in 05c2214 Mar 25, 2015
asfgit pushed a commit that referenced this pull request Mar 25, 2015
[SPARK-6469] Improving documentation on YARN local directories usage
6af9408 
Clarify the local directories usage in YARN

Author: Christophe Préaud <[email protected]>

Closes #5165 from preaudc/yarn-doc-local-dirs and squashes the following commits:

6912b90 [Christophe Préaud] Fix some formatting issues.
4fa8ec2 [Christophe Préaud] Merge remote-tracking branch 'upstream/master' into yarn-doc-local-dirs
eaaf519 [Christophe Préaud] Clarify the local directories usage in YARN
436fb7d [Christophe Préaud] Revert "Clarify the local directories usage in YARN"
876ae5e [Christophe Préaud] Clarify the local directories usage in YARN
608dbfa [Christophe Préaud] Merge remote-tracking branch 'upstream/master'
a49a2ce [Christophe Préaud] Merge remote-tracking branch 'upstream/master'
9ba89ca [Christophe Préaud] Ensure that files are fetched atomically
54419ae [Christophe Préaud] Merge remote-tracking branch 'upstream/master'
c6a5590 [Christophe Préaud] Revert commit 8ea871f
7456a33 [Christophe Préaud] Merge remote-tracking branch 'upstream/master'
8ea871f [Christophe Préaud] Ensure that files are fetched atomically
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

7 participants

@preaudc @SparkQA @AmplabJenkins @tgravescs @sryza @andrewor14 @srowen