Skip to content

[SPARK-3696]Do not override the user-difined conf_dir #2541

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
WangTaoTheTonic wants to merge 1 commit into from
Closed

[SPARK-3696]Do not override the user-difined conf_dir #2541

WangTaoTheTonic wants to merge 1 commit into from

Conversation

@WangTaoTheTonic
Copy link
Contributor

@WangTaoTheTonic WangTaoTheTonic commented Sep 26, 2014

https://issues.apache.org/jira/browse/SPARK-3696

We see if SPARK_CONF_DIR is already defined before assignment.

@SparkQA
Copy link

SparkQA commented Sep 26, 2014

QA tests have started for PR 2541 at commit c3f31e0.

  • This patch merges cleanly.

@SparkQA
Copy link

SparkQA commented Sep 26, 2014

QA tests have finished for PR 2541 at commit c3f31e0.

  • This patch passes unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@AmplabJenkins
Copy link

AmplabJenkins commented Sep 26, 2014

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/20844/

JoshRosen
JoshRosen reviewed Oct 2, 2014
View reviewed changes
sbin/spark-config.sh
Copy link
Contributor

@JoshRosen JoshRosen Oct 2, 2014

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not a bash expert, so I'm curious: does the nesting of double quotes work properly here? Here, we have double quotes inside of the ${} and surrounding it, too.

Copy link
Contributor

@andrewor14 andrewor14 Oct 3, 2014

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's fine. The outer double quotes guarantee that the end result (whatever the user SPARK_CONF_DIR is set to) is also safe.

@WangTaoTheTonic
Copy link
Contributor Author

WangTaoTheTonic commented Oct 3, 2014

@JoshRosen I have tested and it worked fine. You can also have a try simply.

1 similar comment
@WangTaoTheTonic
Copy link
Contributor Author

WangTaoTheTonic commented Oct 3, 2014

@JoshRosen I have tested and it worked fine. You can also have a try simply.

@JoshRosen JoshRosen mentioned this pull request Oct 3, 2014
Closed
@andrewor14
Copy link
Contributor

andrewor14 commented Oct 3, 2014

Ok, I'm merging this into master and 1.1. Thanks @WangTaoTheTonic!

@asfgit asfgit closed this in 9d320e2 Oct 3, 2014
asfgit pushed a commit that referenced this pull request Oct 3, 2014
@WangTaoTheTonic @andrewor14
[SPARK-3696]Do not override the user-difined conf_dir
d5af9e1 
https://issues.apache.org/jira/browse/SPARK-3696

We see if SPARK_CONF_DIR is already defined before assignment.

Author: WangTaoTheTonic <[email protected]>

Closes #2541 from WangTaoTheTonic/confdir and squashes the following commits:

c3f31e0 [WangTaoTheTonic] Do not override the user-difined conf_dir

(cherry picked from commit 9d320e2)
Signed-off-by: Andrew Or <[email protected]>

Conflicts:
	sbin/spark-config.sh
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@WangTaoTheTonic @SparkQA @AmplabJenkins @andrewor14 @JoshRosen