Skip to content

Conversation

@dhruve
Copy link
Contributor

@dhruve dhruve commented Apr 12, 2016

What changes were proposed in this pull request?

The configuration docs are updated to reflect the changes introduced with SPARK-12384. This allows the user to specify initial heap memory settings through the extraJavaOptions for executor, driver and am.

How was this patch tested?

The changes are tested in SPARK-12384. This is just documenting the changes made.

@srowen
Copy link
Member

srowen commented Apr 12, 2016

I don't think this is useful to state, per JIRA.

@dhruve
Copy link
Contributor Author

dhruve commented Apr 12, 2016

@srowen Even if we don't specify explicitly that -Xms can be set, we will need to update the doc which currently states that it is illegal to specify heap memory settings.

@tgravescs
Copy link
Contributor

So I agree its not explicitly necessary to state Xms here, the only reason I think to mention it is the behavior is different from 1.X to 2.X, but there will be many other things different to so not a big deal to me on that.

I do think some of the other text he added here is useful though and makes the descriptions across driver/executor/am more consistent.

@srowen What do you think about just removing the sentence "However, specifying an initial heap

  • size (-Xms) is allowed" from the config descriptions but leaving the rest?

@tgravescs
Copy link
Contributor

@dhruve you should put the jira name int he pr description.

Please see guidelines at: https://cwiki.apache.org/confluence/display/SPARK/Contributing+to+Spark#ContributingtoSpark-PullRequest

@srowen
Copy link
Member

srowen commented Apr 13, 2016

@tgravescs I agree with that suggestion

@dhruve
Copy link
Contributor Author

dhruve commented Apr 13, 2016

I have updated the docs and removed the explicit mentions of Xms being allowed.

@srowen
Copy link
Member

srowen commented Apr 13, 2016

LGTM

@dhruve dhruve changed the title Update config docs to allow -Xms in extraJavaOptions [SPARK-14572]Update config docs to allow -Xms in extraJavaOptions Apr 13, 2016
@dhruve dhruve changed the title [SPARK-14572]Update config docs to allow -Xms in extraJavaOptions [SPARK-14572] [Doc] Update config docs to allow -Xms in extraJavaOptions Apr 13, 2016
@tgravescs
Copy link
Contributor

Jenkins, test this please

@tgravescs
Copy link
Contributor

+1, just doc changes so shouldn't really need jenkins, but kicked anyway.

@SparkQA
Copy link

SparkQA commented Apr 14, 2016

Test build #55822 has finished for PR 12333 at commit 253200c.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@asfgit asfgit closed this in f83ba45 Apr 14, 2016
@dhruve dhruve deleted the doc/SPARK-14572 branch June 17, 2016 02:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants