Skip to content

Conversation

@liyinan926
Copy link
Contributor

What changes were proposed in this pull request?

Updated documentation for Spark on Kubernetes for the upcoming 2.4.0.

Please review http://spark.apache.org/contributing.html before opening a pull request.

@mccheah @erikerlandson

@SparkQA
Copy link

SparkQA commented Aug 24, 2018

@SparkQA
Copy link

SparkQA commented Aug 24, 2018

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/2536/

@SparkQA
Copy link

SparkQA commented Aug 24, 2018

Test build #95220 has finished for PR 22224 at commit 48ecb1e.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

```

## Using Kubernetes Volumes
Starting Spark 2.4.0, users can mount the following types of Kubernetes [volumes](https://kubernetes.io/docs/concepts/storage/volumes/) into the driver and executor pods:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"Starting with Spark ...". Also put a blank line above this.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

There are several Spark on Kubernetes features that are currently being incubated in a fork -
[apache-spark-on-k8s/spark](https://github.com/apache-spark-on-k8s/spark), which are expected to eventually make it into
future versions of the spark-kubernetes integration.
There are several Spark on Kubernetes features that are currently being worked on or planned to be worked on. Those features are expected to eventually make it into future versions of the spark-kubernetes integration.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is fine; could just be removed too.

Copy link
Contributor Author

@liyinan926 liyinan926 Aug 27, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You meant removing the entire Future Work section or just this sentence?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just these two sentences, not the whole section. I figured it goes without saying there are features that may be worked on the the future that may go into Spark. But I don't feel strongly about it; this is fine.

@SparkQA
Copy link

SparkQA commented Aug 27, 2018

@SparkQA
Copy link

SparkQA commented Aug 27, 2018

Test build #95278 has finished for PR 22224 at commit 7e8144b.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Aug 27, 2018

Kubernetes integration test status success
URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/2564/

@liyinan926
Copy link
Contributor Author

Can this be merged?

@srowen
Copy link
Member

srowen commented Aug 27, 2018

Yeah, was just seeing if there were more comments. Merged to master

@asfgit asfgit closed this in dac099d Aug 27, 2018
bogdanrdc pushed a commit to bogdanrdc/spark that referenced this pull request Aug 28, 2018
## What changes were proposed in this pull request?

Updated documentation for Spark on Kubernetes for the upcoming 2.4.0.

Please review http://spark.apache.org/contributing.html before opening a pull request.

mccheah erikerlandson

Closes apache#22224 from liyinan926/master.

Authored-by: Yinan Li <[email protected]>
Signed-off-by: Sean Owen <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants