diff --git a/R/README.md b/R/README.md index c808ca88f72dc..31174c73526f2 100644 --- a/R/README.md +++ b/R/README.md @@ -20,7 +20,7 @@ export R_HOME=/home/username/R Build Spark with [Maven](https://spark.apache.org/docs/latest/building-spark.html#buildmvn) and include the `-Psparkr` profile to build the R package. For example to use the default Hadoop versions you can run ```bash -build/mvn -DskipTests -Psparkr package +./build/mvn -DskipTests -Psparkr package ``` #### Running sparkR diff --git a/README.md b/README.md index 9759559e6cf6f..29777a5962bc2 100644 --- a/README.md +++ b/README.md @@ -25,7 +25,7 @@ This README file only contains basic setup instructions. Spark is built using [Apache Maven](https://maven.apache.org/). To build Spark and its example programs, run: - build/mvn -DskipTests clean package + ./build/mvn -DskipTests clean package (You do not need to do this if you downloaded a pre-built package.) diff --git a/docs/README.md b/docs/README.md index 670e9e01130df..da531321aa5da 100644 --- a/docs/README.md +++ b/docs/README.md @@ -84,7 +84,7 @@ $ PRODUCTION=1 jekyll build ## API Docs (Scaladoc, Javadoc, Sphinx, roxygen2, MkDocs) -You can build just the Spark scaladoc and javadoc by running `build/sbt unidoc` from the `$SPARK_HOME` directory. +You can build just the Spark scaladoc and javadoc by running `./build/sbt unidoc` from the `$SPARK_HOME` directory. Similarly, you can build just the PySpark docs by running `make html` from the `$SPARK_HOME/python/docs` directory. Documentation is only generated for classes that are listed as @@ -94,7 +94,7 @@ after [building Spark](https://github.com/apache/spark#building-spark) first. When you run `jekyll build` in the `docs` directory, it will also copy over the scaladoc and javadoc for the various Spark subprojects into the `docs` directory (and then also into the `_site` directory). We use a -jekyll plugin to run `build/sbt unidoc` before building the site so if you haven't run it (recently) it +jekyll plugin to run `./build/sbt unidoc` before building the site so if you haven't run it (recently) it may take some time as it generates all of the scaladoc and javadoc using [Unidoc](https://github.com/sbt/sbt-unidoc). The jekyll plugin also generates the PySpark docs using [Sphinx](http://sphinx-doc.org/), SparkR docs using [roxygen2](https://cran.r-project.org/web/packages/roxygen2/index.html) and SQL docs diff --git a/docs/running-on-kubernetes.md b/docs/running-on-kubernetes.md index d4efb52e0fbba..769eed1e6f6b7 100644 --- a/docs/running-on-kubernetes.md +++ b/docs/running-on-kubernetes.md @@ -121,7 +121,7 @@ $ ./bin/docker-image-tool.sh -r -t my-tag -R ./kubernetes/dockerfiles/spa To launch Spark Pi in cluster mode, ```bash -$ bin/spark-submit \ +$ ./bin/spark-submit \ --master k8s://https://: \ --deploy-mode cluster \ --name spark-pi \ diff --git a/docs/running-on-mesos.md b/docs/running-on-mesos.md index 907f414e5dc4c..cf51620a700bc 100644 --- a/docs/running-on-mesos.md +++ b/docs/running-on-mesos.md @@ -212,7 +212,7 @@ protected (port 7077 by default). By setting the Mesos proxy config property (requires mesos version >= 1.4), `--conf spark.mesos.proxy.baseURL=http://localhost:5050` when launching the dispatcher, the mesos sandbox URI for each driver is added to the mesos dispatcher UI. -If you like to run the `MesosClusterDispatcher` with Marathon, you need to run the `MesosClusterDispatcher` in the foreground (i.e: `bin/spark-class org.apache.spark.deploy.mesos.MesosClusterDispatcher`). Note that the `MesosClusterDispatcher` not yet supports multiple instances for HA. +If you like to run the `MesosClusterDispatcher` with Marathon, you need to run the `MesosClusterDispatcher` in the foreground (i.e: `./bin/spark-class org.apache.spark.deploy.mesos.MesosClusterDispatcher`). Note that the `MesosClusterDispatcher` not yet supports multiple instances for HA. The `MesosClusterDispatcher` also supports writing recovery state into Zookeeper. This will allow the `MesosClusterDispatcher` to be able to recover all submitted and running containers on relaunch. In order to enable this recovery mode, you can set SPARK_DAEMON_JAVA_OPTS in spark-env by configuring `spark.deploy.recoveryMode` and related spark.deploy.zookeeper.* configurations. For more information about these configurations please refer to the configurations [doc](configuration.html#deploy). @@ -362,7 +362,7 @@ The External Shuffle Service to use is the Mesos Shuffle Service. It provides sh on top of the Shuffle Service since Mesos doesn't yet support notifying another framework's termination. To launch it, run `$SPARK_HOME/sbin/start-mesos-shuffle-service.sh` on all slave nodes, with `spark.shuffle.service.enabled` set to `true`. -This can also be achieved through Marathon, using a unique host constraint, and the following command: `bin/spark-class org.apache.spark.deploy.mesos.MesosExternalShuffleService`. +This can also be achieved through Marathon, using a unique host constraint, and the following command: `./bin/spark-class org.apache.spark.deploy.mesos.MesosExternalShuffleService`. # Configuration diff --git a/docs/sql-data-sources-jdbc.md b/docs/sql-data-sources-jdbc.md index 3119ec004b2a1..c3502cbdea8e7 100644 --- a/docs/sql-data-sources-jdbc.md +++ b/docs/sql-data-sources-jdbc.md @@ -36,7 +36,7 @@ spark classpath. For example, to connect to postgres from the Spark Shell you wo following command: {% highlight bash %} -bin/spark-shell --driver-class-path postgresql-9.4.1207.jar --jars postgresql-9.4.1207.jar +./bin/spark-shell --driver-class-path postgresql-9.4.1207.jar --jars postgresql-9.4.1207.jar {% endhighlight %} Tables from the remote database can be loaded as a DataFrame or Spark SQL temporary view using diff --git a/docs/streaming-kinesis-integration.md b/docs/streaming-kinesis-integration.md index fd6d776045cd7..55acec53302e4 100644 --- a/docs/streaming-kinesis-integration.md +++ b/docs/streaming-kinesis-integration.md @@ -222,17 +222,17 @@ To run the example,
- bin/run-example --packages org.apache.spark:spark-streaming-kinesis-asl_{{site.SCALA_BINARY_VERSION}}:{{site.SPARK_VERSION_SHORT}} streaming.KinesisWordCountASL [Kinesis app name] [Kinesis stream name] [endpoint URL] + ./bin/run-example --packages org.apache.spark:spark-streaming-kinesis-asl_{{site.SCALA_BINARY_VERSION}}:{{site.SPARK_VERSION_SHORT}} streaming.KinesisWordCountASL [Kinesis app name] [Kinesis stream name] [endpoint URL]
- bin/run-example --packages org.apache.spark:spark-streaming-kinesis-asl_{{site.SCALA_BINARY_VERSION}}:{{site.SPARK_VERSION_SHORT}} streaming.JavaKinesisWordCountASL [Kinesis app name] [Kinesis stream name] [endpoint URL] + ./bin/run-example --packages org.apache.spark:spark-streaming-kinesis-asl_{{site.SCALA_BINARY_VERSION}}:{{site.SPARK_VERSION_SHORT}} streaming.JavaKinesisWordCountASL [Kinesis app name] [Kinesis stream name] [endpoint URL]
- bin/spark-submit --jars external/kinesis-asl/target/scala-*/\ + ./bin/spark-submit --jars external/kinesis-asl/target/scala-*/\ spark-streaming-kinesis-asl-assembly_*.jar \ external/kinesis-asl/src/main/python/examples/streaming/kinesis_wordcount_asl.py \ [Kinesis app name] [Kinesis stream name] [endpoint URL] [region name] @@ -244,7 +244,7 @@ To run the example, - To generate random string data to put onto the Kinesis stream, in another terminal, run the associated Kinesis data producer. - bin/run-example streaming.KinesisWordProducerASL [Kinesis stream name] [endpoint URL] 1000 10 + ./bin/run-example streaming.KinesisWordProducerASL [Kinesis stream name] [endpoint URL] 1000 10 This will push 1000 lines per second of 10 random numbers per line to the Kinesis stream. This data should then be received and processed by the running example. diff --git a/resource-managers/kubernetes/integration-tests/README.md b/resource-managers/kubernetes/integration-tests/README.md index ea8286124a68c..d7ad35a175a61 100644 --- a/resource-managers/kubernetes/integration-tests/README.md +++ b/resource-managers/kubernetes/integration-tests/README.md @@ -11,7 +11,7 @@ is subject to change. Note that currently the integration tests only run with Ja The simplest way to run the integration tests is to install and run Minikube, then run the following from this directory: - dev/dev-run-integration-tests.sh + ./dev/dev-run-integration-tests.sh The minimum tested version of Minikube is 0.23.0. The kube-dns addon must be enabled. Minikube should run with a minimum of 4 CPUs and 6G of memory: @@ -62,11 +62,11 @@ By default, the test framework will build new Docker images on every test execut and it is written to file at `target/imageTag.txt`. To reuse the images built in a previous run, or to use a Docker image tag that you have built by other means already, pass the tag to the test script: - dev/dev-run-integration-tests.sh --image-tag + ./dev/dev-run-integration-tests.sh --image-tag where if you still want to use images that were built before by the test framework: - dev/dev-run-integration-tests.sh --image-tag $(cat target/imageTag.txt) + ./dev/dev-run-integration-tests.sh --image-tag $(cat target/imageTag.txt) ### Customising the Image Names @@ -74,11 +74,11 @@ If your image names do not follow the standard Spark naming convention - `spark` If you use the same basic pattern but a different prefix for the name e.g. `apache-spark` you can just set `--base-image-name ` e.g. - dev/dev-run-integration-tests.sh --base-image-name apache-spark + ./dev/dev-run-integration-tests.sh --base-image-name apache-spark Alternatively if you use completely custom names then you can set each individually via the `--jvm-image-name `, `--python-image-name ` and `--r-image-name ` arguments e.g. - dev/dev-run-integration-tests.sh --jvm-image-name jvm-spark --python-image-name pyspark --r-image-name sparkr + ./dev/dev-run-integration-tests.sh --jvm-image-name jvm-spark --python-image-name pyspark --r-image-name sparkr ## Spark Distribution Under Test diff --git a/sql/README.md b/sql/README.md index 70cc7c637b58d..f0ea848a41d09 100644 --- a/sql/README.md +++ b/sql/README.md @@ -9,4 +9,4 @@ Spark SQL is broken up into four subprojects: - Hive Support (sql/hive) - Includes an extension of SQLContext called HiveContext that allows users to write queries using a subset of HiveQL and access data from a Hive Metastore using Hive SerDes. There are also wrappers that allow users to run queries that include Hive UDFs, UDAFs, and UDTFs. - HiveServer and CLI support (sql/hive-thriftserver) - Includes support for the SQL CLI (bin/spark-sql) and a HiveServer2 (for JDBC/ODBC) compatible server. -Running `sql/create-docs.sh` generates SQL documentation for built-in functions under `sql/site`. +Running `./sql/create-docs.sh` generates SQL documentation for built-in functions under `sql/site`.