SPARK-3337 Paranoid quoting in shell to allow install dirs with spaces within. #2229

ScrapCodes · 2014-09-01T12:26:21Z

...

Tested ! TBH, it isn't a great idea to have directory with spaces within. Because emacs doesn't like it then hadoop doesn't like it. and so on...

SparkQA · 2014-09-01T12:29:06Z

QA tests have started for PR 2229 at commit 4bcd7e8.

This patch merges cleanly.

SparkQA · 2014-09-01T12:29:18Z

QA tests have finished for PR 2229 at commit 4bcd7e8.

This patch fails unit tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
- "$FWDIR"/bin/spark-submit --class org.apache.spark.repl.Main "$
- "$FWDIR"/bin/spark-submit --class org.apache.spark.repl.Main "$
- exec "$SPARK_HOME"/bin/spark-class org.apache.spark.deploy.SparkSubmit "$

andrewor14 · 2014-09-02T01:58:02Z

Great idea. I've been itching to do this myself. retest this please

SparkQA · 2014-09-02T02:04:07Z

QA tests have started for PR 2229 at commit 4bcd7e8.

This patch merges cleanly.

SparkQA · 2014-09-02T02:04:18Z

QA tests have finished for PR 2229 at commit 4bcd7e8.

This patch fails unit tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
- "$FWDIR"/bin/spark-submit --class org.apache.spark.repl.Main "$
- "$FWDIR"/bin/spark-submit --class org.apache.spark.repl.Main "$
- exec "$SPARK_HOME"/bin/spark-class org.apache.spark.deploy.SparkSubmit "$

SparkQA · 2014-09-02T09:14:07Z

QA tests have started for PR 2229 at commit 11642ad.

This patch merges cleanly.

ScrapCodes · 2014-09-02T09:17:46Z

Tested it by have spark directory as "Apache Spark". all scripts seemed to work.

SparkQA · 2014-09-02T09:19:10Z

QA tests have started for PR 2229 at commit 3b9ff28.

This patch merges cleanly.

SparkQA · 2014-09-02T10:59:53Z

QA tests have finished for PR 2229 at commit 11642ad.

This patch passes unit tests.
This patch merges cleanly.
This patch adds no public classes.

SparkQA · 2014-09-02T11:09:45Z

QA tests have finished for PR 2229 at commit 3b9ff28.

This patch passes unit tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
- class ByteArrayChunkOutputStream(chunkSize: Int) extends OutputStream

SparkQA · 2014-09-03T06:59:32Z

QA tests have started for PR 2229 at commit ded203f.

This patch merges cleanly.

SparkQA · 2014-09-03T08:59:33Z

Tests timed out after a configured wait of 120m.

ScrapCodes · 2014-09-03T09:32:11Z

retest this please

SparkQA · 2014-09-03T09:39:11Z

QA tests have started for PR 2229 at commit ded203f.

This patch merges cleanly.

SparkQA · 2014-09-03T11:29:09Z

QA tests have finished for PR 2229 at commit ded203f.

This patch passes unit tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
- "$FWDIR"/bin/spark-submit --class org.apache.spark.repl.Main "$
- "$FWDIR"/bin/spark-submit --class org.apache.spark.repl.Main "$
- exec "$SPARK_HOME"/bin/spark-class org.apache.spark.deploy.SparkSubmit "$

andrewor14 · 2014-09-04T00:22:55Z

bin/compute-classpath.sh

You probably don't need to fix this one since it's only a number (not a big deal)

You are right !

SparkQA · 2014-09-04T06:49:21Z

QA tests have started for PR 2229 at commit 8002918.

This patch merges cleanly.

SparkQA · 2014-09-04T08:31:28Z

QA tests have finished for PR 2229 at commit 8002918.

This patch fails unit tests.
This patch merges cleanly.
This patch adds the following public classes (experimental):
- case class SparkListenerBlockManagerAdded(time: Long, blockManagerId: BlockManagerId, maxMem: Long)
- case class SparkListenerBlockManagerRemoved(time: Long, blockManagerId: BlockManagerId)
- case class SparkListenerApplicationStart(appName: String, appId: Option[String], time: Long,
- class BlockManagerMaster(
- class AttributeMap[A](baseMap: Map[ExprId, (Attribute, A)])

andrewor14 · 2014-09-05T00:06:12Z

sbt/sbt-launch-lib.bash

Him what about this one? If we have multiple arguments this is gonna group them together. Do we want that?

Ah yes, getting rid of them(and alike) as well.

andrewor14 · 2014-09-05T18:16:11Z

retest this please

SparkQA · 2014-09-05T23:53:09Z

QA tests have started for PR 2229 at commit b35fe1c.

This patch merges cleanly.

SparkQA · 2014-09-06T01:53:10Z

Tests timed out after a configured wait of 120m.

ScrapCodes · 2014-09-08T09:22:26Z

test this please.

…s within.

SparkQA · 2014-09-08T09:47:23Z

QA tests have started for PR 2229 at commit d4ad660.

This patch merges cleanly.

ScrapCodes · 2014-09-08T10:27:18Z

This patch is most prone to merge conflicts, would be good to merge it soon. @andrewor14 thoughts?

SparkQA · 2014-09-08T11:27:16Z

QA tests have finished for PR 2229 at commit d4ad660.

This patch passes unit tests.
This patch merges cleanly.
This patch adds no public classes.

andrewor14 · 2014-09-08T17:24:57Z

Yup. I played around with this locally with spaces and it also works for me. Merging this into master. Thanks.

ScrapCodes force-pushed the SPARK-3337/quoting-shell-scripts branch from 4bcd7e8 to 11642ad Compare September 2, 2014 09:08

ScrapCodes force-pushed the SPARK-3337/quoting-shell-scripts branch from 11642ad to 3b9ff28 Compare September 2, 2014 09:14

ScrapCodes force-pushed the SPARK-3337/quoting-shell-scripts branch from 3b9ff28 to ded203f Compare September 3, 2014 06:56

andrewor14 reviewed Sep 4, 2014
View reviewed changes

andrewor14 reviewed Sep 5, 2014
View reviewed changes

ScrapCodes force-pushed the SPARK-3337/quoting-shell-scripts branch from 7b6eee7 to b35fe1c Compare September 5, 2014 06:21

ScrapCodes force-pushed the SPARK-3337/quoting-shell-scripts branch from b35fe1c to 5596f9d Compare September 8, 2014 09:22

SPARK-3337 Paranoid quoting in shell to allow install dirs with space…

d4ad660

…s within.

ScrapCodes force-pushed the SPARK-3337/quoting-shell-scripts branch from 5596f9d to d4ad660 Compare September 8, 2014 09:24

asfgit closed this in e16a8e7 Sep 8, 2014

JoshRosen mentioned this pull request Nov 7, 2014

[SPARK-4275]fix for path including space #3156

Closed

ScrapCodes deleted the SPARK-3337/quoting-shell-scripts branch June 3, 2015 06:02

SPARK-3337 Paranoid quoting in shell to allow install dirs with spaces within. #2229

SPARK-3337 Paranoid quoting in shell to allow install dirs with spaces within. #2229

Uh oh!

Conversation

ScrapCodes commented Sep 1, 2014

Uh oh!

SparkQA commented Sep 1, 2014

Uh oh!

SparkQA commented Sep 1, 2014

Uh oh!

andrewor14 commented Sep 2, 2014

Uh oh!

SparkQA commented Sep 2, 2014

Uh oh!

SparkQA commented Sep 2, 2014

Uh oh!

SparkQA commented Sep 2, 2014

Uh oh!

ScrapCodes commented Sep 2, 2014

Uh oh!

SparkQA commented Sep 2, 2014

Uh oh!

SparkQA commented Sep 2, 2014

Uh oh!

SparkQA commented Sep 2, 2014

Uh oh!

SparkQA commented Sep 3, 2014

Uh oh!

SparkQA commented Sep 3, 2014

Uh oh!

ScrapCodes commented Sep 3, 2014

Uh oh!

SparkQA commented Sep 3, 2014

Uh oh!

SparkQA commented Sep 3, 2014

Uh oh!

andrewor14 Sep 4, 2014

Choose a reason for hiding this comment

Uh oh!

ScrapCodes Sep 4, 2014

Choose a reason for hiding this comment

Uh oh!

SparkQA commented Sep 4, 2014

Uh oh!

SparkQA commented Sep 4, 2014

Uh oh!

andrewor14 Sep 5, 2014

Choose a reason for hiding this comment

Uh oh!

ScrapCodes Sep 5, 2014

Choose a reason for hiding this comment

Uh oh!

andrewor14 commented Sep 5, 2014

Uh oh!

SparkQA commented Sep 5, 2014

Uh oh!

SparkQA commented Sep 6, 2014

Uh oh!

ScrapCodes commented Sep 8, 2014

Uh oh!

SparkQA commented Sep 8, 2014

Uh oh!

ScrapCodes commented Sep 8, 2014

Uh oh!

SparkQA commented Sep 8, 2014

Uh oh!

andrewor14 commented Sep 8, 2014

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants