Skip to content

[SPARK-26076][Build][Minor] Revise ambiguous error message from load-spark-env.sh #23049

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
gengliangwang wants to merge 5 commits into from
Closed

[SPARK-26076][Build][Minor] Revise ambiguous error message from load-spark-env.sh #23049

Changes from 2 commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
bf94264
fix
gengliangwang Nov 15, 2018
1289df1
address comments
gengliangwang Nov 15, 2018
daf5e33
address comments
gengliangwang Nov 16, 2018
3269862
revise
gengliangwang Nov 16, 2018
07cd7f5
add doc link
gengliangwang Nov 17, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 11 additions & 10 deletions bin/load-spark-env.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -42,19 +42,20 @@ fi
# Setting SPARK_SCALA_VERSION if not already set.

if [ -z "$SPARK_SCALA_VERSION" ]; then

ASSEMBLY_DIR2="${SPARK_HOME}/assembly/target/scala-2.11"
ASSEMBLY_DIR1="${SPARK_HOME}/assembly/target/scala-2.12"

if [[ -d "$ASSEMBLY_DIR2" && -d "$ASSEMBLY_DIR1" ]]; then
echo -e "Presence of build for multiple Scala versions detected." 1>&2
echo -e 'Either clean one of them or, export SPARK_SCALA_VERSION in spark-env.sh.' 1>&2
SCALA_VERSION1=2.12
SCALA_VERSION2=2.11

ASSEMBLY_DIR1="${SPARK_HOME}/assembly/target/scala-${SCALA_VERSION1}"
ASSEMBLY_DIR2="${SPARK_HOME}/assembly/target/scala-${SCALA_VERSION2}"
if [[ -d "$ASSEMBLY_DIR1" && -d "$ASSEMBLY_DIR2" ]]; then
echo -e "Presence of build for multiple Scala versions detected($ASSEMBLY_DIR1 and $ASSEMBLY_DIR2)." 1>&2

@srowen srowen Nov 15, 2018

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: space after 'detected'. I think you can put this in one string if you like.

echo -e "Remove one of them, or export SPARK_SCALA_VERSION=$SCALA_VERSION1 in load-spark-env.sh." 1>&2
exit 1
fi

if [ -d "$ASSEMBLY_DIR2" ]; then
export SPARK_SCALA_VERSION="2.11"
if [ -d "$ASSEMBLY_DIR1" ]; then
export SPARK_SCALA_VERSION=$SCALA_VERSION1
else
export SPARK_SCALA_VERSION="2.12"
export SPARK_SCALA_VERSION=$SCALA_VERSION2
fi
fi