Skip to content

Commit aea7a99

Browse files
SaintBacchuspwendell
authored andcommitted
[SPARK-4623]Add the some error infomation if using spark-sql in yarn-cluster mode
If using spark-sql in yarn-cluster mode, print an error infomation just as the spark shell in yarn-cluster mode. Author: carlmartin <[email protected]> Author: huangzhaowei <[email protected]> Closes #3479 from SaintBacchus/sparkSqlShell and squashes the following commits: 35829a9 [carlmartin] improve the description of comment e6c1eb7 [carlmartin] add a comment in bin/spark-sql to remind user who wants to change the class f1c5c8d [carlmartin] Merge branch 'master' into sparkSqlShell 8e112c5 [huangzhaowei] singular form ec957bc [carlmartin] Add the some error infomation if using spark-sql in yarn-cluster mode 7bcecc2 [carlmartin] Merge branch 'master' of https://github.com/apache/spark into codereview 4fad75a [carlmartin] Add the Error infomation using spark-sql in yarn-cluster mode
1 parent 048ecca commit aea7a99

File tree

2 files changed

+11
-0
lines changed

2 files changed

+11
-0
lines changed

bin/spark-sql

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -23,6 +23,8 @@
2323
# Enter posix mode for bash
2424
set -o posix
2525

26+
# NOTE: This exact class name is matched downstream by SparkSubmit.
27+
# Any changes need to be reflected there.
2628
CLASS="org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver"
2729

2830
# Figure out where Spark is installed

core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -142,6 +142,8 @@ object SparkSubmit {
142142
printErrorAndExit("Cluster deploy mode is currently not supported for python applications.")
143143
case (_, CLUSTER) if isShell(args.primaryResource) =>
144144
printErrorAndExit("Cluster deploy mode is not applicable to Spark shells.")
145+
case (_, CLUSTER) if isSqlShell(args.mainClass) =>
146+
printErrorAndExit("Cluster deploy mode is not applicable to Spark SQL shell.")
145147
case _ =>
146148
}
147149

@@ -393,6 +395,13 @@ object SparkSubmit {
393395
primaryResource == SPARK_SHELL || primaryResource == PYSPARK_SHELL
394396
}
395397

398+
/**
399+
* Return whether the given main class represents a sql shell.
400+
*/
401+
private[spark] def isSqlShell(mainClass: String): Boolean = {
402+
mainClass == "org.apache.spark.sql.hive.thriftserver.SparkSQLCLIDriver"
403+
}
404+
396405
/**
397406
* Return whether the given primary resource requires running python.
398407
*/

0 commit comments

Comments
 (0)