-
Notifications
You must be signed in to change notification settings - Fork 29k
[SPARK-4623]Add the some error infomation if using spark-sql in yarn-cluster mode #3479
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
“shells" should be in singular form.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you add comment int he SparkSQLCLIDriver class that explains there is a dependency here on the exact name? Otherwise that class could be renamed and this would break inadvertently.
|
I had a small comment to make this less prone to being broken. Looks good other than that comment. |
|
add to whitelist. This LGTM |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think elsewhere we use SQL (all caps). Maybe we should say "not applicable to the Spark SQL shell" here
|
Test build #23953 has finished for PR 3479 at commit
|
bin/spark-sql
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A minor improvement:
# NOTE: This exact class name is matched downstream by SparkSubmit. Any changes need to be reflected there.
|
Just proposed a minor change to the comment. If you feel the current one is better you can leave it there. Either way LGTM. |
|
Test build #23958 has finished for PR 3479 at commit
|
|
Test build #23959 has finished for PR 3479 at commit
|
|
LGTM - thanks I'll pull this in |
|
Thanks I merged this. I couldn't find you on the ASF jira to give you credit as the author of this. Can you comment here so that I can make you the assignee? |
|
Thanks for mergeing. The jira was also reported by me, and I had rename it as github. |
If using spark-sql in yarn-cluster mode, print an error infomation just as the spark shell in yarn-cluster mode.