[SPARK-11191] [SPARK-11311] [SQL] Backports #9664 and #9277 to branch-1.5

[liancheng]

The main purpose of this PR is to backport #9664, which depends on #9277.

[liancheng]

HiveQl tries to resolve UDTFs using Hive FunctionRegistry during parsing, must wrap it with execution Hive state.

[SparkQA]

Test build #45756 has finished for PR 9671 at commit 5502d69.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[liancheng]

Hm, I suspect maybe the Jenkins build was executed before #9570 (the ADD JAR issue) was merged to branch-1.5, and thus caused the failure.

[liancheng]

retest this please

[SparkQA]

Test build #45797 has finished for PR 9671 at commit 5502d69.

• This patch fails Spark unit tests.
• This patch merges cleanly.
• This patch adds no public classes.

[liancheng]

The test failure was because a previous test case (HiveThriftBinaryServerSuite.test jdbc cancel) sets spark.sql.hive.thriftServer.async to false. Haven't looked into why this fails ADD JAR, but I think we can address this issue in a separate PR.

[SparkQA]

Test build #45930 has finished for PR 9671 at commit fd7cddb.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[liancheng]

cc @yhuai

[yhuai]

Merge to branch 1.5.

[yhuai]

Merged.