Skip to content

[SPARK-21589][SQL][DOC] Add documents about Hive UDF/UDTF/UDAF#18792

Closed
maropu wants to merge 3 commits intoapache:masterfrom
maropu:HOTFIX-20170731
Closed

[SPARK-21589][SQL][DOC] Add documents about Hive UDF/UDTF/UDAF#18792
maropu wants to merge 3 commits intoapache:masterfrom
maropu:HOTFIX-20170731

Conversation

@maropu
Copy link
Member

@maropu maropu commented Aug 1, 2017

What changes were proposed in this pull request?

This pr added documents about unsupported functions in Hive UDF/UDTF/UDAF.
This pr relates to #18768 and #18527.

How was this patch tested?

N/A

@maropu
Copy link
Member Author

maropu commented Aug 1, 2017

@gatorsmile If you get time, could you check this? Thanks!

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

to to -> to

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

functions with MapredContext, which is inapplicable to Spark.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

finished -> finish

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please remove Spark SQL does not support this optimization yet

@SparkQA
Copy link

SparkQA commented Aug 1, 2017

Test build #80103 has finished for PR 18792 at commit 1434bde.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

Copy link
Member

@gatorsmile gatorsmile Aug 1, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about simplifying the whole paragraph to?

Not all the APIs of the Hive UDF/UDTF/UDAF are supported by Spark SQL. Below are the unsupported APIs:

@gatorsmile
Copy link
Member

Thanks for working on it! Just left some minor comments.

@maropu
Copy link
Member Author

maropu commented Aug 1, 2017

@gatorsmile ok, fixed.

@SparkQA
Copy link

SparkQA commented Aug 1, 2017

Test build #80106 has finished for PR 18792 at commit 7d07e6b.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Aug 1, 2017

Test build #80107 has finished for PR 18792 at commit c703d57.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: But looks redundant here, because there's inapplicable before. Looks like negative to negative...

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

removed. Thanks!

@gatorsmile
Copy link
Member

LGTM pending Jenkins

@SparkQA
Copy link

SparkQA commented Aug 1, 2017

Test build #80109 has finished for PR 18792 at commit 29f1108.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@asfgit asfgit closed this in 110695d Aug 1, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

Comments