Skip to content

Commit 032f006

Browse files
committed
[SPARK-5193][SQL] Reconcile Java and Scala UDFRegistration.
1. Removed UDFRegistration as a mixin in SQLContext and made it a field ("udf"). 2. For Java UDFs, renamed dataType to returnType, and made that the 2nd argument since the UDF itself can be long. 3. Added all Java UDF registration methods to Scala's UDFRegistration. 4. Documentation
1 parent 4b325c7 commit 032f006

File tree

6 files changed

+383
-44
lines changed

6 files changed

+383
-44
lines changed

sql/core/src/main/scala/org/apache/spark/sql/SQLContext.scala

Lines changed: 27 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -54,7 +54,6 @@ class SQLContext(@transient val sparkContext: SparkContext)
5454
extends org.apache.spark.Logging
5555
with CacheManager
5656
with ExpressionConversions
57-
with UDFRegistration
5857
with Serializable {
5958

6059
self =>
@@ -338,6 +337,33 @@ class SQLContext(@transient val sparkContext: SparkContext)
338337
*/
339338
val experimental: ExperimentalMethods = new ExperimentalMethods(this)
340339

340+
/**
341+
* A collection of methods for registering user-defined functions (UDF).
342+
*
343+
* The following example registers a Scala closure as UDF:
344+
* {{{
345+
* sqlContext.udf.register("myUdf", (arg1: Int, arg2: String) => arg2 + arg1)
346+
* }}}
347+
*
348+
* The following example registers a UDF in Java:
349+
* {{{
350+
* sqlContext.udf().register("myUDF", DataTypes.StringType,
351+
* new UDF2<Integer, String, String>() {
352+
* @Override
353+
* public String call(Integer arg1, String arg2) {
354+
* return arg2 + arg1;
355+
* }
356+
* });
357+
* }}}
358+
*
359+
* Or, to use Java 8 lambda syntax:
360+
* {{{
361+
* sqlContext.udf().register("myUDF", DataTypes.StringType,
362+
* (Integer arg1, String arg2) -> arg2 + arg1));
363+
* }}}
364+
*/
365+
val udf: UDFRegistration = new UDFRegistration(this)
366+
341367
protected[sql] class SparkPlanner extends SparkStrategies {
342368
val sparkContext: SparkContext = self.sparkContext
343369

0 commit comments

Comments
 (0)