Add a function for splitting array into slices of given length#23264
Add a function for splitting array into slices of given length#23264abhinavmuk04 merged 1 commit intoprestodb:masterfrom abhinavmuk04:splitudf
Conversation
presto-main/src/main/java/com/facebook/presto/operator/scalar/sql/ArraySqlFunctions.java
Outdated
Show resolved
Hide resolved
elharo
left a comment
There was a problem hiding this comment.
Is there a reason this is a UDF and not just a function?
presto-main/src/main/java/com/facebook/presto/operator/scalar/sql/ArraySqlFunctions.java
Outdated
Show resolved
Hide resolved
@elharo With things such as preventing division by zero and checking for larger element it allows for a bit more customization and can be beneficial as a UDF |
elharo
left a comment
There was a problem hiding this comment.
As far as I can tell this is NOT a UDF. It is a new built-in function. Is there something I'm missing?
|
we use the term loosely also SQL UDFs is a term we use for all these functions that we add now. No need to split hairs |
elharo
left a comment
There was a problem hiding this comment.
presto-docs/src/main/sphinx/functions/array.rst needs to be updated in this PR to reflect the new function
presto-main/src/main/java/com/facebook/presto/operator/scalar/sql/ArraySqlFunctions.java
Outdated
Show resolved
Hide resolved
presto-main/src/main/java/com/facebook/presto/operator/scalar/sql/ArraySqlFunctions.java
Outdated
Show resolved
Hide resolved
presto-main/src/test/java/com/facebook/presto/operator/scalar/sql/TestArraySqlFunctions.java
Show resolved
Hide resolved
presto-main/src/test/java/com/facebook/presto/operator/scalar/sql/TestArraySqlFunctions.java
Show resolved
Hide resolved
|
@feilong-liu : Thanks for this function. Could we add a test in https://github.com/prestodb/presto/blob/master/presto-native-execution/src/test/java/com/facebook/presto/nativeworker/AbstractTestNativeArrayFunctionQueries.java as well. Since this is a SQL function, the native engine should be able to execute it without any changes. |
@aditi-pandit We have disabled inline-sql-function in all our production systems. There is a historical reason for it, I heard its due to regresssion in planning time, but I am not totally sure (CC: @tdcmeehan). In java, this means planning time the function is passed as-is to worker. In worker, inline-sql-function config is not respected and java worker can expand and execute the functions. In case of C++, the expansion is not there so C++ worker fails to run the function, and in some cases, functions can be implemented in worker (both native and java) to be very efficient. Hence, we are trying to add the functions as a regular function if possible. |
This is not acceptable. SQL functions is a firstclass feature and we should support them. |
By definition we should be able to support all presto features in the backend especially sql functions. There are tests added and native should run all tests if they want to be compatible. It's too much burden to test both. It should be part of CI |
|
@amitkdutta : We too have disabled inline-sql-function based on your guidance. Though the current setup is very hacky. We are also on the wrong track to add new functionality as SQL functions. Might be better to make them Java functions, and expect a C++ implementation as well. Agree with @kaikalur, that SQL functions should be supported at the backend. Another option is to have a per-function annotation to enable/disable individual SQL functions. |
Description
Add a function for splitting array into slices of given length
Motivation and Context
(#22996)
Impact
UDF for more features within Presto
Test Plan
Various unit tests, similar to other existing UDF tests
Contributor checklist
Release Note