-
Notifications
You must be signed in to change notification settings - Fork 1.8k
Description
Is your feature request related to a problem or challenge?
In my case, I would like to extend a default function's supporting signature. For example, the avg function in DataFusion doesn't support calculating the Duration average.
However, suppose I want to register a custom avg for Duration. In that case, it means that I need to re-implement the whole avg function because the new implementation will override the previous one when registered. I want to propose an approach to register the custom implementation and keep the original implementation at the same time.
Describe the solution you'd like
In Postgres or DuckDB, their function definitions have catalog and schema. We can create a macro or UDF with the specific catalog or schema prefix. Then, we can set up the search_path to decide where to find the functions. If no matching candidates are found in the search path, we can find the candidate in the default path.
If we have similar features, I imagine I can follow these steps to extend an existing function easily:
- Create a schema
self. - Implement an
avgfunction for Duration. - Register the
self.avgfunction. - Set
search_pathtoself. - Query the SQL
select avg(time_col1 - time_col2) from table1(Invoke self.avg internally). - Query the SQL
select avg(int_col) from table1(Invoke default avg internally).
To approach this proposal, I think we need two new features:
- Register and use UDFs with the catalog and schema. I think we can do something at:
datafusion/datafusion/core/src/execution/session_state.rs
Lines 1773 to 1778 in d460abb
fn register_udf( &mut self, udf: Arc<ScalarUDF>, ) -> datafusion_common::Result<Option<Arc<ScalarUDF>>> { udf.aliases().iter().for_each(|alias| { self.scalar_functions - Implement
search_pathfor searching function candidates.
Describe alternatives you've considered
No response
Additional context
No response