feat(cuDF): Register not in cuDF function registry#16503
feat(cuDF): Register not in cuDF function registry#16503pramodsatya wants to merge 1 commit intofacebookincubator:mainfrom
not in cuDF function registry#16503Conversation
✅ Deploy Preview for meta-velox canceled.
|
|
This is understandable. We should have a standalone Function implementation for everything that's supported in AST and JIT. I'm still curious where the AST failed to take over and |
not as a cudf functionnot in cuDF function registry
|
@devavret, the AST did not fail to take over. All registered cuDF scalar and aggregate functions will be consumed by Presto sidecar for better function management at coordinator; this will be followed by #16504 and Presto sidecar changes (prototyped in prestodb/presto#27164). Could you help review this change? |
This should've already happened. ExpressionEvaluator's canBeEvaluatedByCudf delegates this to AstExpression::canEvaluate and if that fails, it falls back to checking FunctionExpression::canEvaluate. In this case, |
|
When cuDF function management is enabled via sidecar, the queries will fail on the coordinator during planning before getting to The function validation via sidecar retrieves cuDF functions from the function registry, which is exposed to Presto in #16504. If |
Motivation
Presto's planner and optimizer build
not(...)call expressions during analysis and rewrite phases and resolve them using functions in the active function namespace. In native execution, the function namespace is populated from the sidecar's/v1/functionsendpoint and when cuDF execution is enabled this list contains cuDF functions only. Ifnotis missing in this list, function resolution fails on coordinator for simple queries with sidecar.Change
Adding
notto the cuDF built-in registry ensures it appears in the sidecar's function list and is resolvable during planning in cuDF-backed Presto C++ deployments with sidecar.