Add expression evaluation in Presto sidecar #24126
Closed
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
To support constant folding and consistent semantics between the Presto coordinator (Java) and the Presto C++ worker, it is necessary to use consistent expression evaluation. To support this, a native expression evaluation endpoint,
v1/expressions, has been added to the Presto sidecar process, and a plugin has been created which can utilize Velox expression evaluation behind a standardExpressionOptimizer.Adds
ExpressionOptimizerin Velox to optimize and constant foldTypedExprs via APIoptimizeExpression(); depends on Velox changes. Optimizes PrestoRowExpressions in the Presto native sidecar with following conversion:Adds helper class
RowExpressionConverterto convert the optimized/constant folded Veloxcore::TypedExprto a Prestoprotocol::RowExpression.These changes are being reviewed in PR: #22927.
Motivation and Context
Consistency between C++ and Java semantics. Support for using C++ functions during constant folding of expressions in the planner.
Impact
Fix some lingering bugs with C++ clusters.
Test Plan
Tests have been added by extending the
TestRowExpressionInterpreterclass to also test native expression evaluation inTestNativeExpressionOptimizer.java. The Velox expression fuzzer is extended to ensure the expression optimizations are tested exhaustively. However, this feature is still in Beta, and to support production workloads with complete certainty a fuzzer will be created to surface any remaining bugs with the integration at a later time.Release Notes