Load lazy blocks before using them in page filter/projection

[raunaqmorarka]

Helps to avoid calls to LazyData#getTopLevelBlock in
generated page filter and projection methods.
PageFieldsToInputParametersRewriter now also records which
channels are evaluated unconditionally so that LazyBlock can
be loaded for those channel before expression evaluation.

[skrzypo987]

This seems way beyond my Trino capabilities.
At least I found a missing requireNonNull, which makes me a proper reviewer.
I'll try to have another look tomorrow but you should seek feedback from professionals.

[sopel39]

lgtm % comments

@martint do you want to peek?

[martint]

This really depends on the function being applied. It's possible that not all of the references in the lambda expression is used.

[raunaqmorarka]

I see, I've updated visitCall to now assume that all the inputs to lambda expression can be conditionally evaluated (i.e. keep existing behaviour of all blocks being lazily loaded).

[sopel39]

This really depends on the function being applied. It's possible that not all of the references in the lambda expression is used.

@martint could you put an example here?

Should we just assume that lambda body is unconditionallyEvaluated=false and add a TODO?

[sopel39]

I don't fully understand why call expression has special handling of lambdas? Shouldn't lambdas be fully covered via visitLambda?

[raunaqmorarka]

When the visitor is traversing lambda expression, the leaf nodes always seem to be ConstantExpression or VariableReferenceExpression. So setting any value for unconditionallyEvaluated in visitLambda seems to be a no-op.
I think lambda expression is supposed to be always part of a function like transform, reduce etc. The InputReferenceExpression for the fields potentially accessed by the lambda are found in the initial arguments of CallExpression.

[sopel39]

I will let this one for @martint to look at.
Could you search for a code that creates CallExpression for lambdas and see what are the arguments? According to antrl lambdas are primary expressions, so I'm not sure where CallExpression comes from

[martint]

the leaf nodes always seem to be ConstantExpression or VariableReferenceExpression. So setting any value for unconditionallyEvaluated in visitLambda seems to be a no-op.

You would need to link those references to the corresponding arguments of the call to the higher-order function.

For example, in a hypothetical apply2 function: apply2(a, b, (x, y) -> x AND y), b is conditionally loaded, but a isn't due to the conditional nature of x AND y.

[raunaqmorarka]

@martint in the current code my intent was to fallback to existing behaviour for lambda expression. The motivating scenario for this PR was to improve efficiency for simple filters and projections (e.g. tpch/q01). Can we skip optimising this scenario or would you still consider this a blocker ?

[martint]

Yes, we can skip it for now. But add a TODO so we know the implementation is not yet complete.

[raunaqmorarka]

I've added a TODO here now

[sopel39]

This really depends on the function being applied. It's possible that not all of the references in the lambda expression is used.

@martint could you put an example here?

Should we just assume that lambda body is unconditionallyEvaluated=false and add a TODO?

[sopel39]

lgtm % comments. Waiting for @martint input

[sopel39]

I will let this one for @martint to look at.
Could you search for a code that creates CallExpression for lambdas and see what are the arguments? According to antrl lambdas are primary expressions, so I'm not sure where CallExpression comes from

[sopel39]

lgtm % @martint review

[sopel39]

@martint can you look?

[martint]

I'll take a look in the next couple of days.

[raunaqmorarka]

Load lazy blocks TPCH unpartitioned ORC sf1000.pdf

@martint I've attached results from TPCH unpartitioned ORC sf1000 run