Fusing multiple operations into one kernel

[ozgrakkurt]

Hey!

Does datafusion already do something like this or was this considered?

For example arr0+arr1*arr2 can be fused into a single kernel to avoid allocation and which might even end up being vectorized into a fmadd SIMD instruction. Also for something like a && b && c could be easily fused to avoid intermediary allocation, I guess this might be common in filter operations.

[tustvold]

was this considered

In general the model of arrow-rs and DataFusion is to favour vectorised dispatch over JIT-based execution. This saves a significant amount of implementation complexity, whilst yielding comparable performance in most cases

The discussion on apache/arrow-rs#3708 might be related here.

As with any optimisation, I'd suggest getting profiles showing this to be a significant overhead, and go from there. I personally would be surprised to see such low-level operations registering in profiles, compared to the likes of sorts and aggregates.

[alamb]

I think there is a similar discussion on #5944

One technique for fusing kernels is "JIT" just in time compilation.

Another technique to avoid the intermediate copying in columnar engines is called "selection vectors" (where you pass along an additional mask that says to ignore some of the values, which allows you to "filter" data without actually moving values)

DataFusion does neither at the moment, but there was some code in https://github.com/apache/arrow-datafusion/tree/main/datafusion/jit that starts on the JIT aproach