Optimize the performance of MOR on Trino

[shidayang]

I had done a chbenchmark of iceberg on Trino. I found that the performance of MOR is very low when have many delete files. The scale of data is 10 warehouse. The average duration is less than 10 second when no have delete files, but when I add some delete file to every tables some queries spent over one hour.

Reasons:

1. Make predicates of delete only initialize once #5195 The Trino every page will call DeleteFilter#filter, every calling of DeleteFilter#filter will initialize delete files.
2. Add StructLikeWrapperFactory to generate StructLikeWrapper #5244 Add InternalRecordWrapperFactory to generate InternalRecordWrapper #5242 We found that the cost of creating StructLikeWrapper and InternalRecordWrapper is high.
   this is Flame Graph:

The query performance improved when we made these optimizations. such as the query "select count(*) from stock", before optimize spent 8 minutes, after optimize only spent 20 seconds.

[shidayang]

@jackye1995 @chenjunjiedada @rdblue cc

[findinpath]

Please see trinodb/trino#13112 which is very likely related to your problem.

[shidayang]

@findinpath I think this problem is not only for Trino. Iceberg core should be responsible for delete file loading once

[lhofhansl]

I think Trino is special here as it operated a page at a time. Spark does not have this problem. And, as state in the PR, caching the filters outright unconditionally could be detrimental on Spark (increase memory usage)

[flyrain]

Hi @shidayang, how many delete files were there in your test?

I did benchmark multiple delete files, you can see the result here #3287 (comment).

with 25% rows are deleted and distribute these deletes to 1, 2, 5, 10 delete files

The perf doesn’t degrade much with more delete files. Please be ware that non-vectorized read is using the path without caching the filter. I am guessing Trino could be different from Spark in terms of read pattern.

[shidayang]

@flyrain About running 5 minutes TPCC on 10 warehouse of chbenchmark

[rdblue]

Thanks for working on this, @shidayang! The PRs are all resolved so I'm going to close this now.