Support data_size when analyzing in Delta Lake

[ebyhr]

Description

Add support for data_size when analyzing in Delta Lake

Documentation

(x) Sufficient documentation is included in this PR.

Release notes

(x) Release notes entries required with the following suggested text:

# Delta Lake
* Add support for `data_size` when analyzing a table. ({issue}`12814`)

[ebyhr]

CI hit #12818

[findepi]

It doesn't make sense to calculate TOTAL_SIZE_IN_BYTES for eg numbers, since engine doesn't use this value for fixed-width data types.

[findepi]

also, if we decide we cannot merge existing stats with no data size (see other commnent), we should ask engine to collect them

[ebyhr]

if we decide we cannot merge existing stats with no data size (see other commnent), we should ask engine to collect them

I don't understand how to achieve this. Could you share the details?

[findepi]

In io.trino.plugin.deltalake.DeltaLakeMetadata#getStatisticsCollectionMetadata we already read the ExtendedStatistics. We can check whether we have data size for selected columns.
If we don't, we just don't create ColumnStatisticMetadata asking to collect TOTAL_SIZE_IN_BYTES

[findepi]

Don't invoke this when computedStatistics.containsKey(NUMBER_OF_DISTINCT_VALUES_SUMMARY), since this may be somewhat expensive allocation.

BTW what is the case when ! computedStatistics.containsKey(NUMBER_OF_DISTINCT_VALUES_SUMMARY)?
We ask engine to calculate the HLL, so we may expect it to be present, right?

[findepi]

@losipiuk PTAL

[alexjo2144]

There's a version number in ExtendedStats we should bump with this change

[findepi]

There's a version number in ExtendedStats we should bump with this change

We could, but we would need to update the logic here

trino/plugin/trino-delta-lake/src/main/java/io/trino/plugin/deltalake/DeltaLakeMetadata.java

Line 1980 in fa83b65

statistics.getModelVersion() == ExtendedStatistics.CURRENT_MODEL_VERSION,

(since the change is quite backwards compatible, i though we're going to leave the current number as is, but maybe i under-appreciate some consequences of doing so)

[ebyhr]

I didn't increase the number since I thought this is backward compatible as @findepi already said. Let me know if we should increment the number.

[ebyhr]

CI hit #12858