[Iceberg]Support setting the max number of columns for which metrics are collected

[hantangwangd]

Description

For a very-wide-table, there will be too much metadata stored as stats in manifest files if we always collect stats for all the columns. It will occupy too much memory when loading the table metadata into memory. To cope with this, Iceberg lib support configuring the max columns that can have metrics for a table, referring to: apache/iceberg#3959, apache/iceberg#5215, apache/iceberg#5916

This PR add an Iceberg table property metrics_max_inferred_column to set the max columns number for which metrics are collected, and implement recognizing and supporting metrics_max_inferred_column on metrics collection for Iceberg table with PARQUET format.

Motivation and Context

Support setting the max number of columns for which metrics are collected

Impact

Iceberg tables with PARQUET format now only collect statistics information from the first 100 columns by default

Test Plan

• Newly added test case to show the behavior of stats collection with columns count limit

Contributor checklist

• Please make sure your submission complies with our development, formatting, commit message, and attribution guidelines.
• PR description addresses the issue accurately and concisely. If the change is non-trivial, a GitHub Issue is referenced.
• Documented new properties (with its default value), SQL syntax, functions, or other functionality.
• If release notes are required, they follow the release notes guidelines.
• Adequate tests were added if applicable.
• CI passed.

Release Notes

Please follow release notes guidelines and fill in the release notes below.

== RELEASE NOTES ==

Iceberg Connector Changes
* Add table property `metrics_max_inferred_column` to configure the max columns number for which metrics are collected, and support `metrics_max_inferred_column` for Iceberg table with `PARQUET` format :pr:`23468`

[steveburnett]

LGTM! (docs)

Pull branch, local doc build, looks good. Thanks!

[tdcmeehan]

Can you help explain why we need these changes to IcebergPageSink, and why do we need to pass in the Iceberg Table when it seems we're just using the Presto PrestoIcebergTable implementation?

[hantangwangd]

The problem encountered when creating IcebergFileWriter in IcebergPageSink is that, among all the static methods provided by Iceberg Lib for building a MetricsConfig instance, we can only pass an iceberg Table object in order to specify the table's schema. Refer to the static methods in MetricsConfig.

However, at present, there isn't a way on the presto worker side for us to load an iceberg Table object through metadata manager. So we just built an iceberg Table interface's implementation that is only used for simple encapsulation of schema, properties, 'spec', and 'sortOrder'. Then pass to and use it in IcebergPageSink.

[tdcmeehan]

Suggested change

	public class PrestoIcebergTable
	public class PrestoIcebergTableForMetricsConfig

Please add a comment that this is a dummy class and is required for MetricsConfig#forTable. Consider an Iceberg issue and PR adding a more direct method.

[hantangwangd]

Please add a comment that this is a dummy class and is required for MetricsConfig#forTable.

The comment is added, please take a look when available.

Consider an Iceberg issue and PR adding a more direct method.

Thanks for the suggestion, will consult with the Iceberg community later on whether a more direct method should be added.

[tdcmeehan]

Suggested change

	/**
	* This is a dummy class required for {@link org.apache.iceberg.MetricsConfig#forTable}
	*
	* */
	/**
	* This is a dummy class required for {@link org.apache.iceberg.MetricsConfig#forTable}
	**/