Use Iceberg Metadata table's API for snapshot metadata table

[osscm]

Description

This will avoid using custom code to populate the Snapshot metadata table. This improvement will use the Iceberg's metadata table.scan, hence any changes to the underlying Iceberg's metadata table's scan logic will be used here. Spark engine also uses the Iceberg's metadata table.scan instead of custom code, so this will also help to reduce regression between Trino and Spark engines.

Is this change a fix, improvement, new feature, refactoring, or other?

Improvement

Is this a change to the core query engine, a connector, client library, or the SPI interfaces? (be specific)

Iceberg connector

How would you describe this change to a non-technical end user or system administrator?

No change in the syntax or the usage of the $snapshots metadata table

Related issues, pull requests, and links

• Fixes Use Iceberg's Metadata table API's for the SnapshotsTable #12649

Documentation

(x) No documentation is needed.

[ebyhr]

CI failure relates to this change.

[findepi]

cc @alexjo2144

[osscm]

CI failure relates to this change.

Error:  io.trino.plugin.iceberg.TestIcebergReadVersionedTable.testSelectTableWithEndShortTimestampWithTimezone  Time elapsed: 0.006 s  <<< FAILURE!
java.lang.AssertionError: 
For query: 
 SELECT * FROM test_iceberg_read_versioned_table FOR TIMESTAMP AS OF TIMESTAMP '+54409-04-16 13:40:21.000000000 Z'
not equal
Actual rows (up to 100 of 1 extra rows shown, 2 rows in total):
    [b, 2]
Expected rows (up to 100 of 0 missing rows shown, 1 rows in total):

yes, I see this query is failing, this seems to be not related to the $snapshot code change.
but let me check one more time.

[alexjo2144]

The test does look related, it reads from the Snapshots table here: https://github.com/trinodb/trino/blob/master/plugin/trino-iceberg/src/test/java/io/trino/plugin/iceberg/TestIcebergReadVersionedTable.java#L104

If you're sure though, just push an empty commit to trigger a rerun

[osscm]

The test does look related, it reads from the Snapshots table here: https://github.com/trinodb/trino/blob/master/plugin/trino-iceberg/src/test/java/io/trino/plugin/iceberg/TestIcebergReadVersionedTable.java#L104

If you're sure though, just push an empty commit to trigger a rerun

@alexjo2144 there was an issue with the commit_at timestamp's long value length. Fixed it, and all test cases are green now. Can you please review when you get a chance. tests are cleared now, fault_tolerance tests are still red, checking that.

[alexjo2144]

Sorry for the delay on the review. I'm okay with this change, pending the two small comments.

I'm just not sure if this adds much value unless we're deriving the table schema from the Iceberg SnapshotsTable#schema

[cla-bot]

Thank you for your pull request and welcome to our community. We could not parse the GitHub identity of the following contributors: Manish Malhotra.
This is most likely caused by a git client misconfiguration; please make sure to:

1. check if your git client is configured with an email to sign commits git config --list | grep email
2. If not, set it up using git config --global user.email email@example.com
3. Make sure that the git commit email is configured in your GitHub account settings, see https://github.com/settings/emails

[cla-bot]

Thank you for your pull request and welcome to our community. We could not parse the GitHub identity of the following contributors: Manish Malhotra.
This is most likely caused by a git client misconfiguration; please make sure to:

1. check if your git client is configured with an email to sign commits git config --list | grep email
2. If not, set it up using git config --global user.email email@example.com
3. Make sure that the git commit email is configured in your GitHub account settings, see https://github.com/settings/emails

[cla-bot]

Thank you for your pull request and welcome to our community. We could not parse the GitHub identity of the following contributors: Manish Malhotra.
This is most likely caused by a git client misconfiguration; please make sure to:

1. check if your git client is configured with an email to sign commits git config --list | grep email
2. If not, set it up using git config --global user.email email@example.com
3. Make sure that the git commit email is configured in your GitHub account settings, see https://github.com/settings/emails

[cla-bot]

Thank you for your pull request and welcome to our community. We could not parse the GitHub identity of the following contributors: Manish Malhotra.
This is most likely caused by a git client misconfiguration; please make sure to:

1. check if your git client is configured with an email to sign commits git config --list | grep email
2. If not, set it up using git config --global user.email email@example.com
3. Make sure that the git commit email is configured in your GitHub account settings, see https://github.com/settings/emails

[cla-bot]

Thank you for your pull request and welcome to our community. We could not parse the GitHub identity of the following contributors: Manish Malhotra.
This is most likely caused by a git client misconfiguration; please make sure to:

1. check if your git client is configured with an email to sign commits git config --list | grep email
2. If not, set it up using git config --global user.email email@example.com
3. Make sure that the git commit email is configured in your GitHub account settings, see https://github.com/settings/emails

[osscm]

@ebyhr resolved the conflitcs

[ebyhr]

Commit granularity is wrong. Please squash commits into one or separate them into appropriate commits.

[osscm]

Commit granularity is wrong. Please squash commits into one or separate them into appropriate commits.

thanks @ebyhr
Squashed the commits.

[osscm]

Commit granularity is wrong. Please squash commits into one or separate them into appropriate commits.

thanks @ebyhr Squashed the commits.

thanks for the review @ebyhr
Do you think we can push this PR now?

[osscm]

Thanks a lot @ebyhr @electrum @colebow and @mosabua !!