Support snapshot queries on Iceberg system tables - IcebergSystemTableHandle is a ConnectorTableHandle with custom instanceof handling in IcebergMetadata#13497
Closed
findinpath wants to merge 1 commit intotrinodb:masterfrom
Conversation
70701a4 to
8879436
Compare
findinpath
commented
Aug 4, 2022
Contributor
Author
There was a problem hiding this comment.
In io.trino.connector.system.SystemTablesMetadata#applyFilter there is made the call
to verify whether the table has disappeared in the meantime.
Should we do it here as well?
77dfbbe to
fb3cab1
Compare
fb3cab1 to
c47291e
Compare
c47291e to
e651e40
Compare
Use `IcebergSystemTableHandle` for dealing with system tables because it encodes for the table being queried also: - snapshot id - table type Previously there were supported versioned queries on system tables because the snapshot id could be encoded within the table name: ``` SELECT * FROM table1$files@1242141 ``` The syntax mentioned previously is now deprecated and replaced by a richer syntax: ``` SELECT * FROM table1$files FOR VERSION AS OF 1242141 SELECT * FROM table1$files FOR TIMESTAMP AS OF TIMESTAMP '2022-01-01' ```
e651e40 to
80ff57c
Compare
instanceof handling in IcebergMetadata
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Use
IcebergSystemTableHandlefor dealing with system tablesbecause it encodes for the table being queried also:
NOTE that this change reuses
IcebergMetadatacolumns for performing queries related to system tables.Bugfix
Iceberg connector
The metadata tables exposed by the Iceberg connector for its tables should support snapshot/versioned queries in the same fashion as the Iceberg tables.
Related issues, pull requests, and links
Fixes #12736
This PR is built on top of #12737 and uses
IcebergSystemTableHandlefor dealing with system tables, instead ofIcebergTableHandle.This PR could pave the way to the functionality of having distributed
$filescalculation. See #4840Having a distributed retrieval for
$filescould be beneficial in debugging scenarios when dealing with tables with a lot of files. Probably other scenarios could follow.Documentation
(x) No documentation is needed.
( ) Sufficient documentation is included in this PR.
( ) Documentation PR is available with #prnumber.
( ) Documentation issue #issuenumber is filed, and can be handled later.
Release notes
() No release notes entries required.
(x) Release notes entries required with the following suggested text: