Skip to content

Conversation

@ebyhr
Copy link
Member

@ebyhr ebyhr commented Dec 2, 2024

Description

all_manifests metadata table provides data for the current snapshot as well as all snapshots.
This table will be helpful when we support rewriting manifests in the future.

Added the null handling just in case though I couldn't find how to get NULL from contains_nan field.
https://iceberg.apache.org/docs/latest/spark-queries/#all-manifests

  1. contains_nan could return null, which indicates that this information is not available from the file's metadata. This usually occurs when reading from V1 table, where contains_nan is not populated.

Docs PR can be handled in different PR. We may want to separate a section for all_ metadata tables.

Release notes

## Iceberg
* Add the `all_manifests` metadata table. ({issue}`24330`)

@cla-bot cla-bot bot added the cla-signed label Dec 2, 2024
@github-actions github-actions bot added the iceberg Iceberg connector label Dec 2, 2024
@ebyhr ebyhr merged commit db88992 into trinodb:master Dec 3, 2024
42 checks passed
@ebyhr ebyhr deleted the ebi/iceberg-all-manifests branch December 3, 2024 04:51
@github-actions github-actions bot added this to the 467 milestone Dec 3, 2024
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package io.trino.plugin.iceberg;
Copy link
Contributor

@findinpath findinpath Dec 4, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the documenation for this table on https://trino.io/docs/current/connector/iceberg.html#metadata-tables coming as a follow-up ?

cc @mosabua

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I should .. I think @Ordinant @jhlodin and team are working on it .. I wont be able to

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But yes .. it would have been better to do the docs as part of this PR so the release notes can link to docs and users get a feature with docs at the same time

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cla-signed iceberg Iceberg connector

Development

Successfully merging this pull request may close these issues.

4 participants