Add Iceberg format version 3 table support

[Copilot]

Description

Adds support for Iceberg format version 3 tables: creation, v2→v3 upgrade, and inserts.

Unsupported v3 features now fail fast with NOT_SUPPORTED:

• Row-level mutations (DELETE, UPDATE, MERGE) on v3 tables
• OPTIMIZE (rewrite_data_files) on v3 tables
• PUFFIN deletion vectors
• Column default values (initial-default, write-default)
• Table encryption

Changes:

• IcebergUtil.java: Add MAX_FORMAT_VERSION_FOR_ROW_LEVEL_OPERATIONS=2, MAX_SUPPORTED_FORMAT_VERSION=3
• IcebergAbstractMetadata.java: Add validateTableForPresto() for v3 feature validation; update beginMerge/Delete/Update/Insert to enforce version limits
• IcebergSplitSource.java: Reject PUFFIN format delete files
• RewriteDataFilesProcedure.java: Block OPTIMIZE on v3 tables

Motivation and Context

Mirrors Trino PR #27786. Enables incremental v3 adoption while preventing spec violations from unsupported features.

Impact

• New: v3 tables can be created with "format-version" = '3'
• New: Inserts into v3 tables work
• Breaking: DELETE/UPDATE/MERGE/OPTIMIZE on v3 tables now throw NOT_SUPPORTED

Test Plan

New TestIcebergV3 with 9 tests covering:

• v3 table creation and v2→v3 upgrade
• Inserts and selects on v3 tables
• Partitioned v3 tables
• Rejection of DELETE/UPDATE/MERGE/OPTIMIZE on v3 tables

Existing tests pass: TestRewriteDataFilesProcedure, TestIcebergLogicalPlanner, TestIcebergSystemTables, TestIcebergUtil

Contributor checklist

• Please make sure your submission complies with our contributing guide, in particular code style and commit standards.
• PR description addresses the issue accurately and concisely. If the change is non-trivial, a GitHub Issue is referenced.
• Documented new properties (with its default value), SQL syntax, functions, or other functionality.
• If release notes are required, they follow the release notes guidelines.
• Adequate tests were added if applicable.
• CI passed.
• If adding new dependencies, verified they have an OpenSSF Scorecard score of 5.0 or higher (or obtained explicit TSC approval for lower scores).

Release Notes

== RELEASE NOTES ==

Iceberg Connector Changes
* Add support for creating and inserting into Iceberg format version 3 tables.
* Row-level operations (DELETE, UPDATE, MERGE) and OPTIMIZE are not yet supported on v3 tables and will fail with a clear error message.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

• repo.hortonworks.com
  • Triggering command: /usr/lib/jvm/temurin-17-jdk-amd64/bin/java /usr/lib/jvm/temurin-17-jdk-amd64/bin/java --enable-native-access=ALL-UNNAMED -Xmx8192m -classpath /usr/share/apache-maven-3.9.12/boot/plexus-classworlds-2.9.0.jar -Dclassworlds.conf=/usr/share/apache-maven-3.9.12/bin/m2.conf -Dmaven.home=/usr/share/apache-maven-3.9.12 -Dlibrary.jansi.path=/usr/share/apache-maven-3.9.12/lib/jansi-native -Dmaven.multiModuleProjectDirectory=/home/REDACTED/work/presto/presto org.codehaus.plexus.classworlds.launcher.Launcher install -DskipTests -pl presto-iceberg -am -q (dns block)
• repository.apache.org
  • Triggering command: /usr/lib/jvm/temurin-17-jdk-amd64/bin/java /usr/lib/jvm/temurin-17-jdk-amd64/bin/java --enable-native-access=ALL-UNNAMED -Xmx8192m -classpath /usr/share/apache-maven-3.9.12/boot/plexus-classworlds-2.9.0.jar -Dclassworlds.conf=/usr/share/apache-maven-3.9.12/bin/m2.conf -Dmaven.home=/usr/share/apache-maven-3.9.12 -Dlibrary.jansi.path=/usr/share/apache-maven-3.9.12/lib/jansi-native -Dmaven.multiModuleProjectDirectory=/home/REDACTED/work/presto/presto org.codehaus.plexus.classworlds.launcher.Launcher install -DskipTests -pl presto-iceberg -am -q (dns block)
• repository.cloudera.com
  • Triggering command: /usr/lib/jvm/temurin-17-jdk-amd64/bin/java /usr/lib/jvm/temurin-17-jdk-amd64/bin/java --enable-native-access=ALL-UNNAMED -Xmx8192m -classpath /usr/share/apache-maven-3.9.12/boot/plexus-classworlds-2.9.0.jar -Dclassworlds.conf=/usr/share/apache-maven-3.9.12/bin/m2.conf -Dmaven.home=/usr/share/apache-maven-3.9.12 -Dlibrary.jansi.path=/usr/share/apache-maven-3.9.12/lib/jansi-native -Dmaven.multiModuleProjectDirectory=/home/REDACTED/work/presto/presto org.codehaus.plexus.classworlds.launcher.Launcher install -DskipTests -pl presto-iceberg -am -q (dns block)
• repository.jboss.org
  • Triggering command: /usr/lib/jvm/temurin-17-jdk-amd64/bin/java /usr/lib/jvm/temurin-17-jdk-amd64/bin/java --enable-native-access=ALL-UNNAMED -Xmx8192m -classpath /usr/share/apache-maven-3.9.12/boot/plexus-classworlds-2.9.0.jar -Dclassworlds.conf=/usr/share/apache-maven-3.9.12/bin/m2.conf -Dmaven.home=/usr/share/apache-maven-3.9.12 -Dlibrary.jansi.path=/usr/share/apache-maven-3.9.12/lib/jansi-native -Dmaven.multiModuleProjectDirectory=/home/REDACTED/work/presto/presto org.codehaus.plexus.classworlds.launcher.Launcher install -DskipTests -pl presto-iceberg -am -q (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

• Configure Actions setup steps to set up my environment, which run before the firewall is enabled
• Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

Original prompt

trinodb/trino#27786 Implement the same

Add support for creating Iceberg format version 3 tables, upgrading v2 tables to v3, and inserting into v3 tables.

This change intentionally does not implement Iceberg v3 features beyond allowing v3 metadata and validating that inserts produce the required row-lineage metadata (as observed through the Iceberg library).

To avoid spec violations while v3 support is incomplete, the connector now explicitly rejects v3 features that are not yet supported. The goal is to safely unlock v3 table creation and incremental adoption, while making unsupported behavior fail fast and predictably.

Unsupported v3 features that now throw NOT_SUPPORTED include:

Row-level mutations on v3 tables: DELETE, UPDATE, MERGE OPTIMIZE on v3 tables add_files / add_files_from_table procedures on v3 tables Deletion vectors (PUFFIN delete files) Column default values (initial-default, write-default) Iceberg table encryption (encryption-keys / snapshot key-id) Tests:

Add TestIcebergV3 to cover:

create v3 tables and upgrade v2→v3 inserts into v3 tables produce required lineage metadata (nextRowId, firstRowId, dataSequenceNumber) unsupported v3 features fail with clear exceptions

---

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

[Joe-Abraham]

@copilot testOptimizeOnV3TableNotSupported is failing. with below warning

java.lang.AssertionError: Expecting code to raise a throwable.

at com.facebook.presto.iceberg.TestIcebergV3.testOptimizeOnV3TableNotSupported(TestIcebergV3.java:180)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:569)
at org.testng.internal.invokers.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:135)
at org.testng.internal.invokers.TestInvoker.invokeMethod(TestInvoker.java:673)
at org.testng.internal.invokers.TestInvoker.invokeTestMethod(TestInvoker.java:220)
at org.testng.internal.invokers.MethodRunner.runInSequence(MethodRunner.java:50)
at org.testng.internal.invokers.TestInvoker$MethodInvocationAgent.invoke(TestInvoker.java:945)
at org.testng.internal.invokers.TestInvoker.invokeTestMethods(TestInvoker.java:193)
at org.testng.internal.invokers.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:146)
at org.testng.internal.invokers.TestMethodWorker.run(TestMethodWorker.java:128)
at java.base/java.util.ArrayList.forEach(ArrayList.java:1511)
at org.testng.TestRunner.privateRun(TestRunner.java:808)
at org.testng.TestRunner.run(TestRunner.java:603)
at org.testng.SuiteRunner.runTest(SuiteRunner.java:429)
at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:423)
at org.testng.SuiteRunner.privateRun(SuiteRunner.java:383)
at org.testng.SuiteRunner.run(SuiteRunner.java:326)
at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:95)
at org.testng.TestNG.runSuitesSequentially(TestNG.java:1249)
at org.testng.TestNG.runSuitesLocally(TestNG.java:1169)
at org.testng.TestNG.runSuites(TestNG.java:1092)
at org.testng.TestNG.run(TestNG.java:1060)
at com.intellij.rt.testng.IDEARemoteTestNG.run(IDEARemoteTestNG.java:65)
at com.intellij.rt.testng.RemoteTestNGStarter.main(RemoteTestNGStarter.java:105)