Update to Iceberg 1.2.1

[electrum]

Fixes #15372

Release notes

(x) This is not user-visible or docs only and no release notes are required.

[findepi]

LGTM
when it comes to rest integration I'd defer to @alexjo2144

[electrum]

The TestIcebergSparkCompatibility test failed with

Cannot determine whether the commit was successful or not, the underlying data files may or may not be needed. Manual intervention via the Remove Orphan Files Action can remove these files when a connection to the Catalog can be re-established if the commit was actually unsuccessful.
Please check to see whether or not your commit was successful before retrying this commit. Retrying an already successful operation will result in duplicate records or unintentional modifications.
At this time no files will be deleted including possibly unused manifest lists.
	at org.apache.iceberg.rest.ErrorHandlers$CommitErrorHandler.accept(ErrorHandlers.java:85)
	at org.apache.iceberg.rest.ErrorHandlers$CommitErrorHandler.accept(ErrorHandlers.java:71)
	at org.apache.iceberg.rest.HTTPClient.throwFailure(HTTPClient.java:177)
	at org.apache.iceberg.rest.HTTPClient.execute(HTTPClient.java:286)
	at org.apache.iceberg.rest.HTTPClient.execute(HTTPClient.java:220)
	at org.apache.iceberg.rest.HTTPClient.post(HTTPClient.java:331)
	at org.apache.iceberg.rest.RESTClient.post(RESTClient.java:112)
	at org.apache.iceberg.rest.RESTTableOperations.commit(RESTTableOperations.java:144)
	at org.apache.iceberg.RemoveSnapshots.lambda$commit$2(RemoveSnapshots.java:307)
	at org.apache.iceberg.util.Tasks$Builder.runTaskWithRetry(Tasks.java:413)
	at org.apache.iceberg.util.Tasks$Builder.runSingleThreaded(Tasks.java:219)
	at org.apache.iceberg.util.Tasks$Builder.run(Tasks.java:203)
	at org.apache.iceberg.util.Tasks$Builder.run(Tasks.java:196)
	at org.apache.iceberg.RemoveSnapshots.commit(RemoveSnapshots.java:304)
	at io.trino.plugin.iceberg.IcebergMetadata.executeExpireSnapshots(IcebergMetadata.java:1326)
	at io.trino.plugin.iceberg.IcebergMetadata.executeTableExecute(IcebergMetadata.java:1259)
	at io.trino.plugin.base.classloader.ClassLoaderSafeConnectorMetadata.executeTableExecute(ClassLoaderSafeConnectorMetadata.java:208)
	at io.trino.metadata.MetadataManager.executeTableExecute(MetadataManager.java:358)
	at io.trino.operator.SimpleTableExecuteOperator.getOutput(SimpleTableExecuteOperator.java:128)
	at io.trino.operator.Driver.processInternal(Driver.java:402)
	at io.trino.operator.Driver.lambda$process$8(Driver.java:305)
	at io.trino.operator.Driver.tryWithLock(Driver.java:701)
	at io.trino.operator.Driver.process(Driver.java:297)
	at io.trino.operator.Driver.processForDuration(Driver.java:268)
	at io.trino.execution.SqlTaskExecution$DriverSplitRunner.processFor(SqlTaskExecution.java:845)
	at io.trino.execution.executor.PrioritizedSplitRunner.process(PrioritizedSplitRunner.java:165)
	at io.trino.execution.executor.TaskExecutor$TaskRunner.run(TaskExecutor.java:537)
	at io.trino.$gen.Trino_410_265_g372c594____20230328_053742_2.run(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:833)
Caused by: org.apache.iceberg.exceptions.ServiceFailureException: Service failed: 500: {
"cause1":"java.lang.IllegalArgumentException: Cannot parse to an integer value: snapshot-id: 2168225879919926620",
"servlet":"org.apache.iceberg.rest.RESTCatalogServlet-3bbc39f8",
"cause0":"com.fasterxml.jackson.databind.JsonMappingException: Cannot parse to an integer value: snapshot-id: 2168225879919926620 (through reference chain: org.apache.iceberg.rest.requests.UpdateTableRequest["updates"]->java.util.ArrayList[2])",
"message":"com.fasterxml.jackson.databind.JsonMappingException: Cannot parse to an integer value: snapshot-id: 2168225879919926620 (through reference chain: org.apache.iceberg.rest.requests.UpdateTableRequest["updates"]->java.util.ArrayList[2])",
"url":"/v1/namespaces/default/tables/test_spark_reads_trino_partitioned_table_after_expiring_snapshotsparquet",
"status":"500"
}
	... 31 more

I'm going to try reverting the REST server container update.

[electrum]

I'm guessing we need to update the Iceberg version for Spark as well: trinodb/docker-images#164

This seems to indicate that data written with Iceberg 1.2 is not readable by Iceberg 1.1, which is concerning.

[nastra]

I looked at the failures in TestIcebergSparkCompatibility. It seems that all of the cleanup tests are affected that call RemoveSnapshots. The underlying issue is that the snapshot-id is being parsed as an integer rather than a long here.
I've opened apache/iceberg#7235 to address that.

/cc @findepi

[electrum]

Nice find. Does this mean that Iceberg 1.2.0 is unusable until this is fixed? Do we need to wait for 1.2.1 to upgrade?

[electrum]

Hmm, it appears that this commit went into Iceberg 1.1.0. Any idea why it didn't break before? Or thoughts on a workaround?

[nastra]

Nice find. Does this mean that Iceberg 1.2.0 is unusable until this is fixed? Do we need to wait for 1.2.1 to upgrade?

I think this is only an issue with the removal of statistics files, but I don't know why it wasn't an issue with Iceberg 1.1.0. @findepi any ideas?

[nastra]

We might get a fix for this into 1.2.1

[electrum]

@nastra @rdblue Any idea on when 1.2.1 could be released?

I'm wondering if snapshot ID generation was changed, such that IDs were previously valid 32-bit integers.

[danielcweeks]

@electrum We're working on pulling together the 1.2.1 release (there's a small set of issues, so hopefully we'll have a quick turnaround for you). I believe the fix for this is already in mainline, so it'll be included in 1.2.1.

[nastra]

@electrum Iceberg 1.2.1 has been officially released now. I've also created a new version of the REST server image: tabulario/iceberg-rest:0.4.0

[electrum]

@nastra Thanks, updated. Hopefully all the tests pass this time, then I'll finally merge this