Support Iceberg default warehouse location config

[jackye1995]

Support using a configurable warehouse location to determine the default table location.

This is to match the Iceberg side catalog behavior: https://github.com/apache/iceberg/blob/master/hive-metastore/src/main/java/org/apache/iceberg/hive/HiveCatalog.java#L452-L459.

@losipiuk @findepi

[findepi]

In Iceberg, I can define location property on the schema.
@jackye1995 Why do we need this?

cc @losipiuk @phd3 @alexjo2144

[jackye1995]

In Iceberg, I can define location property on the schema. Why do we need this?

There are 2 reasons for this config:

1. Iceberg has the concept of a default warehouse location for catalog, so that users do not have to specify table location. This is similar to Hive's hive.metastore.warehouse.dir. But as you know Iceberg does not prefer using Hadoop config, so a warehouse location config is used. Adding this config in Trino brings Iceberg and Trino catalog concept closer.
2. Hadoop catalog actually requires such a config. If user supplies an alternative table location, it should throw an exception. This is because Hadoop catalog relies on the notion of warehouse/db/table to store catalog architecture. It does file listing to find all the databases and tables within a database. So this config is necessary for Hadoop catalog implementation.

[findepi]

Add a test covering creation of tables without location, within schema without location set.
This is what becomes possible with this change.

[findepi]

Make the property Optional<String>

[jackye1995]

updated

[findepi]

not sure about the name. something like "default storage location"?

cc @losipiuk @alexjo2144 @phd3

[jackye1995]

This is to match https://github.com/apache/iceberg/blob/master/core/src/main/java/org/apache/iceberg/CatalogProperties.java#L31. Please let me know if other name is preferred, I can update to that.

[losipiuk]

For schemas and tables we use "location". So probably we should use it also here. iceberg.default-schema-location?

[findepi]

This should be named more meaningfully. Let's figure config property name first.

[jackye1995]

sure, I updated to catalogWarehouse for now

[findepi]

Let's not build a fake Database object.
Instead, let's modify getTableDefaultLocation (which already has related logic).

[findepi]

why .db?

[jackye1995]

This is to match https://github.com/apache/iceberg/blob/master/hive-metastore/src/main/java/org/apache/iceberg/hive/HiveCatalog.java#L433-L440, not sure if Trino has any preference for this

[jackye1995]

@findepi test is actually quite difficult to add, the file metastore has a hard-coded database location, so that code path would never be exercised. Not sure if there is any other good way to trigger that.

[jackye1995]

We can also choose to not have this feature for Hive catalog and only support it with Hadoop and Glue catalog, because based on the current logic of HiveWriteUtils.getTableDefaultLocation, table name creation should fail if the provided path does not exist in non-S3 HDFS environment. So it seems like Trino assumes the existence of a database location for this to work for Hive. Any thoughts?

[jackye1995]

close in favor of #10151

[findepi]

This is to match the Iceberg side catalog behavior: https://github.com/apache/iceberg/blob/master/hive-metastore/src/main/java/org/apache/iceberg/hive/HiveCatalog.java#L452-L459.

The link no longer seems to be adequate. @jackye1995 can you link to a file in a commit, instead of a branch?