[iceberg] non-staged table creation fails on non-s3-tables REST catalog

[twuebi]

Hi all,

with a REST catalog which does not return a namespace location but at the same time is not s3tables, it's not possible to create a table.

The reason is that trino:

1. uses the existence of the namespace property location as a switch to decide if a table should be created as staged or not
2. then uses the --table-s3 suffix as a switch to decide if it should check that a table location is empty after creation
3. any non s3tables catalog that doesn't return a namespace property location now fails on table creation since the conditions for non-staged creation & exemption from the is-empty-check are not the same

This early return makes the location become empty:

    public String defaultTableLocation(ConnectorSession session, SchemaTableName schemaTableName)
    {
        String tableName = createLocationForTable(schemaTableName.getTableName());

        Map<String, Object> properties = loadNamespaceMetadata(session, schemaTableName.getSchemaName());
        String databaseLocation = (String) properties.get(IcebergSchemaProperties.LOCATION_PROPERTY);
        if (databaseLocation == null) {
            // Iceberg REST catalog doesn't require location property.
            // S3 Tables doesn't return the property.
            return null;
        }
        ...

This if decides on staged / non-staged create:

    public Transaction newCreateTableTransaction(
            ...
    )
    {
            ...
            if (location.isEmpty()) {
                // TODO Replace with createTransaction once S3 Tables supports stage-create option
                return tableBuilder.create().newTransaction();
            }
            return tableBuilder.withLocation(location.get()).createTransaction();
        ...
    }

The non-staged create ends up writing to the table location which then leads this check to fail:

if (!isS3Tables(location.toString())) {
                TrinoFileSystem fileSystem = fileSystemFactory.create(session.getIdentity(), transaction.table().io().properties());
                if (!replace && fileSystem.listFiles(location).hasNext()) {
                    throw new TrinoException(ICEBERG_FILESYSTEM_ERROR, format("" +
                            "Cannot create a table on a non-empty location: %s, set 'iceberg.unique-table-location=true' in your Iceberg catalog properties " +
                            "to use unique table locations for every table.", location));
                }
            }

Potential solutions:

1. add a property which lets us disable the location-is-empty check
2. remember that location.isEmpty() was true and replace the isS3Tables condition with that

As an aside, would it also be acceptable to add a property to disable Trino's attempts at setting table locations entirely?

[ebyhr]

@twuebi Which REST catalog are you using?

[twuebi]

@ebyhr AIStor Tables, I'm not sure if relying on an external dependency is the correct approach to test this, external dependencies may change their behavior and make this test fail.