Revert "Add column add/drop/rename events"

[findepi]

Reverts #15225

The current implementation has the following problems

• does not work for Hive connector with hive.security=system (Add column add/drop/rename events #15225 (comment))
• notifications are propagated even if transaction doesn't end up committing the DDL changes (Add column add/drop/rename events #15225 (comment))
• notification propagation may disrupt transaction finish even if changes already applied in the external system (Adding column in SQL Server connector sometimes fails even though column has been added successfully #12535 (comment)), for any connector that's using SYSTEM security (Iceberg, Delta with system security, or any JDBC connector)

The revert is only temporary, the functionality will be reintroduced soon.

Fixes #12535

[findepi]

Of course, we can also fix the functionality, instead of revert, if a fix exists.
Generally I would prefer the master branch to be always release-ready, and reverting the revert is trivial, so we lose nothing with this PR.

cc @trinodb/maintainers

[findepi]

CI failed with an Ubuntu update problem, going to be fixed by #15840
(cc @MiguelWeezardo that's why it would be nice to have an issue, #15840 (comment))

[findepi]

(updated commit message, no other changes)

[findepi]

CI #15992

[djsagain]

Sorry that this caused trouble.

[findepi]

Sorry that this caused trouble.

No worries @djsstarburst and thank you for your review!

[electrum]

It seems like we can fix this trivially by doing the getTableMetadata() before the column operation. It should be guarded by the security management check, so that we don't fetch it unnecessarily.

So add column would look like

@Override
public void addColumn(Session session, TableHandle tableHandle, ColumnMetadata column)
{
    CatalogHandle catalogHandle = tableHandle.getCatalogHandle();
    CatalogMetadata catalogMetadata = getCatalogMetadataForWrite(session, catalogHandle);
    ConnectorMetadata metadata = catalogMetadata.getMetadata(session);

    SchemaTableName tableName = null;
    if (catalogMetadata.getSecurityManagement() != CONNECTOR) {
        tableName = getTableMetadata(session, tableHandle).getTable();
    }

    metadata.addColumn(session.toConnectorSession(catalogHandle), tableHandle.getConnectorHandle(), column);

    if (catalogMetadata.getSecurityManagement() != CONNECTOR) {
        CatalogSchemaTableName sourceTableName = new CatalogSchemaTableName(catalogHandle.getCatalogName(), tableName);
        systemSecurityMetadata.columnCreated(session, sourceTableName, column.getName());
    }
}

[findepi]

It seems like we can fix this trivially by doing the getTableMetadata() before the column operation.

That solves some, but not all, problems outlined above.

Anyway, looking forward in seeing this in a PR.

[findepi]

Marked this as a correctness because this fixes a correctness issue #12535.
Marked this as a RELEASE-BLOCKER because the correctness issue is a regression introduced in the last release (406; #12535 (comment))

Talking with @electrum @djsstarburst offline about this PR.

[electrum]

I didn’t realize this is correctness issue in an existing connector, when not using this feature. We should revert.

[findepi]

Thank you @electrum for your review!

[findepi]

CI #14637