Skip to content

feat(plugin-iceberg): Add support for engine.hive.lock-enabled property when creating iceberg tables #26234

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
hantangwangd merged 1 commit into from
Oct 26, 2025
Merged

feat(plugin-iceberg): Add support for engine.hive.lock-enabled property when creating iceberg tables #26234

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 11 additions & 2 deletions presto-docs/src/main/sphinx/connector/iceberg.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -477,6 +477,10 @@ Property Name Description
``write.update.mode`` Optionally specifies the write update mode of the Iceberg ``merge-on-read`` Yes No, write is not supported yet
specification to use for new tables, either ``copy-on-write``
or ``merge-on-read``.

``engine.hive.lock-enabled`` Whether to use Hive metastore locks when committing to Yes No
a Hive metastore

======================================================== =============================================================== ===================== =================== =============================================

The table definition below specifies format ``ORC``, partitioning by columns ``c1`` and ``c2``,
Expand Down Expand Up @@ -1495,8 +1499,13 @@ Use ``ARRAY[...]`` instead of a string to specify multiple partition transforms

ALTER TABLE iceberg.web.page_views ADD COLUMN dt date WITH (partitioning = ARRAY['year', 'bucket(16)', 'identity']);

Some Iceberg table properties can be modified using an ALTER TABLE SET PROPERTIES statement. The modifiable table properties are
``commit.retry.num-retries``, ``read.split.target-size``, ``write.metadata.delete-after-commit.enabled``, and ``write.metadata.previous-versions-max``.
Some Iceberg table properties can be modified using an ``ALTER TABLE SET PROPERTIES`` statement. The modifiable table properties are:

* ``commit.retry.num-retries``
* ``read.split.target-size``
* ``write.metadata.delete-after-commit.enabled``
* ``engine.hive.lock-enabled``
* ``write.metadata.previous-versions-max``

For example, to set ``commit.retry.num-retries`` to 6 for the table ``iceberg.web.page_views_v2``, use::

Expand Down
12 changes: 12 additions & 0 deletions presto-iceberg/src/main/java/com/facebook/presto/iceberg/IcebergTableProperties.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@
import java.util.Collection;
import java.util.List;
import java.util.Map;
import java.util.Optional;
import java.util.Set;
import java.util.function.Function;

Expand All @@ -44,6 +45,7 @@
import static java.lang.String.format;
import static java.util.Locale.ENGLISH;
import static org.apache.iceberg.TableProperties.COMMIT_NUM_RETRIES;
import static org.apache.iceberg.TableProperties.HIVE_LOCK_ENABLED;
import static org.apache.iceberg.TableProperties.METADATA_DELETE_AFTER_COMMIT_ENABLED;
import static org.apache.iceberg.TableProperties.METRICS_MAX_INFERRED_COLUMN_DEFAULTS;
import static org.apache.iceberg.TableProperties.UPDATE_MODE;
Expand Down Expand Up @@ -112,6 +114,7 @@ public class IcebergTableProperties
.add(METADATA_DELETE_AFTER_COMMIT)
.add(METADATA_DELETE_AFTER_COMMIT_ENABLED)
.add(METADATA_PREVIOUS_VERSIONS_MAX)
.add(HIVE_LOCK_ENABLED)
.add(TableProperties.METADATA_PREVIOUS_VERSIONS_MAX)
.build();

Expand Down Expand Up @@ -198,6 +201,11 @@ public IcebergTableProperties(IcebergConfig icebergConfig)
"The maximum number of columns for which metrics are collected",
icebergConfig.getMetricsMaxInferredColumn(),
false))
.add(booleanProperty(
HIVE_LOCK_ENABLED,
"Whether to enable hive locks",
null,
false))
.add(new PropertyMetadata<>(
UPDATE_MODE,
"Update mode for the table",
Expand Down Expand Up @@ -298,6 +306,10 @@ public static String getWriteDataLocation(Map<String, Object> tableProperties)
{
return (String) tableProperties.get(WRITE_DATA_LOCATION);
}
public static Optional<String> isHiveLocksEnabled(Map<String, Object> tableProperties)
{
return tableProperties.containsKey(HIVE_LOCK_ENABLED) ? Optional.of(String.valueOf(tableProperties.get(HIVE_LOCK_ENABLED))) : Optional.empty();
}

public String getFormatVersion(ConnectorSession session, Map<String, Object> tableProperties)
{
Expand Down
4 changes: 4 additions & 0 deletions presto-iceberg/src/main/java/com/facebook/presto/iceberg/IcebergUtil.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -146,6 +146,7 @@
import static com.facebook.presto.iceberg.IcebergSessionProperties.getCompressionCodec;
import static com.facebook.presto.iceberg.IcebergSessionProperties.isMergeOnReadModeEnabled;
import static com.facebook.presto.iceberg.IcebergTableProperties.getWriteDataLocation;
import static com.facebook.presto.iceberg.IcebergTableProperties.isHiveLocksEnabled;
import static com.facebook.presto.iceberg.TypeConverter.toIcebergType;
import static com.facebook.presto.iceberg.TypeConverter.toPrestoType;
import static com.facebook.presto.iceberg.util.IcebergPrestoModelConverters.toIcebergTableIdentifier;
Expand Down Expand Up @@ -191,6 +192,7 @@
import static org.apache.iceberg.TableProperties.DELETE_MODE;
import static org.apache.iceberg.TableProperties.DELETE_MODE_DEFAULT;
import static org.apache.iceberg.TableProperties.FORMAT_VERSION;
import static org.apache.iceberg.TableProperties.HIVE_LOCK_ENABLED;
import static org.apache.iceberg.TableProperties.MERGE_MODE;
import static org.apache.iceberg.TableProperties.METADATA_DELETE_AFTER_COMMIT_ENABLED;
import static org.apache.iceberg.TableProperties.METADATA_DELETE_AFTER_COMMIT_ENABLED_DEFAULT;
Expand Down Expand Up @@ -1210,6 +1212,8 @@ public static Map<String, String> populateTableProperties(IcebergAbstractMetadat

propertiesBuilder.put(SPLIT_SIZE, String.valueOf(IcebergTableProperties.getTargetSplitSize(tableMetadata.getProperties())));

isHiveLocksEnabled(tableMetadata.getProperties()).ifPresent(value -> propertiesBuilder.put(HIVE_LOCK_ENABLED, value));

return propertiesBuilder.build();
}

Expand Down
17 changes: 17 additions & 0 deletions ...to-iceberg/src/test/java/com/facebook/presto/iceberg/hive/TestIcebergDistributedHive.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,7 @@
import static com.google.common.io.Files.createTempDir;
import static java.lang.String.format;
import static org.apache.iceberg.TableMetadata.newTableMetadata;
import static org.apache.iceberg.TableProperties.HIVE_LOCK_ENABLED;
import static org.apache.iceberg.Transactions.createTableTransaction;
import static org.testng.Assert.assertEquals;
import static org.testng.Assert.assertFalse;
Expand Down Expand Up @@ -115,6 +116,21 @@ public void testStatisticsFileCache()
// so this test won't complete successfully.
}

@Test
public void testCreateAlterTableWithHiveLocksDisabled()
{
assertQuerySucceeds("CREATE TABLE test_table(i int) WITH (\"engine.hive.lock-enabled\" = false)");
assertEquals(getQueryRunner().execute("SELECT value FROM \"test_table$properties\" WHERE key = 'engine.hive.lock-enabled'").getOnlyValue(),
"false");
assertQuerySucceeds("CREATE TABLE sample_table(i int)");
assertEquals(getQueryRunner().execute("SELECT value FROM \"sample_table$properties\" WHERE key = 'engine.hive.lock-enabled'").getRowCount(),
0);
assertUpdate("ALTER TABLE sample_table SET PROPERTIES(\"engine.hive.lock-enabled\" = false)");

assertEquals(getQueryRunner().execute("SELECT value FROM \"sample_table$properties\" WHERE key = 'engine.hive.lock-enabled'").getOnlyValue(),
"false");
}

@Test
public void testManifestFileCaching()
throws Exception
Expand Down Expand Up @@ -220,6 +236,7 @@ public void testCommitTableMetadataForNoLock()
{
createTable("iceberg-test-table", createTempDir().toURI().toString(), ImmutableMap.of("engine.hive.lock-enabled", "false"), 2);
BaseTable table = (BaseTable) loadTable("iceberg-test-table");
assertEquals(table.properties().get(HIVE_LOCK_ENABLED), "false");
Copy link
Copy Markdown
Contributor

@tdcmeehan tdcmeehan Oct 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should the deault be true, as it is for Spark?

Copy link
Copy Markdown
Contributor Author

@infvg infvg Oct 14, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The default is true but I set it to false for this test

HiveTableOperations operations = (HiveTableOperations) table.operations();
TableMetadata currentMetadata = operations.current();

Expand Down
Loading