Skip to content

Hive encryption nits #14659

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
smaheshwar-pltr wants to merge 5 commits into
base: main
Choose a base branch
from
Open

Hive encryption nits #14659

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
8e980d5
Hive encryption clean-ups
Nov 22, 2025
a835b40
Fix tests
Nov 22, 2025
be2a7e5
Address review comments
Nov 23, 2025
a10b264
Merge
smaheshwar-pltr Dec 5, 2025
69c127a
Nit improvements
smaheshwar-pltr Dec 5, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 22 additions & 19 deletions hive-metastore/src/main/java/org/apache/iceberg/hive/HiveTableOperations.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@
import org.apache.hadoop.hive.metastore.api.Table;
import org.apache.iceberg.BaseMetastoreOperations;
import org.apache.iceberg.BaseMetastoreTableOperations;
import org.apache.iceberg.CatalogProperties;
import org.apache.iceberg.ClientPool;
import org.apache.iceberg.LocationProviders;
import org.apache.iceberg.TableMetadata;
Expand All @@ -55,8 +56,9 @@
import org.apache.iceberg.io.FileIO;
import org.apache.iceberg.io.LocationProvider;
import org.apache.iceberg.relocated.com.google.common.annotations.VisibleForTesting;
import org.apache.iceberg.relocated.com.google.common.base.Preconditions;
import org.apache.iceberg.relocated.com.google.common.collect.ImmutableMap;
import org.apache.iceberg.relocated.com.google.common.collect.Lists;
import org.apache.iceberg.relocated.com.google.common.collect.Maps;
import org.apache.iceberg.util.PropertyUtil;
import org.apache.thrift.TException;
import org.slf4j.Logger;
Expand Down Expand Up @@ -146,15 +148,17 @@ public EncryptionManager encryption() {
}

if (tableKeyId != null) {
if (keyManagementClient == null) {
throw new RuntimeException(
"Cant create encryption manager, because key management client is not set");
}

Map<String, String> encryptionProperties = Maps.newHashMap();
encryptionProperties.put(TableProperties.ENCRYPTION_TABLE_KEY, tableKeyId);
encryptionProperties.put(
TableProperties.ENCRYPTION_DEK_LENGTH, String.valueOf(encryptionDekLength));
Preconditions.checkArgument(
keyManagementClient != null,
"Cannot create encryption manager without a key management client. Consider setting the '%s' catalog property",
CatalogProperties.ENCRYPTION_KMS_IMPL);

Map<String, String> encryptionProperties =
ImmutableMap.of(
Copy link
Contributor Author

@smaheshwar-pltr smaheshwar-pltr Nov 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In #13225, I felt like ImmutableMap.of would be more natural here

TableProperties.ENCRYPTION_TABLE_KEY,
tableKeyId,
TableProperties.ENCRYPTION_DEK_LENGTH,
String.valueOf(encryptionDekLength));

List<EncryptedKey> keys = Lists.newLinkedList();
encryptedKeysFromMetadata.ifPresent(keys::addAll);
Expand Down Expand Up @@ -318,17 +322,16 @@ protected void doCommit(TableMetadata base, TableMetadata metadata) {
base.properties().keySet().stream()
.filter(key -> !metadata.properties().containsKey(key))
.collect(Collectors.toSet());
}

if (removedProps.contains(TableProperties.ENCRYPTION_TABLE_KEY)) {
throw new IllegalArgumentException("Cannot remove key in encrypted table");
}
Preconditions.checkArgument(
!removedProps.contains(TableProperties.ENCRYPTION_TABLE_KEY),
"Cannot remove key ID from an encrypted table");

if (base != null
&& !Objects.equals(
base.properties().get(TableProperties.ENCRYPTION_TABLE_KEY),
metadata.properties().get(TableProperties.ENCRYPTION_TABLE_KEY))) {
throw new IllegalArgumentException("Cannot modify key in encrypted table");
Preconditions.checkArgument(
Objects.equals(
base.properties().get(TableProperties.ENCRYPTION_TABLE_KEY),
metadata.properties().get(TableProperties.ENCRYPTION_TABLE_KEY)),
"Cannot modify key ID of an encrypted table");
}

HMSTablePropertyHelper.updateHmsTableForIcebergTable(
Expand Down
19 changes: 17 additions & 2 deletions spark/v4.0/spark/src/test/java/org/apache/iceberg/spark/sql/TestTableEncryption.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,7 @@
import org.apache.iceberg.spark.SparkCatalogConfig;
import org.apache.iceberg.types.Types;
import org.apache.parquet.crypto.ParquetCryptoRuntimeException;
import org.apache.spark.SparkException;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.TestTemplate;
Expand Down Expand Up @@ -166,14 +167,28 @@ public void testInsertAndDelete() {
public void testKeyDelete() {
assertThatThrownBy(
() -> sql("ALTER TABLE %s UNSET TBLPROPERTIES (`encryption.key-id`)", tableName))
.hasMessageContaining("Cannot remove key in encrypted table");
.isInstanceOf(SparkException.class)
.hasMessage("Unsupported table change: Cannot remove key ID from an encrypted table");
}

@TestTemplate
public void testKeyAlter() {
assertThatThrownBy(
() -> sql("ALTER TABLE %s SET TBLPROPERTIES ('encryption.key-id'='abcd')", tableName))
.hasMessageContaining("Cannot modify key in encrypted table");
.isInstanceOf(SparkException.class)
.hasMessage("Unsupported table change: Cannot modify key ID of an encrypted table");
}

@TestTemplate
public void testReplaceKeyChange() {
Copy link
Contributor Author

@smaheshwar-pltr smaheshwar-pltr Dec 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Motiviation: #14723 (comment)

// Replacing a table with a different encryption key is disallowed
assertThatThrownBy(
() ->
sql(
"REPLACE TABLE %s (id bigint) USING iceberg TBLPROPERTIES ('encryption.key-id'='%s')",
tableName, UnitestKMS.MASTER_KEY_NAME2))
.isInstanceOf(IllegalArgumentException.class)
.hasMessage("Cannot modify key ID of an encrypted table");
}

@TestTemplate
Expand Down