Skip to content

Rename last-assigned-partition-id to last-partition-id in table metadata #2188

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
rdblue merged 2 commits into from
Feb 1, 2021
Merged

Rename last-assigned-partition-id to last-partition-id in table metadata #2188

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
5b42e92
rename last-assigned-partition-id to last-partition-id in table metadata
jun-he Jan 31, 2021
4ee3638
roll back the field and and method name changes in other classes
jun-he Feb 1, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions core/src/main/java/org/apache/iceberg/TableMetadataParser.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -91,7 +91,7 @@ private TableMetadataParser() {
static final String PARTITION_SPEC = "partition-spec";
static final String PARTITION_SPECS = "partition-specs";
static final String DEFAULT_SPEC_ID = "default-spec-id";
static final String LAST_ASSIGNED_PARTITION_ID = "last-assigned-partition-id";
static final String LAST_PARTITION_ID = "last-partition-id";
static final String DEFAULT_SORT_ORDER_ID = "default-sort-order-id";
static final String SORT_ORDERS = "sort-orders";
static final String PROPERTIES = "properties";
Expand Down Expand Up @@ -179,7 +179,7 @@ private static void toJson(TableMetadata metadata, JsonGenerator generator) thro
}
generator.writeEndArray();

generator.writeNumberField(LAST_ASSIGNED_PARTITION_ID, metadata.lastAssignedPartitionId());
generator.writeNumberField(LAST_PARTITION_ID, metadata.lastAssignedPartitionId());

generator.writeNumberField(DEFAULT_SORT_ORDER_ID, metadata.defaultSortOrderId());
generator.writeArrayFieldStart(SORT_ORDERS);
Expand Down Expand Up @@ -291,10 +291,10 @@ static TableMetadata fromJson(FileIO io, InputFile file, JsonNode node) {
schema, TableMetadata.INITIAL_SPEC_ID, node.get(PARTITION_SPEC)));
}

Integer lastAssignedPartitionId = JsonUtil.getIntOrNull(LAST_ASSIGNED_PARTITION_ID, node);
Integer lastAssignedPartitionId = JsonUtil.getIntOrNull(LAST_PARTITION_ID, node);
Copy link
Member

@RussellSpitzer RussellSpitzer Feb 1, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Won't this break backwards compatibility?

Copy link
Contributor

@rdblue rdblue Feb 1, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This property was just added, so the rename is safe.

if (lastAssignedPartitionId == null) {
Preconditions.checkArgument(formatVersion == 1,
"%s must exist in format v%s", LAST_ASSIGNED_PARTITION_ID, formatVersion);
"%s must exist in format v%s", LAST_PARTITION_ID, formatVersion);
lastAssignedPartitionId = specs.stream().mapToInt(PartitionSpec::lastAssignedFieldId).max().orElse(999);
}

Expand Down
6 changes: 3 additions & 3 deletions core/src/test/java/org/apache/iceberg/TestTableMetadata.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -483,9 +483,9 @@ public void testParserV2PartitionSpecsValidation() throws Exception {

@Test
public void testParserV2LastAssignedFieldIdValidation() throws Exception {
String unsupportedVersion = readTableMetadataInputFile("TableMetadataV2MissingLastAssignedPartitionId.json");
AssertHelpers.assertThrows("Should reject v2 metadata without last assigned partition id",
IllegalArgumentException.class, "last-assigned-partition-id must exist in format v2",
String unsupportedVersion = readTableMetadataInputFile("TableMetadataV2MissingLastPartitionId.json");
AssertHelpers.assertThrows("Should reject v2 metadata without last assigned partition field id",
IllegalArgumentException.class, "last-partition-id must exist in format v2",
() -> TableMetadataParser.fromJson(
ops.io(), null, JsonUtil.mapper().readValue(unsupportedVersion, JsonNode.class))
);
Expand Down
0 ...dataV2MissingLastAssignedPartitionId.json → ...ableMetadataV2MissingLastPartitionId.json
View file
Open in desktop
File renamed without changes.
2 changes: 1 addition & 1 deletion core/src/test/resources/TableMetadataV2MissingSortOrder.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@
]
}
],
"last-assigned-partition-id": 1000,
"last-partition-id": 1000,
"properties": {},
"current-snapshot-id": -1,
"snapshots": [],
Expand Down
2 changes: 1 addition & 1 deletion core/src/test/resources/TableMetadataV2Valid.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@
]
}
],
"last-assigned-partition-id": 1000,
"last-partition-id": 1000,
"default-sort-order-id": 3,
"sort-orders": [
{
Expand Down