Skip to content

Commit 76985e1

Browse files
committed
Document field-id requirements for partition specs and fix doc formatting
1 parent 4e282c3 commit 76985e1

File tree

1 file changed

+15
-0
lines changed

1 file changed

+15
-0
lines changed

site/docs/spec.md

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -646,6 +646,18 @@ Each partition field in the fields list is stored as an object. See the table fo
646646

647647
In some cases partition specs are stored using only the field list instead of the object format that includes the spec ID, like the deprecated `partition-spec` field in table metadata. The object format should be used unless otherwise noted in this spec.
648648

649+
#### Partition Field ID handling
650+
651+
A partition field id is an integer (starting at 1000) used to identify a partition field.
652+
653+
Since iceberg release 0.8.0, partition fields are present in every partition field of partition specs in a table metadata.
654+
655+
* For backward compatibility, if field ids are missing in a table metadata, iceberg will sequentially generate ids for each field starting at 1000 based on its position in the list of fields.
656+
* For forward compatibility, if field ids are not supported, iceberg will ignore field ids.
657+
658+
Additionally, in table metadata format v2, partition fields are required to have unique field IDs to support partition spec evolution.
659+
660+
For tables without partition field IDs, iceberg will generate an auto-increment unique field id starting at 1000 for every partition field.
649661

650662
### Table Metadata and Snapshots
651663

@@ -698,14 +710,17 @@ This serialization scheme is for storing single values as individual binary valu
698710
### Version 2
699711

700712
Writing v1 metadata:
713+
701714
* Table metadata field `last-sequence-number` should not be written.
702715
* Snapshot field `sequence-number` should not be written.
703716

704717
Reading v1 metadata:
718+
705719
* Table metadata field `last-sequence-number` must default to 0.
706720
* Snapshot field `sequence-number` must default to 0.
707721

708722
Writing v2 metadata:
723+
709724
* Table metadata added required field `last-sequence-number`.
710725
* Table metadata now requires field `table-uuid`.
711726
* Table metadata now requires field `partition-specs`.

0 commit comments

Comments
 (0)