Skip to content

Open-API: Add table updates for statistics #9564

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
nastra merged 2 commits into from
Feb 1, 2024
Merged

Open-API: Add table updates for statistics #9564

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
3a434aa
Open-API: Add table updates for statistics
mrcnc Jan 26, 2024
e50f500
Make BlobMetadata properties optional
mrcnc Feb 1, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 29 additions & 0 deletions open-api/rest-catalog-open-api.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -342,6 +342,11 @@ class SetCurrentViewVersionUpdate(BaseUpdate):
)


class RemoveStatisticsUpdate(BaseUpdate):
action: Literal['remove-statistics']
snapshot_id: int = Field(..., alias='snapshot-id')


class TableRequirement(BaseModel):
type: str

Expand Down Expand Up @@ -596,6 +601,14 @@ class UpdateNamespacePropertiesResponse(BaseModel):
)


class BlobMetadata(BaseModel):
type: str
snapshot_id: int = Field(..., alias='snapshot-id')
sequence_number: int = Field(..., alias='sequence-number')
fields: List[int]
properties: Optional[Dict[str, Any]] = None


class CreateNamespaceRequest(BaseModel):
namespace: Namespace
properties: Optional[Dict[str, str]] = Field(
Expand All @@ -620,10 +633,24 @@ class ReportMetricsRequest2(CommitReport):
report_type: str = Field(..., alias='report-type')


class StatisticsFile(BaseModel):
snapshot_id: int = Field(..., alias='snapshot-id')
statistics_path: str = Field(..., alias='statistics-path')
file_size_in_bytes: int = Field(..., alias='file-size-in-bytes')
file_footer_size_in_bytes: int = Field(..., alias='file-footer-size-in-bytes')
blob_metadata: List[BlobMetadata] = Field(..., alias='blob-metadata')


class Term(BaseModel):
__root__: Union[Reference, TransformTerm]


class SetStatisticsUpdate(BaseUpdate):
action: Literal['set-statistics']
snapshot_id: int = Field(..., alias='snapshot-id')
statistics: StatisticsFile


class UnaryExpression(BaseModel):
type: ExpressionType
term: Term
Expand Down Expand Up @@ -758,6 +785,8 @@ class TableUpdate(BaseModel):
SetLocationUpdate,
SetPropertiesUpdate,
RemovePropertiesUpdate,
SetStatisticsUpdate,
RemoveStatisticsUpdate,
]


Expand Down
83 changes: 83 additions & 0 deletions open-api/rest-catalog-open-api.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2150,6 +2150,8 @@ components:
remove-properties: '#/components/schemas/RemovePropertiesUpdate'
add-view-version: '#/components/schemas/AddViewVersionUpdate'
set-current-view-version: '#/components/schemas/SetCurrentViewVersionUpdate'
set-statistics: '#/components/schemas/SetStatisticsUpdate'
remove-statistics: '#/components/schemas/RemoveStatisticsUpdate'
type: object
required:
- action
Expand Down Expand Up @@ -2394,6 +2396,37 @@ components:
type: integer
description: The view version id to set as current, or -1 to set last added view version id

SetStatisticsUpdate:
allOf:
- $ref: '#/components/schemas/BaseUpdate'
required:
- action
- snapshot-id
- statistics
properties:
action:
type: string
enum: [ "set-statistics" ]
snapshot-id:
type: integer
format: int64
statistics:
$ref: '#/components/schemas/StatisticsFile'

RemoveStatisticsUpdate:
allOf:
- $ref: '#/components/schemas/BaseUpdate'
required:
- action
- snapshot-id
properties:
action:
type: string
enum: [ "remove-statistics" ]
snapshot-id:
type: integer
format: int64

TableUpdate:
anyOf:
- $ref: '#/components/schemas/AssignUUIDUpdate'
Expand All @@ -2411,6 +2444,8 @@ components:
- $ref: '#/components/schemas/SetLocationUpdate'
- $ref: '#/components/schemas/SetPropertiesUpdate'
- $ref: '#/components/schemas/RemovePropertiesUpdate'
- $ref: '#/components/schemas/SetStatisticsUpdate'
- $ref: '#/components/schemas/RemoveStatisticsUpdate'

ViewUpdate:
anyOf:
Expand Down Expand Up @@ -3158,6 +3193,54 @@ components:
metadata:
$ref: '#/components/schemas/TableMetadata'

StatisticsFile:
type: object
required:
- snapshot-id
- statistics-path
- file-size-in-bytes
- file-footer-size-in-bytes
- blob-metadata
properties:
snapshot-id:
type: integer
format: int64
statistics-path:
type: string
file-size-in-bytes:
type: integer
format: int64
file-footer-size-in-bytes:
type: integer
format: int64
blob-metadata:
type: array
items:
$ref: '#/components/schemas/BlobMetadata'

BlobMetadata:
type: object
required:
- type
- snapshot-id
- sequence-number
- fields
properties:
type:
type: string
snapshot-id:
type: integer
format: int64
sequence-number:
type: integer
format: int64
fields:
type: array
items:
type: integer
properties:
type: object


#############################
# Reusable Response Objects #
Expand Down