Skip to content
Merged
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 32 additions & 0 deletions docs/docs/spark-procedures.md
Original file line number Diff line number Diff line change
Expand Up @@ -974,6 +974,38 @@ Collect statistics of the snapshot with id `snap1` of table `my_table` for colum
CALL catalog_name.system.compute_table_stats(table => 'my_table', snapshot_id => 'snap1', columns => array('col1', 'col2'));
```

## Partition Statistics

### `compute_partition_stats`

This procedure computes the stats incrementally from the last snapshot that has partition stats file until the given

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do we have 'partitionStatisticsFile' in some place and not in the first place?

snapshot (uses current snapshot if not specified) and writes the combined result into a `PartitionStatisticsFile`
after merging the partition stats. Does a full compute if previous statistics file does not exist. Also registers the

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: It also registers the 'PartitionStatisticsFile' to the table metadata. (complete sentence)

`PartitionStatisticsFile` to table metadata.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: the table metadata (need article)


| Argument Name | Required? | Type | Description |
|---------------|-----------|---------------|-------------------------------------|
| `table` | ✔️ | string | Name of the table |
| `snapshot_id` | | string | Id of the snapshot to collect stats |

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

describe optional behavior


#### Output

| Output Name | Type | Description |
|-------------------|--------|-------------------------------------------------|
| `partition_statistics_file` | string | Path to stats file created from by this command |

#### Examples

Collect statistics of the latest snapshot of table `my_table`

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should it be 'partition stats' to clarify

```sql
CALL catalog_name.system.compute_partition_stats('my_table');
```

Collect statistics of the snapshot with id `snap1` of table `my_table`
```sql
CALL catalog_name.system.compute_partition_stats(table => 'my_table', snapshot_id => 'snap1' );

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

small nit: extra space at end.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done.

```

## Table Replication

The `rewrite_table_path` procedure prepares an Iceberg table for copying to another location.
Expand Down