From 384e041ac98cac35caf0c97b764033ea27ad887e Mon Sep 17 00:00:00 2001 From: Rajarshi Sarkar Date: Mon, 13 Dec 2021 19:49:47 +0530 Subject: [PATCH 1/6] Add stream-from-timestamp in spark-configuration.md --- site/docs/spark-configuration.md | 1 + 1 file changed, 1 insertion(+) diff --git a/site/docs/spark-configuration.md b/site/docs/spark-configuration.md index 67befeab574e..b4c3d866e2b7 100644 --- a/site/docs/spark-configuration.md +++ b/site/docs/spark-configuration.md @@ -155,6 +155,7 @@ spark.read | file-open-cost | As per table property | Overrides this table's read.split.open-file-cost | | vectorization-enabled | As per table property | Overrides this table's read.parquet.vectorization.enabled | | batch-size | As per table property | Overrides this table's read.parquet.vectorization.batch-size | +| stream-from-timestamp | Long.MIN_VALUE | Timestamp in milliseconds; start a stream from the snapshot that occurs after this timestamp. | ### Write options From 1d1582f2d3f1f4ff7497f5762eff3591dd1ae442 Mon Sep 17 00:00:00 2001 From: Rajarshi Sarkar Date: Tue, 14 Dec 2021 20:41:11 +0530 Subject: [PATCH 2/6] Implement review comments --- site/docs/spark-configuration.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/site/docs/spark-configuration.md b/site/docs/spark-configuration.md index b4c3d866e2b7..ea731a13d5da 100644 --- a/site/docs/spark-configuration.md +++ b/site/docs/spark-configuration.md @@ -155,7 +155,7 @@ spark.read | file-open-cost | As per table property | Overrides this table's read.split.open-file-cost | | vectorization-enabled | As per table property | Overrides this table's read.parquet.vectorization.enabled | | batch-size | As per table property | Overrides this table's read.parquet.vectorization.batch-size | -| stream-from-timestamp | Long.MIN_VALUE | Timestamp in milliseconds; start a stream from the snapshot that occurs after this timestamp. | +| stream-from-timestamp | Long.MIN_VALUE | Timestamp in milliseconds, start streaming this table from the first snapshot that occurs after this timestamp | ### Write options From 7a1957c746c77dfa49334aa0307943e228ddbebc Mon Sep 17 00:00:00 2001 From: Rajarshi Sarkar Date: Wed, 15 Dec 2021 11:45:25 +0530 Subject: [PATCH 3/6] Implement review comments --- site/docs/spark-configuration.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/site/docs/spark-configuration.md b/site/docs/spark-configuration.md index ea731a13d5da..569535d3264a 100644 --- a/site/docs/spark-configuration.md +++ b/site/docs/spark-configuration.md @@ -155,7 +155,7 @@ spark.read | file-open-cost | As per table property | Overrides this table's read.split.open-file-cost | | vectorization-enabled | As per table property | Overrides this table's read.parquet.vectorization.enabled | | batch-size | As per table property | Overrides this table's read.parquet.vectorization.batch-size | -| stream-from-timestamp | Long.MIN_VALUE | Timestamp in milliseconds, start streaming this table from the first snapshot that occurs after this timestamp | +| stream-from-timestamp | Long.MIN_VALUE | Timestamp in milliseconds, start streaming this table from the first snapshot that occurs at or after this timestamp | ### Write options From 88c89e88c70256ad3db7b32daec8d6133833b95a Mon Sep 17 00:00:00 2001 From: Rajarshi Sarkar Date: Wed, 22 Dec 2021 10:17:16 +0530 Subject: [PATCH 4/6] Implement review comments --- site/docs/spark-configuration.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/site/docs/spark-configuration.md b/site/docs/spark-configuration.md index 569535d3264a..3c0e2c72c888 100644 --- a/site/docs/spark-configuration.md +++ b/site/docs/spark-configuration.md @@ -155,7 +155,7 @@ spark.read | file-open-cost | As per table property | Overrides this table's read.split.open-file-cost | | vectorization-enabled | As per table property | Overrides this table's read.parquet.vectorization.enabled | | batch-size | As per table property | Overrides this table's read.parquet.vectorization.batch-size | -| stream-from-timestamp | Long.MIN_VALUE | Timestamp in milliseconds, start streaming this table from the first snapshot that occurs at or after this timestamp | +| stream-from-timestamp | Long.MIN_VALUE | Timestamp in milliseconds, start streaming this table from the first known ancestor snapshot that occurs at or after this timestamp. !!! Note If `stream-from-timestamp` is before the oldest ancestor snapshot in the table, the oldest ancestor will be used. | ### Write options From ad98e2d1f5ead38beb50911433405093d9e599be Mon Sep 17 00:00:00 2001 From: Rajarshi Sarkar Date: Tue, 4 Jan 2022 17:45:11 +0530 Subject: [PATCH 5/6] Implement review comments --- site/docs/spark-configuration.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/site/docs/spark-configuration.md b/site/docs/spark-configuration.md index 3c0e2c72c888..9ad022c07bc2 100644 --- a/site/docs/spark-configuration.md +++ b/site/docs/spark-configuration.md @@ -155,7 +155,7 @@ spark.read | file-open-cost | As per table property | Overrides this table's read.split.open-file-cost | | vectorization-enabled | As per table property | Overrides this table's read.parquet.vectorization.enabled | | batch-size | As per table property | Overrides this table's read.parquet.vectorization.batch-size | -| stream-from-timestamp | Long.MIN_VALUE | Timestamp in milliseconds, start streaming this table from the first known ancestor snapshot that occurs at or after this timestamp. !!! Note If `stream-from-timestamp` is before the oldest ancestor snapshot in the table, the oldest ancestor will be used. | +| stream-from-timestamp | (stream from the oldest ancestor snapshot) | Timestamp in milliseconds, start streaming this table from the first known ancestor snapshot that occurs at or after this timestamp. !!! Note If `stream-from-timestamp` is before the oldest ancestor snapshot in the table, the oldest ancestor will be used. | ### Write options From f9a65dd0d9a27dde0a2940e49e6b34c624e8f4b0 Mon Sep 17 00:00:00 2001 From: Ryan Blue Date: Tue, 4 Jan 2022 08:24:19 -0800 Subject: [PATCH 6/6] Update spark-configuration.md --- site/docs/spark-configuration.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/site/docs/spark-configuration.md b/site/docs/spark-configuration.md index 9ad022c07bc2..d0a4c5af3196 100644 --- a/site/docs/spark-configuration.md +++ b/site/docs/spark-configuration.md @@ -155,7 +155,7 @@ spark.read | file-open-cost | As per table property | Overrides this table's read.split.open-file-cost | | vectorization-enabled | As per table property | Overrides this table's read.parquet.vectorization.enabled | | batch-size | As per table property | Overrides this table's read.parquet.vectorization.batch-size | -| stream-from-timestamp | (stream from the oldest ancestor snapshot) | Timestamp in milliseconds, start streaming this table from the first known ancestor snapshot that occurs at or after this timestamp. !!! Note If `stream-from-timestamp` is before the oldest ancestor snapshot in the table, the oldest ancestor will be used. | +| stream-from-timestamp | (none) | A timestamp in milliseconds to stream from; if before the oldest known ancestor snapshot, the oldest will be used | ### Write options