Skip to content

Commit af8b6cc

Browse files
felixcheunggatorsmile
authored andcommitted
[SPARK-20689][PYSPARK] python doctest leaking bucketed table
## What changes were proposed in this pull request? It turns out pyspark doctest is calling saveAsTable without ever dropping them. Since we have separate python tests for bucketed table, and there is no checking of results, there is really no need to run the doctest, other than leaving it as an example in the generated doc ## How was this patch tested? Jenkins Author: Felix Cheung <[email protected]> Closes #17932 from felixcheung/pytablecleanup.
1 parent 5c2c4dc commit af8b6cc

File tree

1 file changed

+2
-2
lines changed

1 file changed

+2
-2
lines changed

python/pyspark/sql/readwriter.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -575,7 +575,7 @@ def bucketBy(self, numBuckets, col, *cols):
575575
.. note:: Applicable for file-based data sources in combination with
576576
:py:meth:`DataFrameWriter.saveAsTable`.
577577
578-
>>> (df.write.format('parquet')
578+
>>> (df.write.format('parquet') # doctest: +SKIP
579579
... .bucketBy(100, 'year', 'month')
580580
... .mode("overwrite")
581581
... .saveAsTable('bucketed_table'))
@@ -602,7 +602,7 @@ def sortBy(self, col, *cols):
602602
:param col: a name of a column, or a list of names.
603603
:param cols: additional names (optional). If `col` is a list it should be empty.
604604
605-
>>> (df.write.format('parquet')
605+
>>> (df.write.format('parquet') # doctest: +SKIP
606606
... .bucketBy(100, 'year', 'month')
607607
... .sortBy('day')
608608
... .mode("overwrite")

0 commit comments

Comments
 (0)