[SPARK-16266][SQL][STREAING] Moved DataStreamReader/Writer from pyspark.sql to pyspark.sql.streaming #13955
[SPARK-16266][SQL][STREAING] Moved DataStreamReader/Writer from pyspark.sql to pyspark.sql.streaming #13955tdas wants to merge 7 commits intoapache:masterfrom
Conversation
|
@zsxwing Can you take a look? |
|
Test build #61417 has finished for PR 13955 at commit
|
|
Test build #61419 has finished for PR 13955 at commit
|
|
Test build #61420 has finished for PR 13955 at commit
|
|
Test build #61431 has finished for PR 13955 at commit
|
| self.interval) | ||
|
|
||
|
|
||
| class DataStreamReader(OptionUtils): |
There was a problem hiding this comment.
can we create a separate file? this file is already very long.
There was a problem hiding this comment.
then probably we have to create directory streaming and move all the code in streaming.py into separate files. I wanted to minimize the changes. we can do the bookkeeping later. what do you think?
|
Test build #61432 has finished for PR 13955 at commit
|
|
Test build #61439 has finished for PR 13955 at commit
|
|
LGTM. Merging to master and 2.0. Thanks! |
…rk.sql to pyspark.sql.streaming ## What changes were proposed in this pull request? - Moved DataStreamReader/Writer from pyspark.sql to pyspark.sql.streaming to make them consistent with scala packaging - Exposed the necessary classes in sql.streaming package so that they appear in the docs - Added pyspark.sql.streaming module to the docs ## How was this patch tested? - updated unit tests. - generated docs for testing visibility of pyspark.sql.streaming classes. Author: Tathagata Das <tathagata.das1565@gmail.com> Closes #13955 from tdas/SPARK-16266.
What changes were proposed in this pull request?
How was this patch tested?