[SPARK-20855][Docs][DStream] Update the Spark kinesis docs to use the KinesisInputDStream builder instead of deprecated KinesisUtils

[yssharma]

What changes were proposed in this pull request?

The examples and docs for Spark-Kinesis integrations use the deprecated KinesisUtils. We should update the docs to use the KinesisInputDStream builder to create DStreams.

How was this patch tested?

The patch primarily updates the documents. The patch will also need to make changes to the Spark-Kinesis examples. The examples need to be tested.

[SparkQA]

Test build #77250 has finished for PR 18071 at commit 1e7e879.

• This patch fails Scala style tests.
• This patch merges cleanly.
• This patch adds no public classes.

[yssharma]

Hey @budde , drawing your attention here please.
Just wanted to update the docs with the new Builder interface by you. Few questions on top of my mind -

• Is it compatible with the python API as well like KinesisUtils used to be ?
• How can I test that on the py spark-shell. Trying to use KinesisInputDStream gives me Builder not found execptions. I might be missing something obvious here.

[HyukjinKwon]

Hi @yssharma, is it still WIP? I think we should make the PR on the mergable state at least.

[yssharma]

@HyukjinKwon The PR is ready. Just waiting for some 👍 from the reviewers.

[HyukjinKwon]

It looks the last test was failed by style checking. Would you have some time to fix them up?

[yssharma]

I noticed that now. Yes I will post an updated patch today. Thanks @HyukjinKwon

[SparkQA]

Test build #79904 has finished for PR 18071 at commit 9d50bd9.

• This patch passes all tests.
• This patch merges cleanly.
• This patch adds no public classes.

[yssharma]

Updated the patch @HyukjinKwon . Thanks for the note. 👍

[srowen]

I'm OK to merge something like this but don't know about the kinesis integration, not enough to review the content. Who could give it a second look, @budde ?

[budde]

LGTM overall. There are slight inconsistencies in the indent level between the old code samples in streaming-kinesis-integration.md and the new one but I'm not too familiar with how this gets rendered so perhaps this doesn't make a difference. Thanks for picking up my slack and updating the docs BTW— I should've really gotten to this when I added the builder interface.

[yssharma]

Thanks @budde @srowen

[srowen]

Merged to master