Add support for Spring Cloud Stream Kafka Streams binder

[sobychacko]

As a developer, I would like to go to the initializr and generate a project to write boot apps against the Kafka Streams support in Spring Cloud Stream. [1]

I would also like to generate a project that contains both the kafka streams binder (spring-cloud-stream-binder-kafka-streams) and the MessageChannel based kafka binder (spring-cloud-stream-binder-kafka) [2]

We only want these to be supported for Spring Boot 2.0.0.RELEASE.

[1] Here is a sample with the Kafka Streams binder - https://github.com/spring-cloud/spring-cloud-stream-samples/tree/master/kafka-streams-samples/kafka-streams-aggregate
[2] Here is a sample that uses Kafka Streams and regular Kafka binder - https://github.com/spring-cloud/spring-cloud-stream-samples/tree/master/kafka-streams-samples/kafka-streams-word-count

[sabbyanandan]

Hi, @snicoll. As we are gearing up for the SCSt 2.0 GA release this week, we would appreciate if you can help add the KStreams binder to Initializr.

As for the background, we are promoting this binder implementation actively (screencasts/blogs etc.,) and it'd help the community if it shows up a top-level binder option in the Initializr. We can simplify our docs and as well as the other promotion material. It makes it easy to get started.

Please let us know if we can help clarify anything.

[snicoll]

There is no dependency management for org.apache.kafka:kafka-streams so if we decided to pursue this, we'll have to hardcode a version in the metadata

[snicoll]

Also, there are other extension hook points that aren't managed centrally, potentially leading to version conflicts:

• kafka-clients (same version as Kafka Streams)
• The Kafka extension for Spring Integration

I've pushed a proposal in 1412a0b but we need to discuss first how we're going to deal with the dependency management situation.

[wilkinsona]

Spring Integration Kafka was removed from the IO Platform in Cairo on the recommendation of @garyrussell. As such, I do not think we should be adding dependency management for it to Boot. While not ideal, that leaves us with "managing" it here.

[wilkinsona]

Boot's KafkaProperties depends on 6 Kafka classes. Those classes are only on the classpath via a transitive dependency and we're not managing the version. I think that's a mistake; if Boot's using a dependency directly we should at least manage its version. I'll open a Boot issue to fix that.