Proposition for interface

[aneudecker]

We have some options how to define an interface for the producer and consumer objects

• use the raw rcpp modules
• enhance the rcpp modules with some usability functions like print, value, etc. "functional style"
• use an oop approach and hide the consumer and producer objects within the class

I prefer the last approach and like the idea to use R6 classes. Even though this is not the most R idiomatic it helps us to provide a clean interface and separate between rcpp and R functionality. Example of usage:

config <- list("bootstrap.servers" = "my.kafka.server")
consumer <- Consumer$new(config)
consumer$subscribe("topic")

msg <- consumer$poll(timeout = 1)

rm(consumer) # calls finalizer of R6 class (unsubscribe, close connection)

For the message we also need to think about the interface as it not only contains the result but possible also keys, error codes, error messages. I would think of messages as list objects of class kafka_message

message_content(msg) 
message_error_code(msg)
message_error_(msg)
message_key(msg)

Producing messages would work the same. I would give the user the possibility to both publish using key and value or using a message object

publisher$publish(key="my_key", value="my_value")

msg <- kafka_message(key="my_key", value="my_value")
publisher$publish(message=msg)

@wahani happy to hear your thoughts on this

[aneudecker]

We discussed this:

• use R6 for publisher and consumer
• nest message within kafka_result object like this (misusing python type annotations :))

kafka_result <- structure(
    list(
        message = structure(
            list(
                key="",
                value""
            ),
            class = "KafkaMessage"
        ),
        error_code = "",
        error_message = ""
    ),
    class = "KafkaResult"
)

result_error_code <- function(...) -> character | NULL
result_has_error <- function(...) -> logical
result_error_message <- function() -> character | NULL

result_message <- function(...) -> KafkaMessage | NULL

Access key and value of message using $

message$key: character | None
message$value: character

[aneudecker]

This has been merged already into the main branch