Kafka

FastProto built-in Kafka serializer & deserializer.

FastProto provides ProtoKafkaConfig class to help developers configure Kafka. Assuming Weather is a data object, the configuration code examples for producers and consumers are as follows.

Kafka Producer

Use Kafka's built-in string serializer as key serializer, and use FastProto serializer as value serializer. protocol.class and datagram.length are two required configuration properties.

Properties props = new Properties();

props.put("bootstrap.servers", ...);
props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");
props.put("value.serializer", ProtoKafkaConfig.SERIALIZER_NAME_VALUE);
props.put(ProtoKafkaConfig.PROTOCOL_CLASS_KEY, Weather.class);
props.put(ProtoKafkaConfig.DATAGRAM_LENGTH_KEY, 26);

val producer = new KafkaProducer<String, Weather>(props);
producer.send(record);

Kafka Consumer

Use Kafka's built-in string deserializer as key deserializer, and use FastProto deserializer as value deserializer. protocol.class is a required configuration property.

Properties props = new Properties();

props.put("bootstrap.servers", ...);
props.put("key.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");
props.put("value.deserializer", ProtoKafkaConfig.DESERIALIZER_NAME_VALUE);
props.put(ProtoKafkaConfig.PROTOCOL_CLASS_KEY, Weather.class);
props.put("group.id", "1");

val producer = new KafkaProducer<String, Weather>(props);
producer.send(record);