shaded-protobuf-classes

This project creates shaded Java classes for Protobuf files placed in src/main/protobuf directory. These classes are suitable for use with Protobuf functions in Spark.

Java class support in Spark Protobuf connector requires the classes to be shaded. Specifically references to com.google.protobuf.* classes should be rewritten to org.sparkproject.spark_protobuf.protobuf.*.

This project is useful to generate shaded classes in adhoc manner for a set of Protobuf files. Production use cases utilize build system support for generating Java classes and shading them. Common build tools like mvn, bazel, sbt, and others support shading.

How to create shaded Java classes for your Protobufs

• Clone or download this repo
• Add Protobuf files to src/main/protobuf directory
• Run mvn clean package
  • Once the above is successful, target/shaded-protobuf-classes-1.0.jar contains shaded Java classes.

How to verify proper shading

The jar file can be inspected with a Java decompiler like JD-GUI. The generated classes should have import statements that look similar to

   import org.sparkproject.spark_protobuf.protobuf.AbstractMessage;

as shown in screenshot for AppEvent below: