diff --git a/README.md b/README.md index b348d77d..70dceba4 100644 --- a/README.md +++ b/README.md @@ -35,6 +35,7 @@ This library is work in progress so the API may change before the official relea - [Dataframe](doc/dataframe.md) - [Streaming](doc/streaming.md) - [Cluster](doc/cluster.md) + - [Java](doc/java.md) - [Python](doc/python.md) - [Configuration](doc/configuration.md) diff --git a/doc/java.md b/doc/java.md new file mode 100644 index 00000000..aab522d2 --- /dev/null +++ b/doc/java.md @@ -0,0 +1,115 @@ +# Using the library in Java + +The library is written in Scala and the API is primarily intended to be used with Scala. But you can also use the library in +Java because of the Scala/Java interoperability. + + +## RDD + +Please, refer to the detailed documentation of [RDD support](rdd.md) for the full list of available features. +The RDD functions are available in `RedisContext`. Example: + +```java +SparkConf sparkConf = new SparkConf() + .setAppName("MyApp") + .setMaster("local[*]") + .set("spark.redis.host", "localhost") + .set("spark.redis.port", "6379"); + +RedisConfig redisConfig = RedisConfig.fromSparkConf(sparkConf); +ReadWriteConfig readWriteConfig = ReadWriteConfig.fromSparkConf(sparkConf); + +JavaSparkContext jsc = new JavaSparkContext(sparkConf); +RedisContext redisContext = new RedisContext(jsc.sc()); + +JavaRDD> rdd = jsc.parallelize(Arrays.asList(Tuple2.apply("myKey", "Hello"))); +int ttl = 0; + +redisContext.toRedisKV(rdd.rdd(), ttl, redisConfig, readWriteConfig); + +``` + +## Datasets and DataFrames + +The Dataset/DataFrame API is the same in Java and Scala. Please, refer to [DataFrame page](dataframe.md) for details. Here is an +example with Java: + +```Java +public class Person { + + private String name; + private Integer age; + + public Person() { + } + + public Person(String name, Integer age) { + this.name = name; + this.age = age; + } + + public String getName() { + return name; + } + + public void setName(String name) { + this.name = name; + } + + public Integer getAge() { + return age; + } + + public void setAge(Integer age) { + this.age = age; + } +} + +``` + +```Java +SparkSession spark = SparkSession + .builder() + .appName("MyApp") + .master("local[*]") + .config("spark.redis.host", "localhost") + .config("spark.redis.port", "6379") + .getOrCreate(); + +Dataset df = spark.createDataFrame(Arrays.asList( + new Person("John", 35), + new Person("Peter", 40)), Person.class); + +df.write() + .format("org.apache.spark.sql.redis") + .option("table", "person") + .option("key.column", "name") + .mode(SaveMode.Overwrite) + .save(); +``` + +## Streaming + +The following example demonstrates how to create a stream from Redis list `myList`. Please, refer to [Streaming](streaming.md) for more details. + +```java +SparkConf sparkConf = new SparkConf() + .setAppName("MyApp") + .setMaster("local[*]") + .set("redis.host", "localhost") + .set("redis.port", "6379"); + +JavaStreamingContext jssc = new JavaStreamingContext(sparkConf, Durations.seconds(1)); + +RedisConfig redisConfig = new RedisConfig(new RedisEndpoint(sparkConf)); + +RedisStreamingContext redisStreamingContext = new RedisStreamingContext(jssc.ssc()); +String[] keys = new String[]{"myList"}; +RedisInputDStream> redisStream = + redisStreamingContext.createRedisStream(keys, StorageLevel.MEMORY_ONLY(), redisConfig); + +redisStream.print(); + +jssc.start(); +jssc.awaitTermination(); +``` \ No newline at end of file