Skip to content

Commit 63b141e

Browse files
committed
updated README and bumped version of dependencies
1 parent b2bdbca commit 63b141e

File tree

3 files changed

+25
-3
lines changed

3 files changed

+25
-3
lines changed

.gitignore

+2
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,4 @@
11
target/
22
dependency-reduced-pom.xml
3+
.idea/
4+
*.iml

README.md

+21-1
Original file line numberDiff line numberDiff line change
@@ -1,2 +1,22 @@
11
# lmdb-to-hdfs
2-
Small utility program to load data from LMDB database into an HDFS sequence file
2+
Small utility to load LMDB datastore into HDFS as a sequence file.
3+
4+
Pre-configured to run on Linux 64-bit. To run on other architectures (Windows, OSX, Android), change the Maven dependency in [this line within pom.xml](https://github.com/zach-m/lmdb-to-hdfs/blob/master/pom.xml#L20). Possible values are:
5+
* `lmdbjni-linux64`
6+
* `lmdbjni-win64`
7+
* `lmdbjni-osx64`
8+
* `lmdbjni-android`
9+
10+
See the [lmdbjni](https://libraries.io/github/deephacks/lmdbjni) project home page for more information.
11+
12+
The Maven page for this library is [here](http://mvnrepository.com/artifact/org.deephacks.lmdbjni).
13+
14+
### Usage
15+
```bash
16+
java -jar lmdbToHdfs.jar <lmdb-path> <hdfs-path> [<resource-uri>..]
17+
```
18+
* `lmdb-path` is a directory containing an LMDB database, _e.g. /var/imagenet/lmdb_
19+
* `hdfs-path` is a path for (new) sequence file on HDFS, _e.g. hdfs://master:9000/imagenet/lmdb_
20+
* `resource-uri` (0 or more values) are URIs of Hadoop configuration file, _e.g. file:///usr/local/hadoop/etc/hadoop/core-site.xml_
21+
22+
> NOTE: When not specifying any Hadoop configuration files, Hadoop's default [Configuration](https://hadoop.apache.org/docs/r2.6.4/api/org/apache/hadoop/conf/Configuration.html) will be used

pom.xml

+2-2
Original file line numberDiff line numberDiff line change
@@ -18,12 +18,12 @@
1818
<dependency>
1919
<groupId>org.deephacks.lmdbjni</groupId>
2020
<artifactId>lmdbjni-linux64</artifactId>
21-
<version>0.4.5</version>
21+
<version>0.4.6</version>
2222
</dependency>
2323
<dependency>
2424
<groupId>org.apache.hadoop</groupId>
2525
<artifactId>hadoop-client</artifactId>
26-
<version>2.6.3</version>
26+
<version>2.6.4</version>
2727
</dependency>
2828
</dependencies>
2929

0 commit comments

Comments
 (0)