Skip to content

Commit

Permalink
[hbase] update HBase bindings for eom
Browse files Browse the repository at this point in the history
* remove 0.98, 1.0, 1.2, and 2.0 bindings
* add 2.2 binding
* incorporate README from 0.98 binding into current bindings
* incorporate README on bigtable testing from 1.0 binding into 1.4 binding
* incorporate implementation from 1.0 client into current bindings
* updated asynchbase binding to include parts of removed bindings it referenced
* update 1.4 and 2.2 to current releases
* use shaded client test for all hbase bindings.
* make hbase bindings consistently use log4j
* fixes brianfrankcooper#1173
* fixes brianfrankcooper#1172
  • Loading branch information
busbey committed Feb 3, 2020
1 parent 96eb2c4 commit 35fe020
Show file tree
Hide file tree
Showing 40 changed files with 737 additions and 2,235 deletions.
22 changes: 17 additions & 5 deletions asynchbase/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,9 +21,24 @@ This driver provides a YCSB workload binding for Apache HBase using an alternati

## Quickstart

### 1. Setup Hbase
### 1. Start a HBase Server
You need to start a single node or a cluster to point the client at. Please see [Apache HBase Reference Guide](http://hbase.apache.org/book.html) for more details and instructions.

### 2. Set up YCSB

Download the [latest YCSB](https://github.com/brianfrankcooper/YCSB/releases/latest) file. Follow the instructions.

### 3. Create a HBase table for testing

For best results, use the pre-splitting strategy recommended in [HBASE-4163](https://issues.apache.org/jira/browse/HBASE-4163):

```
hbase(main):001:0> n_splits = 200 # HBase recommends (10 * number of regionservers)
hbase(main):002:0> create 'usertable', 'family', {SPLITS => (1..n_splits).map {|i| "user#{1000+i*(9999-1000)/n_splits}"}}
```

*Failing to do so will cause all writes to initially target a single region server*.

Follow directions 1 to 3 from ``hbase098``'s readme.

### 2. Load a Workload

Expand Down Expand Up @@ -54,6 +69,3 @@ The following options can be configured using CLI (using the `-p` parameter) or
* `durable`: When set to false, writes and deletes bypass the WAL for quicker responses. Default is true.
* `jointimeout`: A timeout value, in milliseconds, for waiting on operations synchronously before an error is thrown.
* `prefetchmeta`: Whether or not to read meta for all regions in the table and connect to the proper region servers before starting operations. Defaults to false.


Note: This module includes some Google Guava source files from version 12 that were later removed but are still required by HBase's test modules for setting up the mini cluster during integration testing.
63 changes: 26 additions & 37 deletions asynchbase/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -32,12 +32,34 @@ LICENSE file.
</properties>

<dependencies>
<!-- We switch out moving everything to slf4j in asynchbase
because the facade of log4j it provides doesn't work
with the internal access hbase minicluster does.
-->
<dependency>
<groupId>org.hbase</groupId>
<artifactId>asynchbase</artifactId>
<version>${asynchbase.version}</version>
<exclusions>
<exclusion>
<groupId>org.slf4j</groupId>
<artifactId>log4j-over-slf4j</artifactId>
</exclusion>
</exclusions>
</dependency>

<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
<version>1.7.25</version>
</dependency>

<dependency>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
<version>1.2.17</version>
</dependency>

<dependency>
<groupId>site.ycsb</groupId>
<artifactId>core</artifactId>
Expand All @@ -50,14 +72,6 @@ LICENSE file.
<artifactId>zookeeper</artifactId>
<version>3.4.5</version>
<exclusions>
<exclusion>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
</exclusion>
<exclusion>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
</exclusion>
<exclusion>
<groupId>jline</groupId>
<artifactId>jline</artifactId>
Expand All @@ -82,43 +96,18 @@ LICENSE file.

<dependency>
<groupId>org.apache.hbase</groupId>
<artifactId>hbase-testing-util</artifactId>
<version>${hbase10.version}</version>
<artifactId>hbase-shaded-testing-util</artifactId>
<version>${hbase14.version}</version>
<scope>test</scope>
<exclusions>
<exclusion>
<groupId>jdk.tools</groupId>
<artifactId>jdk.tools</artifactId>
</exclusion>
</exclusions>
</dependency>

<dependency>
<groupId>org.apache.hbase</groupId>
<artifactId>hbase-client</artifactId>
<version>${hbase10.version}</version>
<artifactId>hbase-shaded-client</artifactId>
<version>${hbase14.version}</version>
<scope>test</scope>
<exclusions>
<exclusion>
<groupId>jdk.tools</groupId>
<artifactId>jdk.tools</artifactId>
</exclusion>
</exclusions>
</dependency>

<dependency>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
<version>1.2.17</version>
<scope>test</scope>
</dependency>

<dependency>
<groupId>org.slf4j</groupId>
<artifactId>log4j-over-slf4j</artifactId>
<version>1.7.7</version>
<scope>test</scope>
</dependency>
</dependencies>

<build>
Expand Down
Loading

0 comments on commit 35fe020

Please sign in to comment.