Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[db voltdb] Add VoltDB client #1319

Merged
merged 35 commits into from
Sep 17, 2019
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
89e848f
[voltdb] Release Candidate
srmadscience Jul 1, 2019
9200099
[voltdb] JavadocStyle: First sentence should end with a period.
srmadscience Jul 1, 2019
0a534e8
[db voltdb] First pass at resolves issues raised around #1319
srmadscience Jul 6, 2019
3b375e8
[db voltdb] Fix procedure class name issue
srmadscience Jul 6, 2019
3eac920
[voltdb] Fixes for pull request #feedback
srmadscience Jul 16, 2019
74f8bd2
[db voltdb] Fixes for pull request #1319 feedback
srmadscience Jul 16, 2019
cf9fe36
[voltdb] changed default value for threads in voltdb.properties to 1 …
srmadscience Jul 24, 2019
363f724
[voltdb] fixed bin diretory SNAFU
srmadscience Jul 25, 2019
1424ea4
[voltdb] fixed bindings
srmadscience Sep 3, 2019
da873b3
[voltdb] fixed jdbc README.md
srmadscience Sep 3, 2019
0204c7c
Merge branch 'db_voltdb' of https://github.com/srmadscience/YCSB into…
srmadscience Sep 3, 2019
e8ddb53
[voltdb] fixed sortedvolttable package name
srmadscience Sep 3, 2019
0ebc1ec
[voltdb] added SLF4J + Log4j 2
srmadscience Sep 3, 2019
52fbced
[voltdb] Reverted cleint & server
srmadscience Sep 4, 2019
05feedb
[voltdb] checkstyle violations
srmadscience Sep 4, 2019
91e529a
[voltdb] checkstyle violations
srmadscience Sep 4, 2019
d120449
[voltdb] checkstyle violations
srmadscience Sep 4, 2019
84e59b7
Fixed bin directory
srmadscience Sep 4, 2019
209df09
[voltdb] Rewrote client handling, changed logic for creating DDL
srmadscience Sep 6, 2019
870b322
[voltdb] checkstyle fixes
srmadscience Sep 6, 2019
f6eb76a
[voltdb] Moved to MIT compliant licences for VoltDB libs
srmadscience Sep 6, 2019
7a5b2e6
[voltdb] Fixed %s formatting issue in log message
srmadscience Sep 6, 2019
944ebca
[voltdb] Updated pom.xml to 9.1
srmadscience Sep 6, 2019
69b199f
[voltdb] Various code changes
srmadscience Sep 6, 2019
8c61359
[voltdb] switch to singleton holder idiom.
srmadscience Sep 6, 2019
b2b866f
[voltdb] change voltdb lib to runtime dependency
srmadscience Sep 6, 2019
f9e5b11
[voltdb] change voltdb lib to default compile dependency
srmadscience Sep 6, 2019
b84606d
[voltdb] change voltdb lib to provided dependency
srmadscience Sep 6, 2019
4df5c83
[voltdb] now uses Assume.assumeTrue(haveDb)
srmadscience Sep 6, 2019
bbb968b
[voltdb] remove clientId
srmadscience Sep 7, 2019
06a9656
[voltdb] Re-arranged calls to Assume.assumeTrue(haveDb);
srmadscience Sep 7, 2019
4be8943
[voltdb] Addded call to Assume.assumeTrue(haveDb) to teardown
srmadscience Sep 7, 2019
a9a82b0
[voltdb] Put old Assume.assumeTrue(haveDb)'s back
srmadscience Sep 7, 2019
ff5ba0f
[voltdb] Remove Assume.assumeTrue from setup and teardown methods
srmadscience Sep 7, 2019
fffe7e7
[voltdb] Change all VoltDB versions to 9.1.1
srmadscience Sep 17, 2019
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions distribution/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -286,6 +286,12 @@ LICENSE file.
<version>${project.version}</version>
</dependency>
-->
<dependency>
<groupId>com.yahoo.ycsb</groupId>
<artifactId>voltdb-binding</artifactId>
<version>${project.version}</version>
</dependency>

</dependencies>
<build>
<plugins>
Expand Down
2 changes: 2 additions & 0 deletions pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -110,6 +110,7 @@ LICENSE file.
<thrift.version>0.8.0</thrift.version>
<voldemort.version>0.81</voldemort.version>
<tablestore.version>4.8.0</tablestore.version>
<voltdb.version>9.0</voltdb.version>
srmadscience marked this conversation as resolved.
Show resolved Hide resolved
</properties>

<modules>
Expand Down Expand Up @@ -168,6 +169,7 @@ LICENSE file.
<module>tarantool</module>
<module>tablestore</module>
<!--<module>voldemort</module>-->
<module>voltdb</module>
</modules>

<build>
Expand Down
68 changes: 68 additions & 0 deletions voltdb/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
# YCSB for VoltDB
srmadscience marked this conversation as resolved.
Show resolved Hide resolved

![VoltDB Logo](https://www.voltdb.com/wp-content/uploads/2016/10/VoltDB-logo-320x132.jpg?ycsb=true "VoltDB")

## README

This README describes how to run YCSB on VoltDB.

Here at VoltDB we use 4 machines for testing - 1 client and a 3 node cluster.

### 1. Install Java on all the machines involved.

VoltDB uses Java. Either [Oracle Java](https://www.oracle.com/technetwork/java/javase/downloads/index.html) or [OpenJDK](https://openjdk.java.net/) will work.


### 3. Install and configure VoltDB

If you don't already have a copy of VoltDB you should [download](https://www.voltdb.com/try-voltdb/) and install it.
Make sure you know the hostnames/ip addresses of all the nodes in the cluster and that [port 21212](https://docs.voltdb.com/AdminGuide/HostConfigPortOpts.php) is open for your client.

A representative VoltDB cluster would have 3 nodes and a '[K factor](https://docs.voltdb.com/UsingVoltDB/KSafeEnable.php)' of 1. This configuration allows work to continue if a server dies.

Note: If you contact us we can give you access to AWS Cloudformation scripts and a matching AMI to create a cluster for you.

### 4. Install YCSB

Download the [latest YCSB](https://github.com/brianfrankcooper/YCSB/releases/latest) file. Follow the instructions.

### 5. Configure VoltDB parameters

Create a file called (for example) voltdb.properties:

recordcount=10000000
operationcount=10000000
voltdb.servers=localhost
threadcount=40

Other possible entries would be:

- `voltdb.user`
- Username. Only needed if username/passwords enabled.
- `voltdb.password`
- Password. Only needed if username/passwords enabled.
- `voltdb.ratelimit`
- Maximum number of transactions allowed per second per 50 threads - e.g. 'voltdb.ratelimit=70000'. Note that as you increase the workload you eventually get to a point where throwing more and more transactions at a given configuration is counterproductive. For the three node configuration we mentioned above 70000 would be a good starting point for this value.
- `voltdb.scanall`
- Uses a single query to return data for 'Scan' operations ('workload e') instead of a separate query per partition. The later is much more scalable but generates more network traffic.

### 6. Run YCSB

See: [Running a Workload](https://github.com/brianfrankcooper/YCSB/wiki/Running-a-Workload)

Make sure you keep an eye on the voltdb GUI. If you installed VoltDB on the same computer you are running YCSB on it will be at:

http://localhost:8080/

First load the data:

bin/ycsb.sh load voltdb -P workloads/workloada -P voltdb.properties

Then run the different workloads - 'a' through 'e':

bin/ycsb.sh run voltdb -P workloads/workloada -P voltdb.properties
bin/ycsb.sh run voltdb -P workloads/workloadb -P voltdb.properties
bin/ycsb.sh run voltdb -P workloads/workloadc -P voltdb.properties
bin/ycsb.sh run voltdb -P workloads/workloadd -P voltdb.properties
bin/ycsb.sh run voltdb -P workloads/workloade -P voltdb.properties

59 changes: 59 additions & 0 deletions voltdb/pom.xml
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
<?xml version="1.0" encoding="UTF-8"?>
<!--
Copyright (c) 2012 - 2016 YCSB contributors. All rights reserved.

Licensed under the Apache License, Version 2.0 (the "License"); you
may not use this file except in compliance with the License. You
may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied. See the License for the specific language governing
permissions and limitations under the License. See accompanying
LICENSE file.
-->

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>com.yahoo.ycsb</groupId>
<artifactId>binding-parent</artifactId>
<version>0.17.0-SNAPSHOT</version>
<relativePath>../binding-parent</relativePath>
</parent>

<artifactId>voltdb-binding</artifactId>
<name>VoltDB Binding</name>
<packaging>jar</packaging>
<dependencies>
<dependency>
<groupId>com.yahoo.ycsb</groupId>
<artifactId>core</artifactId>
<version>${project.version}</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.12</version>
<scope>test</scope>
</dependency>
<!-- https://mvnrepository.com/artifact/org.voltdb/voltdbclient -->
<dependency>
<groupId>org.voltdb</groupId>
<artifactId>voltdbclient</artifactId>
srmadscience marked this conversation as resolved.
Show resolved Hide resolved
<version>9.0</version>
</dependency>
<!-- https://mvnrepository.com/artifact/org.voltdb/voltdb -->
<dependency>
<groupId>org.voltdb</groupId>
<artifactId>voltdb</artifactId>
srmadscience marked this conversation as resolved.
Show resolved Hide resolved
<version>9.0</version>
</dependency>
</dependencies>
</project>

Loading