Release version 0.8.0

[cmatser]

For the next release, 0.8.0, the following items have already been merged:

• Accumulo improvements:[accumulo] don't run Accumulo tests on windows build. #626, Revert "[accumulo] don't run Accumulo tests on windows build." #627, [accumulo] don't run Accumulo tests on windows build. #628
• Cassandra improvements: Update cassandra2-cql driver to only run tests on jdk8 #669, [cassandra] properly clear cached containers. #673
• Core improvements: [core] added real version printout #641, [core] Add a UnixEpochTimestampGenerator #656, [core] Add byte to double and long conversion methods to Utils. #657
• Couchbase improvements: [couchbase] nit fix in Couchbase README #631
• Dynamodb improvements: [dynamodb] Checkstyle updates for DynamoDB. #647
• Elasticsearch improvements: Elasticsearch cleanup #644
• Geode improvements: [geode] Checkstyle updates for Apache Geode (incubating) #648
• HBase10 improvements: [hbase10] Turn off info servers during tests #621, [hbase10] Fix the init() method to properly construct a single… #651
• Infinispan improvements: [infinispan] Checkstyle updates for the Infinispan binding. #650
• Orientdb improvements: [orientdb] fixed scan with readallfields #629
• S3 improvements: [S3] Close opened S3Object #652
• Tarantool improvements: [tarantool] Checkstyle updates for the Tarantool binding. #649

Breaking changes:

• None.

Bindings to test:

• Accumulo [fixes]
• Cassandra [fixes]
• Couchbase [fixes]
• Dynamodb [fixes]
• Elasticsearch [fixes]
• Geode [fixes]
• HBase10 [fixes]
• Infinispan [fixes]
• Orientdb [fixes]
• S3 [fixes]
• Tarantool [fixes]

Release plan:

• RC1 - Friday Apr 1st
• 0.8.0 - by Sunday Apr 10th

[cmatser]

RC1 is available: https://github.com/brianfrankcooper/YCSB/releases/tag/0.8.0-RC1

Please chime in if you can help test and which platforms you will work on. THANKS!

[busbey]

I should have Accumulo and works-on-windows tested this week.

[cmatser]

Thanks @busbey! Would you be available to also check the HBase10 binding?

For this release candidate 0.8.0-RC1:
@sbawaska can you please check Geode ?
@kruthar can you please check Cassandra, Couchbase, OrientDB ?
@yuyantingzero can you please check the DynamoDB ?
@jasontedor can you please check Elasticsearch ?
@zx-zheng can you please check S3 ?
Is there someone that can sign up for Infinispan or Tarantool?

[yuyantingzero]

Ran ycsb 0.8.0-RC1 against workloada for both HASH key and HASH_AND_RANGE key on dynamodb-binding. Looks good.

[jasontedor]

can you please check Elasticsearch ?

@cmatser Yes, I'll check it tomorrow. Thanks for pinging me.

[allanbank]

@cmaster - I will try and big up my notes in Tarantool.

[allanbank]

I am having a hard time building due to the com.yahoo.ycsb.db.CassandraCQLClientTest test failing due to Cassandra not starting. That seems to be caused by the lack of the Sigar native library:

Running com.yahoo.ycsb.db.CassandraCQLClientTest
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
no libsigar-amd64-linux.so in java.library.path
org.hyperic.sigar.SigarException: no libsigar-amd64-linux.so in java.library.path
        at org.hyperic.sigar.Sigar.loadLibrary(Sigar.java:172)
        at org.hyperic.sigar.Sigar.<clinit>(Sigar.java:100)
        at org.apache.cassandra.utils.SigarLibrary.<init>(SigarLibrary.java:47)
        at org.apache.cassandra.utils.SigarLibrary.<clinit>(SigarLibrary.java:28)
        at org.apache.cassandra.service.StartupChecks$6.execute(StartupChecks.java:197)
        at org.apache.cassandra.service.StartupChecks.verify(StartupChecks.java:106)
        at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:169)
        at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:551)
        at org.cassandraunit.utils.EmbeddedCassandraServerHelper$1.run(EmbeddedCassandraServerHelper.java:125)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 10.618 sec <<< FAILURE!
com.yahoo.ycsb.db.CassandraCQLClientTest  Time elapsed: 10.618 sec  <<< FAILURE!
java.lang.AssertionError: Cassandra daemon did not start within timeout
        at org.cassandraunit.utils.EmbeddedCassandraServerHelper.startEmbeddedCassandra(EmbeddedCassandraServerHelper.java:132)
        at org.cassandraunit.utils.EmbeddedCassandraServerHelper.startEmbeddedCassandra(EmbeddedCassandraServerHelper.java:87)
        at org.cassandraunit.utils.EmbeddedCassandraServerHelper.startEmbeddedCassandra(EmbeddedCassandraServerHelper.java:66)
        at org.cassandraunit.utils.EmbeddedCassandraServerHelper.startEmbeddedCassandra(EmbeddedCassandraServerHelper.java:58)
        at org.cassandraunit.BaseCassandraUnit.before(BaseCassandraUnit.java:31)
        at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:46)
        at org.junit.rules.RunRules.evaluate(RunRules.java:20)
        at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
        at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:252)
        at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:141)
        at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:112)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)

Anyone else have this issue?

I tried both the OpenJDK 8 and Oracle JDK 8 with the same error.

[busbey]

It's supposed to just start without native sigar if needed.

What OS and version?

[allanbank]

Fedora 23.

There are a number of Cassandra tickets that if the machine is older the timeout might need to be increased. This desktop is getting older but not that old. I noticed that this test is not running (that I could find) as part of the Travis builds. Any idea how that was turned off?

[busbey]

See #669 and #676

[allanbank]

Building with Java 7 works.

[busbey]

Java 7 will totally skip testing the cassandra2 module, so it should work fine. :)

[allanbank]

I ran the basic set of workloads against Tarantool and saw no issues.

[cmatser]

@yuyantingzero Thank you. Can you verify what version of DynamoDB?

@allanbank Thank you. Can you verify Tarantool version used?

[allanbank]

Tarantool 1.6.6-249

[zx-zheng]

@cmatser I checked the S3 binding and it works well.

[cmatser]

@zx-zheng Thank you. Is there are specific version of S3 you tested with?

[zx-zheng]

@cmatser No, I tested with our private S3 compatible system.

[yuyantingzero]

@cmatser aws sdk version is 1.10.48. Dynamodb is managed and doesn't have a version.

[allanbank]

For fun I tested the mongodb and mongodb-async bindings again the following versions of MongoDB:

• 3.2.4
• 3.0.11
• 2.6.12
• 2.4.14
• 2.2.7
• 2.0.9
• 1.8.5 (mongodb-async only)

No issues.

[bigbes]

I'll check latest Tarantool version

[jasontedor]

@cmatser I opened #684 to bring Elasticsearch to the latest stable version and address some issues.

[cmatser]

We're nearing the end of the week and I wanted to get some input on the remaining pieces that still need to be tested.

I didn't get strong commitments on the following bindings. Please reply to this thread if you can pickup one or more of these bindings to test for the release:
Cassandra
Couchbase
Geode
HBase10
Infinispan
Orientdb

These bindings are tested and ready for the release:
Dynamodb
S3
Tarantool

These bindings are in process by the following people.
Accumulo @busbey
Elasticsearch @jasontedor - Do you think that pull request #684 should be merged in for this release 0.8.0?

Please reply to this thread with any updates. Thanks for all the help. Keeping YCSB up-to-date would not happen without the contributions from all of you.

[jasontedor]

Do you think that pull request #684 should be merged in for this release 0.8.0?

@cmatser Yes, I do. During testing I noticed two issues that deserve attention:

• the Elasticsearch client enables you to specify that you want to start with a new index, but if you do this on an empty data directory you'll get an IndexNotFoundException; instead, the code should check for existence of the index first and only attempt to delete if the index exists and a new index was requested
• the Elasticsearch client enables you to specify that you want to run against a remote node, but if you try to run this in the simplest possible way (-p elasticsearch.remote=true) it will fail because the transport client is not constructed properly (it's constructed as a local node (i.e., does not use the network), and it doesn't configure the cluster name). It is possible to work around these issues but I don't think that is going to be readily apparent to most end-users.

Given that, I would like to see #684 to address those two issues and clean up some other things as well.

[busbey]

I'm testing hbase10 now and should get Accumulo and running on Windows done before the 10th.

@daschl could you test the current couchbase binding included in this release candidate?

[sbawaska]

@cmatser will check and report back by EOD.

[busbey]

hbase10 binding checks out fine. tested against a cluster running a derivative of HBase 1.0.0 (CDH5.5.2, specifically)

[bigbes]

It works with Tarantool 1.6.8 and Tarantool 1.7 (Alpha)

[daschl]

@busbey couchbase1 looks good (tested against Server 4.1)! Do you think we can get in couchbase2 for 0.8 as well?

[busbey]

@busbey couchbase1 looks good (tested against Server 4.1)! Do you think we can get in couchbase2 for 0.8 as well?

There's already a lot going into this release, I would just aim for the start-of-May release so that we needn't rush getting things done.

Usually the creation of the staging branch marks feature-complete for the release. Adding support for a new major version of a datastore feels like a new feature to me.

[cmatser]

RC3 is available: https://github.com/brianfrankcooper/YCSB/releases/tag/0.8.0-RC3

This release candidate contains the recent Elasticsearch fix for race condition. @jasontedor, can you please test this release candidate on the Elasticsearch platform and verify that I've pulled in your commit appropriately. Thanks!

Good catch on the race condition! They can be difficult to diagnose and frustrating for the end user.

[busbey]

RC2 works as expected from binary form when following our Windows instructions.

When building from source I had to skip tests with JDK8 due to the same timeouts in Cassandra Unit as @allanbank ran into. Those tests built successfully for the 0.7.0 RC, so I'm going to try to get a patch together to tune the timeouts. I don't think this issue needs to hold up the release, we'll just have to be mindful if folks with older windows machines start showing up asking for help.

[cmatser]

I tested RC3 with cassandra 2.2.5. Looks good.

First draft of release notes: https://gist.github.com/cmatser/1fc8a4aa451947dc8df73c1302fc45e4

I think we'll be ready to go with the release after I hear back on Accumulo and Elasticsearch and their versions.

[jasontedor]

@jasontedor, can you please test this release candidate on the Elasticsearch platform and verify that I've pulled in your commit appropriately.

Elasticsearch is good. I tested RC3 again against a local node, a remote node, and a remote three-node cluster and all is well.

[cmatser]

Thanks @jasontedor! What version?

[jasontedor]

What version?

@cmatser I tested the Elasticsearch bindings in 0.8.0 RC3 (e9378c7) against remote nodes running 2.0.2, 2.1.2, 2.2.2, and 2.3.1 of Elasticsearch (local nodes always run 2.3.1).

[busbey]

we should include #692 once it merges.

[busbey]

👍 accumulo binding works as expected. I ran into a couple of issues, but not things specific to this release or that we have a solid approach to fixing.

• As described in my comment on Benchmarking Runs Forever [Question] #618, I had a cluster failure during the load phase and my client configuration didn't allow the enforcement of maxexecutiontime to work. I think we might be able to improve the Accumulo client specifically to set timeouts on its operations (the default is retry-forever, I think), but it's not a blocker.
• I also realized that the Accumulo client is claiming all OK status even if you run a read workload before there are any records loaded. I tracked this in [accumulo] Accumulo client only ever returns success for operations that ought to require a record exist #699. It's existed as long as we've had the accumulo client, so I also don't think this is a blocker.

[busbey]

We should still wait on #692 before releasing.

The release notes look good, a couple of feedback points:

• it might be worth adding Benchmarking Runs Forever [Question] #618 to the known issue list. Something like "Datastore drivers that lack timeouts may result in YCSB failing to honor the maxexecutiontime parameter. See Benchmarking Runs Forever [Question] #618 for details and discussion."
• the mongodb bindings should be listed as tested-supported since @allanbank tested them above.
• I don't see a disposition for the hbase098 binding. It should probably be in the supported category, since it was tested/supported in 0.7.0.
• It looks like OrientDB was in the list to-test, but hasn't gotten any. I think this means it should move to the untested category, since we don't know if the improvements have caused a regression.

[busbey]

The list of Cassandra related improvements should call out #614, since it added compatibility for Cassandra 3. It was pulled in via the merge of #669 (which is already listed), but we should make that more obvious.

It would have been great to get a test against a Cassandra 3 cluster in, since that should work now and is new, but I don't think I have time before #692 lands. Anyone else have ~30 minutes to run through things?

[cmatser]

@busbey thanks for going over the release notes.

I'm a little unsure how to word the Cassandra 3 update. How do we say we have updates to support Cassandra 3, but it is untested and unverified? There's still issue #677 to verify Cassandra 3.

What version of Accumulo did you test?

...waiting on the #692 merge. Then I'll create RC4 and ask you to retest HBase10.

[busbey]

We could just list Cassandra 3.x via the cassandra2-cql binding under untested (presuming I have the name right)

[busbey]

I tested with Accumulo 1.6.0. If folks end up here and would like precise details, it was the vendor packaged 1.6.0-cdh5.1.4 running on top of CDH 5.7.0.

[sbawaska]

@cmatser Geode was tested on 1.0.0-incubating.M1

[cmatser]

RC4 is available: https://github.com/brianfrankcooper/YCSB/releases/tag/0.8.0-RC4

This release candidate contains the recent HBase10 fix for synchronization. @busbey , can you please test this release candidate on the HBase platform and verify that I've pulled in the appropriate commit appropriately. Thanks!

Thanks @sbawaska, I've captured the Geode version.

I believe this HBase test is the last hurdle before closing out the release.

@busbey, should we call out attention to the follow-on work for HBase regarding the syncronization fix?

[busbey]

testing RC4 now.

I don't think we need to call out the follow-on work. It doesn't impact correctness and any impact it has on performance will be seen in the next release. (we expect negligible impact)

[busbey]

+1 tested RC4 against hbase10 binding against the same cluster as last time.

[busbey]

release notes look good too. 👍

[cmatser]

YCSB 0.8.0 is out https://github.com/brianfrankcooper/YCSB/releases/tag/0.8.0

Will make the official announcements, wiki, and readme updates soon.

[cmatser]

This is the planned announcement: https://gist.github.com/cmatser/ff88389c26f2cd0b9eddc4734aedc007

[busbey]

I personally don't think the style enhancements need to be called out at all, but I can see why some folks might disagree.

Please remember to call the Apache projects "Apache Foo" in their first occurrence in the message.

• Accumulo improvements including Windows build testing

I do the windows build testing every release, I don't think it's worth calling out. I also only use the basic binding when testing on windows.

• Cassandra improvements including adding compatibility with Cassandra3

Since Cassandra3 didn't make it to supported, I'd recommend phrasing this more along the lines of "Cassandra binding updated to DataStax driver 3.0.0, tested with Cassandra 2.2.5."

In the announcement specific to the Cassandra community, I'd recommend including a note about how we'd love some help testing it against Cassandra 3 for the next release.

[risdenk]

@cmatser the updates to the announcement look good to me.

[busbey]

• Apache Accumulo improvements including proper initialization of mini clusters on Windows

The change was to skip the Accumulo tests on Windows entirely because we can't initialize the minicluster. Can probably leave it out.

[busbey]

other than that Accumulo bit, 👍

[cmatser]

Announced on the following lists:

Aerospike Forum
Amazon DynamoDB Forum
Amazon S3 Forum
Apache Accumulo User Group
Apache Cassandra User Group
Apache Geode (incubating) User Group
Apache HBase User Group
Apache Kudu (incubating) User Group
Apache Solr User Group
Couchbase Forum
Elasticsearch Forum
Google Cloud Datastore
Memcached Forum
MongoDB Forum
OrientDB Forum
Tarantool Forum
Tarantool-ru Forum

Done. Thanks to everyone for helping out.