Use Datastax Cassandra java driver v4#7828
Conversation
|
This PR is an initial draft (although the trino-cassandra project builds and should have green tests) and has the purpose of getting feedback on how to shape the connector with Cassandra java driver v4 in order to be production ready. |
|
Useful documentation resources:
|
|
Should we have split cassandra plugin into two like we did with phoenix, so we could have two plugins cassandra3 and cassadra5. Higher maintenance cost but no backward compatible changes. WDYT? |
|
The Cassandra version being dropped (2.0.x) has had no new releases since 2015.
This refers to the driver version. The 4.x driver is compatible with Cassandra 2.1.x which is the oldest active release (released in 2014). (@findinpath It would help to clarify this in the commit message and PR description. Instead of talking about driver versions we can talk about Cassandra versions which is more clear.) |
@hashhar I am pretty sure that this commit should probably be split in several other commits to provide more insights for the maintainer in why certain changes have been made. My initial intention was to have a decent draft on which we can start the discussion. |
@kokosing
I am actually more in favor of doing the update of the cassandra client library and take advantage of the richness of configuration options for CqlSession builder. |
There was a problem hiding this comment.
I understand that V2 no longer works, am I right? If so we should not allow users to set this.
There was a problem hiding this comment.
Indeed @kokosing . Dropping the support for V2 of the protocol was a design decision from Datastax
Please see also: #7729 (comment)
There was a problem hiding this comment.
The approach for setting the load balancing policy in the v4 version of the driver implies to rethink the appropriate settings for the cassandra trino plugin.
Have a look over https://github.com/datastax/java-driver/blob/4.x/core/src/main/resources/reference.conf for the available options for load balancing policy:
DefaultLoadBalancingPolicyBasicLoadBalancingPolicyDcInferringLoadBalancingPolicy- ...
NOTE that the current implementation hardcodes the usage of DefaultLoadBalancingPolicy load balancing policy.
|
@findinpath Sorry for my late review. Could you rebase on upstream to resolve conflicts? |
|
@ebyhr Just did a rebase here: https://github.com/robd003/trino |
|
@robd003 thanks for the help. @ebyhr concerning this PR, the tests were passing at the time when I wrote the PR, but I have identified at the time the fact that cassandra v4 driver offers much much more configuration capabilities compared to cassandra v3 driver. See #7828 (comment) The branch has now been rebased on top of |
afdb0ab to
f7efc82
Compare
plugin/trino-cassandra/src/main/java/io/trino/plugin/cassandra/util/CassandraCqlUtils.java
Outdated
Show resolved
Hide resolved
plugin/trino-cassandra/src/main/java/io/trino/plugin/cassandra/BackoffRetryPolicy.java
Outdated
Show resolved
Hide resolved
plugin/trino-cassandra/src/main/java/io/trino/plugin/cassandra/CassandraType.java
Outdated
Show resolved
Hide resolved
plugin/trino-cassandra/src/main/java/io/trino/plugin/cassandra/util/HostAddressFactory.java
Outdated
Show resolved
Hide resolved
plugin/trino-cassandra/src/main/java/io/trino/plugin/cassandra/BackoffRetryPolicy.java
Outdated
Show resolved
Hide resolved
plugin/trino-cassandra/src/test/java/io/trino/plugin/cassandra/TestCassandraClientConfig.java
Outdated
Show resolved
Hide resolved
plugin/trino-cassandra/src/test/java/io/trino/plugin/cassandra/TestingScyllaServer.java
Outdated
Show resolved
Hide resolved
plugin/trino-cassandra/src/test/java/io/trino/plugin/cassandra/TestingScyllaServer.java
Outdated
Show resolved
Hide resolved
plugin/trino-cassandra/src/test/java/io/trino/plugin/cassandra/TestingScyllaServer.java
Outdated
Show resolved
Hide resolved
plugin/trino-cassandra/src/test/java/io/trino/plugin/cassandra/CassandraQueryRunner.java
Outdated
Show resolved
Hide resolved
b7fe0e5 to
5db3b3f
Compare
|
@ebyhr regarding the comments related to having more commits, I definitely agree with you. Initially I considered this PR just a draft and wanted to first gather feedback regarding the implementation. Moreover, I wasn't sure whether this PR has any chance on landing on
After analysing the content of the commit, it seems that all the files changed are linked to upgrading the cassandra driver version and for this reason I think they should share the same commit. Please advice whether you see a better approach on how to split the commit in more readable / independent commits (which still compile and have the tests running successfully). |
There was a problem hiding this comment.
Is here IAE appropriate or rather TrinoException ?
de7f747 to
416642d
Compare
dc9bff7 to
ea1f1e3
Compare
ebyhr
left a comment
There was a problem hiding this comment.
Could you update cassandra.rst where mentions protocol versions?
plugin/trino-cassandra/src/main/java/io/trino/plugin/cassandra/BackoffRetryPolicy.java
Outdated
Show resolved
Hide resolved
plugin/trino-cassandra/src/main/java/io/trino/plugin/cassandra/CassandraType.java
Outdated
Show resolved
Hide resolved
plugin/trino-cassandra/src/main/java/io/trino/plugin/cassandra/FallthroughRetryPolicy.java
Outdated
Show resolved
Hide resolved
plugin/trino-cassandra/src/main/java/io/trino/plugin/cassandra/CassandraPageSink.java
Outdated
Show resolved
Hide resolved
|
Rebasing on |
plugin/trino-cassandra/src/main/java/io/trino/plugin/cassandra/BackoffRetryPolicy.java
Outdated
Show resolved
Hide resolved
plugin/trino-cassandra/src/main/java/io/trino/plugin/cassandra/BackoffRetryPolicy.java
Outdated
Show resolved
Hide resolved
plugin/trino-cassandra/src/main/java/io/trino/plugin/cassandra/BackoffRetryPolicy.java
Outdated
Show resolved
Hide resolved
...trino-cassandra/src/test/java/io/trino/plugin/cassandra/BaseCassandraConnectorSmokeTest.java
Outdated
Show resolved
Hide resolved
...trino-cassandra/src/test/java/io/trino/plugin/cassandra/BaseCassandraConnectorSmokeTest.java
Outdated
Show resolved
Hide resolved
...trino-cassandra/src/test/java/io/trino/plugin/cassandra/BaseCassandraConnectorSmokeTest.java
Outdated
Show resolved
Hide resolved
plugin/trino-cassandra/src/main/java/io/trino/plugin/cassandra/BackoffRetryPolicy.java
Outdated
Show resolved
Hide resolved
244754b to
0891a63
Compare
After extracting the logic related to SSLContext creation to plugin-toolkit these methods became leftover in `CassandraClientModule`.
Upgrade to latest java-driver-core version 4.14.0 in order to make use of the latest improvements in the Cassandra java driver. The version 4 of the Cassandra java driver is not backwards compatible with the version 3 although most of the concepts still overlap. Most of the changes revolve to a very customizable way of building the CqlSession (which is the replacement for Cluster & Session from the version 3 of the driver). This is why there are a great deal of changes in the CassandraClientModule class. Drop the support for WhiteListPolicy because this load balancing policy is not anymore present in v4 of the Cassandra driver. HostAddressFactory deals currently only with nodes with endpoints which resolve to InetSocketAddress. Keep working with internal String names for tables and schema instead of the newly introduced CqlIdentifier in order to keep the compatibility to trino-spi layer. When specifying explicit contact points for Cassandra, it is mandatory to specify also the local data center by setting correspondingly the properties: - cassandra.load-policy.use-dc-aware - cassandra.load-policy.dc-aware.local-dc
With the introduction of Cassandra driver v4 via PR trinodb#7828 there are a few properties that are not anymore needed: - cassandra.load-policy.use-token-aware - cassandra.load-policy.token-aware.shuffle-replicas - cassandra.load-policy.allowed-addresses
With the introduction of Cassandra driver v4 via PR #7828 there are a few properties that are not anymore needed: - cassandra.load-policy.use-token-aware - cassandra.load-policy.token-aware.shuffle-replicas - cassandra.load-policy.allowed-addresses
With the introduction of Cassandra driver v4 via PR trinodb#7828 there are a few properties that are not anymore needed: - cassandra.load-policy.use-token-aware - cassandra.load-policy.token-aware.shuffle-replicas - cassandra.load-policy.allowed-addresses
Description
This PR addresses #7729
Upgrade to latest java-driver-core version 4.14.0 in order to make use
of the latest improvements in the Cassandra java driver.
The version 4 of the Cassandra java driver is not backwards compatible
with the version 3 although most of the concepts still overlap.
Most of the changes revolve to a very customizable way of building the CqlSession
(which is the replacement for Cluster & Session from the version 3 of the driver).
This is why there are a great deal of changes in the CassandraClientModule class.
Drop the support for WhiteListPolicy because this load balancing policy
is not anymore present in v4 of the Cassandra driver.
HostAddressFactory deals currently only with nodes with endpoints which
resolve to InetSocketAddress.
Keep working with internal String names for tables and schema instead of
the newly introduced CqlIdentifier in order to keep the compatibility
to trino-spi layer.
NOTE that these changes introduce breaking changes in the configuration of the
trino-cassandra connector for the following sections:
Refactoring
trino-cassandra connector
The Datastax Cassandra Java driver is described like:
Pragmatically speaking, the driver for Cassandra 3.x won't get new functionality, only critical bug fixes. So it would be wise to switch to the actively maintained library.
Related issues, pull requests, and links
Fixes #7729
Once this PR is being merged and actively used in trino, the repository https://github.com/trinodb/trino-cassandra-driver can be archived because it will no longer be in use.
Documentation
(x) No documentation is needed.
( ) Sufficient documentation is included in this PR.
( ) Documentation PR is available with #prnumber.
( ) Documentation issue #issuenumber is filed, and can be handled later.
Release notes
( ) No release notes entries required.
(x) Release notes entries required with the following suggested text: