update to work with cosmosdb #922
Conversation
woodsaj
force-pushed
the
cosmosdb
branch
2 times, most recently
from
988cd10 to
baa8a5b
Compare
| @@ -356,6 +358,8 @@ password = cassandra | |||
| create-keyspace = true | |||
| # File containing the needed schemas in case database needs initializing | |||
| schema-file = /etc/metrictank/schema-idx-cassandra.toml | |||
| # instruct the driver to not attempt to get host info from the system.peers table | |||
| disable-initial-host-lookup = false | |||
There was a problem hiding this comment.
i'm not a big fan of double negatives. can we call this initial-host-lookup = true ?
There was a problem hiding this comment.
the config setting used by gocql is "DisableInitialHostLookup" so i think it makes sense to also use that.
| @@ -170,7 +170,7 @@ func GetTTLTable(ttl uint32, windowFactor int, nameFormat string) ttlTable { | |||
| } | |||
| } | |||
|
|
|||
| func NewCassandraStore(addrs, keyspace, consistency, CaPath, Username, Password, hostSelectionPolicy string, timeout, readers, writers, readqsize, writeqsize, retries, protoVer, windowFactor, omitReadTimeout int, ssl, auth, hostVerification bool, createKeyspace bool, schemaFile string, ttls []uint32) (*CassandraStore, error) { | |||
| func NewCassandraStore(addrs, keyspace, consistency, CaPath, Username, Password, hostSelectionPolicy string, timeout, readers, writers, readqsize, writeqsize, retries, protoVer, windowFactor, omitReadTimeout int, ssl, auth, hostVerification bool, createKeyspace bool, schemaFile string, ttls []uint32, disableInitialHostLookup bool) (*CassandraStore, error) { | |||
There was a problem hiding this comment.
we can make this more compact by bundling the bool params together
There was a problem hiding this comment.
I will fix up this constructor in a separate PR.
store/cassandra/store_cassandra.go
Outdated
| var err error | ||
| tmpSession, err := cluster.CreateSession() | ||
| if err != nil { | ||
| log.Info("cassandra_store: session timeout: %s", cluster.Timeout.String()) | ||
| log.Fatal(4, err.Error()) |
There was a problem hiding this comment.
it's up to the caller to handle this error. (I checked and current callers already will exit when they get an error)
| @@ -190,11 +190,15 @@ func NewCassandraStore(addrs, keyspace, consistency, CaPath, Username, Password, | |||
| } | |||
| cluster.Consistency = gocql.ParseConsistency(consistency) | |||
| cluster.Timeout = time.Duration(timeout) * time.Millisecond | |||
| cluster.ConnectTimeout = cluster.Timeout | |||
There was a problem hiding this comment.
this seems useful, but warrants a separate commit so we can mark it in the changelog
store/cassandra/store_cassandra.go
Outdated
| var err error | ||
| tmpSession, err := cluster.CreateSession() | ||
| if err != nil { | ||
| log.Info("cassandra_store: session timeout: %s", cluster.Timeout.String()) |
There was a problem hiding this comment.
we generally don't log config params elsewhere, so i don't think we should do it for this value either
| placeholders[i] = strconv.Itoa(int(p)) | ||
| } | ||
| q := fmt.Sprintf("SELECT id, orgid, partition, name, interval, unit, mtype, tags, lastupdate from metric_idx where partition in (%s)", strings.Join(placeholders, ",")) | ||
| iter := c.session.Query(q).Iter() |
There was a problem hiding this comment.
can you shed some more light on this? is this merely because cosmos doesn't understand the former syntax, or are there other reasons such as performance?
There was a problem hiding this comment.
This is just an optimization. I ran into issues on a dev cosmosdb instance due to how it limits requests. Basically, making the request with a single query rather then separate queries for each partition uses fewer resources.
|
can you tell me which other stack the docker-cosmos is based on? |
it is not based on any of them. I built a stack that had everything I needed to test and verify things were working. |
Connecting to a cassandra cluster can sometimes take longer then the default 600ms, especially on a dev cluster and when using TLS.
when using some managed Cassandra services, like Azure Cosmos DB, the Cassandra services doesn't expose the internal cluster topology, so we need to disable initial host lookup and just send requests to the configured cluster address.
add config settings for gocql's DisableInitialHostLookup setting. This is needed when using Cosmosdb as it does not report the internal topology of the service.