StatefulSet clustering in Kube

[sholdee]

I have been trying to create a Gravity cluster with a 3 replica StatefulSet, but the methods documented for joining the cluster are not possible in this configuration. Various methods I have tried to manually create the cluster have all failed. I was wondering how difficult it would be to have some environment variable, for instance, that could be set that would bootstrap the cluster.

One thing I have done is set the INSTANCE_IDENTIFIER for each pod dynamically to it's DNS name from a headless service (manifest below), which works, and then clustering manually via etcdctl, but I haven't been able to get that part working.

gravity-statefulset.yml: https://pastebin.com/raw/2AgK5xGV

[BeryJu]

Most of the node joining mechanics are attempts to make it easier for etcd newcomers and hence probably don't have enough dynamic-ness for K8s. It would probably be easier to use something like https://operatorhub.io/operator/etcd to run etcd, and run 3 gravity instances without etcd

[sholdee]

Most of the node joining mechanics are attempts to make it easier for etcd newcomers and hence probably don't have enough dynamic-ness for K8s. It would probably be easier to use something like https://operatorhub.io/operator/etcd to run etcd, and run 3 gravity instances without etcd

You are right; that was the ticket. I have it working now. Thanks!

For anyone else who wants to try and get this going, here are my manifests:

https://pastebin.com/raw/HNzCbP6U

[bw210343]

Thank you for sharing. This config worked for me with microk8s. One thing that may be useful for future readers - the cluster only showed 2 replicas (number of replicas - 1) until the statefulset was restarted.