Skip to content
This repository has been archived by the owner on Jan 20, 2022. It is now read-only.

Recommendation for ETCD node type for a large cluster #349

Open
jsonmp-k8 opened this issue Oct 16, 2020 · 0 comments
Open

Recommendation for ETCD node type for a large cluster #349

jsonmp-k8 opened this issue Oct 16, 2020 · 0 comments

Comments

@jsonmp-k8
Copy link

This is more of a question than an issue.

We were trying to do an perf test on the ETCD using the ETCDCTL command. This is a KOPS cluster that we use and it keeps failing for a large workload performance test. Do you have any recommendations for the node sizes for the master nodes for a large cluster? Master node is where the etcd manager, ETCD and the control plane components reside.

I used the below command to run a performance test with the workload size l (large).

ETCDCTL_API=3 etcdctl --cacert=/etc/kubernetes/pki/kube-apiserver/etcd-ca.crt --cert=/etc/kubernetes/pki/kube-apiserver/etcd-client.crt --key=/etc/kubernetes/pki/kube-apiserver/etcd-client.key --endpoints https://127.0.0.1:4001 check perf --load="l"
 60 / 60 Boooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooooo! 100.00% 1m0s
FAIL: Throughput too low: 7183 writes/s
PASS: Slowest request took 0.074107s
PASS: Stddev is 0.004803s
FAIL

This cluster is created on AWS and the master node size is r5.4xlarge. I do have the following config for the ETCD manager and events

      volumeType: io1
      volumeIops: 3000
      volumeSize: 200

Any recommendations are welcome!

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant