Updated documentation, cleaned up old files, reworked secrets. #2
Conversation
Bunch of changes: - Made documentation as painless as possible - Set default network to 10.244.0.0/16, this is what kube-up does by default, along with other deployers in cluster/ - Removed templates for kubernetes output - Generating configs to files, and secrets from the files. - Removed hostpaths. This makes things brittle but portable, warning is in documentation. This is nowhere near ready yet, but lets people poke at it easier. Needs more automation around the IP range variables.
| kubectl create secret generic ceph-client-key --from-file=ceph-client-key --namespace=ceph | ||
|
|
||
| cd .. | ||
| ``` |
There was a problem hiding this comment.
Can I be greedy and ask for this to be in one generate_ceph_secrets.sh?
There was a problem hiding this comment.
Yeah, I'll leave this for the tutorial but add in a script that just does everything.
|
Great job @elsonrodriguez! I appreciated the resolv.conf changes being documented too! 👍 |
|
|
||
| kubectl create namespace ceph | ||
|
|
||
| kubectl create secret generic ceph-conf-combined --from-file=ceph.conf --from-file=ceph.client.admin.keyring --from-file=ceph.mon.keyring --namespace=ceph |
There was a problem hiding this comment.
Minor change but should ceph.conf be placed in a ConfigMap instead of a secret?
There was a problem hiding this comment.
The /etc/ceph directory is mounted from this secret and contains keys. ceph.conf is the only insecure thing in there.
Would need a refactor, not worth it just yet.
|
Also I opted to delete the DaemonSet for the MON instead of the RC, but you Ceph experts should chime in on whether or not we need MON for every OSD. |
| If your pod has issues mounting, make sure mount.ceph is installed on all nodes. | ||
|
|
||
| ``` | ||
| apt-get install ceph-fs-common |
There was a problem hiding this comment.
To avoid having to install packages (which won't work for things like CoreOS), I wonder if it would make sense (and actually work) to modify the ceph-tools example to copy mount.ceph to the node - https://github.com/ceph/ceph-docker/tree/master/examples/coreos/tools
There was a problem hiding this comment.
Looks like CoreOS can mount cephfs volumes without additional configuration.
Tested on CoreOS 1032.1.0
|
Also need to get RBD working. EDIT: Nope, there's an upstream bug regarding Secrets and RBD: kubernetes/kubernetes#25490 Only other way is to put a keyring.conf on each node, much nope. |
elsonrodriguez
force-pushed
the
kubernetes-mds
branch
from
c16d59e
to
9eb6a34
Compare
This lets `kubectl rollout` be used for managing everything.
|
I went ahead and tested the pending RBD PR and it works, so I added an RBD example. @hunter @cornelius-keller What do you think? |
|
Also doing some work upstream to make the salt-based kube-up providers more ceph-friendly: |
|
Hi @elsonrodriguez sorry I was on vacation with no mobile broadband access. Right now I am busy catching up, but I will look deeper into it as soon as possible. |
|
@cornelius-keller No worries. Let me know what else you think we need in order to merge this all upstream. |
Bunch of changes:
This is nowhere near ready yet, but lets people poke at it easier.
Needs more automation around the IP range variables.