checkpoint installer should source manifests from api-object

[aaronlevy]

Taken from discussion in: #180

I'm not super happy with how we essentially are installing a somewhat "hidden" manifest - which requires re-building an image to change anything about it. If it's updating the binary, that's one thing, but needing a rebuild just to change options in the manifest feels wrong.

One option might be to make use of the podTemplate api. Essentially a cold-storage for a pod from which we can essentially make cookie-cutter copies.

This might look like:

checkpoint-pod-template.yaml

apiVersion: v1
kind: PodTemplate
metadata:
  name: pod-checkpointer
  namespace: kube-system
template:
  metadata:
    name: pod-checkpointer
    namespace: kube-system
    labels:
      k8s-app: pod-checkpointer
  spec:
    containers:
    - name: checkpoint
      image: quay.io/coreos/pod-checkpointer:foo
      command:
      - /checkpoint
      imagePullPolicy: Always
      volumeMounts:
      - mountPath: /etc/kubernetes
        name: etc-kubernetes
      - mountPath: /srv/kubernetes
        name: srv-kubernetes
    hostNetwork: true
    restartPolicy: Always
    volumes:
    - name: etc-kubernetes
      hostPath:
        path: /etc/kubernetes
    - name: srv-kubernetes
      hostPath:
        path: /srv/kubernetes

Then we create that template entry:

kubectl create -f checkpoint-pod-template.yaml

Then our "checkpoint-installer" is changed to (in psuedo steps):

while true:
    watch for change: /api/v1/namespaces/kube-system/podtemlates/pod-checkpointer
    create static pod from pod-checkpointer template

The "checkpoint-installer" might even just become a generic golang "static-pod installer" which can be told what pod-template to use.

Any customizations that a user needs to make can then just be made to an actual api-object rather than relying on re-building an image just to change the manifest.

[lblackstone]

It looks like PodTemplates are not directly supported right now: kubernetes/kubernetes#170 (comment)

I suppose it would still be possible to construct a Pod manifest using the spec from the template in its current state.

[aaronlevy]

Actually might want to close this (or rename). @yifan-gu came up with a more elegant solution which is essentially the pod-checkpointer checkpointing itself (and therefore "installing" itself): #366

[aaronlevy]

An alternative resolution was merged in: #206