[WIP] Vitess Operator example

examples/vitess/README.md

@@ -0,0 +1,126 @@
+## Vitess Operator
+
+This is an example of an app-specific [Operator](https://coreos.com/operators/), 
+in this case for [Vitess](http://vitess.io), built with Metacontroller.
+
+It's meant to demonstrate the following patterns:
+
+* Building an Operator for a complex, stateful application out of a set of small
+  Lambda Controllers that each do one thing well.
+  * In addition to presenting a k8s-style API to users, this Operator uses
+    custom k8s API objects to coordinate within itself.
+  * Each controller manages one layer of the hierarchical Vitess cluster topology.
+    The user only needs to create and manage a single, top-level VitessCluster
+    object.
+* Replacing static, client-side template rendering with Lambda Controllers,
+  which can adjust based on dynamic cluster state.
+  * Each controller aggregates status and orchestrates app-specific rolling
+    updates for its immediate children.
+  * The top-level object contains a continuously-updated, aggregate "Ready"
+    condition for the whole app, and can be directly edited to trigger rolling
+    updates throughout the app.
+* Using a functional-style language ([Jsonnet](http://jsonnet.org)) to
+  define Lambda Controllers in terms of template-like transformations on JSON
+  objects.
+  * You can use any language to write a Lambda Controller webhook, but the
+    functional style is a good fit for a process that conceptually consists of
+    declarative input, declarative output, and no side effects.
+  * As a JSON templating language, Jsonnet is a particularly good fit for
+    generating k8s manifests, providing functionality missing from pure
+    JavaScript, such as first-class *merge* and *deep equal* operations.
+* Using the "Apply" update strategy feature of CompositeController, which
+  emulates the behavior of `kubectl apply`, except that it attempts to do
+  pseudo-strategic merges for CRDs.
+
+### Vitess Components
+
+A typical VitessCluster might expand to the following tree once it's fully
+deployed.
+Objects in **bold** are custom resource kinds defined by this Operator.
+
+* **VitessCluster**: The top-level specification for a Vitess cluster.
+  This is the only one the user creates.
+  * **VitessCell**: Each Vitess [cell](http://vitess.io/overview/concepts/#cell-data-center)
+    represents an independent failure domain (e.g. a Zone or Availability Zone).
+    * EtcdCluster ([etcd-operator](https://github.com/coreos/etcd-operator)):
+      Vitess needs its own etcd cluster to coordinate its built-in load-balancing
+      and automatic shard routing.
+    * Deployment ([orchestrator](https://github.com/github/orchestrator)):
+      An optional automated failover tool that works with Vitess.
+    * Deployment ([vtctld](http://vitess.io/overview/#vtctld)):
+      A pool of stateless Vitess admin servers, which serve a dashboard UI as well
+      as being an endpoint for the Vitess CLI tool (vtctlclient).
+    * Deployment ([vtgate](http://vitess.io/overview/#vtgate)):
+      A pool of stateless Vitess query routers.
+      The client application can use any one of these vtgate Pods as the entry
+      point into Vitess, through a MySQL-compatible interface.
+    * **VitessKeyspace** (db1): Each Vitess [keyspace](http://vitess.io/overview/concepts/#keyspace)
+      is a logical database that may be composed of many MySQL databases (shards).
+      * **VitessShard** (db1/0): Each Vitess [shard](http://vitess.io/overview/concepts/#shard)
+      is a single-master tree of replicating MySQL instances.
+        * Pod(s) ([vttablet](http://vitess.io/overview/#vttablet)): Within a shard, there may be many Vitess [tablets](http://vitess.io/overview/concepts/#tablet)
+          (individual MySQL instances).
+          VitessShard acts like an app-specific [replacement for StatefulSet](https://github.com/GoogleCloudPlatform/kube-metacontroller/tree/master/examples/catset),
+          creating both Pods and PersistentVolumeClaims.
+        * PersistentVolumeClaim(s)
+      * **VitessShard** (db1/1)
+        * Pod(s) (vttablet)
+        * PersistentVolumeClaim(s)
+    * **VitessKeyspace** (db2)
+      * **VitessShard** (db2/0)
+        * Pod(s) (vttablet)
+        * PersistentVolumeClaim(s)
+
+### Prerequisites
+
+* Kubernetes 1.8+ is required for its improved CRD support, especially garbage
+  collection.
+  * This config currently requires a dynamic PersistentVolume provisioner and a
+    default StorageClass.
+  * The example `my-vitess.yaml` config results in a lot of Pods.
+    If the Pods don't schedule due to resource limits, you can try lowering the
+    limits, lowering `replicas` values, or removing the `batch` config under
+    `tablets`.
+* Install [kube-metacontroller](https://github.com/GoogleCloudPlatform/kube-metacontroller).
+* Install [etcd-operator](https://github.com/coreos/etcd-operator) in the
+  namespace where you plan to create a VitessCluster.
+
+### Deploy the Operator
+
+```sh
+kubectl create configmap vitess-operator-hooks -n metacontroller --from-file=hooks
+kubectl apply -f vitess-operator.yaml
+```
+
+### Create a VitessCluster
+
+```sh
+kubectl apply -f my-vitess.yaml
+```
+
+### View the Vitess Dashboard
+
+Wait until the cluster is ready:
+
+```sh
+kubectl get vitessclusters -o 'custom-columns=NAME:.metadata.name,READY:.status.conditions[?(@.type=="Ready")].status'
+```
+
+You should see:
+
+```console
+NAME      READY
+vitess    True
+```
+
+Start a kubectl proxy:
+
+```sh
+kubectl proxy --port=8001
+```
+
+Then visit:
+
+```
+http://localhost:8001/api/v1/namespaces/default/services/vitess-global-vtctld:web/proxy/app/
+```

examples/vitess/hooks/etcd.libsonnet

@@ -0,0 +1,26 @@
+local k8s = import "k8s.libsonnet";
+local metacontroller = import "metacontroller.libsonnet";
+
+{
+  local etcd = self,
+
+  apiVersion: "etcd.database.coreos.com/v1beta2",
+
+  // EtcdClusters
+  clusters(observed, specs)::
+    metacontroller.collection(observed, specs, etcd.apiVersion, "EtcdCluster", etcd.cluster),
+
+  // Create/update an EtcdCluster child for a VitessCell parent.
+  cluster(observed, spec):: {
+    apiVersion: etcd.apiVersion,
+    kind: "EtcdCluster",
+    metadata: {
+      name: observed.parent.metadata.name + "-etcd",
+      labels: observed.parent.spec.template.metadata.labels,
+    },
+    spec: {
+      version: spec.version,
+      size: spec.size,
+    }
+  },
+}

examples/vitess/hooks/k8s.libsonnet

@@ -0,0 +1,68 @@
+// Library for working with Kubernetes objects.
+{
+  local k8s = self,
+
+  // Fill in a conventional status condition object.
+  condition(type, status):: {
+    type: type,
+    status:
+      if std.type(status) == "string" then (
+        status
+      ) else if std.type(status) == "boolean" then (
+        if status then "True" else "False"
+      ) else (
+        "Unknown"
+      ),
+  },
+
+  // Extract the status of a given condition type.
+  // Returns null if the condition doesn't exist.
+  conditionStatus(obj, type)::
+    if obj != null && "status" in obj && "conditions" in obj.status then
+      // Filter conditions with matching "type" field.
+      local matches = [
+        cond.status for cond in obj.status.conditions if cond.type == type
+      ];
+      // Take the first one, if any.
+      if std.length(matches) > 0 then matches[0] else ""
+    else
+      null,
+
+  // Returns only the objects from a given list that have the
+  // "Ready" condition set to "True".
+  filterReady(list)::
+    std.filter(function(x) self.conditionStatus(x, "Ready") == "True", list),
+
+  // Returns only the objects from a given list that have the
+  // "Available" condition set to "True".
+  filterAvailable(list)::
+    std.filter(function(x) self.conditionStatus(x, "Available") == "True", list),
+
+  // Returns whether the object matches the given label values.
+  matchLabels(obj, labels)::
+    local keys = std.objectFields(labels);
+
+    "metadata" in obj && "labels" in obj.metadata &&
+      [
+        obj.metadata.labels[k]
+          for k in keys if k in obj.metadata.labels
+      ]
+      ==
+      [labels[k] for k in keys],
+
+  // Get the value of a label from object metadata.
+  // Returns null if the label doesn't exist.
+  getLabel(obj, key)::
+    if "metadata" in obj && "labels" in obj.metadata && key in obj.metadata.labels then
+      obj.metadata.labels[key]
+    else
+      null,
+
+  // Get the value of an annotation from object metadata.
+  // Returns null if the annotation doesn't exist.
+  getAnnotation(obj, key)::
+    if "metadata" in obj && "annotations" in obj.metadata && key in obj.metadata.annotations then
+      obj.metadata.annotations[key]
+    else
+      null,
+}

examples/vitess/hooks/metacontroller.libsonnet

@@ -0,0 +1,98 @@
+local k8s = import "k8s.libsonnet";
+
+// Library for working with kube-metacontroller.
+{
+  local metacontroller = self,
+
+  // Extend a metacontroller request object with extra fields and functions.
+  observed(request):: request + {
+    children+: {
+      // Get a map of children of a given kind, by child name.
+      getMap(apiVersion, kind)::
+        self[kind + "." + apiVersion],
+
+      // Get a list of children of a given kind.
+      getList(apiVersion, kind)::
+        local map = self.getMap(apiVersion, kind);
+        [map[key] for key in std.objectFields(map)],
+
+      // Get a child object of a given kind and name.
+      get(apiVersion, kind, name)::
+        local map = self.getMap(apiVersion, kind);
+        if name in map then map[name] else null,
+    },
+  },
+
+  // Helpers for managing spec, observed, and desired states
+  // for a collection of objects of a given Kind.
+  collection(observed, specs, apiVersion, kind, desired):: {
+    specs: if specs != null then specs else [],
+
+    observed: observed.children.getList(apiVersion, kind),
+    desired: [
+      {apiVersion: apiVersion, kind: kind} + desired(observed, spec)
+        for spec in self.specs
+    ],
+
+    getObserved(name): observed.children.get(apiVersion, kind, name),
+    updateStrategy:
+      if "updateStrategy" in observed.controller.spec then
+        observed.controller.spec.updateStrategy
+      else
+        null,
+  },
+
+  // Mix-in for collection that filters observed objects.
+  // This may be needed if a given parent has multiple collections of children
+  // of the same Kind.
+  collectionFilter(filter):: {
+    observed: std.filter(filter, super.observed),
+  },
+
+  // Mix-in for collection that causes the child objects to be treated as
+  // "immutable". That is, they will be created if they don't exist, but left
+  // untouched if they do.
+  //
+  // Note that in the case of the "Apply" strategy, this means we return the
+  // last applied config. Metacontroller might still attempt an update in
+  // response, if a third party has mutated fields to which we've previously
+  // applied values. In other words, it continues to actively maintain the last
+  // applied config; it doesn't become totally passive.
+  collectionImmutable:: {
+    desired: [
+      local observed = super.getObserved(desired.metadata.name);
+      if observed == null then
+        desired
+      else (
+        if super.updateStrategy == "Apply" then
+          metacontroller.getLastApplied(observed)
+        else
+          observed
+      )
+      for desired in super.desired
+    ],
+  },
+
+  // Unmarshal the Metacontroller last applied config annotation from an object.
+  // If the annotation doesn't exist, it returns an "empty" last applied config.
+  // Returning an empty config when using the "Apply" strategy will tell
+  // Metacontroller, "I want this to exist, but I don't care what's in it."
+  getLastApplied(obj)::
+    local lastApplied = k8s.getAnnotation(obj, "metacontroller.k8s.io/last-applied-configuration");
+    if lastApplied != null then
+      metacontroller.jsonUnmarshal(lastApplied)
+    else {
+      apiVersion: obj.apiVersion,
+      kind: obj.kind,
+      metadata: {name: obj.metadata.name},
+    },
+
+  // Unmarshal JSON into a Jsonnet value.
+  // This function is defined as a native extension in jsonnetd.
+  jsonUnmarshal(jsonString):: std.native("jsonUnmarshal")(jsonString),
+
+  // Convert an integer string in the given base to "int" (actually double).
+  // Should be precise up to 2^53.
+  // This function is defined as a native extension in jsonnetd.
+  parseInt(intStr, base):: std.native("parseInt")(intStr, base),
+}