Add documentation for helm, application sources, and parameter overrides

CONTRIBUTING.md

@@ -1,10 +1,18 @@
 ## Requirements
-Make sure you have following tools installed [docker](https://docs.docker.com/install/#supported-platforms), [golang](https://golang.org/), [dep](https://github.com/golang/dep), [protobuf](https://developers.google.com/protocol-buffers/), [ksonnet](https://github.com/ksonnet/ksonnet#install), [go-swagger](https://github.com/go-swagger/go-swagger/blob/master/docs/install.md), and [jq](https://stedolan.github.io/jq/)
-[kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/).
+Make sure you have following tools installed
+* [docker](https://docs.docker.com/install/#supported-platforms)
+* [golang](https://golang.org/)
+* [dep](https://github.com/golang/dep)
+* [protobuf](https://developers.google.com/protocol-buffers/)
+* [ksonnet](https://github.com/ksonnet/ksonnet#install)
+* [helm](https://github.com/helm/helm/releases)
+* [go-swagger](https://github.com/go-swagger/go-swagger/blob/master/docs/install.md)
+* [jq](https://stedolan.github.io/jq/)
+* [kubectl](https://kubernetes.io/docs/tasks/tools/install-kubectl/).
 
 ```
 $ brew tap go-swagger/go-swagger
-$ brew install go dep protobuf kubectl ksonnet/tap/ks jq go-swagger
+$ brew install go dep protobuf kubectl ksonnet/tap/ks kubernetes-helm jq go-swagger
 $ go get -u github.com/golang/protobuf/protoc-gen-go
 $ go get -u github.com/go-swagger/go-swagger/cmd/swagger
 $ go get -u github.com/grpc-ecosystem/grpc-gateway/protoc-gen-grpc-gateway

README.md

@@ -3,7 +3,7 @@
 
 ## What is Argo CD?
 
-Argo CD is a declarative, continuous delivery service based on **ksonnet** for Kubernetes.
+Argo CD is a declarative, continuous delivery for Kubernetes.
 
 ![Argo CD UI](docs/argocd-ui.gif)
 
@@ -19,21 +19,25 @@ is provided for additional features.
 
 ## How it works
 
-Argo CD follows the **GitOps** pattern of using git repositories as the source of truth for defining the
-desired application state. Kubernetes manifests are specified as [ksonnet](https://ksonnet.io)
-applications. Argo CD automates the deployment of the desired
-application states in the specified target environments.
-
-![Argo CD Architecture](docs/argocd_architecture.png)
+Argo CD follows the **GitOps** pattern of using git repositories as the source of truth for defining
+the desired application state. Kubernetes manifests can be specified in several ways:
+* [ksonnet](https://ksonnet.io) applications
+* [helm](https://helm.sh) charts
+* Simple directory of YAML/json manifests
 
+Argo CD automates the deployment of the desired application states in the specified target environments.
 Application deployments can track updates to branches, tags, or pinned to a specific version of 
 manifests at a git commit. See [tracking strategies](docs/tracking_strategies.md) for additional
 details about the different tracking strategies available.
 
+## Architecture
+
+![Argo CD Architecture](docs/argocd_architecture.png)
+
 Argo CD is implemented as a kubernetes controller which continuously monitors running applications
 and compares the current, live state against the desired target state (as specified in the git repo).
-A deployed application whose live state deviates from the target state is considered out-of-sync.
-Argo CD reports & visualizes the differences as well as providing facilities to automatically or
+A deployed application whose live state deviates from the target state is considered `OutOfSync`.
+Argo CD reports & visualizes the differences, while providing facilities to automatically or
 manually sync the live state back to the desired target state. Any modifications made to the desired
 target state in the git repo can be automatically applied and reflected in the specified target
 environments.
@@ -51,41 +55,7 @@ For additional details, see [architecture overview](docs/architecture.md).
 * SSO Integration (OIDC, OAuth2, LDAP, SAML 2.0, GitLab, Microsoft, LinkedIn)
 * Webhook Integration (GitHub, BitBucket, GitLab)
 * PreSync, Sync, PostSync hooks to support complex application rollouts (e.g.blue/green & canary upgrades)
-
-## What is ksonnet?
-
-* [Jsonnet](http://jsonnet.org), the basis for ksonnet, is a domain specific configuration language,
-which provides extreme flexibility for composing and manipulating JSON/YAML specifications.
-* [Ksonnet](http://ksonnet.io) goes one step further by applying Jsonnet principles to Kubernetes
-manifests. It provides an opinionated file & directory structure to organize applications into
-reusable components, parameters, and environments. Environments can be hierarchical, which promotes
-both re-use and granular customization of application and environment specifications.
-
-## Why ksonnet?
-
-Application configuration management is a hard problem and grows rapidly in complexity as you deploy
-more applications, against more and more environments. Current templating systems, such as Jinja,
-and Golang templating, are unnatural ways to maintain kubernetes manifests, and are not well suited to
-capture subtle configuration differences between environments. Its ability to compose and re-use
-application and environment configurations is also very limited.
-
-Imagine we have a single guestbook application deployed in following environments:
-
-| Environment   | K8s Version | Application Image      | DB Connection String  | Environment Vars | Sidecars      |
-|---------------|-------------|------------------------|-----------------------|------------------|---------------|
-| minikube      | 1.10.0      | jesse/guestbook:latest | sql://locahost/db     | DEBUG=true       |               |
-| dev           | 1.11.0      | app/guestbook:latest   | sql://dev-test/db     | DEBUG=true       |               |
-| staging       | 1.10.0      | app/guestbook:e3c0263  | sql://staging/db      |                  | istio,dnsmasq |
-| us-west-1     | 1.9.0       | app/guestbook:abc1234  | sql://prod/db         | FOO_FEATURE=true | istio,dnsmasq |
-| us-west-2     | 1.10.0      | app/guestbook:abc1234  | sql://prod/db         |                  | istio,dnsmasq |
-| us-east-1     | 1.9.0       | app/guestbook:abc1234  | sql://prod/db         | BAR_FEATURE=true | istio,dnsmasq |
-
-Ksonnet:
-* Enables composition and re-use of common YAML specifications
-* Allows overrides, additions, and subtractions of YAML sub-components specific to each environment
-* Guarantees proper generation of K8s manifests suitable for the corresponding Kubernetes API version
-* Provides [kubernetes-specific jsonnet libraries](https://github.com/ksonnet/ksonnet-lib) to enable
-concise definition of kubernetes manifests
+* Parameter overrides for overriding ksonnet/helm parameters in git
 
 ## Development Status
 * Argo CD is being used in production to deploy SaaS services at Intuit

docs/README.md

@@ -7,6 +7,8 @@
 * [Tracking Strategies](tracking_strategies.md)
 
 ## Features
+* [Application Sources](application_sources.md)
+* [Application Parameters](parameters.md)
 * [Resource Health](health.md)
 * [Resource Hooks](resource_hooks.md)
 * [Single Sign On](sso.md)

docs/application_sources.md

@@ -0,0 +1,102 @@
+# Application Source Types
+
+ArgoCD supports several different ways in which kubernetes manifests can be defined:
+
+* [ksonnet](https://ksonnet.io) applications
+* [helm](https://helm.sh) charts
+* Simple directory of YAML/json manifests
+
+Some additional considerations should be made when deploying apps of a particular type:
+
+## Ksonnet
+
+### Environments
+Ksonnet has a first class concept of an "environment." To create an application from a ksonnet
+app directory, an environment must be specified. For example, the following command creates the
+"guestbook-default" app, which points to the `default` environment:
+
+```
+argocd app create --name guestbook-default --repo https://github.com/argoproj/argocd-example-apps.git --path guestbook --env default
+```
+
+### Parameters
+Ksonnet parameters all belong to a component. For example, the following are the parameters
+available in the guestbook app, all of which belong to the `guestbook-ui` component:
+
+```
+$ ks param list
+COMPONENT    PARAM         VALUE
+=========    =====         =====
+guestbook-ui containerPort 80
+guestbook-ui image         "gcr.io/heptio-images/ks-guestbook-demo:0.1"
+guestbook-ui name          "guestbook-ui"
+guestbook-ui replicas      1
+guestbook-ui servicePort   80
+guestbook-ui type          "LoadBalancer"
+```
+
+When overriding ksonnet parameters in ArgoCD, the component name should also be specified in the
+`argocd app set` command, in the form of `-p COMPONENT=PARAM=VALUE`. For example:
+```
+argocd app set guestbook-default -p guestbook-ui=image=gcr.io/heptio-images/ks-guestbook-demo:0.1
+```
+
+## Helm
+
+### Values Files
+
+Helm has the ability to use a different, or even multiple "values.yaml" files to derive its
+parameters from. Alternate or multiple values file(s), can be specified using the `--values`
+flag. The flag can be repeated to support multiple values files:
+
+```
+argocd app set helm-guestbook --values values-production.yaml
+```
+
+### Helm Parameters
+
+Helm has the ability to set parameter values, which override any values in
+a `values.yaml`. For example, `service.type` is a common parameter which is exposed in a Helm chart:
+```
+helm template . --set service.type=LoadBalancer
+```
+Similarly ArgoCD can override values in the `values.yaml` parameters using `argo app set` command,
+in the form of `-p PARAM=VALUE`. For example:
+```
+argocd app set helm-guestbook -p service.type=LoadBalancer
+```
+
+### Helm Hooks
+
+Helm hooks are equivalent in concept to [ArgoCD resource hooks](resource_hooks.md). In helm, a hook
+is any normal kubernetes resource annotated with the `helm.sh/hook` annotation. When ArgoCD deploys
+helm application which contains helm hooks, all helm hook resources are currently ignored during
+the `kubectl apply` of the manifests. There is an 
+[open issue](https://github.com/argoproj/argo-cd/issues/355) to map Helm hooks to ArgoCD's concept
+of Pre/Post/Sync hooks.
+
+### Random Data
+
+Helm templating has the ability to generate random data during chart rendering via the 
+`randAlphaNum` function. Many helm charts from the [charts repository](https://github.com/helm/charts)
+make use of this feature. For example, the following is the secret for the
+[redis helm chart](https://github.com/helm/charts/blob/master/stable/redis/templates/secrets.yaml):
+
+```
+data:
+  {{- if .Values.password }}
+  redis-password: {{ .Values.password | b64enc | quote }}
+  {{- else }}
+  redis-password: {{ randAlphaNum 10 | b64enc | quote }}
+  {{- end }}
+```
+
+The ArgoCD application controller periodically compares git state against the live state, running
+the `helm template <CHART>` command to generate the helm manifests. Because the random value is
+regenerated every time the comparison is made, any application which makes use of the `randAlphaNum`
+function will always be in an `OutOfSync` state. This can be mitigated by explicitly setting a
+value, in the values.yaml such that the value is stable between each comparison. For example:
+
+```
+argocd app set redis -p password=abc123
+```

docs/architecture.md

@@ -22,15 +22,31 @@ manifests when provided the following inputs:
 * repository URL
 * git revision (commit, tag, branch)
 * application path
-* application environment
+* template specific settings: parameters, ksonnet environments, helm values.yaml
 
 ### Application Controller
 The application controller is a Kubernetes controller which continuously monitors running
 applications and compares the current, live state against the desired target state (as specified in
-the git repo). It detects out-of-sync application state and optionally takes corrective action. It
-is responsible for invoking any user-defined handlers (argo workflows) for Sync, OutOfSync events
+the git repo). It detects `OutOfSync` application state and optionally takes corrective action. It
+is responsible for invoking any user-defined hooks for lifcecycle events (PreSync, Sync, PostSync)
 
 ### Application CRD (Custom Resource Definition)
 The Application CRD is the Kubernetes resource object representing a deployed application instance
-in an environment. It holds a reference to the desired target state (repo, revision, app, environment)
-of which the application controller will enforce state against.
+in an environment. It is defined by two key pieces of information:
+* `source` reference to the desired state in git (repository, revision, path, environment)
+* `destination` reference to the target cluster and namespace.
+
+An example spec is as follows:
+
+```
+spec:
+  project: default
+  source:
+    repoURL: https://github.com/argoproj/argocd-example-apps.git
+    targetRevision: HEAD
+    path: guestbook
+    environment: default
+  destination:
+    server: https://kubernetes.default.svc
+    namespace: default
+```

docs/getting_started.md

@@ -1,6 +1,6 @@
 # ArgoCD Getting Started
 
-An example Ksonnet guestbook application is provided to demonstrates how ArgoCD works.
+An example guestbook application is provided to demonstrate how ArgoCD works.
 
 ## Requirements
 * Installed [minikube](https://github.com/kubernetes/minikube#installation)
@@ -157,17 +157,13 @@ Service     guestbook-ui  service "guestbook-ui" created
 Deployment  guestbook-ui  deployment.apps "guestbook-ui" created
 ```
 
-This command retrieves the manifests from the ksonnet app in the git repository and performs a 
-`kubectl apply` of the manifests. The guestbook app is now running and you can now view its resource
+This command retrieves the manifests from git repository and performs a `kubectl apply` of the 
+manifests. The guestbook app is now running and you can now view its resource
 components, logs, events, and assessed health:
 
 ![view app](assets/guestbook-tree.png)
 
-
 ## 8. Next Steps
 
-ArgoCD supports additional features such as SSO, WebHooks, RBAC. See the following guides on setting
-these up:
-* [Configuring SSO](sso.md)
-* [Configuring RBAC](rbac.md)
-* [Configuring WebHooks](webhook.md)
+ArgoCD supports additional features such as SSO, WebHooks, RBAC, Projects. See the rest of 
+the [documentation](./) for details.

docs/parameters.md

@@ -0,0 +1,39 @@
+# Parameter Overrides
+
+ArgoCD provides a mechanism to override the parameters of a ksonnet/helm app. This gives some extra
+flexibility in having most of the application manifests defined in git, while leaving room for
+*some* parts of the k8s manifests determined dynamically, or outside of git. It also serves as an
+alternative way of redeploying an application by changing application parameters via ArgoCD, instead
+of making the changes to the manifests in git.
+
+**NOTE:** many consider this mode of operation as an anti-pattern to GitOps, since the source of
+truth becomes a union of the git repository, and the application overrides. The ArgoCD parameter
+overrides feature is provided mainly convenience to developers and is intended to be used more for
+dev/test environments, vs. production environments.
+
+To use parameter overrides, run the `argocd app set -p (COMPONENT=)PARAM=VALUE` command:
+```
+argocd app set guestbook -p guestbook=image=example/guestbook:abcd123
+argocd app sync guestbook
+```
+
+The following are situations where parameter overrides would be useful: 
+
+1. A team maintains a "dev" environment, which needs to be continually updated with the latest
+version of their guestbook application after every build in the tip of master. To address this use
+case, the application would expose an parameter named `image`, whose value used in the `dev`
+environment contains a placeholder value (e.g. `example/guestbook:replaceme`). The placeholder value
+would be determined externally (outside of git) such as a build systems. Then, as part of the build
+pipeline, the parameter value of the `image` would be continually updated to the freshly built image 
+(e.g. `argocd app set guestbook -p guestbook=image=example/guestbook:abcd123`). A sync operation
+would result in the application being redeployed with the new image.
+
+2. A repository of helm manifests is already publicly available (e.g. https://github.com/helm/charts).
+Since commit access to the repository is unavailable, it is useful to be able to install charts from
+the public repository, customizing the deployment with different parameters, without resorting to
+forking the repository to make the changes. For example, to install redis from the helm chart
+repository and customize the the database password, you would run:
+
+```
+argocd app create --name redis --repo https://github.com/helm/charts.git --path stable/redis --dest-server https://kubernetes.default.svc --dest-namespace default -p password=abc123
+```

docs/rbac.md

@@ -44,15 +44,19 @@ Kubernetes clusters which can be used by applications belonging to the project.
 
 ### 1. Create new project
 
-Following command creates project `myproject` which can deploy applications to namespace `default` of cluster `https://kubernetes.default.svc`. The source ksonnet application
-should be defined in `https://github.com/argoproj/argocd-example-apps.git` repository.
+Following command creates project `myproject` which can deploy applications to namespace `default` of cluster `https://kubernetes.default.svc`. The valid application source is defined in the `https://github.com/argoproj/argocd-example-apps.git` repository.
 
 ```
 argocd proj create myproject -d https://kubernetes.default.svc,default -s https://github.com/argoproj/argocd-example-apps.git
 ```
 
-Project sources and destinations can be managed using commands `argocd project add-destination`, `argocd project remove-destination`, `argocd project add-source`
-and `argocd project remove-source`.
+Project sources and destinations can be managed using commands
+```
+argocd project add-destination
+argocd project remove-destination
+argocd project add-source
+argocd project remove-source
+```
 
 ### 2. Assign application to a project