chore: first spike of submitting 2 enhancements

README.md

@@ -1,4 +1,4 @@
-# enhancements
+# Enhancements
 
 This repository contains design proposals for [Jenkins X](https://jenkins-x.io/) enhancements.
 
@@ -7,3 +7,8 @@ All proposals are welcome, please follow this process:
 - Raise an issue on this repo describing at high level what the enhancement looks to achieve and whether you would like any help with the proposal.
 - If you would like some early collaboration from the Jenkins X community you might want to start with Google Docs for a faster feedback cycle.
 - Open a pull request containing markdown giving some context, describing the problem the enhancement aims to solve along with possible solutions.
+
+## Proposals
+
+* [1: Apps and Addons](proposals/1/README.md) - simplifying the language and concepts in Jenkins X
+* [1: Boot Apps](proposals/2/README.md) - provide a generic mechanism to install/upgrade any chart in boot

proposals/1/README.md

@@ -0,0 +1,47 @@
+# 1: Consolidate Apps and Addons
+
+This proposal tries to conslidate Apps and Addons inside Jenkins X to avoid confusion
+
+## Background
+
+Two years ago we added the addon framework to Jenkins X as a way of extending the Jenkins X platform to provide additional capabilities. 
+
+### We created Addons
+
+`Addons` were either generic helm charts or a combination of helm chart plus some custom go code to install and integrate them with the Jenkins X platform.
+
+At this point we referred to `apps` as the charts developers created. e.g. the output of `jx create quickstart` was an app and `jx get apps` would list the quickstarts folks had added to `Staging` or `Production`.
+
+### We created Apps 
+
+We wanted to avoid having to change the `jx` binary to add a new extension to the Jenkins X platform; so we then introduced the `Apps` framework which provided a more generic way of adding charts chatting to Jenkins X such that anybody could create an app without having to modify the underlying `jx` code.
+
+
+### Confusion
+
+That's now left us with confusing over what an App is.
+
+This is made worse by having support for `apps` and `addons` in the current CLI.
+
+Developers like to talk about the `apps` they are developing. 
+
+
+## Proposal
+
+Going forward lets refer to everything as an `App` and deprecate the use of the word `Addon`. 
+
+Then all of these things would be an `App`:
+
+* system charts like knative, gloo, nginx-controller, flagger, prometheus, external-dns, cert-manager
+* instantiations of Quickstarts or repositories that folks import
+
+## Status: Proposed
+
+## Implementation
+
+We would be able to reuse the current `jx add app` and `jx delete app` commands and folks could reuse them to add any app to any environment `dev`, `Staging` or `Production`.
+
+One possible confusion is between `jx promote` and `jx add app`. In many ways `promote` is for promoting a different version to a different environment; `jx add app` adds an app to the current environment (usually `Dev` by default).
+
+
+

proposals/2/README.md

@@ -0,0 +1,223 @@
+# 2: Boot Apps
+
+Issue: https://github.com/jenkins-x/jx/issues/6442
+Old design doc:: https://docs.google.com/document/d/1DL1oEeAZtWFfpJOWk5UVO54D88BnlqZDf5IDaOkGxCk/edit?usp=sharing
+
+### Current Implementation
+
+See the [Implementation Documentation](docs/README.md) for an overview of how to use helmfile and helm 3
+
+## Requirements
+
+We want a simple canonical way to install any helm chart in any namespace without having to manually hack the `jx boot` pipeline file: `jenkins-x.yml`.
+
+This will make it easy for people or software to easily add/remove apps to any git repository for any Environment (dev / staging / production).
+
+The boot config repo has become quite complex and hard to manage upgrades to and share common config between different flavours of base boot config repos.  It’s also hard to experiment by adding new helm charts / features / applications that require changes to the boot install pipeline.
+
+We want a simpler more modular system that lets us, say, swap out nginx-controller and use gloo/knative/istio for ingress without a deep knowledge of the jenkins-x.yml pipeline for boot.
+
+What we’d really like is for folks to type commands like…
+
+```
+jx delete app nginx-controller
+jx add app istio
+jx delete app knative
+```
+
+And for it to just work and generically add those to the right namespace.
+
+## Background
+
+The current apps model lets you add/remove charts fine - but there are a number of limitations:
+
+### Only works in 1 namespace
+
+Currently apps can only work inside the dev namespace - we can’t support things like knative, gloo, cert-manager, nginx, velero which tend to be installed in separate namespaces.
+
+### Does not handle boot phases
+
+This is not surprising as we created the App extension model before boot. But right now boot has a number of distinct phases with pipeline steps between them where we may want to add apps:
+
+* pre/post ingress setup (in the case of nip.io / nginx-controller)
+* pre/post vault setup
+* pre/post vault population of parameters
+* pre/post setup of certmanager/external dns
+
+There is currently no way for an apps to specify a namespace + specify which phase its to be installed in & have (say) the domain / certs / secrets injected easily.
+
+So rather than adding steps at the right point in a list of steps in the jenkins-x.yml it would be nice to have a more declarative YAML file to describe which apps are to be enabled.
+
+E.g. we need a new YAML file with list of charts to be installed along with metadata for which phase to install them. We also want to invoke a selection of those appss to be installed at the different phases of the boot pipeline.
+
+### Limitations of the current deploy model
+
+We have some issues with the current way we deploy apps in Staging / Production:
+
+#### Composite charts hide the individual versions
+
+Right now we deploy all charts in Staging/Production as a single chart. That means if you use `helm list` we don't show any actual versions of the dependent charts - its just 1 chart (with no version).
+
+#### Removing Apps on Dev doesn't work
+
+If you remove a chart from `env/requirements.yaml`  in the Dev repository the pipeline does not remove the chart. e.g. if you add `lighthouse` and remove `prow` you need to manually delete the prow resources yourself.
+
+
+### Limitation of boot
+
+We currently manually configure in the boot pipeline the install of multiple systems usually in separate namespaces:
+
+* velero (if enabled)
+* nginx
+* externaldns
+* cert-manager
+* acme certs chart
+
+Ideally those should be modular and optional. e.g it should be really trivial to disable nginx if folks are using, say, istio - without having to hack a pipeline yaml.
+
+We'd also like to make it easy to add a number of other systems in order in separate namespaces independently of the `Staging / Production` environment namespaces:
+
+* knative https://github.com/jenkins-x/jx/issues/6331
+* gloo
+* istio
+* linkerd https://github.com/jenkins-x/jx/issues/6330
+* flagger
+
+so it'd be nice to have a simple app model where anyone can add any systems/charts to any namespace at any point in the flow before we setup the dev/staging/production environment in the traditional way.
+
+
+## Proposal
+
+We propose we combine the extensibility of Jenkins X using Apps with `jx boot` via a simple declarative YAML file declaring the charts to be installed. 
+
+e.g. a file `apps.yaml`  something like this:
+
+```yaml    
+defaultNamespace: jx
+
+releases: 
+- alias: velero
+  name: velero
+  repository: https://kubernetes-charts.storage.googleapis.com
+  phase: system  
+  namespace: velero   
+- alias: nginx-ingress
+  name: nginx-ingress
+  repository: https://kubernetes-charts.storage.googleapis.com
+  phase: system  
+  namespace: kube-system   
+- name: external-dns
+  repository: https://charts.bitnami.com/bitnami
+  phase: post-ingress  
+- alias: cert-manager
+  name: cert-manager
+  repository: https://charts.jetstack.io
+  namespace: cert-manager  
+  phase: post-ingress  
+- name: jxboot-resources
+  repository: http://chartmuseum.jenkins-x.io
+- alias: tekton
+  name: tekton
+  repository: http://chartmuseum.jenkins-x.io
+- alias: prow
+  condition: prow.enabled
+  name: prow
+  repository: http://chartmuseum.jenkins-x.io
+- alias: lighthouse
+  condition: lighthouse.enabled
+  name: lighthouse
+  repository: http://chartmuseum.jenkins-x.io
+- alias: bucketrepo
+  condition: bucketrepo.enabled
+  name: bucketrepo
+  repository: http://chartmuseum.jenkins-x.io
+- name: jenkins-x-platform
+  repository: http://chartmuseum.jenkins-x.io
+```
+
+Using a yaml file in a boot config repo we list the apps that should be installed.  This means we can instead have a bare minimum base boot config repo and list the apps we want installed and different points of the installation process.
+
+
+The above YAML looks quite like the `env/requirements.yaml` file only: 
+
+* it allows the `namespace` to be specified if its different from the `defaultNamespace`
+* we add the `phase` label so that we can filter the apps by phase so we can install different charts at different points in the boot pipeline.
+
+
+### Strawman Solutions
+
+The above `apps.yaml` proposal looks very much like helmfile. 
+
+So lets look into helmfile and try using it with a stripped back boot config repo to see how things might fit together 
+
+One possible solution might be to add a single step the boot pipeline to invoke helmfile: https://github.com/roboll/helmfile#configuration
+
+e.g. given the `apps.yaml` file in the file system then run helmfile as a step on the file.
+
+It looks like helmfile already supports tillerless + helm3 and lets us define namespaces and orders etc.
+
+We may need to build our own step to wrap helmfile so that we can do some of the things we do with `jx step helm apply` right now (e.g. exposing the `jx-requirements.yml` file as a `values.yaml` its values can be used inside any `values.yaml` we use in the charts).
+
+
+* should we support access to secrets in vault / local file system via URLs or Parameters injection?
+* do we need to support exposing the properties from the `jx-requirements.yml` as values.yaml files that can be reused inside the helmfile templates?
+* do we need to improve access to certs / domain after we've set those up in boot?
+* allow the use of version streams to manage versions of things if no version is specified in the `helmfile.yaml` - like we do with helm charts requirements.yaml file right now (in `jx step helm apply`)
+
+So if we do need anything other than vanilla helmfile we could maybe do a similar thing to `jx step helm apply` where we copy the directory where helmfile lives, then generate any secret yamls + `jx-requirements.yaml` files and then run helmfile in a temporary directory (to avoid accidentally committing any secrets to git). Hopefully we don't need to fork helmfile; but we could just wrap it slightly with a pre-processor?
+
+
+We want to allow the `jx boot` pipeline to invoke a helmfile like thing at the different phases.
+
+e.g. currently in boot we do something like this:
+
+* install a bunch of charts in different namespaces with a defined order
+* then we modify the requirements based on ingress: `jx step create install values` (to handle things like detecting the domain/ingress)
+* install more charts (e.g. external dns/cert manager/vault)
+* populate parameters/secrets (`jx step create values --name parameters`)
+* do the traditional install and maybe other charts
+
+so we already have 3 natural places to invoke a helmfile-like thing to install charts in different phases. So maybe thats 3 places where we invoke helmfile with a selector as @vbehar mentions if thats something helmfile can do? or we use 3 separate helmfiles maybe?
+
+e.g. the pipeline with boot apps could be something like this...
+
+```
+jx step helmfile --selector phase=pre-ingress
+
+# populate ingress stuff (e.g. default domain from nip.io if not using custom domain)
+jx step create install values
+
+# add any charts that need a domain injected
+jx step helmfile --selector phase=post-ingress
+
+
+# populate any missing charts in vault
+jx step create values --name parameters
+
+# lets include generated secrets after they have been populated in vault
+jx step helmfile --selector "!phase" --secrets
+```
+
+Then we could have a list of apps which we group as to where they get added based on what they are & whether they need integration with secrets / TLS / domain / certs etc?
+
+
+We can then modify the existing `jx add app / jx delete app` to detect the `apps.yaml` file and create the necessary code change in a Pull Request.
+
+Maybe over time we move more towards this kind of helmfile-like approach for all environments too (`Dev` / `Staging` and `Production`)?
+
+
+### Proposed Schedule
+
+We can easily try a prototype a new boot config repo where we add helmfile and add some helmfile steps and see how it works and what we think of the general git source code & if it helps us and users have modular boot apps without committing to any significant changes in the jx repos.
+
+
+
+* try out helmfile for installing things like gloo / knative / linkerd / istio?
+* if that works and we are happy with it, we could look at adding the phases approach for easier app composition
+* if thats all looking good try migrate existing things like nginx / cert manager / external dns to the helmfile approach in a new git repository
+* anyone can then try out the helmfile approach without changing the core upstream boot configuration [jenkins-x-boot-config](https://github.com/jenkins-x/jenkins-x-boot-config/tree/master/env)
+* we could try create a new remote cluster boot configuration for multi-cluster which installs a small subset of Jenkins X (e.g. lighthouse + tekton + a single `SourceRepository + Schedule` for replicating the logic of the `environment controller`)
+* create a demo repository which replaces nginx-controller with knative + gloo we can use for Progressive Delivery demos
+
+Maybe we find when we look at migrating the current charts to boot apps (nginx / cert manager / external-dns in particular) we may find its got some limitations and building a simple similar tool might be easier. Or we may find helmfile gets us where we need to go faster.
+

proposals/2/docs/README.md

@@ -0,0 +1,33 @@
+## Helmfile and Helm 3 support
+
+This document outlines the new `jx boot` implementation using [helmfile](https://github.com/roboll/helmfile) and helm 3. 
+
+
+## Benefits of helmfile and helm 3
+
+* We can use vanilla helm 3 now to install, update or delete charts in any namespace without needing tiller or custom code to manage `helm template`
+  * We can avoid all the complexities of the `jx step helm apply` logic using our own helm template generation + post processing logic. We can also move away from boot's use of `{{ .Requirements.foo }}` and `{{ .Parameters.bar }}` expressions
+* It opens the door to a flexible multi-cluster support so that every cluster/environment can be managed in the same canonical GitOps approach (as each cluster can use `jx boot` whether its a dev environment or remote staging/production environment)
+* We can use the `helm list` command line to view versions of each chart/app nicely in the CLI.
+  * we can avoid composite charts to simplfiy configuration and upgrades
+* Everything is now an app. So if you want to remove our `nginx-ingress` chart and replace it with another ingress solution (knative / istio / gloo / ambassador / linkerd or whatever) just go ahead and use the [apps commands](apps.md) to add/remove apps and have boot manage everything in a consistent way
+* The boot git repository is much smaller and simpler; less to keep in sync/rebase/merge with the upstream git repository. Its mostly just 2 YAML files now `jx-requirements.yml` and `jx-apps.yml` which are both pretty much specific to your cluster installation and a shared `jenkins-x.yml` file (which over time we can move into the build pack).
+  * we rely more instead on the [version stream](https://jenkins-x.io/docs/concepts/version-stream/) which can be shared across installations
+* secret handling is currently much simpler - you can provide a `secrets.yaml` file however you want via an environment variable. So it should be easy to mount secrets from any vault / github secret service / cloud provider service or local file.
+  * we've moved population of the secrets outside of the boot process so we should be able to default to running `jx boot` via a helm chart to simplify installation and avoid issues with local laptop configuration + binary packaeg differences.
+
+## Comparison
+
+Here is a [comparison of boot with helmfile and helm 3](comparison.md) in terms of similarities and differences if you are already aware of `jx boot` with helm 2.
+
+## Apps Model
+
+We have enhanced the existing [app extensibility model](apps.md) we have always had with Jenkins X to be more powerful:
+* an app can be in any namespace and can make more use of the [version stream](https://jenkins-x.io/docs/concepts/version-stream/))
+* everything is now an app including the system charts like `nginx-ingress` and `cert-manager`
+
+## How it works
+
+Here is [how boot works with helmfile and helm 3](how-it-works.md) if you want to understand how the implementation works.
+
+

proposals/2/docs/apps.md

@@ -0,0 +1,43 @@
+## Apps framework
+
+The new [helmfile](https://github.com/roboll/helmfile) and helm 3 approach extends the app extension model in Jenkins X.
+
+In essence that means if you are using helmfile you can use the usual Apps commands which create Pull Requests on the [jx-apps.yml](https://github.com/jenkins-x-labs/boot-helmfile-poc/blob/master/jx-apps.yml) file in your environments git repository rather than the traditional `env/requirements.yaml` file.
+
+This also means you can have apps in different namespaces. e.g. its common to put some charts in different namespaces like `nginx-ingress`, `gloo`, `cert-mangager` etc.
+
+### Using the apps commands
+
+you can use [jx add app](https://jenkins-x.io/commands/jx_add_app/) to add apps using the usual helm style notation of `repositoryPrefix/chartName` such as:
+
+```
+jx add app jetstack/cert-manager
+jx add app flagger/flagger
+
+```
+
+these commands will implicity use the [version stream](https://jenkins-x.io/docs/concepts/version-stream/) configuration (via [charts/repositories.yml](https://github.com/jenkins-x/jenkins-x-versions/blob/master/charts/repositories.yml)) to determine the mapping of prefixes to repository URLs.
+
+Then these commands will create Pull Requests on the [jx-apps.yml](https://github.com/jenkins-x-labs/boot-helmfile-poc/blob/master/jx-apps.yml) file in your environments git repository.
+
+Note that usually the Pull Request will only add a simple line of the format to the `applications:` entry:
+
+```
+apps:
+- name: jetstack/cert-manager 
+- name: flagger/flagger
+``` 
+
+This keeps the configuration in the environment git repository nice and concise. The `version` of the chart is then resolved during deployment via the [version stream](https://jenkins-x.io/docs/concepts/version-stream/).
+
+### Viewing apps
+
+You can view your apps across all namespaces via [jx get app](https://jenkins-x.io/commands/jx_get_apps/)
+
+``` 
+jx get app
+```
+
+This will effectively display data from the [jx-apps.yml](https://github.com/jenkins-x-labs/boot-helmfile-poc/blob/master/jx-apps.yml). This data will be pretty close to using a regular `helm list` using helm 3.x or later; only it will show apps across all namespaces by default..
+
+