Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[wip] multi cluster support #6664

Closed
wants to merge 54 commits into from
Closed

[wip] multi cluster support #6664

wants to merge 54 commits into from

Commits on Feb 27, 2020

  1. fix: add helper class for working with environment contexts

    * wrapping up team settings / version resolver / requirements APIs in a simpler facade
    * add a helper method with tests for resolving a chart name with optional prefix and/or repository to a canonical set of data we can use in version streams / apps framework
    
    Signed-off-by: James Strachan <[email protected]>
    jstrachan committed Feb 27, 2020
    Configuration menu
    Copy the full SHA
    5736803 View commit details
    Browse the repository at this point in the history
  2. fix: support helmfile/helm 3 with the app framework

    so that we can add/delete/update/get apps which map to changing the `jx-apps.yml` instead of the `env/requirements.yaml` file.
    
    this implementation also lets you use helm repo prefixes to fully qualify charts; using the version stream to map prefixes <-> URLs.
    
    e.g. to add flagger/cert manager you can do:
    
    ```
    jx add app jetstack/cert-manager
    jx add app flagger/flagger
    
    ```
    
    Signed-off-by: James Strachan <[email protected]>
    jstrachan committed Feb 27, 2020
    Configuration menu
    Copy the full SHA
    41d9c73 View commit details
    Browse the repository at this point in the history
  3. fix: lets resolve fully qualified charts for helmfile

    and reuse the version stream prefixes for repositories in the generated helmfile
    
    also lets add a diff based test case to verify we generate the right helmfile
    
    Signed-off-by: James Strachan <[email protected]>
    jstrachan committed Feb 27, 2020
    Configuration menu
    Copy the full SHA
    ab4eb0e View commit details
    Browse the repository at this point in the history
  4. fix: add version defaulting from the version stream

    so that we always use a fixed version in the generated helmfiles
    
    Signed-off-by: James Strachan <[email protected]>
    jstrachan committed Feb 27, 2020
    Configuration menu
    Copy the full SHA
    2f4fe19 View commit details
    Browse the repository at this point in the history
  5. fix: improve handling of local charts in helmfile generation

    Signed-off-by: James Strachan <[email protected]>
    jstrachan committed Feb 27, 2020
    Configuration menu
    Copy the full SHA
    76f85f5 View commit details
    Browse the repository at this point in the history
  6. fix: add version stream defaults for apps in helmfile

    so that we can default things like namespace / phase / values.yaml* files across cluster configurations via the Version Stream
    
    Signed-off-by: James Strachan <[email protected]>
    jstrachan committed Feb 27, 2020
    Configuration menu
    Copy the full SHA
    35ddb61 View commit details
    Browse the repository at this point in the history
  7. fix: lets allow helmfile to be abled via env vars

    Signed-off-by: James Strachan <[email protected]>
    jstrachan committed Feb 27, 2020
    Configuration menu
    Copy the full SHA
    84d3334 View commit details
    Browse the repository at this point in the history
  8. fix: rebased with latest jx

    also ensured we can default the boot phase from the defaults in the version stream
    
    Signed-off-by: James Strachan <[email protected]>
    jstrachan committed Feb 27, 2020
    Configuration menu
    Copy the full SHA
    85e45d7 View commit details
    Browse the repository at this point in the history
  9. chore: fix hound warnings

    Signed-off-by: James Strachan <[email protected]>
    jstrachan committed Feb 27, 2020
    Configuration menu
    Copy the full SHA
    fa15e69 View commit details
    Browse the repository at this point in the history
  10. fix: add missing classifications

    to fix lint failure
    
    Signed-off-by: James Strachan <[email protected]>
    jstrachan committed Feb 27, 2020
    Configuration menu
    Copy the full SHA
    73ca45b View commit details
    Browse the repository at this point in the history
  11. chore: fix failing test broken due to rebase

    Signed-off-by: James Strachan <[email protected]>
    jstrachan committed Feb 27, 2020
    Configuration menu
    Copy the full SHA
    195ec21 View commit details
    Browse the repository at this point in the history
  12. chore: regenerated

    Signed-off-by: James Strachan <[email protected]>
    jstrachan committed Feb 27, 2020
    Configuration menu
    Copy the full SHA
    d7f3844 View commit details
    Browse the repository at this point in the history
  13. chore: fix some tests broken due to rebase

    Signed-off-by: James Strachan <[email protected]>
    jstrachan committed Feb 27, 2020
    Configuration menu
    Copy the full SHA
    64c17a6 View commit details
    Browse the repository at this point in the history
  14. fix: refactor ApplicationConfig => AppConfig

    so we use Apps rather than Application to refer to the apps framework and the config for `jx-apps.yml` to avoid confusion
    
    Signed-off-by: James Strachan <[email protected]>
    jstrachan committed Feb 27, 2020
    Configuration menu
    Copy the full SHA
    bb2f29a View commit details
    Browse the repository at this point in the history
  15. chore: go mod tidy

    Signed-off-by: James Strachan <[email protected]>
    jstrachan committed Feb 27, 2020
    Configuration menu
    Copy the full SHA
    6b489ec View commit details
    Browse the repository at this point in the history
  16. chore: revert go mod tidy

    as it seems to break lint
    
    Signed-off-by: James Strachan <[email protected]>
    jstrachan committed Feb 27, 2020
    Configuration menu
    Copy the full SHA
    155fba2 View commit details
    Browse the repository at this point in the history
  17. fix: regression in jx add app when not using helmfile

    to more cleanly differentiate between local chart name and full chart name (with repo prefix)
    
    Signed-off-by: James Strachan <[email protected]>
    jstrachan committed Feb 27, 2020
    Configuration menu
    Copy the full SHA
    aa5d816 View commit details
    Browse the repository at this point in the history
  18. fix: lets improve the EnvironmentContext API

    so that we can give an indication of if we prefer local requirements files (for early stages in boot) versus commands used after boot such as (get|add|upgrade|delete) apps
    
    Signed-off-by: James Strachan <[email protected]>
    jstrachan committed Feb 27, 2020
    Configuration menu
    Copy the full SHA
    d3faf81 View commit details
    Browse the repository at this point in the history
  19. fix: regression in jx upgrade app

    lets make sure we populate the repository from the team settings/default repo
    
    Signed-off-by: James Strachan <[email protected]>
    jstrachan committed Feb 27, 2020
    Configuration menu
    Copy the full SHA
    42b5bc5 View commit details
    Browse the repository at this point in the history
  20. fix: lets polish the code to be more clear on chart names

    as we can support full chart names or local chart names; so lets be more specific which name we are using in the code
    
    Signed-off-by: James Strachan <[email protected]>
    jstrachan committed Feb 27, 2020
    Configuration menu
    Copy the full SHA
    969358d View commit details
    Browse the repository at this point in the history
  21. fix: first spike of supporting remote environments with boot and helm…

    …file
    
    Signed-off-by: James Strachan <[email protected]>
    jstrachan committed Feb 27, 2020
    Configuration menu
    Copy the full SHA
    3733c89 View commit details
    Browse the repository at this point in the history
  22. fix: add generated jx-requirements.yml to remote environments

    for remote clusters
    
    Signed-off-by: James Strachan <[email protected]>
    jstrachan committed Feb 27, 2020
    Configuration menu
    Copy the full SHA
    80a45f5 View commit details
    Browse the repository at this point in the history
  23. fix: if no namespace is in jx-apps.yml lets use the default namespace

    which lets us avoid specifying the namespace in remote environments for multi-cluster
    
    Signed-off-by: James Strachan <[email protected]>
    
    #4784
    jstrachan committed Feb 27, 2020
    Configuration menu
    Copy the full SHA
    2d256dc View commit details
    Browse the repository at this point in the history
  24. fix: default the owners + git owner on remote environments

    Signed-off-by: James Strachan <[email protected]>
    
    #4784
    jstrachan committed Feb 27, 2020
    Configuration menu
    Copy the full SHA
    ae6d462 View commit details
    Browse the repository at this point in the history
  25. fix: add support for remote environments to promote

    fixes #4784
    
    Signed-off-by: James Strachan <[email protected]>
    jstrachan committed Feb 27, 2020
    Configuration menu
    Copy the full SHA
    eb11782 View commit details
    Browse the repository at this point in the history
  26. fix: avoid possible nil pointer exception

    Signed-off-by: James Strachan <[email protected]>
    jstrachan committed Feb 27, 2020
    Configuration menu
    Copy the full SHA
    f11fa77 View commit details
    Browse the repository at this point in the history
  27. chore: lets sort the list of k8s providers

    so they appear nicer when we ask users what provider to use
    
    Signed-off-by: James Strachan <[email protected]>
    jstrachan committed Feb 27, 2020
    Configuration menu
    Copy the full SHA
    8fc707c View commit details
    Browse the repository at this point in the history
  28. fix: add support for jx boot --helmfile

    so folks can try out helmfile and helm 3
    
    Signed-off-by: James Strachan <[email protected]>
    jstrachan committed Feb 27, 2020
    Configuration menu
    Copy the full SHA
    b4b3a62 View commit details
    Browse the repository at this point in the history
  29. fix: avoid creating the dev Environment when using helmfile

    as we want to create and manage all k8s resources via helm instead
    
    Signed-off-by: James Strachan <[email protected]>
    jstrachan committed Feb 27, 2020
    Configuration menu
    Copy the full SHA
    93db455 View commit details
    Browse the repository at this point in the history
  30. fix: default the namespace from the jx-requirements.yml file

    also add validation of the enviroment variable `JX_SECRETS_YAML` to ensure its defined to point to the secrets yaml file
    
    Signed-off-by: James Strachan <[email protected]>
    jstrachan committed Feb 27, 2020
    Configuration menu
    Copy the full SHA
    5747dcc View commit details
    Browse the repository at this point in the history
  31. fix: promotion for helmfile and multi-cluster

    so that we handle the public chart museum URL and include it into the `jx-apps.yml` file in the pull request
    
    Signed-off-by: James Strachan <[email protected]>
    jstrachan committed Feb 27, 2020
    Configuration menu
    Copy the full SHA
    be4c1cd View commit details
    Browse the repository at this point in the history
  32. fix: lets generate a default secrets YAML file for helmfile

    so its easier for folks to populate it. Longer term we can help the user populate it like we do with the current boot approach
    jstrachan committed Feb 27, 2020
    Configuration menu
    Copy the full SHA
    baa6e51 View commit details
    Browse the repository at this point in the history
  33. Configuration menu
    Copy the full SHA
    78d44f8 View commit details
    Browse the repository at this point in the history
  34. fix: refactor

    rename the `jx-apps.yml` apps.valueFiles to apps.values to match helmfile's naming convention
    
    also allow discovery of a `myrepo/mychart`'s yaml file via `mychart/values.yaml` as using `myrepo/mychart/values.yaml` seems to confuse helm into thinking the `myrepo/mychart` folder is a local chart source code
    jstrachan committed Feb 27, 2020
    Configuration menu
    Copy the full SHA
    8a3a7ef View commit details
    Browse the repository at this point in the history
  35. fix: avoid creating invalid helmfile

    if we have no releases for a helmfile lets add a dummy empty chart to avoid `helmfile sync` failing in the system or apps dir
    
    refactored the test cases so they are easier to create
    jstrachan committed Feb 27, 2020
    Configuration menu
    Copy the full SHA
    1419770 View commit details
    Browse the repository at this point in the history
  36. fix: jx step create helmfile improvements

    lets lazily create the `jx-requirements.values.yaml.gotmpl` file if it does not exist. This lets us use a simpler PR pipeline on local environments without needing to run the whole `jx step verify preinstall` step.
    
    also lets default the `--values` arguments with nice validation if folks don't specify them so we can simplify the OOTB pipelines
    jstrachan committed Feb 27, 2020
    Configuration menu
    Copy the full SHA
    f9a5339 View commit details
    Browse the repository at this point in the history
  37. Configuration menu
    Copy the full SHA
    b43b3b9 View commit details
    Browse the repository at this point in the history
  38. Configuration menu
    Copy the full SHA
    a9b1e2f View commit details
    Browse the repository at this point in the history
  39. fix: lets not fail if helm version errors

    which it usually does on helm 2 containers
    jstrachan committed Feb 27, 2020
    Configuration menu
    Copy the full SHA
    ed97e29 View commit details
    Browse the repository at this point in the history
  40. Configuration menu
    Copy the full SHA
    ea318b5 View commit details
    Browse the repository at this point in the history
  41. Configuration menu
    Copy the full SHA
    4f7d509 View commit details
    Browse the repository at this point in the history
  42. Configuration menu
    Copy the full SHA
    c3aec4f View commit details
    Browse the repository at this point in the history
  43. Configuration menu
    Copy the full SHA
    640d141 View commit details
    Browse the repository at this point in the history
  44. fix: allow environment variables to reference environment variable ex…

    …pressions
    
    e.g. so we can enable helm 3 in a build pack via `PATH=/opt/bin/helm3-bin:$PATH`
    jstrachan committed Feb 27, 2020
    Configuration menu
    Copy the full SHA
    c51a083 View commit details
    Browse the repository at this point in the history
  45. Configuration menu
    Copy the full SHA
    12339c3 View commit details
    Browse the repository at this point in the history
  46. fix: lets default to helmfile for staging and production

    if we are using helmfile for dev
    jstrachan committed Feb 27, 2020
    Configuration menu
    Copy the full SHA
    49dc84b View commit details
    Browse the repository at this point in the history
  47. chore: avoid warn message with custom built jx binaries as causes iss…

    …ues running steps and capturing system out
    rawlingsj authored and jstrachan committed Feb 27, 2020
    Configuration menu
    Copy the full SHA
    27bd17b View commit details
    Browse the repository at this point in the history
  48. Configuration menu
    Copy the full SHA
    1877cea View commit details
    Browse the repository at this point in the history
  49. Configuration menu
    Copy the full SHA
    734215b View commit details
    Browse the repository at this point in the history
  50. chore: rebase

    jstrachan committed Feb 27, 2020
    Configuration menu
    Copy the full SHA
    a605584 View commit details
    Browse the repository at this point in the history
  51. fix: better validation of helm

    if we have configured helm 3 then lets use that version to validate helm
    
    Signed-off-by: James Strachan <[email protected]>
    jstrachan committed Feb 27, 2020
    Configuration menu
    Copy the full SHA
    8ef772f View commit details
    Browse the repository at this point in the history
  52. fix: lets add a conditional compilation flag

    so that we can reuse the same code in jx alpha / jx 3.x where we default to helm 3
    
    Signed-off-by: James Strachan <[email protected]>
    jstrachan committed Feb 27, 2020
    Configuration menu
    Copy the full SHA
    bafcb8f View commit details
    Browse the repository at this point in the history
  53. fix: lets default to helm 3 in this branch

    Signed-off-by: James Strachan <[email protected]>
    jstrachan committed Feb 27, 2020
    Configuration menu
    Copy the full SHA
    2e16696 View commit details
    Browse the repository at this point in the history

Commits on Mar 1, 2020

  1. fix: allow the secret yaml env var validation to be disabled

    Signed-off-by: James Strachan <[email protected]>
    jstrachan committed Mar 1, 2020
    Configuration menu
    Copy the full SHA
    7b3b393 View commit details
    Browse the repository at this point in the history