Rework kobs architecture (#346)

Rework kobs architecture, by introducing a "hub" and "satellite"
component. This new architecture allows us to run the kobs "hub"
component in a central cluster and access clusters / services (plugins)
through the kobs "satellite" component.

The following tasks are still todo:

- We have to update the documentation with the new architecture.
- We have to update the plugin documentation.
- We have to adjust the demo.

Author: ricoberger

.dockerignore

@@ -1,13 +1,10 @@
 .github
 .vscode
-app/build
-app/node_modules
 bin
 deploy
 docs
 node_modules
-plugins/*/lib
-plugins/*/lib-esm
+plugins/*/build
 plugins/*/node_modules
 CHANGELOG.md
 CODE_OF_CONDUCT.md

.github/workflows/codeql.yaml

@@ -16,7 +16,6 @@ on:
       - 'docs/**'
       - '.dockerignore'
       - '.editorconfig'
-      - '.gitattributes'
       - '.gitignore'
       - 'CHANGELOG.md'
       - 'CODE_OF_CONDUCT.md'
@@ -35,7 +34,6 @@ on:
       - 'docs/**'
       - '.dockerignore'
       - '.editorconfig'
-      - '.gitattributes'
       - '.gitignore'
       - 'CHANGELOG.md'
       - 'CODE_OF_CONDUCT.md'
@@ -58,40 +56,41 @@ jobs:
     strategy:
       fail-fast: false
       matrix:
-        language: ['go', 'javascript']
-        # CodeQL supports [ 'cpp', 'csharp', 'go', 'java', 'javascript', 'python' ]
-        # Learn more:
-        # https://docs.github.com/en/free-pro-team@latest/github/finding-security-vulnerabilities-and-errors-in-your-code/configuring-code-scanning#changing-the-languages-that-are-analyzed
+        language: [ 'go', 'javascript' ]
+        # CodeQL supports [ 'cpp', 'csharp', 'go', 'java', 'javascript', 'python', 'ruby' ]
+        # Learn more about CodeQL language support at https://aka.ms/codeql-docs/language-support
 
     steps:
     - name: Checkout repository
-      uses: actions/checkout@v2
+      uses: actions/checkout@v3
 
     # Initializes the CodeQL tools for scanning.
     - name: Initialize CodeQL
-      uses: github/codeql-action/init@v1
+      uses: github/codeql-action/init@v2
       with:
         languages: ${{ matrix.language }}
         # If you wish to specify custom queries, you can do so here or in a config file.
         # By default, queries listed here will override any specified in a config file.
         # Prefix the list here with "+" to use these queries and those in the config file.
-        # queries: ./path/to/local/query, your-org/your-repo/queries@main
+
+        # Details on CodeQL's query packs refer to : https://docs.github.com/en/code-security/code-scanning/automatically-scanning-your-code-for-vulnerabilities-and-errors/configuring-code-scanning#using-queries-in-ql-packs
+        # queries: security-extended,security-and-quality
+
 
     # Autobuild attempts to build any compiled languages  (C/C++, C#, or Java).
     # If this step fails, then you should remove it and run the build manually (see below)
     - name: Autobuild
-      uses: github/codeql-action/autobuild@v1
+      uses: github/codeql-action/autobuild@v2
 
     # ℹ️ Command-line programs to run using the OS shell.
-    # 📚 https://git.io/JvXDl
+    # 📚 See https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idstepsrun
 
-    # ✏️ If the Autobuild fails above, remove it and uncomment the following three lines
-    #    and modify them (or add more) to build your code if your project
-    #    uses a compiled language
+    #   If the Autobuild fails above, remove it and uncomment the following three lines.
+    #   modify them (or add more) to build your code if your project, please refer to the EXAMPLE below for guidance.
 
-    #- run: |
-    #   make bootstrap
-    #   make release
+    # - run: |
+    #   echo "Run, Build Application using script"
+    #   ./location_of_script_within_repo/buildscript.sh
 
     - name: Perform CodeQL Analysis
-      uses: github/codeql-action/analyze@v1
+      uses: github/codeql-action/analyze@v2

.github/workflows/continuous-delivery.yaml

@@ -46,5 +46,5 @@ jobs:
           push: true
           context: .
           file: ./cmd/kobs/Dockerfile
-          platforms: linux/386,linux/amd64,linux/arm/v6,linux/arm/v7,linux/arm64/v8
+          platforms: linux/386,linux/amd64,linux/arm64
           tags: kobsio/kobs:${{ steps.tag.outputs.tag }}

.github/workflows/continuous-integration.yaml

@@ -8,9 +8,9 @@ on:
       - 'cmd/kobs/Dockerfile'
       - 'deploy/**'
       - 'docs/**'
+      - 'hack/**'
       - '.dockerignore'
       - '.editorconfig'
-      - '.gitattributes'
       - '.gitignore'
       - 'CHANGELOG.md'
       - 'CODE_OF_CONDUCT.md'
@@ -32,7 +32,7 @@ jobs:
       - name: Setup
         uses: actions/setup-go@v2
         with:
-          go-version: 1.17
+          go-version: 1.18
 
       - name: Cache
         uses: actions/cache@v2
@@ -51,7 +51,7 @@ jobs:
       - name: Test
         run: |
           make test-coverage
-          cat coverage.out | grep -v "github.com/kobsio/kobs/pkg/api/apis" | grep -v "github.com/kobsio/kobs/pkg/api/clients" | grep -v "_mock.go" > coverage_modified.out; mv coverage_modified.out coverage.out
+          cat coverage.out | grep -v "github.com/kobsio/kobs/pkg/kube/apis" | grep -v "github.com/kobsio/kobs/pkg/kube/clients" | grep -v "_mock.go" > coverage_modified.out; mv coverage_modified.out coverage.out
 
       - name: Upload Coverage to Codecov
         uses: codecov/codecov-action@v2
@@ -98,4 +98,4 @@ jobs:
 
       - name: Build
         run: |
-          yarn build
+          make generate-assets

.github/workflows/docs.yaml

@@ -33,10 +33,10 @@ jobs:
           echo ::set-output name=aliases::${MIKE_ALIASES}
 
       - name: Install Material for MkDocs
-        run: pip install -Iv mkdocs-material==8.2.9
+        run: pip install -Iv mkdocs-material==8.3.4
 
       - name: Install PyMdown Extensions
-        run: pip install -Iv pymdown-extensions==9.1
+        run: pip install -Iv pymdown-extensions==9.5
 
       - name: Install mike
         run: pip install -Iv mike==1.1.2

.github/workflows/helm.yaml

@@ -3,31 +3,35 @@ name: Helm
 on:
   push:
     paths:
-      - 'deploy/helm/kobs/**'
+      - '.github/workflows/helm.yaml'
+      - 'deploy/helm/**'
 
 jobs:
   helm:
     name: Helm Chart
     runs-on: ubuntu-latest
+    strategy:
+      matrix:
+        chart: [hub, satellite]
     steps:
       - name: Checkout
         uses: actions/checkout@v2
 
       - name: Lint
         run: |
-          helm lint deploy/helm/kobs
+          helm lint deploy/helm/${{ matrix.chart }}
 
       - name: Template
         run: |
-          helm template kobs -n observability deploy/helm/kobs
+          helm template kobs-${{ matrix.chart }} -n kobs deploy/helm/${{ matrix.chart }}
 
       - name: Install
         run: |
           kind create cluster
           sleep 60s
-          kubectl create namespace observability
+          kubectl create namespace kobs
           sleep 10s
-          helm install --namespace observability kobs deploy/helm/kobs
+          helm install --namespace kobs kobs${{ matrix.chart }} deploy/helm/${{ matrix.chart }}
 
       - name: Configure SSH
         if: ${{ github.ref == 'refs/heads/main' }}
@@ -43,7 +47,7 @@ jobs:
       - name: Package Helm Chart
         if: ${{ github.ref == 'refs/heads/main' }}
         run: |
-          helm package ./deploy/helm/kobs
+          helm package ./deploy/helm/${{ matrix.chart }}
 
       - name: Clone Helm Repository
         if: ${{ github.ref == 'refs/heads/main' }}
@@ -53,12 +57,12 @@ jobs:
       - name: Update Helm Repository
         if: ${{ github.ref == 'refs/heads/main' }}
         run: |
-          mv kobs* ./helm-repository/ && helm repo index helm-repository/ --url https://helm.kobs.io/
+          mv ${{ matrix.chart }}* ./helm-repository/ && helm repo index helm-repository/ --url https://helm.kobs.io/
 
       - name: Commit Changes
         if: ${{ github.ref == 'refs/heads/main' }}
         run: |
-          cd helm-repository/ && git add . && git commit -m "Add new release for kobs"
+          cd helm-repository/ && git add . && git commit -m "Add new release for kobs ${{ matrix.chart }}"
 
       - name: Push Changes
         if: ${{ github.ref == 'refs/heads/main' }}

.gitignore

@@ -6,9 +6,10 @@ node_modules
 # testing
 /coverage
 coverage.out
+coverage.html
 
-# production
-/build
+# builds
+build
 
 # misc
 .DS_Store
@@ -24,20 +25,13 @@ npm-debug.log*
 yarn-debug.log*
 yarn-error.log*
 
-# lib
-lib
-lib-esm
-
-# build
-/app/build
-
 # binaries
 /bin
 
 # temporary files
 /tmp
 
-# test files
-/config.yaml
+# test config, manifests and values file
+/config*.yaml
 /test.yaml
 /values.yaml

.vscode/launch.json

@@ -2,27 +2,46 @@
   "version": "0.2.0",
   "configurations": [
     {
-      "name": "Server",
+      "name": "hub",
       "type": "go",
       "request": "launch",
       "mode": "debug",
       "program": "${workspaceFolder}/cmd/kobs",
       "args": [
-        "--log.level=trace",
-        "--config=../../deploy/docker/kobs/config.yaml",
-        "--app.assets=../../app/build",
-        "--development"
+        "--log.level=debug",
+        "--hub.config=../../deploy/docker/kobs/config.yaml",
+        "--app.assets=''"
       ]
     },
     {
-      "name": "Chrome",
+      "name": "satellite",
+      "type": "go",
+      "request": "launch",
+      "mode": "debug",
+      "program": "${workspaceFolder}/cmd/kobs",
+      "args": [
+        "--log.level=debug",
+        "--satellite.config=../../deploy/docker/kobs/config.yaml",
+        "--satellite.plugins=../../bin/plugins",
+        "--satellite.token=unsecuretoken"
+      ]
+    },
+    {
+      "name": "app",
       "type": "chrome",
       "request": "launch",
       "url": "http://localhost:3000",
-      "webRoot": "${workspaceFolder}/app/src",
-      "sourceMapPathOverrides": {
-        "webpack:///src/*": "${webRoot}/*"
-      }
+      "webRoot": "${workspaceFolder}/plugins/app",
+      "sourceMaps": true
+    },
+    {
+      "name": "plugin",
+      "type": "chrome",
+      "request": "launch",
+      "url": "http://localhost:3001",
+      "webRoot": "${workspaceFolder}/plugins",
+      "sourceMaps": true
     }
+
   ]
 }

CHANGELOG.md

@@ -14,6 +14,8 @@ NOTE: As semantic versioning states all 0.y.z releases can contain breaking chan
 
 ### Changed
 
+- [#346](https://github.com/kobsio/kobs/pull/346): [app] :warning: _Breaking change:_ :warning: Rework kobs architecture, by introducing a `hub` and a `satellite` component. This new architecture allows us to run the kobs `hub` component in a central cluster and access clusters / services (plugins) through the kobs `satellite` component. More information regarding the new kobs architecture can be found in the documentation at [kobs.io](https://kobs.io).
+
 ## [v0.8.0](https://github.com/kobsio/kobs/releases/tag/v0.8.0) (2022-03-24)
 
 ### Added

CONTRIBUTING.md

@@ -78,7 +78,8 @@ Most of the kobs functions are implemented via plugins, which can be found in th
 If you adjust the Custom Resource Definitions for kobs, you must install the Kubernetes [code-generator](https://github.com/kubernetes/code-generator) into your `GOPATH`:
 
 ```sh
-go get k8s.io/[email protected]
+go install k8s.io/code-generator/[email protected]
+go install sigs.k8s.io/controller-tools/[email protected]
 ```
 
 ### Components