Security Observability: Add persistent volume, gate on tech preview (#…

…418)

* Work in progress Analysis tools

* Thursday updates

* Friday fixes and improvements

* Friday code

* Separate analyzers into a separate container

* Wire in analyzers container in the Helm chart

* Hide analysis UI features when not enabled

* Fix sidepanel bug with fallback metadata

* Bug fix for change in way tab links are hidden

* Remove debug logging

* Add refresh button to report selector drop down. Change no reports icon

* Add support for adding breadcrumbs in the sub nav bar

* Fix unit tests

* Fix format issues

* Final front-end unit test fixes

* Fix issues when deploying via Helm with mariadb

* Analyzers container fix. Allow helm chart to be packaged.

* Build script fixes

* Remove file

* WIP: Add support for Clair image scanning

* Use klar

* Remove binary

* Add clair helm chart for dev

* Fixes

* USe end var for clair server address

* Latest updates

* Improvements

* Minor fixes

* Tweak

* Fix 1.16 detecton issue with the analyzers

* Chart fixes

* Changes following first run of script

* Changes following npm install

* Update custom-src to new model
- expose custom module's module's
- Add routing module
- Tweak stratos.config.ts log output
- remove custom-src dir

* update naming... custom extensions --> suse extensions

* A few tidyups to help review

* Fix build issue due to merge

* Fixes following merge from upstream

* Remove clair from this PR

* Ignore example packages when there's a stratos config file

* Changes following review

* Changes following merge

* Update dir names, remove examples folder

* Add back in custom-src deploy content, also add product version to config

* Revert change needed downstream... (only needed when suse extension is included)

* Remove unused wip report viewers

* Fix after merge

* Move new terminal & config code to plugin, fix more build files

* Fix imports and add doc

* Fix compilation issues

* Change following merge

* Tweaks to logging

* Fix bug where report can not be deleted

* Fix kube config connect after merge, also fix subtype & error on connect

* Fix e2e

* Improve drop-down menu

* Remove strange merge artifacts

* Remove build file

* Fix graph overview

* Numerous improvements to graph parsing and presentation

* Remove logging. Add no reports message to workload analysis

* Add support for CRDs. KubeCF renders correctly.

* Allow which engines are enabled to be configured

* Fix issue where reports are not filtered by endpoint

* Minor changes following review

* Fixes for a few more issues

* Add Analyzers image build to Concourse CI

* Multiple small fixes
- fix text search in analysis list
- fix title of links in analysers info page
- handle slow connections by only polling analysis list when not already

* Fix kubeGuid for helm world

* Add AnalysisReportRunnerComponent
- Still need to add this to other places

* Delete reports when endpoint is unregistered

* Buf fixes. Use breadcrumbs in sub-nav

* Add run analysis button to workload analyis and graph tabs

* Fix select of overlay in workload graphs page

* Change default sort order of analysis list to age

* Ensure table cell links update on row change

* Align table action's icon better

* Use a side panel for analyzer info

* Add actions/effects for all used analyis actions
- Add new ResetPaginationOfType action, like ResetPagination but applies to all types
- Allows user to refresh reports list after kicking off new report on namespace & workload tabs
- Handle missing report param in reports returned from get all reports

* Remove some console.logs, converted some to console.info

* Update Kube Dashboard, allow download link to be configurable
- Default download link updated to v2.0.3
- Can configured link by setting env var `STRATOS_KUBERNETES_DASHBOARD_IMAGE`
- Can configure env var in helm via `console.kubeDashboardImage`
- Kube Dashboard now expanded by default (to show namespace drop down)

* Fix after merge

* Changes following review

* Fix expand of kube dashboard header by default

* Changes following review

* Fix json-viewer dark mode

* Fix profile page and side nav top position following header diet
- Fix side nav top position
- Update fix for profile page to also work in non-desktop mode

* WIP Wire in alerts to workload graph
- need to understand if namespace should be checked when matching node/resource to alert
- need to apply correct colour

* Fix workload security analysis overlay slide in

* Hide analysis headers info in tech preview & tie in tech preview check to analysisService.hideAnalysis$
- Q should the backend plugins be available in tech preview, see TODO

* Hide the Workload Graph view if in tech preview

* Fix disable of analysis plugin when tech preview is switched off

* Adderss PR feedback

* Gate analyzer deployment on tech preview. Add persistent volume

* Fixes - removed volume for now

* Fix issue with persistent volume

* Fix error message colour in login screen

* Fix after merge

Co-authored-by: Richard Cox <[email protected]>

deploy/kubernetes/console/templates/analyzers.yaml

@@ -1,4 +1,5 @@
 ---
+{{- if .Values.console.techPreview }}
 {{- if semverCompare ">=1.16" (printf "%s.%s" .Capabilities.KubeVersion.Major (trimSuffix "+" .Capabilities.KubeVersion.Minor) )}}
 apiVersion: apps/v1
 {{- else }}
@@ -20,13 +21,20 @@ spec:
       app.kubernetes.io/component: "stratos-analyzers"
   template:
     metadata:
+{{- if .Values.console.podAnnotations }}
+      annotations:
+{{ toYaml .Values.console.podAnnotations | indent 8 }}
+{{- end }}
       labels:
         app.kubernetes.io/name: "stratos"
         app.kubernetes.io/instance: "{{ .Release.Name }}"
         app.kubernetes.io/version: "{{ .Chart.AppVersion }}"
         app.kubernetes.io/component: "stratos-analyzers"
         helm.sh/chart: "{{ .Chart.Name }}-{{ .Chart.Version | replace "+" "_" }}"
         app: "{{ .Release.Name }}"
+        {{- if .Values.console.podExtraLabels}}
+        {{ toYaml .Values.console.podExtraLabels | nindent 8 }}
+        {{- end}}        
     spec:
       containers:
       - name: analyzers
@@ -36,10 +44,25 @@ spec:
         - name: api
           containerPort: 8090
         env:
+        - name: STRATOS_IMAGE_REF
+          value: "{{.Values.consoleVersion}}:{{ .Release.Revision }}"
         - name: ANALYSIS_SCRIPTS_DIR
           value: "/scripts"
         - name: ANALYSIS_REPORTS_DIR
           value: "/reports"
+        volumeMounts:
+        - name: data
+          mountPath: /reports
+      {{- if and .Values.kube.registry.username .Values.kube.registry.password }}
+      imagePullSecrets:
+      - name: {{.Values.dockerRegistrySecret}}
+      {{- end }}
+      {{- if not .Values.console.reportsVolumeDisabled }}
+      volumes:
+      - name: data
+        persistentVolumeClaim:
+          claimName: "{{ .Release.Name }}-reports"
+      {{- end }}
 ---
 apiVersion: v1
 kind: Service
@@ -60,3 +83,27 @@ spec:
   selector:
     app: "{{ .Release.Name }}"
     app.kubernetes.io/component: "stratos-analyzers"
+---
+kind: PersistentVolumeClaim
+apiVersion: v1
+metadata:
+  name: "{{ .Release.Name }}-reports"
+  labels:
+    app.kubernetes.io/name: "stratos"
+    app.kubernetes.io/instance: "{{ .Release.Name }}"
+    app.kubernetes.io/version: "{{ .Chart.AppVersion }}"
+    app.kubernetes.io/component: "stratos-reports-volume"
+    helm.sh/chart: "{{ .Chart.Name }}-{{ .Chart.Version | replace "+" "_" }}"
+  annotations:
+  {{- if .Values.storageClass }}
+    volume.beta.kubernetes.io/storage-class: {{ .Values.storageClass | quote }}
+  {{- else }}
+    volume.alpha.kubernetes.io/storage-class: default
+  {{- end }}
+spec:
+  accessModes:
+    - "ReadWriteOnce"
+  resources:
+    requests:
+      storage: {{ default "1Gi" .Values.console.reportsVolumeSize | quote }}
+{{- end }}

deploy/kubernetes/console/values.yaml

@@ -112,6 +112,12 @@ console:
 
   # Download link when installing the Kubernetes Dashboard in a targetted Kube Endpoint
   kubeDashboardImage:
+
+  # Size for analysis reports volume
+  reportsVolumeSize: 1Gi
+
+  # Do not use a persistent volume for analysis reports
+  reportsVolumeDisabled: false
 
 images:
   console: stratos-console

src/frontend/packages/suse-extensions/src/custom/kubernetes/workloads/release/tabs/helm-release-resource-graph/helm-release-resource-graph.component.ts

@@ -245,16 +245,9 @@ export class HelmReleaseResourceGraphComponent implements OnInit, OnDestroy {
   private getResource(node: CustomHelmReleaseGraphNode): Observable<HelmReleaseResource> {
     return this.helper.fetchReleaseResources().pipe(
       filter(r => !!r),
-      // tap(r => {
-      //   console.log(node);
-      //   console.log(r);
-      // }),
-      map((r: HelmReleaseResources) => Object.values(r.data).find((res) => {
-        // if (!res.metadata) {
-        //   console.log(node, res);
-        // }
-        return res.metadata.name === node.label && res.kind === node.data.kind;
-      })),
+      map((r: HelmReleaseResources) => Object.values(r.data).find((res) =>
+        res.metadata.name === node.label && res.kind === node.data.kind
+      )),
       first(),
     );
   }

src/frontend/packages/suse-extensions/src/custom/suse-login/suse-login.component.scss

@@ -128,6 +128,7 @@
     }
   }
   &__message {
+    color: $suse-gray-fg;
     font-size: 18px;
     font-weight: 300;
     height: 20px;