Issue with pass postgres credentials to grafana.ini (grafana.assertNoLeakedSecrets)

[SergeiCherevko]

I use helm for deploying grafana

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: grafana
  namespace: argocd
spec:
  project: devops
  source:
    repoURL: https://grafana.github.io/helm-charts
    chart: grafana
    targetRevision: 7.3.12
    helm:
      releaseName: grafana
      values: |
        ingress:
          enabled: true
          ingressClassName: nginx-external
          annotations:
            cert-manager.io/cluster-issuer: letsencrypt-example
            nginx.ingress.kubernetes.io/proxy-read-timeout: "120"
            nginx.ingress.kubernetes.io/proxy-send-timeout: "120"
          hosts: 
            - grafana.example.com
          tls:
            - hosts: 
                - grafana.example.com
              secretName: grafana-example-com
        resources:
          requests:
            cpu: 1
            memory: 2Gi
          limits:
            cpu: 2
            memory: 8Gi
        initChownData:
          enabled: false
        env:
          GF_PLUGINS_ALLOW_LOADING_UNSIGNED_PLUGINS: "gcorelabs-cdn-stats-datasource,quickwit-quickwit-datasource"
        extraSecretMounts:
          - name: devops-grafana-mount
            secretName: grafana.grafana-playground-postgres.credentials.postgresql.acid.zalan.do
            defaultMode: 0440
            mountPath: /etc/secrets
            readOnly: true
        plugins:
          - fifemon-graphql-datasource
          - magnesium-wordcloud-panel
          - https://github.com/G-Core/cdn-stats-datasource-plugin/releases/download/v1.0.13/gcorelabs-cdn-stats-datasource-1.0.13.zip;gcorelabs-cdn-stats-datasource
          - https://grafana.com/api/plugins/grafana-clickhouse-datasource/versions/4.0.7/download;grafana-clickhouse-datasource
          - https://grafana.com/api/plugins/vertamedia-clickhouse-datasource/versions/3.1.1/download;vertamedia-clickhouse-datasources
          - https://github.com/quickwit-oss/quickwit-datasource/releases/download/v0.4.6/quickwit-quickwit-datasource-0.4.6.zip;quickwit-quickwit-datasource
        sidecar:
          plugins:
            enabled: true
            skipReload: false
        headlessService: true
        replicas: 2
        rbac:
          namespaced: true
        grafana.ini:
          server:
            domain: grafana.example.com
            root_url: https://grafana.example.com
          database:
            type: postgres
            ssl_mode: require
            host: grafana-playground-postgres:5432
            name: grafana
            user: $__file{/etc/secrets/username}
            password: $__file{/etc/secrets/password}
          auth:
            disable_login_form: false
          auth.basic:
            enabled: true
          dataproxy:
            timeout: 120
  destination:
    name: in-cluster
    namespace: monitoring
  syncPolicy:
    syncOptions:
      - CreateNamespace=true

and in argocd i receive

Failed to load target state: failed to generate manifest for source 1 of 1: rpc error: code = Unknown desc = Manifest generation error (cached): `helm template . --name-template grafana --namespace monitoring --kube-version 1.29 --values /tmp/5d2217c8-33e5-4aaa-8d70-ad4acb2d455e <api versions removed> --include-crds` failed exit status 1: Error: template: grafana/templates/deployment.yaml:36:28: executing "grafana/templates/deployment.yaml" at <include "grafana.configData" .>: error calling include: template: grafana/templates/_config.tpl:5:3: executing "grafana.configData" at <include "grafana.assertNoLeakedSecrets" .>: error calling include: template: grafana/templates/_helpers.tpl:262:94: executing "grafana.assertNoLeakedSecrets" at <$elem>: wrong type for value; expected string; got map[string]interface {} Use --debug flag to render out invalid YAML

as i know its should be fixed since 7.2.2, but i can't understand what im doing wrong

[Ojakoo]

I had the same problem with adding authentication provider secrets and was able to resolve the issue with using envFromSecret in values.yaml

# values.yaml
envFromSecret: "my-secret-name"

Then create a kubernetes secret with data values for the required environment variables. For example the required auth env variables were GF_AUTH_GOOGLE_CLIENT_ID and GF_AUTH_GOOGLE_CLIENT_SECRET. For postgres these are probably something like GF_DATABASE_USER, GF_DATABASE_PASSWORD, etc but im not fully sure about that.

The root problem is that the grafana.ini file is written as plain text in a ConfigMap which can be bypassed with assertNoLeakedSecrets but that is not recommended.

[SergeiCherevko]

I resolved this like this:

      values: |

        envFromSecret: "grafana.grafana-playground-postgres.credentials.postgresql.acid.zalan.do"

        env:
          GF_DATABASE_USER: "$(username)"
          GF_DATABASE_PASSWORD: "$(password)"

However, I think this is more of a workaround because, according to the official documentation, they achieve this using the file provider.

I tried different Helm versions, including the latest, but it still doesn’t work! I couldn't find my mistake.

If the developers see this issue, please explain what I am doing wrong.

[SergeiCherevko]

If you need more than one secret, i use

        envFromSecrets:
          - name: grafana-gsuite-oidc
          - name: grafana.grafana-playground-postgres.credentials.postgresql.acid.zalan.do