Validation Errors not properly printed in v3.0.0-beta.2

[sudermanjr]

Is this a request for help?: No

---

Is this a BUG REPORT or FEATURE REQUEST? (choose one): Bug Report

Version of Helm and Kubernetes:
Helm v3.1.1
Kubernetes - Not in Kube
Chart-testing v3.0.0-beta.2

What happened:

After successfully linting one chart, the other fails and provides only a usage message. The second chart was missing a required maintainers block. This error was not shown to the user.

------------------------------------------------------------------------------------------------------------------------
 Charts to be processed:
------------------------------------------------------------------------------------------------------------------------
 clamav => (version: "0.0.1", path: "incubator/clamav")
 goldilocks => (version: "2.2.3", path: "stable/goldilocks")
------------------------------------------------------------------------------------------------------------------------

"jetstack" has been added to your repositories
"fairwinds-stable" has been added to your repositories
"fairwinds-incubator" has been added to your repositories
Linting chart 'clamav => (version: "0.0.1", path: "incubator/clamav")'
Checking chart 'clamav => (version: "0.0.1", path: "incubator/clamav")' for a version bump...
Unable to find chart on master. New chart detected.
Validating /repo/incubator/clamav/Chart.yaml...
Validation success! 👍
incubator/clamav/values.yaml
  66:1      error    too many blank lines (1 > 0)  (empty-lines)

Linting chart 'goldilocks => (version: "2.2.3", path: "stable/goldilocks")'
Checking chart 'goldilocks => (version: "2.2.3", path: "stable/goldilocks")' for a version bump...
Old chart version: 2.2.2
New chart version: 2.2.3
Chart version ok.
Validating /repo/stable/goldilocks/Chart.yaml...
Validation success! 👍
Validating maintainers...

Linting chart with values file 'stable/goldilocks/ci/test-values.yaml'...

==> Linting stable/goldilocks

1 chart(s) linted, 0 chart(s) failed
Error: Error linting charts: Error processing charts
Usage:
  ct lint [flags]

Flags:
      --all                            Process all charts except those explicitly excluded.
                                       Disables changed charts detection and version increment checking
      --chart-dirs strings             Directories containing Helm charts. May be specified multiple times
                                       or separate values with commas (default [charts])
      --chart-repos strings            Additional chart repositories for dependency resolutions.
                                       Repositories should be formatted as 'name=url' (ex: local=http://127.0.0.1:8879/charts).
                                       May be specified multiple times or separate values with commas
      --chart-yaml-schema string       The schema for chart.yml validation. If not specified, 'chart_schema.yaml'
                                       is searched in the current directory, '$HOME/.ct', and '/etc/ct', in
                                       that order.
      --charts strings                 Specific charts to test. Disables changed charts detection and
                                       version increment checking. May be specified multiple times
                                       or separate values with commas
      --check-version-increment        Activates a check for chart version increments (default: true) (default true)
      --config string                  Config file
      --debug                          Print CLI calls of external tools to stdout (Note: depending on helm-extra-args
                                       passed, this may reveal sensitive data)
      --excluded-charts strings        Charts that should be skipped. May be specified multiple times
                                       or separate values with commas
      --helm-repo-extra-args strings   Additional arguments for the 'helm repo add' command to be
                                       specified on a per-repo basis with an equals sign as delimiter
                                       (e.g. 'myrepo=--username test --password secret'). May be specified
                                       multiple times or separate values with commas
  -h, --help                           help for lint
      --lint-conf string               The config file for YAML linting. If not specified, 'lintconf.yaml'
                                       is searched in the current directory, '$HOME/.ct', and '/etc/ct', in
                                       that order
      --remote string                  The name of the Git remote used to identify changed charts (default "origin")
      --target-branch string           The name of the target branch used to identify changed charts (default "master")
      --validate-chart-schema          Enable schema validation of 'Chart.yaml' using Yamale (default: true) (default true)
      --validate-maintainers           Enable validation of maintainer account names in chart.yml (default: true).
                                       Works for GitHub, GitLab, and Bitbucket (default true)
      --validate-yaml                  Enable linting of 'Chart.yaml' and values files (default: true) (default true)

Error linting charts: Error processing charts

What you expected to happen:
Either a success or a failure and some useful output if there is a failure.

How to reproduce it (as minimally and precisely as possible):
ct lint --config scripts/ct.yaml --chart-yaml-schema scripts/schema.yaml
If you want my exact setup, you can see this build: https://app.circleci.com/jobs/github/FairwindsOps/charts/3866

Anything else we need to know:

• This happened both in CI and in a local environment where I ran the chart-test container and volume-mounted my repo.
• It would seem that perhaps the lint is failing on the second chart in my list, which is a schema v2 chart, wheras the one that passed is v1. Not sure if that helps

[sudermanjr]

EDIT: Redacted because it was a red herring.

[sudermanjr]

also if it helps, here's my schema:

name: str()
home: str(required=False)
version: str()
appVersion: any(str(), num())
description: str()
keywords: list(str(), required=False)
sources: list(str(), required=False)
maintainers: list(include('maintainer'), required=False)
icon: str(required=False)
engine: str(required=False)
condition: str(required=False)
tags: str(required=False)
deprecated: bool(required=False)
kubeVersion: str(required=False)
annotations: map(str(), str(), required=False)
---
maintainer:
  name: str()
  email: str(required=False)
  url: str(required=False)

[sudermanjr]

Started debugging this today. Just to make sure, I installed yamale and made sure that it would validate against my schema.

└─ yamale --schema scripts/schema.yaml stable/goldilocks/Chart.yaml
Validating /Users/asuderma/repos/fairwindsops/charts/stable/goldilocks/Chart.yaml...
Validation success! 👍

So I have definitely narrowed it down to the helm lint step:

>>> helm lint stable/goldilocks --values stable/goldilocks/ci/test-values.yaml
==> Linting stable/goldilocks

1 chart(s) linted, 0 chart(s) failed
Error: Error linting charts: Error processing charts

But if I run that myself:

└─ helm lint stable/goldilocks --values stable/goldilocks/ci/test-values.yaml
==> Linting stable/goldilocks

1 chart(s) linted, 0 chart(s) failed

└─ echo $?
0

[sudermanjr]

Okay, so I figured out the cause of the initial error. It was far harder than it should have been since the error never got caught correctly. I had to go add a bunch of print statements to find it.

My schema requires maintainers, and the clamav chart in that repo had no maintainers.

I have edited the title and updated my initial report

[sudermanjr]

Here is the full debug output of the command in question when it fails to provide the error message.
output.log

[unguiculus]

Looking at you initial log above, it did print the error:

incubator/clamav/values.yaml
66:1 error too many blank lines (1 > 0) (empty-lines)

[sudermanjr]

Yes, however it did not print it at the end (which I believe it used to do), and it printed a usage message which is very misleading.