Use architect 0.8.1 #37

stone-z · 2020-03-09T16:18:30Z

Includes fixes so that linting passes.

Currently fails due to a missing CloudProvider field which was also identified here. I can either fix that bug here or update this PR if that one goes in first

tfussell

LGTM.

MarcelMue · 2020-03-09T22:20:33Z

Why is the build failing though?

tfussell · 2020-03-09T22:57:14Z

Why is the build failing though?

There was a missing variable and the template error was ignored so it was silently truncating the output.

MarcelMue · 2020-03-10T08:41:38Z

Okay build now fails very oddly - any idea what the cause could be?

stone-z · 2020-03-10T08:46:03Z

Okay build now fails very oddly - any idea what the cause could be?

I think chart linting is kind of new, we probably haven't built ignition-operator since it got added

stone-z · 2020-03-10T09:15:02Z

Running ct lint locally I don't get this issue

MarcelMue · 2020-03-10T09:22:22Z

Yes - the error is super confusing to me:

 ✖︎ ignition-operator => (version: "0.0.1-67e7bf3548124c1320e8d11efd2b31d059897052", path: "helm/ignition-operator") > Error reading old chart version: Could not unmarshal 'Chart.yaml': yaml: unmarshal errors:
  line 3: cannot unmarshal !!seq into string

I don't understand how the unmarshal of that file fails - especially why does it refer to some old chart version?

stone-z · 2020-03-10T09:29:57Z

If I set a version in values.yaml and re-lint, I get the error locally

stone-z · 2020-03-10T10:27:53Z

Hey @kopiczko @yasn77 sorry for the ping but are we missing something obvious here regarding how to use the new chart linting in architect-orb?

kopiczko · 2020-03-10T10:49:06Z

It looks like it doesn't like square brackets in version: [[ .Version ]]. @yasn77 is there a way to disable that?

@stone-z have you managed to make linter work locally before templating?
Alternative could be linting after running architect helm template which I believe makes more sense.

stone-z · 2020-03-10T10:50:44Z

Trying with a random string version gives me the same result:

✖︎ ignition-operator => (version: "aversion", path: "helm/ignition-operator") > Error reading old chart version: Could not unmarshal 'Chart.yaml': yaml: unmarshal errors:
  line 3: cannot unmarshal !!seq into string

stone-z · 2020-03-10T10:52:41Z

This open issue makes me think it references the previous chart version

kopiczko · 2020-03-10T10:53:50Z

I'm trying to re-run the job with SSH and see what's wrong there

kopiczko · 2020-03-10T10:55:15Z

------------------------------------------------------------------------------------------------------------------------
 ✖︎ ignition-operator => (version: "0.0.1-67e7bf3548124c1320e8d11efd2b31d059897052", path: "helm/ignition-operator") > Error reading old chart version: Could not unmarshal 'Chart.yaml': yaml: unmarshal errors:
  line 3: cannot unmarshal !!seq into string
------------------------------------------------------------------------------------------------------------------------
68561456ea21:~/project# cat helm/ignition-operator/Chart.yaml
apiVersion: v1
name: ignition-operator
version: "0.0.1-67e7bf3548124c1320e8d11efd2b31d059897052"

So it's already templated and looks ok to me...

kopiczko · 2020-03-10T10:56:36Z

https://github.com/helm/chart-testing/blob/00d8fd274ac43d1020f5a57e11c2b9028aa854dd/pkg/chart/chart.go#L801-L804

kopiczko · 2020-03-10T11:02:31Z

68561456ea21:~/project# ct lint --validate-maintainers=false --charts helm/ignition-operator
Linting charts...
Version increment checking disabled.
------------------------------------------------------------------------------------------------------------------------
 Configuration
------------------------------------------------------------------------------------------------------------------------
Remote: origin
TargetBranch: master
BuildId:
LintConf: /etc/ct/lintconf.yaml
ChartYamlSchema: /etc/ct/chart_schema.yaml
ValidateMaintainers: false
ValidateChartSchema: true
ValidateYaml: true
CheckVersionIncrement: false
ProcessAllCharts: false
Charts: [helm/ignition-operator]
ChartRepos: []
ChartDirs: [charts]
ExcludedCharts: []
HelmExtraArgs:
HelmRepoExtraArgs: []
Debug: false
Upgrade: false
SkipMissingValues: false
Namespace:
ReleaseLabel:
------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------------------------------------------
 Charts to be processed:
------------------------------------------------------------------------------------------------------------------------
 ignition-operator => (version: "0.0.1-67e7bf3548124c1320e8d11efd2b31d059897052", path: "helm/ignition-operator")
------------------------------------------------------------------------------------------------------------------------

$HELM_HOME has been configured at /root/.helm.
Not installing Tiller due to 'client-only' flag having been set
No requirements found in helm/ignition-operator/charts.
Linting chart 'ignition-operator => (version: "0.0.1-67e7bf3548124c1320e8d11efd2b31d059897052", path: "helm/ignition-operator")'
Validating /root/project/helm/ignition-operator/Chart.yaml...

Error!
Schema: /etc/ct/chart_schema.yaml
Data file: /root/project/helm/ignition-operator/Chart.yaml
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/yamale/command_line.py", line 29, in _validate
    yamale.validate(schema, data)
  File "/usr/lib/python2.7/site-packages/yamale/yamale.py", line 39, in validate
    schema.validate(d)
  File "/usr/lib/python2.7/site-packages/yamale/schema/schema.py", line 56, in validate
    raise ValueError(error_str)
ValueError:
Error validating data /root/project/helm/ignition-operator/Chart.yaml with schema /etc/ct/chart_schema.yaml
        description: Required field missing
        appVersion: Required field missing
        home: Required field missing

Traceback (most recent call last):
  File "/usr/bin/yamale", line 11, in <module>
    load_entry_point('yamale==1.8.0', 'console_scripts', 'yamale')()
  File "/usr/lib/python2.7/site-packages/yamale/command_line.py", line 112, in main
    _router(args.path, args.schema, args.cpu_num, args.parser)
  File "/usr/lib/python2.7/site-packages/yamale/command_line.py", line 96, in _router
    _validate_single(root, schema_name, parser)
  File "/usr/lib/python2.7/site-packages/yamale/command_line.py", line 68, in _validate_single
    _validate(s, yaml_path, parser)
  File "/usr/lib/python2.7/site-packages/yamale/command_line.py", line 36, in _validate
    raise ValueError('Validation failed!')
ValueError: Validation failed!
Error linting charts: Error processing charts
------------------------------------------------------------------------------------------------------------------------
 ✖︎ ignition-operator => (version: "0.0.1-67e7bf3548124c1320e8d11efd2b31d059897052", path: "helm/ignition-operator") > Error waiting for process: exit status 1
------------------------------------------------------------------------------------------------------------------------

This gives some more information:

ValueError:
Error validating data /root/project/helm/ignition-operator/Chart.yaml with schema /etc/ct/chart_schema.yaml
        description: Required field missing
        appVersion: Required field missing
        home: Required field missing

Also:

68561456ea21:~/project# cat /etc/ct/chart_schema.yaml
name: str()
home: str()
version: str()
appVersion: any(str(), num())
description: str()
keywords: list(str(), required=False)
sources: list(str(), required=False)
maintainers: list(include('maintainer'), required=False)
icon: str(required=False)
engine: str(required=False)
condition: str(required=False)
tags: str(required=False)
deprecated: bool(required=False)
kubeVersion: str(required=False)
annotations: map(str(), str(), required=False)
---
maintainer:
  name: str()
  email: str(required=False)
  url: str(required=False)

yasn77 · 2020-03-10T11:05:10Z

I've raised a PR in architect-orb that disables version checking -> giantswarm/architect-orb#95

kopiczko · 2020-03-10T11:05:40Z

This pushed me a bit further:

68561456ea21:~/project# ct lint --validate-chart-schema=false --charts=helm/ignition-operator
Linting charts...
Version increment checking disabled.
------------------------------------------------------------------------------------------------------------------------
 Configuration
------------------------------------------------------------------------------------------------------------------------
Remote: origin
TargetBranch: master
BuildId:
LintConf: /etc/ct/lintconf.yaml
ChartYamlSchema: /etc/ct/chart_schema.yaml
ValidateMaintainers: true
ValidateChartSchema: false
ValidateYaml: true
CheckVersionIncrement: false
ProcessAllCharts: false
Charts: [helm/ignition-operator]
ChartRepos: []
ChartDirs: [charts]
ExcludedCharts: []
HelmExtraArgs:
HelmRepoExtraArgs: []
Debug: false
Upgrade: false
SkipMissingValues: false
Namespace:
ReleaseLabel:
------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------------------------------------------
 Charts to be processed:
------------------------------------------------------------------------------------------------------------------------
 ignition-operator => (version: "0.0.1-67e7bf3548124c1320e8d11efd2b31d059897052", path: "helm/ignition-operator")
------------------------------------------------------------------------------------------------------------------------

$HELM_HOME has been configured at /root/.helm.
Not installing Tiller due to 'client-only' flag having been set
No requirements found in helm/ignition-operator/charts.
Linting chart 'ignition-operator => (version: "0.0.1-67e7bf3548124c1320e8d11efd2b31d059897052", path: "helm/ignition-operator")'
Validating maintainers...
Error linting charts: Error processing charts
------------------------------------------------------------------------------------------------------------------------
 ✖︎ ignition-operator => (version: "0.0.1-67e7bf3548124c1320e8d11efd2b31d059897052", path: "helm/ignition-operator") > Chart doesn't have maintainers
------------------------------------------------------------------------------------------------------------------------
68561456ea21:~/project# ct lint --validate-maintainers=false --validate-chart-schema=false --charts=helm/ignition-operator
Linting charts...
Version increment checking disabled.
------------------------------------------------------------------------------------------------------------------------
 Configuration
------------------------------------------------------------------------------------------------------------------------
Remote: origin
TargetBranch: master
BuildId:
LintConf: /etc/ct/lintconf.yaml
ChartYamlSchema: /etc/ct/chart_schema.yaml
ValidateMaintainers: false
ValidateChartSchema: false
ValidateYaml: true
CheckVersionIncrement: false
ProcessAllCharts: false
Charts: [helm/ignition-operator]
ChartRepos: []
ChartDirs: [charts]
ExcludedCharts: []
HelmExtraArgs:
HelmRepoExtraArgs: []
Debug: false
Upgrade: false
SkipMissingValues: false
Namespace:
ReleaseLabel:
------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------------------------------------------
 Charts to be processed:
------------------------------------------------------------------------------------------------------------------------
 ignition-operator => (version: "0.0.1-67e7bf3548124c1320e8d11efd2b31d059897052", path: "helm/ignition-operator")
------------------------------------------------------------------------------------------------------------------------

$HELM_HOME has been configured at /root/.helm.
Not installing Tiller due to 'client-only' flag having been set
No requirements found in helm/ignition-operator/charts.
Linting chart 'ignition-operator => (version: "0.0.1-67e7bf3548124c1320e8d11efd2b31d059897052", path: "helm/ignition-operator")'
==> Linting helm/ignition-operator
[INFO] Chart.yaml: icon is recommended
[ERROR] templates/: render error in "ignition-operator/templates/pull-secret.yaml": template: ignition-operator/templates/pull-secret.yaml:9:31: executing "ignition-operator/templates/pull-secret.yaml" at <.Values.Installation.V1.Secret.Registry.PullSecret.DockerConfigJSON>: nil pointer evaluating interface {}.V1

Error: 1 chart(s) linted, 1 chart(s) failed
Error linting charts: Error processing charts
------------------------------------------------------------------------------------------------------------------------
 ✖︎ ignition-operator => (version: "0.0.1-67e7bf3548124c1320e8d11efd2b31d059897052", path: "helm/ignition-operator") > Error waiting for process: exit status 1
-

We should fix the job to use --charts and specify a single dir instead of --charts-dir. I don't know if disabling schema validation is good thing but it seems to work that way and then this next error...

kopiczko · 2020-03-10T11:06:16Z

I guess SkipMissingValues would fix above

kopiczko · 2020-03-10T11:10:27Z

Yasser 11:09
I raised a PR last night giantswarm/architect-orb#95. It should fi the version issue

Oops he already mentioned that #37 (comment) :)

yasn77 · 2020-03-10T11:27:38Z

Question is, do we care about those missing values? I mean the linter is doing its thing... The values are needed for a Helm Chart to be 'accurate'

If we don't need them, I think then it's better to introduce an option to either use a config file for ct or override options.

My initial PR for ct did have an option to use a config file... I can reintroduce it.

kopiczko · 2020-03-10T13:17:01Z

We can't avoid having missing .Values.Installation values. Maybe besides managed apps. So we need to adapt the job to work in those. I'd rather have on/off switch for that in the job for now (like ignore-missing-values: "true"). It's just a thing for charts templated with installations. I'd like to avoid copying over the same config to 90% of our repos.

yasn77 · 2020-03-10T15:51:16Z

Regarding this error:

==> Linting helm/ignition-operator
[INFO] Chart.yaml: icon is recommended
[ERROR] templates/: render error in "ignition-operator/templates/pull-secret.yaml": template: ignition-operator/templates/pull-secret.yaml:9:31: executing "ignition-operator/templates/pull-secret.yaml" at <.Values.Installation.V1.Secret.Registry.PullSecret.DockerConfigJSON>: nil pointer evaluating interface {}.V1

This is a renderer error and I couldn't see a way to bypass it with a switch to ct. However one of the benefits of ct is that you can easily test/lint the chart against multiple values files. So it is possible to prevent this error from happening by supplying a test values file. To do this create a new file helm/ignition-operator/ci/default-values.yaml with the following contents:

---
Installation:
  V1:
    Secret:
      Registry:
        PullSecret:
          DockerConfigJSON: SomeString

Lint should work fine after this:

# ct lint --validate-maintainers=false --check-version-increment=false --validate-chart-schema=false --charts ignition-operator
Linting charts...
Version increment checking disabled.
------------------------------------------------------------------------------------------------------------------------
 Configuration
------------------------------------------------------------------------------------------------------------------------
Remote: origin
TargetBranch: master
BuildId: 
LintConf: /etc/ct/lintconf.yaml
ChartYamlSchema: /etc/ct/chart_schema.yaml
ValidateMaintainers: false
ValidateChartSchema: false
ValidateYaml: true
CheckVersionIncrement: false
ProcessAllCharts: false
Charts: [ignition-operator]
ChartRepos: []
ChartDirs: [charts]
ExcludedCharts: []
HelmExtraArgs: 
HelmRepoExtraArgs: []
Debug: false
Upgrade: false
SkipMissingValues: false
Namespace: 
ReleaseLabel: 
------------------------------------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------------------------------------------
 Charts to be processed:
------------------------------------------------------------------------------------------------------------------------
 ignition-operator => (version: "1.0", path: "ignition-operator")
------------------------------------------------------------------------------------------------------------------------

$HELM_HOME has been configured at /root/.helm.
Not installing Tiller due to 'client-only' flag having been set
No requirements found in ignition-operator/charts.
Linting chart 'ignition-operator => (version: "1.0", path: "ignition-operator")'

Linting chart with values file 'ignition-operator/ci/default-values.yaml'...

==> Linting ignition-operator
[INFO] Chart.yaml: icon is recommended

1 chart(s) linted, no failures
All charts linted successfully
------------------------------------------------------------------------------------------------------------------------
 ✔︎ ignition-operator => (version: "1.0", path: "ignition-operator")
------------------------------------------------------------------------------------------------------------------------

I'll continue to create a PR which allows the user to make the tests less strict. But skipping missing values will require a ci values file.

yasn77 · 2020-03-11T12:47:55Z

@stone-z I would suggest closing this and waiting for the next orb release. I've also opened #42 which will help when the orb is released.

stone-z · 2020-03-17T12:55:24Z

Replaced by #47

stone-z added 7 commits March 5, 2020 16:21

Test lint job with ignition-operator

e150902

Comment to trigger CI

8540a80

Comment for circle

601ab52

Make Circle run

32998ca

Use architect-orb v0.8.1

0e4f92b

Clean up initial ignition-operator lint findings

55ac165

Handle errors on entry to walk func

709148c

stone-z requested a review from a team March 9, 2020 16:18

stone-z self-assigned this Mar 9, 2020

tfussell approved these changes Mar 9, 2020

View reviewed changes

MarcelMue added 2 commits March 10, 2020 09:18

Merge branch 'master' into test-lint-in-gotest

c0edfcb

fix key func

679a973

Use quotation marks?

67e7bf3

yasn77 mentioned this pull request Mar 10, 2020

Release 8.0.2 giantswarm/architect-orb#97

Merged

2 tasks

yasn77 mentioned this pull request Mar 10, 2020

Options to disable and make helm lint less strict giantswarm/architect-orb#99

Closed

2 tasks

stone-z mentioned this pull request Mar 17, 2020

Use architect-orb 0.8.3 #47

Merged

stone-z closed this Mar 17, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use architect 0.8.1 #37

Use architect 0.8.1 #37

stone-z commented Mar 9, 2020

tfussell left a comment

MarcelMue commented Mar 9, 2020

tfussell commented Mar 9, 2020

MarcelMue commented Mar 10, 2020

stone-z commented Mar 10, 2020

stone-z commented Mar 10, 2020

MarcelMue commented Mar 10, 2020

stone-z commented Mar 10, 2020

stone-z commented Mar 10, 2020 •

edited

Loading

kopiczko commented Mar 10, 2020

stone-z commented Mar 10, 2020

stone-z commented Mar 10, 2020

kopiczko commented Mar 10, 2020

kopiczko commented Mar 10, 2020

kopiczko commented Mar 10, 2020

kopiczko commented Mar 10, 2020

yasn77 commented Mar 10, 2020

kopiczko commented Mar 10, 2020

kopiczko commented Mar 10, 2020

kopiczko commented Mar 10, 2020 •

edited

Loading

yasn77 commented Mar 10, 2020

kopiczko commented Mar 10, 2020

yasn77 commented Mar 10, 2020

yasn77 commented Mar 11, 2020

stone-z commented Mar 17, 2020

Use architect 0.8.1 #37

Use architect 0.8.1 #37

Conversation

stone-z commented Mar 9, 2020

tfussell left a comment

Choose a reason for hiding this comment

MarcelMue commented Mar 9, 2020

tfussell commented Mar 9, 2020

MarcelMue commented Mar 10, 2020

stone-z commented Mar 10, 2020

stone-z commented Mar 10, 2020

MarcelMue commented Mar 10, 2020

stone-z commented Mar 10, 2020

stone-z commented Mar 10, 2020 • edited Loading

kopiczko commented Mar 10, 2020

stone-z commented Mar 10, 2020

stone-z commented Mar 10, 2020

kopiczko commented Mar 10, 2020

kopiczko commented Mar 10, 2020

kopiczko commented Mar 10, 2020

kopiczko commented Mar 10, 2020

yasn77 commented Mar 10, 2020

kopiczko commented Mar 10, 2020

kopiczko commented Mar 10, 2020

kopiczko commented Mar 10, 2020 • edited Loading

yasn77 commented Mar 10, 2020

kopiczko commented Mar 10, 2020

yasn77 commented Mar 10, 2020

yasn77 commented Mar 11, 2020

stone-z commented Mar 17, 2020

stone-z commented Mar 10, 2020 •

edited

Loading

kopiczko commented Mar 10, 2020 •

edited

Loading