Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Regression on 0.6.3: could not find schema due to Draft 4 being used #223

Open
carlossg opened this issue Jul 18, 2023 · 11 comments
Open

Regression on 0.6.3: could not find schema due to Draft 4 being used #223

carlossg opened this issue Jul 18, 2023 · 11 comments

Comments

@carlossg
Copy link
Contributor

Hi, trying 0.6.3 I noticed a regression, caused by #221

with 0.6.3 it breaks

❯ ~/Downloads/kubeconform-063/kubeconform -debug -strict -schema-location target/kubernetes-json-schema/master target/test.yaml
2023/07/18 20:04:30 using schema found at target/kubernetes-json-schema/master/master-standalone-strict/verticalpodautoscaler-autoscaling-v1beta2.json
target/test.yaml - VerticalPodAutoscaler toggleservice-vpa failed validation: could not find schema for VerticalPodAutoscaler

with 0.6.2 it worked fine

❯ ~/Downloads/kubeconform-062/kubeconform -debug -strict -schema-location target/kubernetes-json-schema/master target/test.yaml
2023/07/18 20:04:34 using schema found at target/kubernetes-json-schema/master/master-standalone-strict/verticalpodautoscaler-autoscaling-v1beta2.json

what happens is that the schema is not valid in draft 4 but it is in the default 2020 schema

2023/07/18 20:28:18 error jsonschema file:///.../target/kubernetes-json-schema/master/master-standalone-strict/verticalpodautoscaler-autoscaling-v1beta2.json compilation failed: '/properties/spec/required' does not validate with http://json-schema.org/draft-04/schema#/properties/properties/additionalProperties/$ref/properties/required/$ref/minItems: minimum 1 items required, but found 0 items

from #217 (comment) I assume we need to fix our schemas to be Draft 4 ?

@carlossg
Copy link
Contributor Author

if that's the case it would be good to push this to a 0.7.0 version at least and have a changelog warning

@yannh
Copy link
Owner

yannh commented Jul 18, 2023

i guess it should use the draft version specified in the schema and only fallback to draft 4 🤔

@carlossg
Copy link
Contributor Author

I've tried with some public schemas and it works, so I'll change ours.
It would be good to have more debug entries inside downloadSchema. I see that the debug option is not available there to conditionally print logs

❯ ~/dev/yannh/kubeconform/bin/kubeconform -debug -strict -schema-location 'https://raw.githubusercontent.com/zapier/kubechecks/main/schemas/v1.23.0/{{ .ResourceKind }}{{ .KindSuffix }}.json' target/test.yaml
2023/07/18 20:44:08 using schema found at https://raw.githubusercontent.com/zapier/kubechecks/main/schemas/v1.23.0/verticalpodautoscaler-autoscaling-v1beta2.json

@carlossg
Copy link
Contributor Author

@yannh I cannot find many sources of autoscaler or its swagger definition, do you know where to get them? maybe to contribute them to https://github.com/yannh/kubernetes-json-schema/

@yannh
Copy link
Owner

yannh commented Sep 28, 2023

I agree it would be good to have a way to have debug info in downloadschema, but I ve found it can be really verbose since well, by architecture, when you use multiple schema repositories it will try to find every file in every repository until it finds it...

@yannh
Copy link
Owner

yannh commented Sep 28, 2023

@carlossg any chance you could share the schema so I could reproduce the original issue? I feel like Kubeconform should complain if it downloads an invalid schema!

@carlossg
Copy link
Contributor Author

it was this one I think

{
  "type": "object",
  "properties": {
    "spec": {
      "type": "object",
      "required": [],
      "properties": {
        "targetRef": {
          "type": "object"
        },
        "updatePolicy": {
          "type": "object",
          "properties": {
            "minReplicas": {
              "type": "integer"
            },
            "updateMode": {
              "type": "string"
            }
          },
          "additionalProperties": false
        },
        "resourcePolicy": {
          "type": "object",
          "properties": {
            "containerPolicies": {
              "type": "array",
              "items": {
                "type": "object",
                "properties": {
                  "containerName": {
                    "type": "string"
                  },
                  "controlledValues": {
                    "type": "string",
                    "enum": [
                      "RequestsAndLimits",
                      "RequestsOnly"
                    ]
                  },
                  "mode": {
                    "type": "string",
                    "enum": [
                      "Auto",
                      "Off"
                    ]
                  },
                  "minAllowed": {
                    "type": "object"
                  },
                  "maxAllowed": {
                    "type": "object"
                  },
                  "controlledResources": {
                    "type": "array",
                    "items": {
                      "type": "string",
                      "enum": [
                        "cpu",
                        "memory"
                      ]
                    }
                  }
                },
                "additionalProperties": false
              }
            }
          },
          "additionalProperties": false
        }
      },
      "additionalProperties": false
    }
  }
}

@rino-pupkin
Copy link

Can confirm the issue.
Looks like it found a correct schema but throws error:

root@gitlab-runner-vm:/home/gitlab-runner# cat tmp.yaml | kubeconform -kubernetes-version 1.28.1 -schema-location default -schema-location '/home/gitlab-runner/yamltests/schemas/{{ .ResourceKind }}_{{ .ResourceAPIVersion }}.json' -n 1 -summary -strict -skip Certificate  --debug
2023/10/31 14:32:48 using schema found at https://raw.githubusercontent.com/yannh/kubernetes-json-schema/master/v1.28.1-standalone-strict/configmap-v1.json
2023/10/31 14:32:48 using schema found at https://raw.githubusercontent.com/yannh/kubernetes-json-schema/master/v1.28.1-standalone-strict/service-v1.json
2023/10/31 14:32:48 using schema found at https://raw.githubusercontent.com/yannh/kubernetes-json-schema/master/v1.28.1-standalone-strict/deployment-apps-v1.json
2023/10/31 14:32:48 using schema found at https://raw.githubusercontent.com/yannh/kubernetes-json-schema/master/v1.28.1-standalone-strict/ingress-networking-v1.json
2023/10/31 14:32:48 could not find schema at https://raw.githubusercontent.com/yannh/kubernetes-json-schema/master/v1.28.1-standalone-strict/servicemonitor-monitoring-v1.json
2023/10/31 14:32:48 using schema found at /home/gitlab-runner/yamltests/schemas/servicemonitor_v1.json
2023/10/31 14:32:48 could not find schema at https://raw.githubusercontent.com/yannh/kubernetes-json-schema/master/v1.28.1-standalone-strict/verticalpodautoscaler-autoscaling-v1.json
2023/10/31 14:32:48 using schema found at /home/gitlab-runner/yamltests/schemas/verticalpodautoscaler_v1.json
stdin - VerticalPodAutoscaler xxx failed validation: could not find schema for VerticalPodAutoscaler
2023/10/31 14:32:48 using schema found at https://raw.githubusercontent.com/yannh/kubernetes-json-schema/master/v1.28.1-standalone-strict/job-batch-v1.json
Summary: 11 resources found parsing stdin - Valid: 9, Invalid: 0, Errors: 1, Skipped: 1

root@gitlab-runner-vm:/home/gitlab-runner# echo $?
1

v0.6.2 works great
we use this schema to validate vpa
https://raw.githubusercontent.com/kubernetes/autoscaler/master/vertical-pod-autoscaler/deploy/vpa-v1-crd.yaml

and this to validate servicemonitor
https://raw.githubusercontent.com/prometheus-operator/prometheus-operator/release-0.60/example/prometheus-operator-crd-full/monitoring.coreos.com_servicemonitors.yaml

we use openapi2jsonschema.py to downlad and convert schemas into json.

@dctrwatson
Copy link

@yannh
Copy link
Owner

yannh commented Nov 18, 2023

👋 My understanding is that the schemas should be in draft 4... Am I mistaken? See #217 (comment) - I would close this ticket unless a lot of public CRDs aren't in that format?

@megamorf
Copy link

Hey guys, I'd like to report something that is somewhat related. It might help other users who struggled with this like we did.

Starting with v0.6.3 we got the "could not find schema" for an RDSInstance resource:

RDSInstance dcpc-commerce-markets failed validation: could not find schema for RDSInstance

I checked with:

  • v0.6.2 - ok
  • v0.6.3 - not found
  • v0.6.4 - not found
  • v0.6.5 - not found
  • v0.6.6 - not found

In our case this is a crossplane resource and the CRD is managed by a different team. I converted the CRD spect to json-schema format and added the validation fields:

  {
+   "id": "http://json-schema.org/draft-04/schema#",
+   "$schema": "http://json-schema.org/draft-04/schema#",
    "properties": {
      "apiVersion": {
        "type": "string"
      },
      "kind": {
        "type": "string"
      },
  [...]

As it turns out, there was an enum list with a duplicate in it:

image

As soon as the duplicate was removed the could not find schema error disappeared and the validation started working on all recent versions:

Before

KUBECONFORM [0.6.2]
2024/06/10 16:22:47 using schema found at /Users/megamorf/projects/myapp/app-k8s-crds-jsonschemas/caas-v3/schemas/aws.contoso.com/rdsinstance_v1alpha1.json
/Users/megamorf/projects/myapp/app-infrastructure/clusters/app-dev/rdsinstance-dcpc-commerce-markets.yaml - RDSInstance dcpc-commerce-markets is valid
Summary: 3 resources found in 1 file - Valid: 3, Invalid: 0, Errors: 0, Skipped: 0

KUBECONFORM [0.6.3]
2024/06/10 16:22:47 using schema found at /Users/megamorf/projects/myapp/app-k8s-crds-jsonschemas/caas-v3/schemas/aws.contoso.com/rdsinstance_v1alpha1.json
/Users/megamorf/projects/myapp/app-infrastructure/clusters/app-dev/rdsinstance-dcpc-commerce-markets.yaml - RDSInstance dcpc-commerce-markets failed validation: could not find schema for RDSInstance
Summary: 3 resources found in 1 file - Valid: 2, Invalid: 0, Errors: 1, Skipped: 0

KUBECONFORM [0.6.4]
2024/06/10 16:22:47 using schema found at /Users/megamorf/projects/myapp/app-k8s-crds-jsonschemas/caas-v3/schemas/aws.contoso.com/rdsinstance_v1alpha1.json
/Users/megamorf/projects/myapp/app-infrastructure/clusters/app-dev/rdsinstance-dcpc-commerce-markets.yaml - RDSInstance dcpc-commerce-markets failed validation: could not find schema for RDSInstance
Summary: 3 resources found in 1 file - Valid: 2, Invalid: 0, Errors: 1, Skipped: 0

KUBECONFORM [0.6.5]
2024/06/10 16:22:47 using schema found at /Users/megamorf/projects/myapp/app-k8s-crds-jsonschemas/caas-v3/schemas/aws.contoso.com/rdsinstance_v1alpha1.json
/Users/megamorf/projects/myapp/app-infrastructure/clusters/app-dev/rdsinstance-dcpc-commerce-markets.yaml - RDSInstance dcpc-commerce-markets failed validation: could not find schema for RDSInstance
Summary: 3 resources found in 1 file - Valid: 2, Invalid: 0, Errors: 1, Skipped: 0

KUBECONFORM [0.6.6]
2024/06/10 16:22:47 using schema found at /Users/megamorf/projects/myapp/app-k8s-crds-jsonschemas/caas-v3/schemas/aws.contoso.com/rdsinstance_v1alpha1.json
/Users/megamorf/projects/myapp/app-infrastructure/clusters/app-dev/rdsinstance-dcpc-commerce-markets.yaml - RDSInstance dcpc-commerce-markets failed validation: could not find schema for RDSInstance
Summary: 3 resources found in 1 file - Valid: 2, Invalid: 0, Errors: 1, Skipped: 0

After

KUBECONFORM [0.6.2]
2024/06/10 16:22:47 using schema found at /Users/megamorf/projects/myapp/app-k8s-crds-jsonschemas/caas-v3/schemas/aws.contoso.com/rdsinstance_v1alpha1.json
/Users/megamorf/projects/myapp/app-infrastructure/clusters/app-dev/rdsinstance-dcpc-commerce-markets.yaml - RDSInstance dcpc-commerce-markets is valid
Summary: 3 resources found in 1 file - Valid: 3, Invalid: 0, Errors: 0, Skipped: 0

KUBECONFORM [0.6.3]
2024/06/10 16:22:47 using schema found at /Users/megamorf/projects/myapp/app-k8s-crds-jsonschemas/caas-v3/schemas/aws.contoso.com/rdsinstance_v1alpha1.json
/Users/megamorf/projects/myapp/app-infrastructure/clusters/app-dev/rdsinstance-dcpc-commerce-markets.yaml - RDSInstance dcpc-commerce-markets is valid
Summary: 3 resources found in 1 file - Valid: 3, Invalid: 0, Errors: 0, Skipped: 0

KUBECONFORM [0.6.4]
2024/06/10 16:22:47 using schema found at /Users/megamorf/projects/myapp/app-k8s-crds-jsonschemas/caas-v3/schemas/aws.contoso.com/rdsinstance_v1alpha1.json
/Users/megamorf/projects/myapp/app-infrastructure/clusters/app-dev/rdsinstance-dcpc-commerce-markets.yaml - RDSInstance dcpc-commerce-markets is valid
Summary: 3 resources found in 1 file - Valid: 3, Invalid: 0, Errors: 0, Skipped: 0

KUBECONFORM [0.6.5]
2024/06/10 16:22:47 using schema found at /Users/megamorf/projects/myapp/app-k8s-crds-jsonschemas/caas-v3/schemas/aws.contoso.com/rdsinstance_v1alpha1.json
/Users/megamorf/projects/myapp/app-infrastructure/clusters/app-dev/rdsinstance-dcpc-commerce-markets.yaml - RDSInstance dcpc-commerce-markets is valid
Summary: 3 resources found in 1 file - Valid: 3, Invalid: 0, Errors: 0, Skipped: 0

KUBECONFORM [0.6.6]
2024/06/10 16:22:47 using schema found at /Users/megamorf/projects/myapp/app-k8s-crds-jsonschemas/caas-v3/schemas/aws.contoso.com/rdsinstance_v1alpha1.json
/Users/megamorf/projects/myapp/app-infrastructure/clusters/app-dev/rdsinstance-dcpc-commerce-markets.yaml - RDSInstance dcpc-commerce-markets is valid
Summary: 3 resources found in 1 file - Valid: 3, Invalid: 0, Errors: 0, Skipped: 0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants