Mark Conditions in CustomResourceDefinitionStatus as optional #64996

roycaihw · 2018-06-12T00:37:55Z

What this PR does / why we need it:
Apiserver may respond null for required property Conditions in metav1.CustomResourceDefinitionStatus (e.g. create a crd with kubectl --v=9 and check the response), which fails the validation in generated python client kubernetes-client/python#415. This PR marks the field as optional.

Release note:

Property `conditions` in `apiextensions.v1beta1.CustomResourceDefinitionStatus` and `apiextensions.v1.CustomResourceDefinitionStatus` is now optional instead of required.

/sig api-machinery
/area custom-resources

roycaihw · 2018-06-12T23:07:09Z

/assign @mbohlool @sttts

sttts · 2018-06-13T08:52:11Z

Seeing this being inconsistent for a number of types (when searching for Conditions [] in kube). Can we fix this everywhere?

roycaihw · 2018-06-13T21:46:48Z

@sttts Did a search under staging/src/k8s.io/api. 20 out of 21 Conditions fields have the omitempty json tag. 16 have the +optional tag. The openapi generator considers a field "optional" if either tag exists.

The problem is that we don't (and probably cannot?) have API conventions to reason about if an arbitrary API field should be optional, so we have to fix them one by one.

kubernetes/staging/src/k8s.io/api$ grep -r "Conditions \[\]" -B 4
extensions/v1beta1/types.go-
extensions/v1beta1/types.go-	// Represents the latest available observations of a deployment's current state.
extensions/v1beta1/types.go-	// +patchMergeKey=type
extensions/v1beta1/types.go-	// +patchStrategy=merge
extensions/v1beta1/types.go:	Conditions []DeploymentCondition `json:"conditions,omitempty" patchStrategy:"merge" patchMergeKey:"type" protobuf:"bytes,6,rep,name=conditions"`
--
extensions/v1beta1/types.go-	// Represents the latest available observations of a DaemonSet's current state.
extensions/v1beta1/types.go-	// +optional
extensions/v1beta1/types.go-	// +patchMergeKey=type
extensions/v1beta1/types.go-	// +patchStrategy=merge
extensions/v1beta1/types.go:	Conditions []DaemonSetCondition `json:"conditions,omitempty" patchStrategy:"merge" patchMergeKey:"type" protobuf:"bytes,10,rep,name=conditions"`
--
extensions/v1beta1/types.go-	// Represents the latest available observations of a replica set's current state.
extensions/v1beta1/types.go-	// +optional
extensions/v1beta1/types.go-	// +patchMergeKey=type
extensions/v1beta1/types.go-	// +patchStrategy=merge
extensions/v1beta1/types.go:	Conditions []ReplicaSetCondition `json:"conditions,omitempty" patchStrategy:"merge" patchMergeKey:"type" protobuf:"bytes,6,rep,name=conditions"`
--
batch/v1/types.go-	// More info: https://kubernetes.io/docs/concepts/workloads/controllers/jobs-run-to-completion/
batch/v1/types.go-	// +optional
batch/v1/types.go-	// +patchMergeKey=type
batch/v1/types.go-	// +patchStrategy=merge
batch/v1/types.go:	Conditions []JobCondition `json:"conditions,omitempty" patchStrategy:"merge" patchMergeKey:"type" protobuf:"bytes,1,rep,name=conditions"`
--
certificates/v1beta1/types.go-
certificates/v1beta1/types.go-type CertificateSigningRequestStatus struct {
certificates/v1beta1/types.go-	// Conditions applied to the request, such as approval or denial.
certificates/v1beta1/types.go-	// +optional
certificates/v1beta1/types.go:	Conditions []CertificateSigningRequestCondition `json:"conditions,omitempty" protobuf:"bytes,1,rep,name=conditions"`
--
apps/v1/types.go-	// Represents the latest available observations of a statefulset's current state.
apps/v1/types.go-	// +optional
apps/v1/types.go-	// +patchMergeKey=type
apps/v1/types.go-	// +patchStrategy=merge
apps/v1/types.go:	Conditions []StatefulSetCondition `json:"conditions,omitempty" patchStrategy:"merge" patchMergeKey:"type" protobuf:"bytes,10,rep,name=conditions"`
--
apps/v1/types.go-
apps/v1/types.go-	// Represents the latest available observations of a deployment's current state.
apps/v1/types.go-	// +patchMergeKey=type
apps/v1/types.go-	// +patchStrategy=merge
apps/v1/types.go:	Conditions []DeploymentCondition `json:"conditions,omitempty" patchStrategy:"merge" patchMergeKey:"type" protobuf:"bytes,6,rep,name=conditions"`
--
apps/v1/types.go-	// Represents the latest available observations of a DaemonSet's current state.
apps/v1/types.go-	// +optional
apps/v1/types.go-	// +patchMergeKey=type
apps/v1/types.go-	// +patchStrategy=merge
apps/v1/types.go:	Conditions []DaemonSetCondition `json:"conditions,omitempty" patchStrategy:"merge" patchMergeKey:"type" protobuf:"bytes,10,rep,name=conditions"`
--
apps/v1/types.go-	// Represents the latest available observations of a replica set's current state.
apps/v1/types.go-	// +optional
apps/v1/types.go-	// +patchMergeKey=type
apps/v1/types.go-	// +patchStrategy=merge
apps/v1/types.go:	Conditions []ReplicaSetCondition `json:"conditions,omitempty" patchStrategy:"merge" patchMergeKey:"type" protobuf:"bytes,6,rep,name=conditions"`
--
apps/v1beta1/types.go-	// Represents the latest available observations of a statefulset's current state.
apps/v1beta1/types.go-	// +optional
apps/v1beta1/types.go-	// +patchMergeKey=type
apps/v1beta1/types.go-	// +patchStrategy=merge
apps/v1beta1/types.go:	Conditions []StatefulSetCondition `json:"conditions,omitempty" patchStrategy:"merge" patchMergeKey:"type" protobuf:"bytes,10,rep,name=conditions"`
--
apps/v1beta1/types.go-
apps/v1beta1/types.go-	// Represents the latest available observations of a deployment's current state.
apps/v1beta1/types.go-	// +patchMergeKey=type
apps/v1beta1/types.go-	// +patchStrategy=merge
apps/v1beta1/types.go:	Conditions []DeploymentCondition `json:"conditions,omitempty" patchStrategy:"merge" patchMergeKey:"type" protobuf:"bytes,6,rep,name=conditions"`
--
apps/v1beta2/types.go-	// Represents the latest available observations of a statefulset's current state.
apps/v1beta2/types.go-	// +optional
apps/v1beta2/types.go-	// +patchMergeKey=type
apps/v1beta2/types.go-	// +patchStrategy=merge
apps/v1beta2/types.go:	Conditions []StatefulSetCondition `json:"conditions,omitempty" patchStrategy:"merge" patchMergeKey:"type" protobuf:"bytes,10,rep,name=conditions"`
--
apps/v1beta2/types.go-
apps/v1beta2/types.go-	// Represents the latest available observations of a deployment's current state.
apps/v1beta2/types.go-	// +patchMergeKey=type
apps/v1beta2/types.go-	// +patchStrategy=merge
apps/v1beta2/types.go:	Conditions []DeploymentCondition `json:"conditions,omitempty" patchStrategy:"merge" patchMergeKey:"type" protobuf:"bytes,6,rep,name=conditions"`
--
apps/v1beta2/types.go-	// Represents the latest available observations of a DaemonSet's current state.
apps/v1beta2/types.go-	// +optional
apps/v1beta2/types.go-	// +patchMergeKey=type
apps/v1beta2/types.go-	// +patchStrategy=merge
apps/v1beta2/types.go:	Conditions []DaemonSetCondition `json:"conditions,omitempty" patchStrategy:"merge" patchMergeKey:"type" protobuf:"bytes,10,rep,name=conditions"`
--
apps/v1beta2/types.go-	// Represents the latest available observations of a replica set's current state.
apps/v1beta2/types.go-	// +optional
apps/v1beta2/types.go-	// +patchMergeKey=type
apps/v1beta2/types.go-	// +patchStrategy=merge
apps/v1beta2/types.go:	Conditions []ReplicaSetCondition `json:"conditions,omitempty" patchStrategy:"merge" patchMergeKey:"type" protobuf:"bytes,6,rep,name=conditions"`
--
autoscaling/v2beta1/types.go-	CurrentMetrics []MetricStatus `json:"currentMetrics" protobuf:"bytes,5,rep,name=currentMetrics"`
autoscaling/v2beta1/types.go-
autoscaling/v2beta1/types.go-	// conditions is the set of conditions required for this autoscaler to scale its target,
autoscaling/v2beta1/types.go-	// and indicates whether or not those conditions are met.
autoscaling/v2beta1/types.go:	Conditions []HorizontalPodAutoscalerCondition `json:"conditions" protobuf:"bytes,6,rep,name=conditions"`
--
core/v1/types.go-	// resized then the Condition will be set to 'ResizeStarted'.
core/v1/types.go-	// +optional
core/v1/types.go-	// +patchMergeKey=type
core/v1/types.go-	// +patchStrategy=merge
core/v1/types.go:	Conditions []PersistentVolumeClaimCondition `json:"conditions,omitempty" patchStrategy:"merge" patchMergeKey:"type" protobuf:"bytes,4,rep,name=conditions"`
--
core/v1/types.go-	// More info: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle#pod-conditions
core/v1/types.go-	// +optional
core/v1/types.go-	// +patchMergeKey=type
core/v1/types.go-	// +patchStrategy=merge
core/v1/types.go:	Conditions []PodCondition `json:"conditions,omitempty" patchStrategy:"merge" patchMergeKey:"type" protobuf:"bytes,2,rep,name=conditions"`
--
core/v1/types.go-	// Represents the latest available observations of a replication controller's current state.
core/v1/types.go-	// +optional
core/v1/types.go-	// +patchMergeKey=type
core/v1/types.go-	// +patchStrategy=merge
core/v1/types.go:	Conditions []ReplicationControllerCondition `json:"conditions,omitempty" patchStrategy:"merge" patchMergeKey:"type" protobuf:"bytes,6,rep,name=conditions"`
--
core/v1/types.go-	// More info: https://kubernetes.io/docs/concepts/nodes/node/#condition
core/v1/types.go-	// +optional
core/v1/types.go-	// +patchMergeKey=type
core/v1/types.go-	// +patchStrategy=merge
core/v1/types.go:	Conditions []NodeCondition `json:"conditions,omitempty" patchStrategy:"merge" patchMergeKey:"type" protobuf:"bytes,4,rep,name=conditions"`
--
core/v1/types.go-	// List of component conditions observed
core/v1/types.go-	// +optional
core/v1/types.go-	// +patchMergeKey=type
core/v1/types.go-	// +patchStrategy=merge
core/v1/types.go:	Conditions []ComponentCondition `json:"conditions,omitempty" patchStrategy:"merge" patchMergeKey:"type" protobuf:"bytes,2,rep,name=conditions"`

sttts · 2018-06-20T08:59:08Z

The problem is that we don't (and probably cannot?) have API conventions to reason about if an arbitrary API field should be optional, so we have to fix them one by one.

Which means a number of follow-ups?

sttts · 2018-06-20T09:05:59Z

@roycaihw can you formulate the release-note in JSON terms to be understandable by users?

Otherwise, lgtm.

sttts · 2018-06-20T09:06:08Z

/lgtm
/approve

lavalamp · 2018-06-21T21:35:08Z

/approve

I edited the release note.

roycaihw · 2018-06-22T00:37:51Z

I edited the release note.

@lavalamp Thanks. I was thinking about the point that @smarterclayton made in #65041 (comment). Marking these beta fields omitempty does look backwards incompatible to me if client expects to see the key in the json response from apiserver (even if the value used to be null). Any thought?

/hold

Which means a number of follow-ups?

@sttts Yes I can file an issue for those Conditions [] but I would like more inputs on the convention that we want to enforce (e.g. every Conditions in <RESOURCE>Status should be +optional and omitempty?). Some other API fields tracked in kubernetes-client/gen#52

lavalamp · 2018-06-22T21:56:45Z

I understand apiserver already omits the key completely if the list is empty, and is therefore violating the spec, correct? (per kubernetes-client/gen#52) Is this true for json, proto, both, or neither? Please confirm this before we proceed onto the next step :)

Assuming we are in violation of the spec, we can either fix the spec or fix our behavior. I think fixing the spec is less disruptive overall. Therefore, I agree with this change and think it is allowable, pending double-checking today's behavior.

smarterclayton · 2018-06-24T19:42:30Z

When you say fixing spec, you mean "tell people to just set +optional in their Godoc without setting omitempty, document that existing fields should never be marked omitempty once they hit beta, and new fields can continue to be created as omitempty"? That would still mark the field optional in OpenAPI, and would preserve the semantics for users.

k8s-github-robot · 2018-07-14T08:15:52Z

/test all

Tests are more than 96 hours old. Re-running tests.

fejta-bot · 2018-10-12T09:24:02Z

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

fejta-bot · 2018-11-11T10:11:34Z

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten

fejta-bot · 2019-08-14T01:21:31Z

This PR may require API review.

If so, when the changes are ready, complete the pre-review checklist and request an API review.

Status of requested reviews is tracked in the API Review project.

staging/src/k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1beta1/types.go

liggitt · 2019-08-19T12:52:09Z

needs update to include v1 API type as well

roycaihw · 2019-08-20T21:35:09Z

needs update to include v1 API type as well

thanks. Updated

roycaihw · 2019-08-20T22:35:38Z

/retest

flake:

subresources_test.go:689: unable to update status: Operation cannot be fulfilled on noxus.mygroup.example.com "foo": StorageError: invalid object, Code: 4, Key: /6ce03995-b641-4b07-b305-99a9a6878ed0/mygroup.example.com/noxus/not-the-default/foo, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: ee04977e-9768-4062-87e8-f1000a00570e, UID in object meta: 613ac225-6e00-478a-aae7-b6b3828820c9

quite rare according to: https://storage.googleapis.com/k8s-gubernator/triage/index.html?pr=1&text=Precondition%20failed%3A%20UID%20in%20precondition

roycaihw · 2019-08-21T00:29:18Z

/retest

missed / in my previous comment

liggitt · 2019-08-21T02:56:28Z

/lgtm
/approve

k8s-ci-robot · 2019-08-21T02:57:02Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: lavalamp, liggitt, roycaihw, sttts

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~api/OWNERS~~ [lavalamp,liggitt]
~~staging/src/k8s.io/apiextensions-apiserver/pkg/apis/OWNERS~~ [lavalamp,liggitt]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

fejta-bot · 2019-08-21T07:04:00Z

/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to fejta).

Review the full test history for this PR.

Silence the bot with an /lgtm cancel or /hold comment for consistent failures.

fejta-bot · 2019-08-21T13:01:00Z

/retest
This bot automatically retries jobs that failed/flaked on approved PRs (send feedback to fejta).

Review the full test history for this PR.

Silence the bot with an /lgtm cancel or /hold comment for consistent failures.

k8s-ci-robot requested review from enisoc and luxas June 12, 2018 00:38

roycaihw mentioned this pull request Jun 12, 2018

Required arrays with null value kubernetes-client/gen#52

Open

k8s-ci-robot assigned mbohlool and sttts Jun 12, 2018

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jun 20, 2018

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jun 21, 2018

k8s-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jun 22, 2018

lavalamp mentioned this pull request Jun 22, 2018

Mark DisruptedPods in PodDisruptionBudgetStatus as optional #65041

Closed

k8s-ci-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Oct 12, 2018

k8s-ci-robot added lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. and removed lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. labels Nov 11, 2018

k8s-ci-robot added priority/important-soon Must be staffed and worked on either currently, or very soon, ideally in time for the next release. and removed needs-priority Indicates a PR lacks a `priority/foo` label and requires one. labels Aug 13, 2019

This was referenced Aug 13, 2019

promote CRD to v1 #79604

Merged

Add a test for custom object operations kubernetes-client/python#930

Closed

liggitt reviewed Aug 19, 2019

View reviewed changes

staging/src/k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1beta1/types.go Show resolved Hide resolved

roycaihw added 2 commits August 20, 2019 14:31

mark CRD .status.conditions as optional

80c02b6

generated

b614879

roycaihw force-pushed the optional-crdstatus-conditions branch from 2c6fba8 to b614879 Compare August 20, 2019 21:33

k8s-ci-robot assigned liggitt Aug 21, 2019

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Aug 21, 2019

k8s-ci-robot merged commit d951d19 into kubernetes:master Aug 21, 2019

k8s-ci-robot added this to the v1.16 milestone Aug 21, 2019

roycaihw mentioned this pull request Dec 3, 2019

Creating a CRD fails with "ValueError: Invalid value for conditions, must not be None" kubernetes-client/python#1022

Closed

roycaihw deleted the optional-crdstatus-conditions branch March 3, 2020 16:23

roycaihw mentioned this pull request Jul 20, 2020

create horizontal_pod_autoscaler error, conditions must not be None kubernetes-client/python#1200

Closed

roycaihw mentioned this pull request May 20, 2021

Mark ContainerImage.names as optional #102159

Merged

liggitt mentioned this pull request Jun 30, 2021

Migrate existing repo to kubernetes-client (kube-rs) kubernetes/org#2792

Closed

roycaihw mentioned this pull request Sep 13, 2021

[V1HorizontalPodAutoscaler] - 'Invalid value for conditions, must not be None' kubernetes-client/python#1543

Closed

roycaihw mentioned this pull request Apr 4, 2022

api request hang and stuck kubernetes-client/python#1740

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Mark Conditions in CustomResourceDefinitionStatus as optional #64996

Mark Conditions in CustomResourceDefinitionStatus as optional #64996

roycaihw commented Jun 12, 2018 •

edited

Loading

roycaihw commented Jun 12, 2018

sttts commented Jun 13, 2018

roycaihw commented Jun 13, 2018

sttts commented Jun 20, 2018

sttts commented Jun 20, 2018

sttts commented Jun 20, 2018

lavalamp commented Jun 21, 2018

roycaihw commented Jun 22, 2018

lavalamp commented Jun 22, 2018

smarterclayton commented Jun 24, 2018

k8s-github-robot commented Jul 14, 2018

fejta-bot commented Oct 12, 2018

fejta-bot commented Nov 11, 2018

fejta-bot commented Aug 14, 2019

liggitt commented Aug 19, 2019 •

edited

Loading

roycaihw commented Aug 20, 2019

roycaihw commented Aug 20, 2019 •

edited

Loading

roycaihw commented Aug 21, 2019 •

edited

Loading

liggitt commented Aug 21, 2019

k8s-ci-robot commented Aug 21, 2019

fejta-bot commented Aug 21, 2019

fejta-bot commented Aug 21, 2019

Mark Conditions in CustomResourceDefinitionStatus as optional #64996

Mark Conditions in CustomResourceDefinitionStatus as optional #64996

Conversation

roycaihw commented Jun 12, 2018 • edited Loading

roycaihw commented Jun 12, 2018

sttts commented Jun 13, 2018

roycaihw commented Jun 13, 2018

sttts commented Jun 20, 2018

sttts commented Jun 20, 2018

sttts commented Jun 20, 2018

lavalamp commented Jun 21, 2018

roycaihw commented Jun 22, 2018

lavalamp commented Jun 22, 2018

smarterclayton commented Jun 24, 2018

k8s-github-robot commented Jul 14, 2018

fejta-bot commented Oct 12, 2018

fejta-bot commented Nov 11, 2018

fejta-bot commented Aug 14, 2019

liggitt commented Aug 19, 2019 • edited Loading

roycaihw commented Aug 20, 2019

roycaihw commented Aug 20, 2019 • edited Loading

roycaihw commented Aug 21, 2019 • edited Loading

liggitt commented Aug 21, 2019

k8s-ci-robot commented Aug 21, 2019

fejta-bot commented Aug 21, 2019

fejta-bot commented Aug 21, 2019

roycaihw commented Jun 12, 2018 •

edited

Loading

liggitt commented Aug 19, 2019 •

edited

Loading

roycaihw commented Aug 20, 2019 •

edited

Loading

roycaihw commented Aug 21, 2019 •

edited

Loading