diff --git a/keps/sig-api-machinery/20191001-crd-immutability-pipeline.png b/keps/sig-api-machinery/1101-immutable-fields/20191001-crd-immutability-pipeline.png similarity index 100% rename from keps/sig-api-machinery/20191001-crd-immutability-pipeline.png rename to keps/sig-api-machinery/1101-immutable-fields/20191001-crd-immutability-pipeline.png diff --git a/keps/sig-api-machinery/0030-storage-migration.md b/keps/sig-api-machinery/2330-migrating-api-objects-to-latest-storage-version/README.md similarity index 97% rename from keps/sig-api-machinery/0030-storage-migration.md rename to keps/sig-api-machinery/2330-migrating-api-objects-to-latest-storage-version/README.md index c35d215c488..81cd87fb5c0 100644 --- a/keps/sig-api-machinery/0030-storage-migration.md +++ b/keps/sig-api-machinery/2330-migrating-api-objects-to-latest-storage-version/README.md @@ -1,20 +1,3 @@ ---- -title: Migrating API objects to latest storage version -authors: - - "@xuchao" -owning-sig: sig-api-machinery -reviewers: - - "@deads2k" - - "@yliaog" - - "@lavalamp" -approvers: - - "@deads2k" - - "@lavalamp" -creation-date: 2018-08-06 -last-updated: 2019-03-19 -status: implementable ---- - # Migrating API objects to latest storage version ## Table of Contents @@ -248,7 +231,7 @@ Before upgrading or downgrading the cluster, the cluster administrator must run migrations have completed. Otherwise the apiserver can crash, because it cannot interpret the serialized data in etcd. To mitigate, the cluster administrator can rollback the apiserver to the old version, and wait for the migration to -complete. +complete. With the newly introduced [storageStates API][], the cluster administrator can fast upgrade/downgrade as long as the new apiserver binaries understand all diff --git a/keps/sig-api-machinery/2330-migrating-api-objects-to-latest-storage-version/kep.yaml b/keps/sig-api-machinery/2330-migrating-api-objects-to-latest-storage-version/kep.yaml new file mode 100644 index 00000000000..33a3442155c --- /dev/null +++ b/keps/sig-api-machinery/2330-migrating-api-objects-to-latest-storage-version/kep.yaml @@ -0,0 +1,15 @@ +title: Migrating API objects to latest storage version +kep-number: 2330 +authors: + - "@xuchao" +owning-sig: sig-api-machinery +reviewers: + - "@deads2k" + - "@yliaog" + - "@lavalamp" +approvers: + - "@deads2k" + - "@lavalamp" +creation-date: 2018-08-06 +last-updated: 2019-03-19 +status: implementable diff --git a/keps/sig-api-machinery/20180731-crd-pruning-decoding.png b/keps/sig-api-machinery/2332-pruning-for-custom-resources/20180731-crd-pruning-decoding.png similarity index 100% rename from keps/sig-api-machinery/20180731-crd-pruning-decoding.png rename to keps/sig-api-machinery/2332-pruning-for-custom-resources/20180731-crd-pruning-decoding.png diff --git a/keps/sig-api-machinery/20180731-crd-pruning.md b/keps/sig-api-machinery/2332-pruning-for-custom-resources/README.md similarity index 94% rename from keps/sig-api-machinery/20180731-crd-pruning.md rename to keps/sig-api-machinery/2332-pruning-for-custom-resources/README.md index 3b6086c2682..9c589401082 100644 --- a/keps/sig-api-machinery/20180731-crd-pruning.md +++ b/keps/sig-api-machinery/2332-pruning-for-custom-resources/README.md @@ -1,29 +1,3 @@ ---- -title: Pruning for Custom Resources -authors: - - "@sttts" -owning-sig: sig-api-machinery -participating-sigs: - - sig-api-machinery -reviewers: - - "@deads2k" - - "@lavalamp" - - "@liggitt" - - "@erictune" - - "@mbohlool" - - "@apelisse" -approvers: - - "@deads2k" - - "@lavalamp" -editor: "@sttts" -creation-date: 2018-07-31 -last-updated: 2019-04-30 -status: implemented -see-also: - - "https://github.com/kubernetes/enhancements/pull/1002" - - "/keps/sig-api-machinery/20180415-crds-to-ga.md" ---- - # Pruning for Custom Resources ## Table of Contents @@ -48,9 +22,9 @@ see-also: ## Summary -CustomResources store arbitrary JSON data without following the typical Kubernetes API behaviour to prune unknown fields. This makes CRDs different, but also leads to security and general data consistency concerns because it is unclear what is actually stored in etcd. +CustomResources store arbitrary JSON data without following the typical Kubernetes API behaviour to prune unknown fields. This makes CRDs different, but also leads to security and general data consistency concerns because it is unclear what is actually stored in etcd. -This KEP proposes to add pruning of all fields which are not specified in the OpenAPI validation schemas given in the CRD. +This KEP proposes to add pruning of all fields which are not specified in the OpenAPI validation schemas given in the CRD. Pruning will be opt-in in v1beta1 of `apiextensions.k8s.io` via @@ -62,7 +36,7 @@ spec: ... ``` -i.e. CRDs created in v1beta1 default to disabled pruning. +i.e. CRDs created in v1beta1 default to disabled pruning. **Pruning will be enabled for every CRD created in v1** and we will hide `preserveUnknownFields` in v1 in that case. @@ -91,7 +65,7 @@ We assume the CRD has _structural schemas_ (as defined in [KEP Vanilla OpenAPI S We propose to -1. derive the value-validation-less variant of the structural schema (trivial by definition of _structural schema_) and +1. derive the value-validation-less variant of the structural schema (trivial by definition of _structural schema_) and 2. recursively follow the given CustomResource instance and the structural schema, removing fields from the former if they are not specified in the `properties` of the latter. Skip field removal in that recursion step (not for children) if `additionalProperties` is defined in the schema. 3. return a deserialization error if the CustomResource instance JSON value and the type in the structural schema do not match 4. fields of `metav1.TypeMeta` (`apiVersion` and `kind`) and `metav1.ObjectMeta` at the object root are implicitly specified. `metav1.ObjectMeta` is always pruned. @@ -112,7 +86,7 @@ The vendor extension `x-kubernetes-preserve-unknown-fields: true` proposed in (a The whole JSON subtree `X` at the level of `x-kubernetes-preserve-unknown-fields: true` and below is excluded from pruning, with the following exceptions: 1. pruning starts again at `Y` in the subtree of children of `X` if `Y` is specified by `properties`. -2. pruning starts again in the `metadata` child of `Y` in the subtree of `X` if `Y` is specified by `x-kubernetes-embedded-resource: true`. +2. pruning starts again in the `metadata` child of `Y` in the subtree of `X` if `Y` is specified by `x-kubernetes-embedded-resource: true`. We do not allow `x-kubernetes-preserve-unknown-fields: false`. @@ -125,7 +99,7 @@ Note that (1) does not apply if `X` and `Y` are the same, compare examples (7) ```yaml type: object ``` - + Everything is pruned, i.e. ```json @@ -134,13 +108,13 @@ Note that (1) does not apply if `X` and `Y` are the same, compare examples (7) "json": {"bar": 43} } ``` - + is pruned to - + ```json {} ``` - + 2. `properties` at top-level ```yaml @@ -149,7 +123,7 @@ Note that (1) does not apply if `X` and `Y` are the same, compare examples (7) foo: type: object ``` - + Everything other than `foo` is pruned, i.e. ```json @@ -160,9 +134,9 @@ Note that (1) does not apply if `X` and `Y` are the same, compare examples (7) "json": {"bar": 43} } ``` - + is pruned to - + ```json { "foo": {} @@ -180,7 +154,7 @@ Note that (1) does not apply if `X` and `Y` are the same, compare examples (7) bar: type: object ``` - + Everything other than `foo` and `bar` is pruned, i.e. ```json @@ -194,9 +168,9 @@ Note that (1) does not apply if `X` and `Y` are the same, compare examples (7) "json": {"ghi": 44} } ``` - + is pruned to - + ```json { "foo": { @@ -204,7 +178,7 @@ Note that (1) does not apply if `X` and `Y` are the same, compare examples (7) } } ``` - + 4. `additionalProperties` with a non-empty schema ```yaml @@ -215,7 +189,7 @@ Note that (1) does not apply if `X` and `Y` are the same, compare examples (7) additionalProperties: type: object ``` - + Everything directly inside of `foo` stays (it is considered a string map), but objects further down are pruned again because they are unspecified, i.e. @@ -228,9 +202,9 @@ Note that (1) does not apply if `X` and `Y` are the same, compare examples (7) "json": {"ghi": 44} } ``` - + is pruned to - + ```json { "foo": { @@ -249,7 +223,7 @@ Note that (1) does not apply if `X` and `Y` are the same, compare examples (7) type: object additionalProperties: false ``` - + Everything directly inside of `foo` stays (it is considered a string map), but objects further down are pruned again because they are unspecified, i.e. @@ -262,9 +236,9 @@ Note that (1) does not apply if `X` and `Y` are the same, compare examples (7) "json": {"ghi": 44} } ``` - + is pruned to - + ```json { "foo": { @@ -274,7 +248,7 @@ Note that (1) does not apply if `X` and `Y` are the same, compare examples (7) } ``` but validation will fail. We consider the semantical meaning of `false` as value validation, not structural. - + 6. arbitrary JSON ```yaml @@ -286,22 +260,22 @@ Note that (1) does not apply if `X` and `Y` are the same, compare examples (7) ``` Inside of `.json` nothing is pruned, i.e. - + ```json { "foo": 42, "json": {"bar": 43} } ``` - + is pruned to - + ```json { "json": {"bar": 43} } ``` - + 7. JSON, but with properties at the same level ```yaml @@ -317,7 +291,7 @@ Note that (1) does not apply if `X` and `Y` are the same, compare examples (7) ``` Inside of `.json` nothing is pruned, including everything in `bar`, i.e. - + ```json { "foo": 42, @@ -329,9 +303,9 @@ Note that (1) does not apply if `X` and `Y` are the same, compare examples (7) } } ``` - + is pruned to - + ```json { "json": { @@ -342,7 +316,7 @@ Note that (1) does not apply if `X` and `Y` are the same, compare examples (7) } } ``` - + 8. JSON, but with properties at the same **and lower** levels ```yaml @@ -361,7 +335,7 @@ Note that (1) does not apply if `X` and `Y` are the same, compare examples (7) ``` The `properties` for `bar` "resets" pruning to normal behaviour, i.e. - + ```json { "foo": 42, @@ -374,9 +348,9 @@ Note that (1) does not apply if `X` and `Y` are the same, compare examples (7) } } ``` - + is pruned to - + ```json { "json": { @@ -387,7 +361,7 @@ Note that (1) does not apply if `X` and `Y` are the same, compare examples (7) } } ``` - + 9. `additionalProperties` within JSON ```yaml @@ -402,7 +376,7 @@ Note that (1) does not apply if `X` and `Y` are the same, compare examples (7) ``` The `additionalProperties` disables pruning **at its level**, `x-kubernetes-preserve-unknown-fields: true` already has the same effect. Inside of the additional properties `x-kubernetes-preserve-unknown-fields: true` keeps being effective, i.e. - + ```json { "foo": 42, @@ -415,9 +389,9 @@ Note that (1) does not apply if `X` and `Y` are the same, compare examples (7) } } ``` - + is pruned to - + ```json { "json": { @@ -428,7 +402,7 @@ Note that (1) does not apply if `X` and `Y` are the same, compare examples (7) "def": 45 } } - ``` + ``` 10. embedded resource ```yaml @@ -440,7 +414,7 @@ Note that (1) does not apply if `X` and `Y` are the same, compare examples (7) x-kubernetes-embedded-resource: true x-kubernetes-preserve-unknown-fields: true ``` - + Here, inside of `.object` nothing is pruned with the exception of unknown fields under `.object.metadata`, i.e. ```json @@ -456,9 +430,9 @@ Note that (1) does not apply if `X` and `Y` are the same, compare examples (7) } } ``` - + is pruned to - + ```json { "object": { @@ -470,7 +444,7 @@ Note that (1) does not apply if `X` and `Y` are the same, compare examples (7) } } ``` - + 11. implicit `metav1.TypeMeta` and `metav1.ObjectMeta` ```yaml @@ -478,7 +452,7 @@ Note that (1) does not apply if `X` and `Y` are the same, compare examples (7) ``` Pruning takes place, but `apiVersion`, `kind`, `metadata` and known fields under `metadata` are preserved, i.e. - + ```json { "apiVersion": "example/v1", @@ -490,9 +464,9 @@ Note that (1) does not apply if `X` and `Y` are the same, compare examples (7) "foo": 42 } ``` - + is pruned to - + ```json { "apiVersion": "example/v1", @@ -510,11 +484,11 @@ We will add a `preserveUnknownFields` flag to `CustomResourceDefinitionSpec` of ```go type CustomResourceDefinitionSpec struct { ... - + // PreserveUnknownFields disables pruning of object fields which are not // specified in the OpenAPI schema. apiVersion, kind, metadata and known // fields inside metadata are excluded from pruning. - // Defaults to true in v1beta1, and will default to false in v1. + // Defaults to true in v1beta1, and will default to false in v1. // Setting this field to false is considered an beta API. PreserveUnknownFields *bool } @@ -526,11 +500,11 @@ For `apiextensions.k8s.io/v1` we will change the default to false and forbid tru We will hide `preserveUnknownFields` in v1 objects if it is not true. -When CRD authors switch on pruning for an existing CRD, they are supposed to make their users trigger a data migration of existing objects in etcd, be it via an external migration mechanism, an operator rewriting all objects or manual procedures. +When CRD authors switch on pruning for an existing CRD, they are supposed to make their users trigger a data migration of existing objects in etcd, be it via an external migration mechanism, an operator rewriting all objects or manual procedures. ### References -* Old pruning implementation PR https://github.com/kubernetes/kubernetes/pull/64558, to be adapted +* Old pruning implementation PR https://github.com/kubernetes/kubernetes/pull/64558, to be adapted * [OpenAPI v3 specification](https://github.com/OAI/OpenAPI-Specification/blob/master/versions/3.0.0.md) * [JSON Schema](http://json-schema.org/) @@ -546,7 +520,7 @@ We default `preserveUnknownFields` to true and hence switch off the whole code p * verify that `preserveUnknownFields` is defaulted to true. * verify that pruning happens if `preserveUnknownFields` is false, for all versions in the CRD according to the schema of the respective version. * verify that `metadata`, `apiVersion`, `kind` are preserved if `preserveUnknownFields` is false and there is no schema given in the CRD. - + **blockers for beta:** * we implement and verify that `x-kubernetes-embedded-resource` and `x-kubernetes-preserve-unknown-fields` work as expected. @@ -576,8 +550,8 @@ Hence, we assume to be at beta in 1.15 and GA in 1.16 guided by the graduation c * downgrading from 1.16 (where pruning might be GA) to 1.15 will keep the same behaviour as we don't feature gate `preserveUnknownFields: false`. * upgrading from 1.14 will default to `preserveUnknownFields: true` and hence changes no behaviour. * upgrading from 1.15 will keep the value and hence change no behaviour. -* when v1 of `apiextensions.k8s.io` is added, we will keep the old pruning behaviour for CRDs created in v1beta1 with `preserveUnknownFields: true`, but forbid `preserveUnknownFields: true` for every newly create v1 CRD. Hence, we keep backwards compatibility. - +* when v1 of `apiextensions.k8s.io` is added, we will keep the old pruning behaviour for CRDs created in v1beta1 with `preserveUnknownFields: true`, but forbid `preserveUnknownFields: true` for every newly create v1 CRD. Hence, we keep backwards compatibility. + Technically, it is still possible to get the old behaviour even in v1 by setting `x-kubernetes-preserve-unknown-fields: true` at the root level and in each `properties` statement. But we enforce the definition of a schema, at least with this minimal contents. ### Version Skew Strategy @@ -591,9 +565,9 @@ Hence, we assume to be at beta in 1.15 and GA in 1.16 guided by the graduation c * we could allow nested `x-kubernetes-preserve-unknown-fields: false`, i.e. to switch on pruning again for a subtree. This might encourage non-Kubernetes-like API types. It is unclear whether there are use-cases we want to support which need this. We can add this in the future. * we could allow per-version opt-in/out of pruning via `preserveUnknownFields` in `CustomResourceDefinitionVersion`. For the goal of data consistency and security a CRD with semi-enabled pruning does not make much sense. The main reason to not enable pruning will probably be the lack of a complete structural schema. If this is added for one version, it should be possible for all other versions as well as it is less a technical, but a CRD development life-cycle question. * we intensively considered avoiding a new `x-kubernetes-preserve-unknown-fields` vendor extension in favor of recursive `additionalProperties` semantics. We decided against because: - + None of OpenAPI v3 schema constructs have effects recursively. We would conflict with that pattern. - + E.g. `additionalProperties: false` invalidates unknown fields only at its level in OpenAPI v3, for example: ```yaml type: object @@ -602,21 +576,21 @@ Hence, we assume to be at beta in 1.15 and GA in 1.16 guided by the graduation c foo: {} ``` (note: this is not allowed in CRDs, but in OpenAPI v3) forbids `{"foo":{},"abc":42}`, but not `{"foo":{"abc":42}}`. A recursive interpretation for pruning would diverge from this pattern. - + Another example: ```yaml additionalProperties: minimum: 42 ``` forbids `{"foo":10}`, but not `{"foo":{"bar":10}}`. - + * we stop pruning even for `additionalProperties: false` or any other additional properties schema. We considered to prune for `false`, but not for `true`. We decided against because: - + * it is unclear what should happen with pruning for non-empty schemas between `false` and `true`. * it is infeasible to compute whether an arbitrary schema is empty (and hence equivalent to `false`) or not. Semantically empty schemas and `false` should behave the same behaviour. This includes pruning. - + By not pruning for any explicit value of `additionalProperties` (including `false`) we follow our principle of not trying to consider full semantics of OpenAPI including value validation when doing structural operations like pruning. - + Compare example (5): CustomResource validation will eventually forbid unpruned values. The semantical meaning of the schema `false` is considered non-structural and therefore not relevant for pruning. -## Implementation History \ No newline at end of file +## Implementation History diff --git a/keps/sig-api-machinery/2332-pruning-for-custom-resources/kep.yaml b/keps/sig-api-machinery/2332-pruning-for-custom-resources/kep.yaml new file mode 100644 index 00000000000..99f1992aa2a --- /dev/null +++ b/keps/sig-api-machinery/2332-pruning-for-custom-resources/kep.yaml @@ -0,0 +1,24 @@ +title: Pruning for Custom Resources +kep-number: 2332 +authors: + - "@sttts" +owning-sig: sig-api-machinery +participating-sigs: + - sig-api-machinery +reviewers: + - "@deads2k" + - "@lavalamp" + - "@liggitt" + - "@erictune" + - "@mbohlool" + - "@apelisse" +approvers: + - "@deads2k" + - "@lavalamp" +editor: "@sttts" +creation-date: 2018-07-31 +last-updated: 2019-04-30 +status: implemented +see-also: + - "https://github.com/kubernetes/enhancements/pull/1002" + - "/keps/sig-api-machinery/20180415-crds-to-ga.md" diff --git a/keps/sig-api-machinery/20190129-legacyflag.md b/keps/sig-api-machinery/2333-legacyflags-kflag/README.md similarity index 95% rename from keps/sig-api-machinery/20190129-legacyflag.md rename to keps/sig-api-machinery/2333-legacyflags-kflag/README.md index 6486ca6c9b2..f5ec67b198b 100644 --- a/keps/sig-api-machinery/20190129-legacyflag.md +++ b/keps/sig-api-machinery/2333-legacyflags-kflag/README.md @@ -1,25 +1,3 @@ ---- -title: legacyflags -authors: - - "@mtaufen" -owning-sig: sig-api-machinery -participating-sigs: - - sig-architecture - - sig-cluster-lifecycle - - wg-component-standard -reviewers: - - "@kubernetes/wg-component-standard" -approvers: - - "@luxas" - - "@sttts" -editor: TBD -creation-date: 2019-01-29 -last-updated: 2019-04-02 -status: provisional -see-also: - - KEP-32 ---- - # kflag ## Table of Contents @@ -201,4 +179,4 @@ careful to test properly on a component-by-component basis as we migrate compone ## Implementation History -* 2019-01-29: Initial KEP PR. \ No newline at end of file +* 2019-01-29: Initial KEP PR. diff --git a/keps/sig-api-machinery/2333-legacyflags-kflag/kep.yaml b/keps/sig-api-machinery/2333-legacyflags-kflag/kep.yaml new file mode 100644 index 00000000000..eedaa077c30 --- /dev/null +++ b/keps/sig-api-machinery/2333-legacyflags-kflag/kep.yaml @@ -0,0 +1,20 @@ +title: legacyflags +kep-number: 2333 +authors: + - "@mtaufen" +owning-sig: sig-api-machinery +participating-sigs: + - sig-architecture + - sig-cluster-lifecycle + - wg-component-standard +reviewers: + - "@kubernetes/wg-component-standard" +approvers: + - "@luxas" + - "@sttts" +editor: TBD +creation-date: 2019-01-29 +last-updated: 2019-04-02 +status: provisional +see-also: + - KEP-32 diff --git a/keps/sig-api-machinery/20190322-server-side-get-to-ga.md b/keps/sig-api-machinery/2334-graduate-server-side-get-and-partial-objects-to-GA/README.md similarity index 88% rename from keps/sig-api-machinery/20190322-server-side-get-to-ga.md rename to keps/sig-api-machinery/2334-graduate-server-side-get-and-partial-objects-to-GA/README.md index 2f12c38b179..c5e871aa5ec 100644 --- a/keps/sig-api-machinery/20190322-server-side-get-to-ga.md +++ b/keps/sig-api-machinery/2334-graduate-server-side-get-and-partial-objects-to-GA/README.md @@ -1,27 +1,3 @@ ---- -title: Graduate Server-side Get and Partial Objects to GA -authors: - - "@smarterclayton" -owning-sig: sig-api-machinery -participating-sigs: - - sig-cli -reviewers: - - "@lavalamp" - - "@soltysh" - - "@liggitt" -approvers: - - "@liggitt" - - "@pwittrock" -editor: TBD -creation-date: 2019-03-22 -last-updated: 2019-03-22 -status: implementable -see-also: - - "https://github.com/kubernetes/community/blob/master/contributors/design-proposals/api-machinery/server-get.md" -replaces: -superseded-by: ---- - # Graduate Server-side Get and Partial Objects to GA ## Table of Contents @@ -44,7 +20,7 @@ superseded-by: ## Summary -Server-side columnar formatting and partial object metadata has been in beta since Kube 1.10 and as of 1.15 is consistently implemented and in wide use as part of `kubectl` and other web interfaces. This document outline required steps to graduate it to GA. +Server-side columnar formatting and partial object metadata has been in beta since Kube 1.10 and as of 1.15 is consistently implemented and in wide use as part of `kubectl` and other web interfaces. This document outline required steps to graduate it to GA. ## Motivation @@ -74,7 +50,7 @@ PartialObjectMetadata exposes our full ObjectMeta interface and no API changes a * In the garbage collector, we will remove the need to call `Update` and use a partial object metadata client/informer * In the namespace controller, we will use a partial object metadata informer * In the quota counting code, we will use a partial object metadata informer -* Announce deprecation of `v1beta1` objects and removal in 1.19 +* Announce deprecation of `v1beta1` objects and removal in 1.19 * `kubectl` should switch to using `meta.k8s.io/v1` `Table` (supporting 1.15+ clusters) ### 1.19 @@ -84,7 +60,7 @@ PartialObjectMetadata exposes our full ObjectMeta interface and no API changes a ### Implementation Details A new dynamic client variant capable of supporting read and write operations on PartialObjectMetadata -should be created that hides whether the server supports PartialObjectMetadata. +should be created that hides whether the server supports PartialObjectMetadata. Currently `v1beta1.Table` does not support Protobuf and the generators do not trivially support the serialization of the cells. We need to decide on a serialization format for the Protobuf cells and @@ -120,4 +96,4 @@ We will support N-1 for both `kubectl` and participating controllers by introduc ## Implementation History * First version of this proposal merged. -* Server-side GET objects moved to v1 in 1.15 \ No newline at end of file +* Server-side GET objects moved to v1 in 1.15 diff --git a/keps/sig-api-machinery/2334-graduate-server-side-get-and-partial-objects-to-GA/kep.yaml b/keps/sig-api-machinery/2334-graduate-server-side-get-and-partial-objects-to-GA/kep.yaml new file mode 100644 index 00000000000..9c3b0b05c00 --- /dev/null +++ b/keps/sig-api-machinery/2334-graduate-server-side-get-and-partial-objects-to-GA/kep.yaml @@ -0,0 +1,22 @@ +title: Graduate Server-side Get and Partial Objects to GA +kep-number: 2334 +authors: + - "@smarterclayton" +owning-sig: sig-api-machinery +participating-sigs: + - sig-cli +reviewers: + - "@lavalamp" + - "@soltysh" + - "@liggitt" +approvers: + - "@liggitt" + - "@pwittrock" +editor: TBD +creation-date: 2019-03-22 +last-updated: 2019-03-22 +status: implementable +see-also: + - "https://github.com/kubernetes/community/blob/master/contributors/design-proposals/api-machinery/server-get.md" +replaces: +superseded-by: diff --git a/keps/sig-api-machinery/20190425-structural-openapi-context.png b/keps/sig-api-machinery/2335-vanilla-crd-openapi-subset-structural-schemas/20190425-structural-openapi-context.png similarity index 100% rename from keps/sig-api-machinery/20190425-structural-openapi-context.png rename to keps/sig-api-machinery/2335-vanilla-crd-openapi-subset-structural-schemas/20190425-structural-openapi-context.png diff --git a/keps/sig-api-machinery/20190425-structural-openapi.md b/keps/sig-api-machinery/2335-vanilla-crd-openapi-subset-structural-schemas/README.md similarity index 96% rename from keps/sig-api-machinery/20190425-structural-openapi.md rename to keps/sig-api-machinery/2335-vanilla-crd-openapi-subset-structural-schemas/README.md index c0e00a12c7a..b4b35861958 100644 --- a/keps/sig-api-machinery/20190425-structural-openapi.md +++ b/keps/sig-api-machinery/2335-vanilla-crd-openapi-subset-structural-schemas/README.md @@ -1,29 +1,3 @@ ---- -title: "Vanilla CRD OpenAPI Subset: Structural Schemas" -authors: - - "@sttts" - - "@mbohlool" -owning-sig: sig-api-machinery -participating-sigs: - - sig-api-machinery -reviewers: - - "@liggitt" - - "@apelisse" - - "@mbohlool" - - "@DirectXMan12" -approvers: - - "@lavalamp" - - "@deads2k" -editor: TBD -creation-date: 2019-04-25 -last-updated: 2019-05-09 -status: implemented -see-also: - - "/keps/sig-api-machinery/20180415-crds-to-ga.md" - - "https://github.com/kubernetes/enhancements/pull/926" - - "https://github.com/kubernetes/enhancements/pull/709" ---- - # Vanilla CRD OpenAPI subset: structural schemas OpenAPI has the goal of describing every API possible. CRDs have the goal of presenting a consistent API to users of Kubernetes. This KEP proposes a restriction of permissible CRD OpenAPI validation schemas to match the later and to enable us add advanced CRD features like server-side pruning, defaulting, apply and potentially Protobuf support in the future. @@ -57,7 +31,7 @@ OpenAPI has the goal of describing every API possible. CRDs have the goal of pre ## Summary -The CRD validation schemas today support nearly the whole OpenAPI v3 schema language. Here, we propose to impose a restriction called _structural schema_ to that language, which will +The CRD validation schemas today support nearly the whole OpenAPI v3 schema language. Here, we propose to impose a restriction called _structural schema_ to that language, which will * significantly reduce difficulty in moving forward with advanced features like pruning, defaulting, server-side apply, read-only fields and protobuf in the near and medium term future * while keeping needed expressivity to express Kubernetes-like APIs including powerful value validations. @@ -129,22 +103,22 @@ To allow only a subset of features of OpenAPI v3, we need to have guidelines for * string maps like labels and annotations, but also `map[string]int64` * IPv4 vs IPv6 strings: same fields with different formats * embedded PodSpec (a non-trivial object) - + * discriminated union types (as described in [KEP: Union Types](https://github.com/kubernetes/enhancements/pull/926)). - + Examples: * VolumeSource, DeploymentStrategy, … * a very restrictive, fixed set of polymorphic types. - + Examples: * IntOrString * RawExtensions, i.e. unconstrained, embedded toplevel types * arbitrary JSON - + Note: we explicitly accept that polymorphic types (like IntOrString) are an anti-pattern for Kubernetes-like types and we do not want to encourage its use beyond IntOrString. -As a soft criterion to validate the direction, restricted CRD OpenAPI schemas should represent a model that can also be represented with other parts of the system: +As a soft criterion to validate the direction, restricted CRD OpenAPI schemas should represent a model that can also be represented with other parts of the system: Golang types <=> OpenAPI <=> Protobuf. @@ -202,15 +176,15 @@ type StructuralSchema struct { Generic Extensions - + *ValueValidation } type Extensions struct { - // x-kubernetes-unions lists unions, i.e. sets of fields which are + // x-kubernetes-unions lists unions, i.e. sets of fields which are // mutual exclusive. XUnions []Union - + // x-kubernetes-preserve-unknown-fields stops the API server // decoding step from pruning fields which are not specified // in the validation schema. This affects fields recursively, @@ -218,7 +192,7 @@ type Extensions struct { // properties or additionalProperties are specified in the schema. // This can either be true or null. False is forbidden. XPreserveUnknownFields *bool - + // x-kubernetes-embedded-resource defines that the value is an // embedded Kubernetes runtime.Object, with TypeMeta and // ObjectMeta. The type must be object. It is allowed to further @@ -227,7 +201,7 @@ type Extensions struct { // is allowed to be true, but does not have to be if the object // is fully specified (up to kind, apiVersion, metadata). XEmbeddedResource bool - + // x-kubernetes-int-or-string specifies that this value is // either an integer or a string. If this is true, an empty // type is allowed and type as child of anyOf is permitted @@ -255,11 +229,11 @@ type Union struct { type Generic struct { // type specifies the type of a value. // It can be object, array, number, integer, boolean, string. - // It is optional only if x-kubernetes-preserve-unknown-fields + // It is optional only if x-kubernetes-preserve-unknown-fields // or x-kubernetes-int-or-string is true. Type string AdditionalProperties *StructuralOrBool - + Description string Title string Nullable bool @@ -295,9 +269,9 @@ type NestedValueValidation struct { Items *NestedValueValidation Properties map[string]NestedValueValidation - - // Anything set in the following will make the scheme - // non-structural, with the exception of these two patterns if + + // Anything set in the following will make the scheme + // non-structural, with the exception of these two patterns if // x-kubernetes-int-or-string is true: // // 1) anyOf: @@ -316,12 +290,12 @@ type NestedValueValidation struct { Note: we do not have to change the API types to this schema right away. Take it as hypothetical data structure for the sake of definition, potentially used though during validation. We add the following additional validations for CRDs based on the representation as `StructuralSchema`: - + 1. **structurality:** both `ForbiddenGenerics` and `ForbiddenExtensions` only have zero values, with the two exceptions of (4). 2. **completeness:** the schema is _structurally complete_ 3. **RawExtension:** for every schema with `x-kubernetes-embedded-resource: true`, `type: object` is set, and either properties are specified or `x-kubernetes-preserve-unknown-fields: true` is set. 4. **IntOrString:** for `x-kubernetes-int-or-string: true` either `type` is empty under `anyOf` and `allOf` or the schema structure is either - + ```yaml x-kubernetes-int-or-string: true anyOf: @@ -340,10 +314,10 @@ We add the following additional validations for CRDs based on the representation ... ``` -Structurally complete means that any specified JSON path in a schema has a specified type outside of any logical junctors (`andOf`,`anyOf`,`oneOf`,`not`; compare the guiding principle above and its example). In other words, we validate that +Structurally complete means that any specified JSON path in a schema has a specified type outside of any logical junctors (`andOf`,`anyOf`,`oneOf`,`not`; compare the guiding principle above and its example). In other words, we validate that -- every object field (`properties`), -- every array value (`items`) and +- every object field (`properties`), +- every array value (`items`) and - every string map value (`additionalProperties`) has a non-empty `type` outside of logical junctors, with the possible exceptions of @@ -400,7 +374,7 @@ Hence, we restrict ... ``` is allowed. - + The restrictions do not apply to embedded resources (specified via `x-kubernetes-embedded-resource`) as these are not instances yet which interact with API machinery beyond pure validation. Note, that it is discouraged to completely specify `metadata` for embedded resources, but only those restrictions which go beyond the implicit `metadata` validation induced by `x-kubernetes-embedded-resource`. @@ -416,7 +390,7 @@ During publishing we unfold the new `x-kubernetes-*` extensions: - type: integer - type: string ``` - if `ValueValidation.AnyOf` is unset, otherwise to: + if `ValueValidation.AnyOf` is unset, otherwise to: ```yaml x-kubernetes-int-or-string: true allOf: @@ -461,7 +435,7 @@ During publishing we unfold the new `x-kubernetes-*` extensions: ``` if no `oneOf` is specified, the analog and'ed variant with `allOf` otherwise. -These unfoldings are subject to change and explicitly not part of the API, but merely illustration and a rough sketch here. Code generators and other tools must not depend on the unfolding to happen in one way or another. It is guaranteed though that the `x-kubernetes-*` fields are preserved during publishing and hence can be used by tooling as an abstract hint for semantics which are complex to express in OpenAPI. +These unfoldings are subject to change and explicitly not part of the API, but merely illustration and a rough sketch here. Code generators and other tools must not depend on the unfolding to happen in one way or another. It is guaranteed though that the `x-kubernetes-*` fields are preserved during publishing and hence can be used by tooling as an abstract hint for semantics which are complex to express in OpenAPI. The guiding principle is that we don't want OpenAPI-consuming tooling, e.g. client generators, to implement OpenAPI schema pattern heuristics to recover the original types like unions, RawExtensions or `IntOrString`. We support them with our abstract, officially supported vendor extensions for those cases. @@ -483,7 +457,7 @@ Note: the kubebuilder tags are partially just sketches because the tags in crd-g // +required SharedField string } - + type HostPath struct { // +kubebuilder:pattern:"/.*" // +required @@ -556,13 +530,13 @@ Note: the kubebuilder tags are partially just sketches because the tags in crd-g - pattern: "(([0-9a-fA-F]{1,4}:){7,7}[0-9a-fA-F]{1,4}|...$" ``` and published the same way as OpenAPI v3 and as `type: string` in OpenAPI v2 (neither `anyOf` nor `pattern` exist in OpenAPI v2). - + After dropping the value validation, we get: ```yaml type: string ``` -3. RawExtensions, i.e. unconstrained, embedded toplevel types +3. RawExtensions, i.e. unconstrained, embedded toplevel types ```go type Foo struct { // +nullable @@ -595,7 +569,7 @@ Note: the kubebuilder tags are partially just sketches because the tags in crd-g ``` In OpenAPI v2 `pattern` does not exist and is dropped. - + After dropping the value validation, we get: ```yaml type: object @@ -630,7 +604,7 @@ Note: the kubebuilder tags are partially just sketches because the tags in crd-g nullable: true ``` and published the same way as OpenAPI v3. For OpenAPI v2 we have to drop `type: object` and `nullable: true` as the latter does not exist in v2 and `type: object` is too restrictive (does not allow `null` as value). - + Dropping value validation does not change anything. 5. string maps: @@ -646,7 +620,7 @@ Note: the kubebuilder tags are partially just sketches because the tags in crd-g minimum: 42 ``` and published as OpenAPI v3 and v2 the same way. - + After dropping the value validation, we get: ```yaml type: object @@ -660,7 +634,7 @@ Note: the kubebuilder tags are partially just sketches because the tags in crd-g // +kubebuilder:anyOf=minimum=42,maximum=50;minimum=52,maximum=60 X *IntOrString } - + type IntOrString struct { Integer *int String *string @@ -691,11 +665,11 @@ Note: the kubebuilder tags are partially just sketches because the tags in crd-g maximum: 60 ``` The latter is what is also published as OpenAPI v3, but as `x-kubernetes-int-or-string: true` in OpenAPI v2 (neither `anyOf`, nor `pattern` exists in v2). - + After dropping the value validation, we get: ```yaml x-kubernetes-int-or-string: true - ``` + ``` Note: the OpenAPI v2 output given above is the same we get from the [OpenAPI v2 filtering in Kubernetes 1.14](https://github.com/kubernetes/kubernetes/blob/master/staging/src/k8s.io/apiextensions-apiserver/pkg/controller/openapi/v2/conversion.go#L30): that filtering in the CRD OpenAPI v2 publishing pipeline supports the unrestricted range of the CRD OpenAPI v3 language and turns it into a weaker variant understandable by kubectl 1.13 and 1.14. Hence, our changes above have no influence on the OpenAPI v2 publishing other than passing through of our new `x-kubernetes-*` vendor extensions. @@ -727,9 +701,9 @@ Also note that structural schemas enforce types and properties outside of logica * rejected at CRD validation time if this field is set; user gets validation error This should create natural motivation for the community to phase out old, non-structural schemas without breaking beta compatibility guarantees. - + We add a condition `NonStructuralSchema` to the CRD objects to reflect that information to the user about a non-structural schema. This condition is only created in case its value is `True`. It is removed if it turns to `False` due to schema updates. - + ### Implementation Plan **For Kubernetes 1.15, must haves:** @@ -752,11 +726,11 @@ We add a condition `NonStructuralSchema` to the CRD objects to reflect that info Next to unit tests, we will add integration tests with real CustomResourceDefinitions. Those will: -- create and update CRDs with +- create and update CRDs with * structural * non-structural and - * no schema, - with + * no schema, + with * one version and * with multiple versions and different schema and check that the `NonStructuralSchema` condition is set correctly. @@ -779,7 +753,7 @@ On the API surface we add ### Version Skew Strategy * kubectl 1.14 has to cope with published schemas created from unfolding of the `x-kubernetes-*` fields. The result of unfolding is a valid schema of today's CRD schema language and therefore supported by the OpenAPI v2 publishing algorithm. Hence, unfolding does not add anything new. -* the new condition is ignored by kubectl 1.14. +* the new condition is ignored by kubectl 1.14. ## Implementation History diff --git a/keps/sig-api-machinery/2335-vanilla-crd-openapi-subset-structural-schemas/kep.yaml b/keps/sig-api-machinery/2335-vanilla-crd-openapi-subset-structural-schemas/kep.yaml new file mode 100644 index 00000000000..0547f06026a --- /dev/null +++ b/keps/sig-api-machinery/2335-vanilla-crd-openapi-subset-structural-schemas/kep.yaml @@ -0,0 +1,24 @@ +title: "Vanilla CRD OpenAPI Subset: Structural Schemas" +kep-number: 2335 +authors: + - "@sttts" + - "@mbohlool" +owning-sig: sig-api-machinery +participating-sigs: + - sig-api-machinery +reviewers: + - "@liggitt" + - "@apelisse" + - "@mbohlool" + - "@DirectXMan12" +approvers: + - "@lavalamp" + - "@deads2k" +editor: TBD +creation-date: 2019-04-25 +last-updated: 2019-05-09 +status: implemented +see-also: + - "/keps/sig-api-machinery/20180415-crds-to-ga.md" + - "https://github.com/kubernetes/enhancements/pull/926" + - "https://github.com/kubernetes/enhancements/pull/709" diff --git a/keps/sig-api-machinery/20190607-owner-ref.md b/keps/sig-api-machinery/2336-OwnerReference-resource-field/README.md similarity index 88% rename from keps/sig-api-machinery/20190607-owner-ref.md rename to keps/sig-api-machinery/2336-OwnerReference-resource-field/README.md index 5c2283df557..9fb91e137ca 100644 --- a/keps/sig-api-machinery/20190607-owner-ref.md +++ b/keps/sig-api-machinery/2336-OwnerReference-resource-field/README.md @@ -1,22 +1,4 @@ ---- -title: OwnerReference Resource Field -authors: - - "@deads2k" -owning-sig: sig-api-machinery -participating-sigs: - - sig-api-machinery -reviewers: - - "@sttts" - - "@caesarxuchao" - - "@liggitt" -approvers: - - "@caesarxuchao" - - "@lavalamp" -editor: "@deads2k" -creation-date: 2019-06-07 -last-updated: 2019-06-12 -status: implementable ---- + # OwnerReference Resource Field @@ -50,7 +32,7 @@ and mostly works, but it is inefficient and unnecessarily vulnerable to network This KEP proposes adding an optional `resource` field to OwnerReferences. If provided, this will be the authoritative value and no lookup or approximation of user intent will be required. If not provided, the `kind` will be used as it is today. If both are provided, old clusters continue to work and new clusters will be more efficient and resilient. - + ```yaml ownerReferences: - apiVersion: apps/v1 @@ -62,10 +44,10 @@ ownerReferences: * Allow precision. `kind` is a one to many relationship that makes the exact object reference inefficient or impossible to determine. * Remove remote calls. Because a `kind` to `resource` mapping requires discovery, it is vulnerable to network segmentation - + ### Goals -* Improve reliability. +* Improve reliability. * Improve efficiency. * Improve predictability. * Avoid disrupting existing clients. @@ -89,23 +71,23 @@ type OwnerReference struct { } ``` -The only impact is in the case of a new cluster with both fields set. In this case, the new cluster does not have to +The only impact is in the case of a new cluster with both fields set. In this case, the new cluster does not have to perform any lookups and will trust the client's specification of `resource`. Existing clients are not required to make this change, but it will improve reliability. If a client sets the value improperly, -then the behavior is congruent to behavior with a mis-set `kind`. If the kind (resource) is valid, but incorrect -GC collects the object immediately. If the kind (resource) is invalid, GC reports the error and waits. +then the behavior is congruent to behavior with a mis-set `kind`. If the kind (resource) is valid, but incorrect +GC collects the object immediately. If the kind (resource) is invalid, GC reports the error and waits. ### Behavior of new clusters -1. `resource` set and `kind` both set: only `resource` is honored and no mapping required -2. `resource` set and `kind` not set: validation failure. Allowing this would break old clusters and clients. +1. `resource` set and `kind` both set: only `resource` is honored and no mapping required +2. `resource` set and `kind` not set: validation failure. Allowing this would break old clusters and clients. 3. `resource` not set and `kind` set: `kind` is honored, exactly as before 4. `resource` not set and `kind` not set: validation failure - + ### Behavior of old clusters -1. `resource` set and `kind` both set: only `kind` is honored, exactly as before -2. `resource` set and `kind` not set: validation failure +1. `resource` set and `kind` both set: only `kind` is honored, exactly as before +2. `resource` set and `kind` not set: validation failure 3. `resource` not set and `kind` set: `kind` is honored, exactly as before 4. `resource` not set and `kind` not set: validation failure @@ -113,7 +95,7 @@ GC collects the object immediately. If the kind (resource) is invalid, GC repor Old clients may clear the new `resource` field. This is most likely to happen with the kube-controller-manager in a downgrade scenario, but can logically happen with any client. In this case, we won't consider the request to be mutating the owner reference if there is a valid mapping between the resource and kind. If there is not such a mapping, we will consider the request to be -mutating owner reference for the purpose of admission checks. +mutating owner reference for the purpose of admission checks. There is a case where a controller (not a built in one) running as cluster-admin strips the fields on a blockownerdeletion reference. If this happens, then the check may pass even when it should be rejected. Because it's a synthetic diff --git a/keps/sig-api-machinery/2336-OwnerReference-resource-field/kep.yaml b/keps/sig-api-machinery/2336-OwnerReference-resource-field/kep.yaml new file mode 100644 index 00000000000..b9fdda44d2c --- /dev/null +++ b/keps/sig-api-machinery/2336-OwnerReference-resource-field/kep.yaml @@ -0,0 +1,18 @@ +title: OwnerReference Resource Field +kep-number: 2336 +authors: + - "@deads2k" +owning-sig: sig-api-machinery +participating-sigs: + - sig-api-machinery +reviewers: + - "@sttts" + - "@caesarxuchao" + - "@liggitt" +approvers: + - "@caesarxuchao" + - "@lavalamp" +editor: "@deads2k" +creation-date: 2019-06-07 +last-updated: 2019-06-12 +status: implementable diff --git a/keps/sig-api-machinery/20190612-crd-group-protection.md b/keps/sig-api-machinery/2337-k8s.io-group-protection/README.md similarity index 92% rename from keps/sig-api-machinery/20190612-crd-group-protection.md rename to keps/sig-api-machinery/2337-k8s.io-group-protection/README.md index aee81f661bf..80280b229bb 100644 --- a/keps/sig-api-machinery/20190612-crd-group-protection.md +++ b/keps/sig-api-machinery/2337-k8s.io-group-protection/README.md @@ -1,23 +1,3 @@ ---- -title: k8s.io Group Protection -authors: - - "@deads2k" -owning-sig: sig-api-machinery -participating-sigs: - - sig-api-machinery - - sig-architecture -reviewers: - - "@sttts" - - "@jpbetz" - - "@liggitt" -approvers: - - "@liggitt" - - "@sttts" -editor: "@deads2k" -creation-date: 2019-06-12 -last-updated: 2019-07-03 -status: implemented ---- # k8s.io Group Protection @@ -53,7 +33,7 @@ space inside of the kubernetes API groups, the kube-apiserver needs to be update This KEP proposes adding an `api-approved.kubernetes.io` annotation to CustomResourceDefinition. This is only needed if the CRD group is `k8s.io`, `kubernetes.io`, or ends with `.k8s.io`, `.kubernetes.io`. The value should be a link to a to a URL where the current spec was approved, so updates to the spec should also update the URL. - + ```yaml metadata: annotations: @@ -63,10 +43,10 @@ metadata: ## Motivation * Prevent accidentally including an unapproved API in community owned API groups - + ### Goals -* Ensure API consistency. +* Ensure API consistency. * Prevent accidentally claiming reserved named. ### Non-Goals @@ -78,9 +58,9 @@ metadata: This KEP proposes adding an `api-approved.kubernetes.io` annotation to CustomResourceDefinition. This is only needed if the CRD group is `k8s.io`, `kubernetes.io`, or ends with `.k8s.io`, `.kubernetes.io`. The value should be a link to the pull request where the API has been approved. If the API is unapproved, you may set the annotation to a string starting -with `"unapproved"`. For instance, `"unapproved, temporarily squatting` or `"unapproved, experimental-only"`. This +with `"unapproved"`. For instance, `"unapproved, temporarily squatting` or `"unapproved, experimental-only"`. This is discouraged. - + ```yaml metadata: annotations: @@ -93,13 +73,13 @@ metadata: "api-approved.kubernetes.io": "unapproved, experimental-only" ``` -This field is used by new kube-apiservers to set the `KubeAPIApproved` condition. +This field is used by new kube-apiservers to set the `KubeAPIApproved` condition. 1. If a new server sees a CRD for a resource in a kube group and sees the annotation set to a URL, it will set the `KubeAPIApproved` condition to true. 2. If a new server sees a CRD for a resource in a kube group and sees the annotation set to `"unapproved.*"`, it will set the `KubeAPIApproved` condition to false. 3. If a new server sees a CRD for a resource in a kube group and does not see the annotation, it will set the `KubeAPIApproved` condition to false. 4. If a new server sees a CRD for a resource outside a kube group, it does not set the `KubeAPIApproved` condition at all. -In v1, this annotation will be required in order to create a CRD for a resource in one of the kube API groups. If the `KubeAPIApproved` condition is false, +In v1, this annotation will be required in order to create a CRD for a resource in one of the kube API groups. If the `KubeAPIApproved` condition is false, the condition message will include a link to https://github.com/kubernetes/enhancements/pull/1111 for reference. ### Behavior of new clusters @@ -109,13 +89,13 @@ the condition message will include a link to https://github.com/kubernetes/enhan 4. CRD for a resource outside the kube group creating via CRD.v1 is contains the `api-approved.kubernetes.io` - fail validation and do not store in etcd. 5. In CRD.v1, remove a required `api-approved.kubernetes.io` - fail validation. 6. In all versions, any update that does not change the `api-approved.kubernetes.io` will go through our current validation rules. - - + + ### Behavior of old clusters 1. Nothing changes. The old clusters will persist and keep the annotation This doesn't actively prevent bad actors from simply setting the annotation, but it does prevent accidentally claiming -an inappropriate name. +an inappropriate name. ### What to do if you accidentally put an unapproved API in a protected group 1. Get the current state and future changes approved. For community projects, this is the best choice if the current state @@ -123,7 +103,7 @@ an inappropriate name. 2. If there are structural problems with the API's current state that prevent approval, you have two choices. 1. restructure in a new version, maintaining a conversion webhook, and plan to stop serving the old version. There are some cases where this may not work if the changes are not roundtrippable, but they should be rare. - 2. restructure in a new API group. There will be no connection to existing data. This may be disruptive for non-alpha APIs, but these + 2. restructure in a new API group. There will be no connection to existing data. This may be disruptive for non-alpha APIs, but these names are reserved and the process of API review has been in place for some time. The expectation is that this is the exceptional case of an exceptional case. 3. Indicate that your API is unapproved by setting the `"api-approved.kubernetes.io"` annotation to something starting with diff --git a/keps/sig-api-machinery/2337-k8s.io-group-protection/kep.yaml b/keps/sig-api-machinery/2337-k8s.io-group-protection/kep.yaml new file mode 100644 index 00000000000..abd38027156 --- /dev/null +++ b/keps/sig-api-machinery/2337-k8s.io-group-protection/kep.yaml @@ -0,0 +1,19 @@ +title: k8s.io Group Protection +kep-number: 2337 +authors: + - "@deads2k" +owning-sig: sig-api-machinery +participating-sigs: + - sig-api-machinery + - sig-architecture +reviewers: + - "@sttts" + - "@jpbetz" + - "@liggitt" +approvers: + - "@liggitt" + - "@sttts" +editor: "@deads2k" +creation-date: 2019-06-12 +last-updated: 2019-07-03 +status: implemented diff --git a/keps/sig-api-machinery/20190625-api-compression-to-ga.md b/keps/sig-api-machinery/2338-graduate-API-gzip-compression-support-to-GA/README.md similarity index 94% rename from keps/sig-api-machinery/20190625-api-compression-to-ga.md rename to keps/sig-api-machinery/2338-graduate-API-gzip-compression-support-to-GA/README.md index c67c4861136..7914ca175e6 100644 --- a/keps/sig-api-machinery/20190625-api-compression-to-ga.md +++ b/keps/sig-api-machinery/2338-graduate-API-gzip-compression-support-to-GA/README.md @@ -1,26 +1,3 @@ ---- -title: Graduate API gzip compression support to GA -authors: - - "@smarterclayton" -owning-sig: sig-api-machinery -participating-sigs: - - sig-cli -reviewers: - - "@lavalamp" - - "@liggitt" -approvers: - - "@liggitt" - - "@lavalamp" -editor: TBD -creation-date: 2019-03-22 -last-updated: 2019-03-22 -status: implementable -see-also: - - "https://github.com/kubernetes/kubernetes/issues/44164" -replaces: -superseded-by: ---- - # Graduate API gzip compression to GA ## Table of Contents diff --git a/keps/sig-api-machinery/2338-graduate-API-gzip-compression-support-to-GA/kep.yaml b/keps/sig-api-machinery/2338-graduate-API-gzip-compression-support-to-GA/kep.yaml new file mode 100644 index 00000000000..40fbdaca7a6 --- /dev/null +++ b/keps/sig-api-machinery/2338-graduate-API-gzip-compression-support-to-GA/kep.yaml @@ -0,0 +1,21 @@ +title: Graduate API gzip compression support to GA +kep-number: 2338 +authors: + - "@smarterclayton" +owning-sig: sig-api-machinery +participating-sigs: + - sig-cli +reviewers: + - "@lavalamp" + - "@liggitt" +approvers: + - "@liggitt" + - "@lavalamp" +editor: TBD +creation-date: 2019-03-22 +last-updated: 2019-03-22 +status: implementable +see-also: + - "https://github.com/kubernetes/kubernetes/issues/44164" +replaces: +superseded-by: diff --git a/keps/sig-api-machinery/20190802-dynamic-coordinated-storage-version.md b/keps/sig-api-machinery/2339-storageversion-api-for-ha-api-servers/README.md similarity index 97% rename from keps/sig-api-machinery/20190802-dynamic-coordinated-storage-version.md rename to keps/sig-api-machinery/2339-storageversion-api-for-ha-api-servers/README.md index 3d07ff9180c..ceaa4a05a2f 100644 --- a/keps/sig-api-machinery/20190802-dynamic-coordinated-storage-version.md +++ b/keps/sig-api-machinery/2339-storageversion-api-for-ha-api-servers/README.md @@ -1,20 +1,3 @@ ---- -title: StorageVersion API for HA API servers -authors: - - "@xuchao" -owning-sig: sig-api-machinery -reviewers: - - "@deads2k" - - "@yliaog" - - "@lavalamp" -approvers: - - "@deads2k" - - "@lavalamp" -creation-date: 2019-08-22 -last-updated: 2019-08-22 -status: implementable ---- - # StorageVersion API for HA API servers ## Table of Contents @@ -71,7 +54,7 @@ type StorageVersion struct { TypeMeta // The name is .. ObjectMeta - + // Spec is omitted because there is no spec field. // Spec StorageVersionSpec @@ -94,13 +77,13 @@ type StorageVersionStatus struct { // The latest available observations of the storageVersion's state. // +optional Conditions []StorageVersionCondition - + } // An API server instance reports the version it can decode and the version it // encodes objects to when persisting objects in the backend. type ServerStorageVersion struct { - // The ID of the reporting API server. + // The ID of the reporting API server. // For a kube-apiserver, the ID is configured via a flag. APIServerID string @@ -155,7 +138,7 @@ all participating API servers via some API. ### Updating StorageVersion -During bootstrap, for each resource, the API server +During bootstrap, for each resource, the API server * gets the storageVersion object for this resource, or creates one if it does not exist yet, * gets the list of participating API servers, @@ -229,7 +212,7 @@ performs the following actions for each storageVersion object: storageVersion.status.serverStorageVersions, * after the removal, if all participating API servers have the same encodingVersion, then sets storageVersion.status.AgreedEncodingVersion and - status.condtion. + status.condtion. * checks if the storageVersion.status.serverStorageVersions is empty, * if empty, deletes the storageVersion object (2nd kind of garbage), * otherwise updates the storageVersion object, @@ -345,7 +328,7 @@ accurately reflects the actual storage version used by the apiextension-apiserve Upon storage version changes in the CRD spec, * [one controller][] deletes the existing resource handler of the CRD, so that a new resource handler is created with the latest cached CRD spec is created - upon the next custom resource request. + upon the next custom resource request. * [another controller][] enqueues the CRD, waiting for the worker to updates the discovery document. diff --git a/keps/sig-api-machinery/2339-storageversion-api-for-ha-api-servers/kep.yaml b/keps/sig-api-machinery/2339-storageversion-api-for-ha-api-servers/kep.yaml new file mode 100644 index 00000000000..eb216f3f4d7 --- /dev/null +++ b/keps/sig-api-machinery/2339-storageversion-api-for-ha-api-servers/kep.yaml @@ -0,0 +1,15 @@ +title: StorageVersion API for HA API servers +kep-number: 2339 +authors: + - "@xuchao" +owning-sig: sig-api-machinery +reviewers: + - "@deads2k" + - "@yliaog" + - "@lavalamp" +approvers: + - "@deads2k" + - "@lavalamp" +creation-date: 2019-08-22 +last-updated: 2019-08-22 +status: implementable diff --git a/keps/sig-api-machinery/20191210-consistent-reads-from-cache.md b/keps/sig-api-machinery/2340-Consistent-reads-from-cache/README.md similarity index 98% rename from keps/sig-api-machinery/20191210-consistent-reads-from-cache.md rename to keps/sig-api-machinery/2340-Consistent-reads-from-cache/README.md index eb3a7a239c5..2f3ba1f33f1 100644 --- a/keps/sig-api-machinery/20191210-consistent-reads-from-cache.md +++ b/keps/sig-api-machinery/2340-Consistent-reads-from-cache/README.md @@ -1,27 +1,3 @@ ---- -title: Consistent Reads from Cache -authors: - - "@jpbetz" - - "@wojtek-t" -owning-sig: sig-api-machinery -participating-sigs: - - sig-scalability -reviewers: - - "@wojtek-t" - - "@jingyih" -approvers: - - "@lavalamp" - - "@deads2k" - - "@wojtek-t" -editor: TBD -creation-date: 2019-12-10 -last-updated: 2020-02-19 -status: provisional -see-also: -replaces: -superseded-by: ---- - # Consistent Reads from Cache Kubernetes Get and List requests are guaranteed to be "consistent reads" if the @@ -156,9 +132,9 @@ When an consistent LIST request is received and the watch cache is enabled: - Use the existing `waitUntilFreshAndBlock` function in the watch cache to wait briefly for the watch to catch up to the current revision. - If the block times out, the request will result in rejection. (see "What if the watch cache is stale?" section for details) -To get the revsion we have some options: +To get the revsion we have some options: -- Use an etcd range request with `WithCount` enabled so etcd return only a count and revision +- Use an etcd range request with `WithCount` enabled so etcd return only a count and revision - Use an etcd range request against a known empty range with limit=1 as an additional guard (since etcd does not allow for limit=0) Consistent GET requests will continue to be served directly from etcd. We will @@ -275,7 +251,7 @@ To supporting watch cache pagination: default etcd compaction history of 5 minutes. - The watch cache would need to be resturctured so that is can serve LIST for the resource versions it has returned continuation tokens to clients for. - + Both of these are major changes, and would require scalability validation. Potential approach: @@ -296,7 +272,7 @@ clients must be able to tolerate unpaginated responses for paginated LIST reques The kube-apiserver already switches between serving paginated responses and serving unpaginated responses depending on if the watch cache is enabled for the -types requested. +types requested. User cases that have disabled the watch cache will still receive paginated responses. diff --git a/keps/sig-api-machinery/2340-Consistent-reads-from-cache/kep.yaml b/keps/sig-api-machinery/2340-Consistent-reads-from-cache/kep.yaml new file mode 100644 index 00000000000..08bd1a46c69 --- /dev/null +++ b/keps/sig-api-machinery/2340-Consistent-reads-from-cache/kep.yaml @@ -0,0 +1,22 @@ +title: Consistent Reads from Cache +kep-number: 2340 +authors: + - "@jpbetz" + - "@wojtek-t" +owning-sig: sig-api-machinery +participating-sigs: + - sig-scalability +reviewers: + - "@wojtek-t" + - "@jingyih" +approvers: + - "@lavalamp" + - "@deads2k" + - "@wojtek-t" +editor: TBD +creation-date: 2019-12-10 +last-updated: 2020-02-19 +status: provisional +see-also: +replaces: +superseded-by: diff --git a/keps/sig-api-machinery/34-storage-hash.md b/keps/sig-api-machinery/2341-enabling-clients-to-tell-if-resource-endpoints-serve-the-same-set-of-objects/README.md similarity index 94% rename from keps/sig-api-machinery/34-storage-hash.md rename to keps/sig-api-machinery/2341-enabling-clients-to-tell-if-resource-endpoints-serve-the-same-set-of-objects/README.md index 8328a2c59fb..762afeaa6bc 100644 --- a/keps/sig-api-machinery/34-storage-hash.md +++ b/keps/sig-api-machinery/2341-enabling-clients-to-tell-if-resource-endpoints-serve-the-same-set-of-objects/README.md @@ -1,19 +1,3 @@ ---- -title: Enabling clients to tell if resource endpoints serve the same set of objects -authors: - - "@xuchao" -owning-sig: sig-api-machinery -reviewers: - - "@deads2k" - - "@lavalamp" -approvers: - - "@deads2k" - - "@lavalamp" -creation-date: 2018-10-12 -last-updated: 2018-12-17 -status: provisional ---- - # Enabling clients to tell if resource endpoints serve the same set of objects ## Table of Contents @@ -155,7 +139,7 @@ though the objects are not persisted, the [forward compatibility][] motivation still applies, e.g., admins might configure the admission webhooks to intercept requests sent to all endpoints. Thus, the `resourceID` cannot be left empty, it will be set to `SHA256()`, e.g., for -`tokenReviews`, it's `SHA256(/registry/tokenreviews)`. +`tokenReviews`, it's `SHA256(/registry/tokenreviews)`. [forward compatibility]:#broken-forwards-compatibility @@ -196,7 +180,7 @@ because the discovery API is read-only. 2. Serving a list of all sets of aliasing resources via a new API. Aggregated apiservers make such a design complex. For example, we will need to design how - the aggregated apiserver registers its resource aliases. + the aggregated apiserver registers its resource aliases. 3. Hard coding UUIDs for built-in resources, instead of hashes. This doesn't work for CRDs. diff --git a/keps/sig-api-machinery/2341-enabling-clients-to-tell-if-resource-endpoints-serve-the-same-set-of-objects/kep.yaml b/keps/sig-api-machinery/2341-enabling-clients-to-tell-if-resource-endpoints-serve-the-same-set-of-objects/kep.yaml new file mode 100644 index 00000000000..56cb447fb7c --- /dev/null +++ b/keps/sig-api-machinery/2341-enabling-clients-to-tell-if-resource-endpoints-serve-the-same-set-of-objects/kep.yaml @@ -0,0 +1,14 @@ +title: Enabling clients to tell if resource endpoints serve the same set of objects +kep-number: 2341 +authors: + - "@xuchao" +owning-sig: sig-api-machinery +reviewers: + - "@deads2k" + - "@lavalamp" +approvers: + - "@deads2k" + - "@lavalamp" +creation-date: 2018-10-12 +last-updated: 2018-12-17 +status: provisional diff --git a/keps/sig-api-machinery/35-storage-version-hash.md b/keps/sig-api-machinery/2342-exposing-hashed-storage-versions-via-the-discovery-API/README.md similarity index 95% rename from keps/sig-api-machinery/35-storage-version-hash.md rename to keps/sig-api-machinery/2342-exposing-hashed-storage-versions-via-the-discovery-API/README.md index 1fc46bc8e4a..25de3b087bb 100644 --- a/keps/sig-api-machinery/35-storage-version-hash.md +++ b/keps/sig-api-machinery/2342-exposing-hashed-storage-versions-via-the-discovery-API/README.md @@ -1,19 +1,3 @@ ---- -title: exposing hashed storage versions via the discovery API -authors: - - "@xuchao" -owning-sig: sig-api-machinery -reviewers: - - "@deads2k" - - "@lavalamp" -approvers: - - "@deads2k" - - "@lavalamp" -creation-date: 2019-01-02 -last-updated: 2019-01-04 -status: provisional ---- - # Exposing storage versions in opaque values via the discovery API ## Table of Contents @@ -73,12 +57,12 @@ storage versions are different. ```golang type APIResource struct { // The hash value of the storage version, the version this resource is - // converted to when written to the data store. Value must be treated + // converted to when written to the data store. Value must be treated // as opaque by clients. Only equality comparison on the value is valid. // This is an alpha feature and may change or be removed in the future. // The field is populated by the apiserver only if the // StorageVersionHash feature gate is enabled. - // This field will remain optional even if it graduates. + // This field will remain optional even if it graduates. // +optional StorageVersionHash string `json:"storageVersionHash,omitempty"` // These are the existing fields. diff --git a/keps/sig-api-machinery/2342-exposing-hashed-storage-versions-via-the-discovery-API/kep.yaml b/keps/sig-api-machinery/2342-exposing-hashed-storage-versions-via-the-discovery-API/kep.yaml new file mode 100644 index 00000000000..3c5097e9834 --- /dev/null +++ b/keps/sig-api-machinery/2342-exposing-hashed-storage-versions-via-the-discovery-API/kep.yaml @@ -0,0 +1,14 @@ +title: exposing hashed storage versions via the discovery API +kep-number: 2342 +authors: + - "@xuchao" +owning-sig: sig-api-machinery +reviewers: + - "@deads2k" + - "@lavalamp" +approvers: + - "@deads2k" + - "@lavalamp" +creation-date: 2019-01-02 +last-updated: 2019-01-04 +status: provisional diff --git a/keps/sig-api-machinery/storage-migration-auto-trigger.md b/keps/sig-api-machinery/2343-automated-storage-version-migration-with-storage-version-hash/README.md similarity index 97% rename from keps/sig-api-machinery/storage-migration-auto-trigger.md rename to keps/sig-api-machinery/2343-automated-storage-version-migration-with-storage-version-hash/README.md index 12b8b4af322..7865f18603a 100644 --- a/keps/sig-api-machinery/storage-migration-auto-trigger.md +++ b/keps/sig-api-machinery/2343-automated-storage-version-migration-with-storage-version-hash/README.md @@ -1,19 +1,3 @@ ---- -title: Automated Storage Version Migration with Storage Version Hash -authors: - - "@xuchao" -owning-sig: sig-api-machinery -reviewers: - - "@deads2k" - - "@yliaog" -approvers: - - "@deads2k" - - "@lavalamp" -creation-date: 2019-01-23 -last-updated: 2019-01-23 -status: provisional ---- - ## Table of Contents @@ -78,7 +62,7 @@ type StorageStateStatus { // +optional LastHeartbeatTime metav1.Time } -``` +``` We had considered making `PersistedStorageVersionHashes` part of the `spec` instead of the `status`, because the stored versions cannot be deduced @@ -96,7 +80,7 @@ status because API. [CRD.status.StoredVersions]:https://github.com/kubernetes/kubernetes/blob/697c2316faaabae8ef8371032b60be65d7795e68/staging/src/k8s.io/apiextensions-apiserver/pkg/apis/apiextensions/v1beta1/types.go#L305 - + ## Storage migration triggering controller We will add a controller that monitors the discovery documents and the @@ -204,7 +188,7 @@ versions might be configured differently in different API servers. Storage migration shouldn't start until the rolling upgrade is done. Ideally, for an HA cluster undergoing rolling upgrade, the following should -happen for each resource: +happen for each resource: * the discovery document should expose the lack of consensus by listing all storage version hashes supported by different API servers. * the storage migration triggering controller should diff --git a/keps/sig-api-machinery/2343-automated-storage-version-migration-with-storage-version-hash/kep.yaml b/keps/sig-api-machinery/2343-automated-storage-version-migration-with-storage-version-hash/kep.yaml new file mode 100644 index 00000000000..281c53a55ce --- /dev/null +++ b/keps/sig-api-machinery/2343-automated-storage-version-migration-with-storage-version-hash/kep.yaml @@ -0,0 +1,14 @@ +title: Automated Storage Version Migration with Storage Version Hash +kep-number: 2343 +authors: + - "@xuchao" +owning-sig: sig-api-machinery +reviewers: + - "@deads2k" + - "@yliaog" +approvers: + - "@deads2k" + - "@lavalamp" +creation-date: 2019-01-23 +last-updated: 2019-01-23 +status: provisional