From 1a05aa2a055627f014e534121ef1a04c6c271070 Mon Sep 17 00:00:00 2001 From: Keith Mattix II Date: Fri, 7 Apr 2023 14:39:14 -0500 Subject: [PATCH 01/10] Publsh GEP 1924 Signed-off-by: Keith Mattix II --- geps/gep-1924.md | 76 ++++++++++++++++++++++++++++++++++++++++++++++++ mkdocs.yml | 1 + 2 files changed, 77 insertions(+) create mode 100644 geps/gep-1924.md diff --git a/geps/gep-1924.md b/geps/gep-1924.md new file mode 100644 index 0000000000..522f217222 --- /dev/null +++ b/geps/gep-1924.md @@ -0,0 +1,76 @@ +# GEP-1924: Mesh parentRef binding + +* Issue: [#1824](https://github.com/kubernetes-sigs/gateway-api/issues/1824) +* Status: Provisional + +## Overview + +[GEP-1426](https://gateway-api.sigs.k8s.io/geps/gep-1426/) defines how xRoute resources should be bound (via parentRef) for service mesh implementations. However, the existing spec only explicitly mentions binding to the Service resource which is not the desired end state for the spec’s mesh capabilities. This GEP clarifies what kind of resources xRoutes can bind to for mesh implementations and goes into more detail what qualities xRoute parentRefs must have. + +## Motivation + +The GAMMA initiative has been working to bring service mesh use-cases to the Gateway API spec, taking the best practices and learnings from mesh implementations and codifying them in a spec. Most mesh users are familiar with using the Kubernetes `Service` resource as the foundation for traffic routing. Generaly, this architecture makes perfect sense; unfortunately, `Service` is far too coupled of a resource. It orchestrates IP address allocation, DNS, endpoint collection and propagation, load balancing, etc. For this reason, it **cannot** be the right long-term answer for parentRef binding; however, it is the only feasible option that Kubernetes has for mesh implementations today. We expect this to change (indeed, we hope to be a part of that change), but in the interest of developing a spec now, we must once again lean on the Service resource. Additionally, we provide provisions to support additional resources as a parentRef. + +## The Frontend Role + +As described in [GEP-1324](https://gateway-api.sigs.k8s.io/geps/gep-1324), the GAMMA spec classifies the Kubernetes Service resource as having two roles: frontend and backend. From the GEP glossary: + +> The "service frontend" refers to how we call a service. In a Service, this is an automatically allocated DNS name (name.namespace.svc.cluster.local) and IP address (ClusterIP). Not all services have frontends - for example, "headless" Services. +> The "service backend" refers to the target of a service. In Service, this is Endpoints (or EndpointSlice). + +Because the Service resource’s roles are split in this way, it is both a valid parentRef and backendRef for a given xRoute resource. However, Service is not the only resource that can fulfill the frontend role. While the Gateway API spec couldn’t possibly enumerate every existing (and future) frontend-like resource, it can specify a subset of resources that implementations MUST support as parentRefs under as a part of core conformance. Meshes MAY support other implementation-specific resources as parentRefs. The spec maintainers also reserve the right to add additional resources to core conformance as the spec evolves. + +## Core Conformance + +At present (v0.7.0+), there is only 1 parentRef resources that is under core conformance: Service. The semantics of the Service binding can be found in GEP 1426. + +## Extended Conformance + +In addition to Service, there are other optional parentRef resources that, if used by implementations, MUST adhere to the spec’s prescriptions. At the time of writing (v0.70), there are two resources in extended conformance: ServiceImport (part of the mcs-api, currently in alpha) and Gateway . The semantics of ServiceImport parentRef binding can be found in GEP 1748 (Note: Headless ServiceImports are out of scope and not currently a part of the spec). The semantics for Gateways in the context of GAMMA implementations are laid out below: + +### `Gateways` + +There has been much discussion around cluster local Gateways (i.e. Gateways not associated with a traditional load balancer). While there are various potential UX impairments (e.g. what’s the difference between a GAMMA HTTPRoute with a Gateway parentRef and an ingress implementation’s HTTPRoute?), there is no technical reason why a Gateway cannot be a valid GAMMA parentRef. In this case, the mesh implementation MUST use the ClusterIP of the Gateway for traffic matching (but not necessarily to the exclusion of the External IP). + +### Why not `IPAddress` + +In Kubernetes 1.27, there will be a new IPAddress resource added to networking.k8s.io/v1alpha1 as part of [KEP 1880](https://github.com/kubernetes/enhancements/tree/master/keps/sig-network/1880-multiple-service-cidrs#proposal). Naturally, it makes sense to examine whether or not this new resource makes sense as a GAMMA aware parentRef. At first glance, IPAddress seems to be an appropriate abstraction for the “frontend” role we’ve been discussing; every Kubernetes Service is accessed over the network via one of its ip addresses. Furthermore, the fact that the Service resource auto-creates an IPAddress is encouraging. However, the fact that the name of the IPAddress is simply the decimal/hex ip address and not a human-readable Service name makes the UX untenable as a parentRef. + +## Implementation-specific `parentRef`s + +The remainder of this GEP will detail considerations and guidelines for mesh implementations if they wish to enable an implementation-specific resource as a parentRef. Recall that the frontend role of a (generic) service is how one calls the service. In the service mesh transparent proxy context, the frontend role (and parentRef by extension) is effectively the matching mechanism for the specified route. For the Service parentRef, this means that the mesh should apply a particular xRoute’s configuration if the destination ip address for a given connection is the ClusterIP of that parentRef Service. If a mesh wishes to use an implementation-specific resource for parentRef, that resource MUST contain layer-appropriate information suitable for traffic matching (e.g. no Host header capture in TCPRoute). For example, consider the following HTTPRoute with an Istio ServiceEntry as a parentRef: + +```yaml +apiVersion: networking.istio.io/v1beta1 +kind: ServiceEntry +metadata: + name: external-svc-httpbin + namespace : egress +spec: + hosts: + - example.com + exportTo: + - "." + location: MESH_EXTERNAL + ports: + - number: 80 + name: http + protocol: HTTP + resolution: DNS +--- +apiVersion: gateway.networking.k8s.io/v1beta1 +kind: HTTPRoute +metadata: + name: mongo-external +spec: + parentRefs: + - kind: ServiceEntry + group: networking.istio.io/v1beta1 + name: external-svc-httpbin + namespace: egress + sectionName: http # referencing the port name + rules: + - backendRefs: + - name: internal-example + port: 80 +``` diff --git a/mkdocs.yml b/mkdocs.yml index 4a91c0c6da..20108338f4 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -82,6 +82,7 @@ nav: - geps/gep-1426.md - geps/gep-1324.md - geps/gep-1282.md + - geps/gep-1924.md - Prototyping: - geps/gep-1709.md - Experimental: From 55cb13763f573797dff8c79143a59e7dab062429 Mon Sep 17 00:00:00 2001 From: Keith Mattix II Date: Fri, 7 Apr 2023 14:43:44 -0500 Subject: [PATCH 02/10] Rename GEP to 1926 Signed-off-by: Keith Mattix II --- geps/gep-1924.md | 2 +- mkdocs.yml | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/geps/gep-1924.md b/geps/gep-1924.md index 522f217222..0985302719 100644 --- a/geps/gep-1924.md +++ b/geps/gep-1924.md @@ -1,4 +1,4 @@ -# GEP-1924: Mesh parentRef binding +# GEP-1926: Mesh parentRef binding * Issue: [#1824](https://github.com/kubernetes-sigs/gateway-api/issues/1824) * Status: Provisional diff --git a/mkdocs.yml b/mkdocs.yml index 20108338f4..4096d84edb 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -82,7 +82,7 @@ nav: - geps/gep-1426.md - geps/gep-1324.md - geps/gep-1282.md - - geps/gep-1924.md + - geps/gep-1926.md - Prototyping: - geps/gep-1709.md - Experimental: From 9cc739c277e645ed7cfbe11985367c4abd17803f Mon Sep 17 00:00:00 2001 From: Keith Mattix II Date: Mon, 10 Apr 2023 17:16:01 -0500 Subject: [PATCH 03/10] Merge GEP into 1426 Signed-off-by: Keith Mattix II --- geps/gep-1426.md | 67 +++++++++++++++++++++++++++++++++++++----- geps/gep-1924.md | 76 ------------------------------------------------ mkdocs.yml | 1 - 3 files changed, 60 insertions(+), 84 deletions(-) delete mode 100644 geps/gep-1924.md diff --git a/geps/gep-1426.md b/geps/gep-1426.md index 27b5f1a878..39859d1e5e 100644 --- a/geps/gep-1426.md +++ b/geps/gep-1426.md @@ -6,7 +6,7 @@ ## Overview Similar to how `xRoutes` bind to `Gateways` and manage North/South traffic flows in Gateway API’s ingress use-case, it would be natural to adopt a similar model for traffic routing concerns in service mesh deployments. The purpose of this GEP is to add a mechanism to the Gateway API spec for the purpose of associating the various `xRoute` types to a service mesh and offering a model for service owners to manage traffic splitting configurations. - + This GEP is intended to establish an implementable, but experimental, baseline for supporting basic service mesh traffic routing functionality through the Gateway API spec. ## Personas @@ -43,11 +43,15 @@ This GEP uses the [roles and personas](https://gateway-api.sigs.k8s.io/concepts/ It is proposed that an application owner should configure traffic rules for a mesh service by configuring an `xRoute` with a Kubernetes `Service` resource as a `parentRef`. -This approach is dependent on both the "frontend" role of the Kubernetes `Service` resource as defined in [GEP-1324: Service Mesh in Gateway API](https://gateway-api.sigs.k8s.io/geps/gep-1324/#service) when used as a `parentRef` and the "backend" role of `Service` when used as a `backendRef`. It would use the Kubernetes service name to match traffic for meshes implementing "transparent proxy" functionality, but the `backendRef` endpoints would ultimately be used for the canonical IP address(es) to which traffic should be redirected by rules defined in this `xRoute`. This approach leverages the existing points of extensibility within the Gateway API spec, and would not require introducing any API changes or new resources, only defining expected behavior. +This approach is dependent on both the "frontend" role of the Kubernetes `Service` resource as defined in [GEP-1324: Service Mesh in Gateway API](https://gateway-api.sigs.k8s.io/geps/gep-1324/#service) when used as a `parentRef` and the "backend" role of `Service` when used as a `backendRef`. It would use the Kubernetes `Service` name to match traffic for meshes implementing "transparent proxy" functionality, but the `backendRef` endpoints would ultimately be used for the canonical IP address(es) to which traffic should be redirected by rules defined in this `xRoute`. This approach leverages the existing points of extensibility within the Gateway API spec, and would not require introducing any API changes or new resources, only defining expected behavior. + +### Why Service? + +The GAMMA initiative has been working to bring service mesh use-cases to the Gateway API spec, taking the best practices and learnings from mesh implementations and codifying them in a spec. Most mesh users are familiar with using the Kubernetes `Service` resource as the foundation for traffic routing. Generaly, this architecture makes perfect sense; unfortunately, `Service` is far too coupled of a resource. It orchestrates IP address allocation, DNS, endpoint collection and propagation, load balancing, etc. For this reason, it **cannot** be the right long-term answer for parentRef binding; however, it is the only feasible option that Kubernetes has for mesh implementations today. We expect this to change (indeed, we hope to be a part of that change), but in the interest of developing a spec now, we must once again lean on the Service resource. However, we will provide provisions to support additional resources as a parentRef. ## API -``` +```yaml metadata: name: foo-route namespace: store @@ -65,7 +69,7 @@ spec: weight: 10 ``` -In the example above, routing rules have been configured to direct 90% of traffic for the `foo` `Service` to the default "backend" endpoints specified by the `foo` `Service` [`selector`](https://kubernetes.io/docs/concepts/services-networking/service/#defining-a-service) field, and 10% to the `foo-v2` `Service`. This is determined based on the `ClusterIP` (for `Service`) and `ClusterSetIP` (for `ServiceImport`) matching, and for "transparent proxy" mesh implementations would match all requests to `foo.svc.cluster.local` (or arbitrary custom suffix, as the hostname is not specified manually) from within the same namespace, all requests to `foo.store.svc.cluster.local` from other namespaces, and all requests to `foo.store.svc.clusterset.local` for multicluster services, within the scope of the service mesh. +In the example above, routing rules have been configured to direct 90% of traffic for the `foo` `Service` to the default "backend" endpoints specified by the `foo` `Service` [`selector`](https://kubernetes.io/docs/concepts/services-networking/service/#defining-a-service) field, and 10% to the `foo-v2` `Service`. This is determined based on the `ClusterIP` (for `Service`) and `ClusterSetIP` (for `ServiceImport`) matching, and for "transparent proxy" mesh implementations would match all requests to `foo.store.svc.cluster.local` (or arbitrary custom suffix, as the hostname is not specified manually) from within the same namespace, all requests to `foo.store.svc.cluster.local` from other namespaces, and all requests to `foo.store.svc.clusterset.local` for multicluster services, within the scope of the service mesh. ### Omitting `backendRefs` @@ -109,11 +113,60 @@ Services supported as `backendRefs` SHOULD be consistent with expectations for N An alternate pattern additionally supported by this approach would be to target a `Service` without selectors as the `parentRef`. This could be a clean way to create a pure routing construct and abstract a logical frontend, as traffic would resolve to a `backendRef` `Service` with selectors defined on the `HTTPRoute`, or receive a 4xx/5xx error response if no matching path or valid backend was found. -#### Multicluster support with `ServiceImport` +### `parentRef` Conformance Levels + +Currently (v0.7.0), this spec only considers the `Service` resource to be under Core conformance as a `parentRef`. However, Service is not the only resource that can fulfill the frontend role. While the Gateway API spec couldn’t possibly enumerate every existing (and future) frontend-like resource, it can specify a subset of resources that implementations MUST support as parentRefs under as a part of core conformance. Meshes MAY support other implementation-specific resources as parentRefs. The spec maintainers also reserve the right to add additional resources to core conformance as the spec evolves. + +#### Extended Conformance + +In addition to Service, there are other optional parentRef resources that, if used by implementations, MUST adhere to the spec’s prescriptions. At the time of writing (v0.70), there are two resources in extended conformance: ServiceImport (part of the mcs-api, currently in alpha) and Gateway . The semantics of ServiceImport parentRef binding can be found in GEP 1748 (Note: Headless ServiceImports are out of scope and not currently a part of the spec). The semantics for Gateways in the context of GAMMA implementations are laid out below: + +##### `Gateways` + +There has been much discussion around cluster local Gateways (i.e. Gateways not associated with a traditional load balancer). While there are various potential UX impairments (e.g. what’s the difference between a GAMMA HTTPRoute with a Gateway parentRef and an ingress implementation’s HTTPRoute?), there is no technical reason why a Gateway cannot be a valid GAMMA parentRef. In this case, the mesh implementation MUST use the ClusterIP of the Gateway for traffic matching (but not necessarily to the exclusion of the External IP). + +##### Why not `IPAddress` + +In Kubernetes 1.27, there will be a new IPAddress resource added to networking.k8s.io/v1alpha1 as part of [KEP 1880](https://github.com/kubernetes/enhancements/tree/master/keps/sig-network/1880-multiple-service-cidrs#proposal). Naturally, it makes sense to examine whether or not this new resource makes sense as a GAMMA aware parentRef. At first glance, IPAddress seems to be an appropriate abstraction for the “frontend” role we’ve been discussing; every Kubernetes Service is accessed over the network via one of its ip addresses. Furthermore, the fact that the Service resource auto-creates an IPAddress is encouraging. However, the fact that the name of the IPAddress is simply the decimal/hex ip address and not a human-readable Service name makes the UX untenable as a spec-supported parentRef. However, `IPAddress` is NOT disallowed; implementations may use it if they wish. -`ServiceImport` resources allocate a virtual IP in the cluster, so MAY be allowed as a `parentRef`. +#### Implementation-specific `parentRef`s -`ServiceImport` would remain a valid backend option for `xRoute` resources (but _not_ currently a requirement for core conformance), and could be specified alongside a `Service` `backendRef` to split traffic across clusters within a `ClusterSet` (as defined in the [Multi-cluster Service (MCS) APIs project](https://github.com/kubernetes-sigs/mcs-api)). This could be a way to solve the need described in [A use case for using Gateway APIs in Multi-Cluster](https://docs.google.com/document/d/1bjr0uAVMmEtTX4mpU_aGXXapQFsEz_Qt0OnDoVswEEI/edit). +If mesh implementations wish to enable an implementation-specific resource as a parentRef, they may do so as long as that resource meets certain conditions. Recall that the frontend role of a (generic) service is how one calls the service. In the service mesh transparent proxy context, the frontend role (and parentRef by extension) is effectively the matching mechanism for the specified route. For the Service parentRef, this means that the mesh should apply a particular xRoute’s configuration if the destination ip address for a given connection is the ClusterIP of that parentRef Service. If a mesh wishes to use an implementation-specific resource for parentRef, that resource MUST contain layer-appropriate information suitable for traffic matching (e.g. no Host header capture in TCPRoute). For example, consider the following HTTPRoute with an Istio `ServiceEntry` as a parentRef: + +```yaml +apiVersion: networking.istio.io/v1beta1 +kind: ServiceEntry +metadata: + name: external-svc-httpbin + namespace : egress +spec: + hosts: + - example.com + exportTo: + - "." + location: MESH_INTERNAL + ports: + - number: 80 + name: http + protocol: HTTP + resolution: DNS +--- +apiVersion: gateway.networking.k8s.io/v1beta1 +kind: HTTPRoute +metadata: + name: mongo-external +spec: + parentRefs: + - kind: ServiceEntry + group: networking.istio.io/v1beta1 + name: external-svc-httpbin + namespace: egress + sectionName: http # referencing the port name + rules: + - backendRefs: + - name: internal-example + port: 80 +``` ### Route types diff --git a/geps/gep-1924.md b/geps/gep-1924.md deleted file mode 100644 index 0985302719..0000000000 --- a/geps/gep-1924.md +++ /dev/null @@ -1,76 +0,0 @@ -# GEP-1926: Mesh parentRef binding - -* Issue: [#1824](https://github.com/kubernetes-sigs/gateway-api/issues/1824) -* Status: Provisional - -## Overview - -[GEP-1426](https://gateway-api.sigs.k8s.io/geps/gep-1426/) defines how xRoute resources should be bound (via parentRef) for service mesh implementations. However, the existing spec only explicitly mentions binding to the Service resource which is not the desired end state for the spec’s mesh capabilities. This GEP clarifies what kind of resources xRoutes can bind to for mesh implementations and goes into more detail what qualities xRoute parentRefs must have. - -## Motivation - -The GAMMA initiative has been working to bring service mesh use-cases to the Gateway API spec, taking the best practices and learnings from mesh implementations and codifying them in a spec. Most mesh users are familiar with using the Kubernetes `Service` resource as the foundation for traffic routing. Generaly, this architecture makes perfect sense; unfortunately, `Service` is far too coupled of a resource. It orchestrates IP address allocation, DNS, endpoint collection and propagation, load balancing, etc. For this reason, it **cannot** be the right long-term answer for parentRef binding; however, it is the only feasible option that Kubernetes has for mesh implementations today. We expect this to change (indeed, we hope to be a part of that change), but in the interest of developing a spec now, we must once again lean on the Service resource. Additionally, we provide provisions to support additional resources as a parentRef. - -## The Frontend Role - -As described in [GEP-1324](https://gateway-api.sigs.k8s.io/geps/gep-1324), the GAMMA spec classifies the Kubernetes Service resource as having two roles: frontend and backend. From the GEP glossary: - -> The "service frontend" refers to how we call a service. In a Service, this is an automatically allocated DNS name (name.namespace.svc.cluster.local) and IP address (ClusterIP). Not all services have frontends - for example, "headless" Services. -> The "service backend" refers to the target of a service. In Service, this is Endpoints (or EndpointSlice). - -Because the Service resource’s roles are split in this way, it is both a valid parentRef and backendRef for a given xRoute resource. However, Service is not the only resource that can fulfill the frontend role. While the Gateway API spec couldn’t possibly enumerate every existing (and future) frontend-like resource, it can specify a subset of resources that implementations MUST support as parentRefs under as a part of core conformance. Meshes MAY support other implementation-specific resources as parentRefs. The spec maintainers also reserve the right to add additional resources to core conformance as the spec evolves. - -## Core Conformance - -At present (v0.7.0+), there is only 1 parentRef resources that is under core conformance: Service. The semantics of the Service binding can be found in GEP 1426. - -## Extended Conformance - -In addition to Service, there are other optional parentRef resources that, if used by implementations, MUST adhere to the spec’s prescriptions. At the time of writing (v0.70), there are two resources in extended conformance: ServiceImport (part of the mcs-api, currently in alpha) and Gateway . The semantics of ServiceImport parentRef binding can be found in GEP 1748 (Note: Headless ServiceImports are out of scope and not currently a part of the spec). The semantics for Gateways in the context of GAMMA implementations are laid out below: - -### `Gateways` - -There has been much discussion around cluster local Gateways (i.e. Gateways not associated with a traditional load balancer). While there are various potential UX impairments (e.g. what’s the difference between a GAMMA HTTPRoute with a Gateway parentRef and an ingress implementation’s HTTPRoute?), there is no technical reason why a Gateway cannot be a valid GAMMA parentRef. In this case, the mesh implementation MUST use the ClusterIP of the Gateway for traffic matching (but not necessarily to the exclusion of the External IP). - -### Why not `IPAddress` - -In Kubernetes 1.27, there will be a new IPAddress resource added to networking.k8s.io/v1alpha1 as part of [KEP 1880](https://github.com/kubernetes/enhancements/tree/master/keps/sig-network/1880-multiple-service-cidrs#proposal). Naturally, it makes sense to examine whether or not this new resource makes sense as a GAMMA aware parentRef. At first glance, IPAddress seems to be an appropriate abstraction for the “frontend” role we’ve been discussing; every Kubernetes Service is accessed over the network via one of its ip addresses. Furthermore, the fact that the Service resource auto-creates an IPAddress is encouraging. However, the fact that the name of the IPAddress is simply the decimal/hex ip address and not a human-readable Service name makes the UX untenable as a parentRef. - -## Implementation-specific `parentRef`s - -The remainder of this GEP will detail considerations and guidelines for mesh implementations if they wish to enable an implementation-specific resource as a parentRef. Recall that the frontend role of a (generic) service is how one calls the service. In the service mesh transparent proxy context, the frontend role (and parentRef by extension) is effectively the matching mechanism for the specified route. For the Service parentRef, this means that the mesh should apply a particular xRoute’s configuration if the destination ip address for a given connection is the ClusterIP of that parentRef Service. If a mesh wishes to use an implementation-specific resource for parentRef, that resource MUST contain layer-appropriate information suitable for traffic matching (e.g. no Host header capture in TCPRoute). For example, consider the following HTTPRoute with an Istio ServiceEntry as a parentRef: - -```yaml -apiVersion: networking.istio.io/v1beta1 -kind: ServiceEntry -metadata: - name: external-svc-httpbin - namespace : egress -spec: - hosts: - - example.com - exportTo: - - "." - location: MESH_EXTERNAL - ports: - - number: 80 - name: http - protocol: HTTP - resolution: DNS ---- -apiVersion: gateway.networking.k8s.io/v1beta1 -kind: HTTPRoute -metadata: - name: mongo-external -spec: - parentRefs: - - kind: ServiceEntry - group: networking.istio.io/v1beta1 - name: external-svc-httpbin - namespace: egress - sectionName: http # referencing the port name - rules: - - backendRefs: - - name: internal-example - port: 80 -``` diff --git a/mkdocs.yml b/mkdocs.yml index 4096d84edb..4a91c0c6da 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -82,7 +82,6 @@ nav: - geps/gep-1426.md - geps/gep-1324.md - geps/gep-1282.md - - geps/gep-1926.md - Prototyping: - geps/gep-1709.md - Experimental: From a46034d2c9966027a23aa58b9a968fe3d430adb7 Mon Sep 17 00:00:00 2001 From: Keith Mattix II Date: Wed, 12 Apr 2023 13:39:50 -0500 Subject: [PATCH 04/10] Remove Gateway ClusterIP coupling Signed-off-by: Keith Mattix II --- geps/gep-1426.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/geps/gep-1426.md b/geps/gep-1426.md index 39859d1e5e..d68f8c4fa2 100644 --- a/geps/gep-1426.md +++ b/geps/gep-1426.md @@ -123,7 +123,7 @@ In addition to Service, there are other optional parentRef resources that, if us ##### `Gateways` -There has been much discussion around cluster local Gateways (i.e. Gateways not associated with a traditional load balancer). While there are various potential UX impairments (e.g. what’s the difference between a GAMMA HTTPRoute with a Gateway parentRef and an ingress implementation’s HTTPRoute?), there is no technical reason why a Gateway cannot be a valid GAMMA parentRef. In this case, the mesh implementation MUST use the ClusterIP of the Gateway for traffic matching (but not necessarily to the exclusion of the External IP). +There has been much discussion around cluster local Gateways (i.e. Gateways not associated with a traditional load balancer). While there are various potential UX impairments (e.g. what’s the difference between a GAMMA HTTPRoute with a Gateway parentRef and an ingress implementation’s HTTPRoute?), there is no technical reason why a Gateway cannot be a valid GAMMA parentRef if an implementation wishes to do so. ##### Why not `IPAddress` From 7643128e79860ade9e62f3547a5e4603bb58323f Mon Sep 17 00:00:00 2001 From: Keith Mattix II Date: Wed, 12 Apr 2023 13:41:44 -0500 Subject: [PATCH 05/10] Internal example for ServiceEntry Signed-off-by: Keith Mattix II --- geps/gep-1426.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/geps/gep-1426.md b/geps/gep-1426.md index d68f8c4fa2..4b13b73c96 100644 --- a/geps/gep-1426.md +++ b/geps/gep-1426.md @@ -137,7 +137,7 @@ If mesh implementations wish to enable an implementation-specific resource as a apiVersion: networking.istio.io/v1beta1 kind: ServiceEntry metadata: - name: external-svc-httpbin + name: internal-svc-httpbin namespace : egress spec: hosts: @@ -154,12 +154,12 @@ spec: apiVersion: gateway.networking.k8s.io/v1beta1 kind: HTTPRoute metadata: - name: mongo-external + name: mongo-internal spec: parentRefs: - kind: ServiceEntry group: networking.istio.io/v1beta1 - name: external-svc-httpbin + name: internal-svc-httpbin namespace: egress sectionName: http # referencing the port name rules: From 095ef548db3d91d1a054b41d38f55db4b3ad2bea Mon Sep 17 00:00:00 2001 From: Keith Mattix II Date: Tue, 25 Apr 2023 12:50:53 -0500 Subject: [PATCH 06/10] Update geps/gep-1426.md Co-authored-by: Mike Beaumont --- geps/gep-1426.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/geps/gep-1426.md b/geps/gep-1426.md index 4b13b73c96..2dc984d568 100644 --- a/geps/gep-1426.md +++ b/geps/gep-1426.md @@ -131,7 +131,7 @@ In Kubernetes 1.27, there will be a new IPAddress resource added to networking.k #### Implementation-specific `parentRef`s -If mesh implementations wish to enable an implementation-specific resource as a parentRef, they may do so as long as that resource meets certain conditions. Recall that the frontend role of a (generic) service is how one calls the service. In the service mesh transparent proxy context, the frontend role (and parentRef by extension) is effectively the matching mechanism for the specified route. For the Service parentRef, this means that the mesh should apply a particular xRoute’s configuration if the destination ip address for a given connection is the ClusterIP of that parentRef Service. If a mesh wishes to use an implementation-specific resource for parentRef, that resource MUST contain layer-appropriate information suitable for traffic matching (e.g. no Host header capture in TCPRoute). For example, consider the following HTTPRoute with an Istio `ServiceEntry` as a parentRef: +If mesh implementations wish to enable an implementation-specific resource as a parentRef, they may do so as long as that resource meets certain conditions. Recall that the frontend role of a (generic) service is how one calls the service. In the service mesh transparent proxy context, the frontend role (and parentRef by extension) is effectively the matching mechanism for the specified route. For the Service parentRef, this means that the mesh should apply a particular xRoute’s configuration if the destination ip address for a given connection is the ClusterIP of that parentRef Service. If a mesh wishes to use an implementation-specific resource for parentRef, that resource MUST contain layer-appropriate information suitable for traffic matching (e.g. no Host header capture in TCPRoute). For example, the following HTTPRoute with an Istio `ServiceEntry` as a parentRef would be a valid implementation-specific reference: ```yaml apiVersion: networking.istio.io/v1beta1 From 065965de6e789d5a93f942555eb6171bdc78a48d Mon Sep 17 00:00:00 2001 From: Keith Mattix II Date: Fri, 28 Apr 2023 13:17:39 -0500 Subject: [PATCH 07/10] Update geps/gep-1426.md Co-authored-by: Mike Morris --- geps/gep-1426.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/geps/gep-1426.md b/geps/gep-1426.md index 2dc984d568..03841966c1 100644 --- a/geps/gep-1426.md +++ b/geps/gep-1426.md @@ -47,7 +47,7 @@ This approach is dependent on both the "frontend" role of the Kubernetes `Servic ### Why Service? -The GAMMA initiative has been working to bring service mesh use-cases to the Gateway API spec, taking the best practices and learnings from mesh implementations and codifying them in a spec. Most mesh users are familiar with using the Kubernetes `Service` resource as the foundation for traffic routing. Generaly, this architecture makes perfect sense; unfortunately, `Service` is far too coupled of a resource. It orchestrates IP address allocation, DNS, endpoint collection and propagation, load balancing, etc. For this reason, it **cannot** be the right long-term answer for parentRef binding; however, it is the only feasible option that Kubernetes has for mesh implementations today. We expect this to change (indeed, we hope to be a part of that change), but in the interest of developing a spec now, we must once again lean on the Service resource. However, we will provide provisions to support additional resources as a parentRef. +The GAMMA initiative has been working to bring service mesh use-cases to the Gateway API spec, taking the best practices and learnings from mesh implementations and codifying them in a spec. Most mesh users are familiar with using the Kubernetes `Service` resource as the foundation for traffic routing. Generaly, this architecture makes perfect sense; unfortunately, `Service` is far too coupled of a resource. It orchestrates IP address allocation, DNS, endpoint collection and propagation, load balancing, etc. For this reason, it **cannot** be the right long-term answer for `parentRef` binding; however, it is the only feasible option that Kubernetes has for mesh implementations today. We expect this to change (indeed, we hope to be a part of that change), but in the interest of developing a spec now, we must once again lean on the `Service` resource. However, we will provide provisions to support additional resources as a `parentRef`. ## API From 03294059e5ce80cb41a596a8906bb0723f7bd00e Mon Sep 17 00:00:00 2001 From: Keith Mattix II Date: Fri, 28 Apr 2023 13:17:52 -0500 Subject: [PATCH 08/10] Update geps/gep-1426.md Co-authored-by: Mike Morris --- geps/gep-1426.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/geps/gep-1426.md b/geps/gep-1426.md index 03841966c1..9ef3c83bd6 100644 --- a/geps/gep-1426.md +++ b/geps/gep-1426.md @@ -69,7 +69,7 @@ spec: weight: 10 ``` -In the example above, routing rules have been configured to direct 90% of traffic for the `foo` `Service` to the default "backend" endpoints specified by the `foo` `Service` [`selector`](https://kubernetes.io/docs/concepts/services-networking/service/#defining-a-service) field, and 10% to the `foo-v2` `Service`. This is determined based on the `ClusterIP` (for `Service`) and `ClusterSetIP` (for `ServiceImport`) matching, and for "transparent proxy" mesh implementations would match all requests to `foo.store.svc.cluster.local` (or arbitrary custom suffix, as the hostname is not specified manually) from within the same namespace, all requests to `foo.store.svc.cluster.local` from other namespaces, and all requests to `foo.store.svc.clusterset.local` for multicluster services, within the scope of the service mesh. +In the example above, routing rules have been configured to direct 90% of traffic for the `foo` `Service` to the default "backend" endpoints specified by the `foo` `Service` [`selector`](https://kubernetes.io/docs/concepts/services-networking/service/#defining-a-service) field, and 10% to the `foo-v2` `Service`. This is determined based on the `ClusterIP` (for `Service`) and `ClusterSetIP` (for `ServiceImport`) matching, and for "transparent proxy" mesh implementations would match all requests to `foo.svc.cluster.local` (or arbitrary custom suffix, as the hostname is not specified manually) from within the same namespace, all requests to `foo.store.svc.cluster.local` from other namespaces, and all requests to `foo.store.svc.clusterset.local` for multicluster services, within the scope of the service mesh. ### Omitting `backendRefs` From 3c009a3909f138aba5bd5c6e46ab2263fd8f7879 Mon Sep 17 00:00:00 2001 From: Keith Mattix II Date: Fri, 28 Apr 2023 13:18:00 -0500 Subject: [PATCH 09/10] Update geps/gep-1426.md Co-authored-by: Mike Morris --- geps/gep-1426.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/geps/gep-1426.md b/geps/gep-1426.md index 9ef3c83bd6..f38ac5db3a 100644 --- a/geps/gep-1426.md +++ b/geps/gep-1426.md @@ -119,7 +119,7 @@ Currently (v0.7.0), this spec only considers the `Service` resource to be under #### Extended Conformance -In addition to Service, there are other optional parentRef resources that, if used by implementations, MUST adhere to the spec’s prescriptions. At the time of writing (v0.70), there are two resources in extended conformance: ServiceImport (part of the mcs-api, currently in alpha) and Gateway . The semantics of ServiceImport parentRef binding can be found in GEP 1748 (Note: Headless ServiceImports are out of scope and not currently a part of the spec). The semantics for Gateways in the context of GAMMA implementations are laid out below: +In addition to Service, there are other optional parentRef resources that, if used by implementations, MUST adhere to the spec’s prescriptions. At the time of writing (v0.7.0), there is one resource in extended conformance: `ServiceImport` (part of the [MCS API](https://github.com/kubernetes-sigs/mcs-api), currently in alpha). The semantics of `ServiceImport` `parentRef` binding can be found in [GEP-1748](https://gateway-api.sigs.k8s.io/geps/gep-1748/) (Note: Headless `ServiceImport` is out of scope and not currently a part of the spec). ##### `Gateways` From 10e70d121df5a9189a7f1b829f813947e39187bd Mon Sep 17 00:00:00 2001 From: Keith Mattix II Date: Fri, 28 Apr 2023 13:20:52 -0500 Subject: [PATCH 10/10] Address PR feedback Signed-off-by: Keith Mattix II --- geps/gep-1426.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/geps/gep-1426.md b/geps/gep-1426.md index f38ac5db3a..7b2cdc617f 100644 --- a/geps/gep-1426.md +++ b/geps/gep-1426.md @@ -43,7 +43,7 @@ This GEP uses the [roles and personas](https://gateway-api.sigs.k8s.io/concepts/ It is proposed that an application owner should configure traffic rules for a mesh service by configuring an `xRoute` with a Kubernetes `Service` resource as a `parentRef`. -This approach is dependent on both the "frontend" role of the Kubernetes `Service` resource as defined in [GEP-1324: Service Mesh in Gateway API](https://gateway-api.sigs.k8s.io/geps/gep-1324/#service) when used as a `parentRef` and the "backend" role of `Service` when used as a `backendRef`. It would use the Kubernetes `Service` name to match traffic for meshes implementing "transparent proxy" functionality, but the `backendRef` endpoints would ultimately be used for the canonical IP address(es) to which traffic should be redirected by rules defined in this `xRoute`. This approach leverages the existing points of extensibility within the Gateway API spec, and would not require introducing any API changes or new resources, only defining expected behavior. +This approach is dependent on both the "frontend" role of the Kubernetes `Service` resource as defined in [GEP-1324: Service Mesh in Gateway API](https://gateway-api.sigs.k8s.io/geps/gep-1324/#service) when used as a `parentRef` and the "backend" role of `Service` when used as a `backendRef`. The conformant implementation would use the Kubernetes `Service` name to match traffic for meshes, but the `backendRef` endpoints would ultimately be used for the canonical IP address(es) to which traffic should be redirected by rules defined in this `xRoute`. This approach leverages the existing points of extensibility within the Gateway API spec, and would not require introducing any API changes or new resources, only defining expected behavior. ### Why Service? @@ -121,10 +121,6 @@ Currently (v0.7.0), this spec only considers the `Service` resource to be under In addition to Service, there are other optional parentRef resources that, if used by implementations, MUST adhere to the spec’s prescriptions. At the time of writing (v0.7.0), there is one resource in extended conformance: `ServiceImport` (part of the [MCS API](https://github.com/kubernetes-sigs/mcs-api), currently in alpha). The semantics of `ServiceImport` `parentRef` binding can be found in [GEP-1748](https://gateway-api.sigs.k8s.io/geps/gep-1748/) (Note: Headless `ServiceImport` is out of scope and not currently a part of the spec). -##### `Gateways` - -There has been much discussion around cluster local Gateways (i.e. Gateways not associated with a traditional load balancer). While there are various potential UX impairments (e.g. what’s the difference between a GAMMA HTTPRoute with a Gateway parentRef and an ingress implementation’s HTTPRoute?), there is no technical reason why a Gateway cannot be a valid GAMMA parentRef if an implementation wishes to do so. - ##### Why not `IPAddress` In Kubernetes 1.27, there will be a new IPAddress resource added to networking.k8s.io/v1alpha1 as part of [KEP 1880](https://github.com/kubernetes/enhancements/tree/master/keps/sig-network/1880-multiple-service-cidrs#proposal). Naturally, it makes sense to examine whether or not this new resource makes sense as a GAMMA aware parentRef. At first glance, IPAddress seems to be an appropriate abstraction for the “frontend” role we’ve been discussing; every Kubernetes Service is accessed over the network via one of its ip addresses. Furthermore, the fact that the Service resource auto-creates an IPAddress is encouraging. However, the fact that the name of the IPAddress is simply the decimal/hex ip address and not a human-readable Service name makes the UX untenable as a spec-supported parentRef. However, `IPAddress` is NOT disallowed; implementations may use it if they wish. @@ -168,6 +164,10 @@ spec: port: 80 ``` +##### `Gateways` + +There has been much discussion around cluster local Gateways (i.e. Gateways not associated with a traditional load balancer). While there are various potential UX impairments (e.g. what’s the difference between a GAMMA HTTPRoute with a Gateway parentRef and an ingress implementation’s HTTPRoute?), there is no technical reason why a Gateway cannot be a valid GAMMA parentRef if an implementation wishes to do so. + ### Route types All types currently defined in the gateway-api core (`HTTP`, `GRPC`, `TCP`, `TLS`, and `UDP`) are available for use in a Mesh implementation.