Gateway API doc issues

[ericvn]

First tried with kind clusters (various k8s versions) and run into an issue with:

kubectl get crd gateways.gateway.networking.k8s.io || \                                                                                         
  { kubectl kustomize "github.com/kubernetes-sigs/gateway-api/config/crd?ref=v0.6.0" | kubectl apply -f -; }

Error from server (NotFound): customresourcedefinitions.apiextensions.k8s.io "gateways.gateway.networking.k8s.io" not found
customresourcedefinition.apiextensions.k8s.io/gatewayclasses.gateway.networking.k8s.io created
customresourcedefinition.apiextensions.k8s.io/gateways.gateway.networking.k8s.io created
customresourcedefinition.apiextensions.k8s.io/httproutes.gateway.networking.k8s.io created

The command works fine against a minikube cluster.

Continuing onward with minikube, this command is failing for me

kubectl wait --for=condition=ready gtw bookinfo-gateway
error: timed out waiting for the condition on gateways/bookinfo-gateway

The gateway is there:

kubectl get gtw
NAME               CLASS   ADDRESS   PROGRAMMED   AGE
bookinfo-gateway   istio                          73s

I notice:

kubectl wait --for=condition=accepted gtw bookinfo-gateway
gateway.gateway.networking.k8s.io/bookinfo-gateway condition met

works.

[ericvn]

Does it make sense to at least update the for==condition?

[ericvn]

The initial command that failed also has the same error against an IKS cluster I created.

[frankbu]

Strange that minikube didn't have the error when installing the crds. That implies it already had them, so the question is what version?

The fact that the wait for condition=ready isn't working, implies that the gateway isn't coming up (maybe related to the wrong installed CRD version in the cluster). You can check kubectl describe pod to see why it's not running

[ericvn]

> k describe gtw bookinfo-gateway
Name:         bookinfo-gateway
Namespace:    default
Labels:       <none>
Annotations:  <none>
API Version:  gateway.networking.k8s.io/v1beta1
Kind:         Gateway
Metadata:
  Creation Timestamp:  2023-02-10T21:53:12Z
  Generation:          1
  Managed Fields:
    API Version:  gateway.networking.k8s.io/v1beta1
    Fields Type:  FieldsV1
    fieldsV1:
      f:status:
        f:conditions:
          k:{"type":"Accepted"}:
            .:
            f:lastTransitionTime:
            f:message:
            f:observedGeneration:
            f:reason:
            f:status:
            f:type:
          k:{"type":"Scheduled"}:
            .:
            f:lastTransitionTime:
            f:message:
            f:observedGeneration:
            f:reason:
            f:status:
            f:type:
    Manager:      istio.io/gateway-controller
    Operation:    Apply
    Subresource:  status
    Time:         2023-02-10T21:53:15Z
    API Version:  gateway.networking.k8s.io/v1beta1
    Fields Type:  FieldsV1
    fieldsV1:
      f:metadata:
        f:annotations:
          .:
          f:kubectl.kubernetes.io/last-applied-configuration:
      f:spec:
        .:
        f:gatewayClassName:
        f:listeners:
          .:
          k:{"name":"http"}:
            .:
            f:allowedRoutes:
              .:
              f:namespaces:
                .:
                f:from:
            f:name:
            f:port:
            f:protocol:
    Manager:      kubectl-client-side-apply
    Operation:    Update
    Time:         2023-02-10T21:53:12Z
    API Version:  gateway.networking.k8s.io/v1beta1
    Fields Type:  FieldsV1
    fieldsV1:
      f:status:
        f:conditions:
          k:{"type":"Ready"}:
            .:
            f:lastTransitionTime:
            f:message:
            f:observedGeneration:
            f:reason:
            f:status:
            f:type:
        f:listeners:
          .:
          k:{"name":"http"}:
            .:
            f:attachedRoutes:
            f:conditions:
              .:
              k:{"type":"Accepted"}:
                .:
                f:lastTransitionTime:
                f:message:
                f:observedGeneration:
                f:reason:
                f:status:
                f:type:
              k:{"type":"Conflicted"}:
                .:
                f:lastTransitionTime:
                f:message:
                f:observedGeneration:
                f:reason:
                f:status:
                f:type:
              k:{"type":"Detached"}:
                .:
                f:lastTransitionTime:
                f:message:
                f:observedGeneration:
                f:reason:
                f:status:
                f:type:
              k:{"type":"Programmed"}:
                .:
                f:lastTransitionTime:
                f:message:
                f:observedGeneration:
                f:reason:
                f:status:
                f:type:
              k:{"type":"Ready"}:
                .:
                f:lastTransitionTime:
                f:message:
                f:observedGeneration:
                f:reason:
                f:status:
                f:type:
              k:{"type":"ResolvedRefs"}:
                .:
                f:lastTransitionTime:
                f:message:
                f:observedGeneration:
                f:reason:
                f:status:
                f:type:
            f:name:
            f:supportedKinds:
    Manager:         pilot-discovery
    Operation:       Update
    Subresource:     status
    Time:            2023-02-10T21:53:15Z
  Resource Version:  1018
  UID:               6df098c6-dff7-4d6a-bff1-045063b297e8
Spec:
  Gateway Class Name:  istio
  Listeners:
    Allowed Routes:
      Namespaces:
        From:  Same
    Name:      http
    Port:      80
    Protocol:  HTTP
Status:
  Conditions:
    Last Transition Time:  2023-02-10T21:53:15Z
    Message:               Deployed gateway to the cluster
    Observed Generation:   1
    Reason:                Accepted
    Status:                True
    Type:                  Accepted
    Last Transition Time:  2023-02-10T21:53:15Z
    Message:               Deployed gateway to the cluster
    Observed Generation:   1
    Reason:                ResourcesAvailable
    Status:                True
    Type:                  Scheduled
    Last Transition Time:  2023-02-10T21:53:15Z
    Message:               Assigned to service(s) bookinfo-gateway-istio.default.svc.cluster.local:80, but failed to assign to all requested addresses: address pending for hostname "bookinfo-gateway-istio.default.svc.cluster.local"
    Observed Generation:   1
    Reason:                AddressNotAssigned
    Status:                False
    Type:                  Ready
  Listeners:
    Attached Routes:  1
    Conditions:
      Last Transition Time:  2023-02-10T21:53:12Z
      Message:               No errors found
      Observed Generation:   1
      Reason:                Accepted
      Status:                True
      Type:                  Accepted
      Last Transition Time:  2023-02-10T21:53:12Z
      Message:               No errors found
      Observed Generation:   1
      Reason:                NoConflicts
      Status:                False
      Type:                  Conflicted
      Last Transition Time:  2023-02-10T21:53:12Z
      Message:               No errors found
      Observed Generation:   1
      Reason:                Attached
      Status:                False
      Type:                  Detached
      Last Transition Time:  2023-02-10T21:53:12Z
      Message:               No errors found
      Observed Generation:   1
      Reason:                Programmed
      Status:                True
      Type:                  Programmed
      Last Transition Time:  2023-02-10T21:53:12Z
      Message:               No errors found
      Observed Generation:   1
      Reason:                Ready
      Status:                True
      Type:                  Ready
      Last Transition Time:  2023-02-10T21:53:12Z
      Message:               No errors found
      Observed Generation:   1
      Reason:                ResolvedRefs
      Status:                True
      Type:                  ResolvedRefs
    Name:                    http
    Supported Kinds:
      Group:  gateway.networking.k8s.io
      Kind:   HTTPRoute
Events:       <none>

[ericvn]

So looks like Message: Assigned to service(s) bookinfo-gateway-istio.default.svc.cluster.local:80, but failed to assign to all requested addresses: address pending for hostname "bookinfo-gateway-istio.default.svc.cluster.local"

I'll close this for now as this is working once the EXTERNAL-IP gets set.

[frankbu]

Gateway API requires external loadbalancer, so you MUST use minikube tunnel, but even there minikube isn't great because you can only have one loadbalancer-type service, so if you already have a gateway (gateway-api or plain Istio), a second one won't work (or it steals the LB and the first one stops working). Bottom line is that minikube isn't the best env for Gateway API. Also kind will absolutely need MetalLB installed (which doesn't work on a Mac).