Wrong health check with end-to-end https scheme

[sdouche]

Hi,
I'm trying to set https end-to-end (GCP LB to K8S Pod). On the Pod I have gunicorn that doesn't support HTTP and HTTPS in the same time, but only one. So I must use HTTPS everywhere.

I set this in the deployment object:

        readinessProbe:
          httpGet:
            scheme: HTTPS
            path: /heartbeat/
            port: 443
          initialDelaySeconds: 5
          timeoutSeconds: 1
          periodSeconds: 10
          successThreshold: 1
          failureThreshold: 10

I add this to the service object:

  annotations:
    service.alpha.kubernetes.io/app-protocols: '{"my-https-port":"HTTPS"}'

Unfortunately, the health check is set to check / on HTTP, and the backend service is on HTTP.

Note: The same configuration with HTTP works well (the LB is up and running).

Did I miss something?

Regards.

[pwistrand]

I've been having a similar issue over the last couple of days. Have you tried removing the readinessProbe from the deployment? I've witnessing that the GCE ingress doesn't pay any attention to it despite what the documentation says (especially as its still being discussed as a feature #42)

[nicksardo]

I can't reproduce. Post a simple reproducible manifest of the ingress/service/deployment. Make sure you're actually naming the port "my-https-port".

[sdouche]

Hi @nicksardo,
Apologize for the late reply. I use the same named port in service:

apiVersion: v1
kind: Service
metadata:
  name: booking
  namespace: booking
  annotations:
    service.alpha.kubernetes.io/app-protocols: '{"booking-port":"HTTPS"}'  
spec:
  type: NodePort
  ports:
  - port: 8001
    targetPort: 8001
    protocol: TCP
    name: booking-port
  selector:
    app: booking

And ingress:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: booking
  namespace: booking
  annotations:
    kubernetes.io/ingress.class: "gce"
    kubernetes.io/ingress.allow-http: "false"
    kubernetes.io/ingress.global-static-ip-name: booking
    service.alpha.kubernetes.io/app-protocols: '{"booking-port":"HTTPS"}' 
spec:
  tls:
  - hosts:
    - xxx
    secretName: booking
  rules:
  - host: xxx
    http:
      paths:
      - path: /*
        backend:
          serviceName: booking
          servicePort: booking-port

But I have this strange behavior:

Kubernetes says backends aren't healthy but the LB says they're good.

But surprise after 5 minutes, it also good for k8s:
ingress.kubernetes.io/backends: {"k8s-be-30656--d41670d42012045e":"HEALTHY","k8s-be-32505--d41670d42012045e":"HEALTHY"}

[nicksardo]

The console shows that the LB is correctly using backend HTTPS.

I suggest you disregard that ingress.kubernetes.io/backends annotation. It's faulty in a few ways and most certainly outdated when compared to the Google Cloud Console. Also, setting service.alpha.kubernetes.io/app-protocols on the ingress object is ignored - it's only relevant on the service.

[sdouche]

I suggest you disregard that ingress.kubernetes.io/backends annotation. It's faulty in a few ways and most certainly outdated when compared to the Google Cloud Console.

Understood.

Also, setting service.alpha.kubernetes.io/app-protocols on the ingress object is ignored - it's only relevant on the service.

You're right, wrong copy / paste.

Thanks.