Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How can I redirect 80 to 443, and redirect 8000 to 4443 in the same ingress and same load balancer? #4060

Open
monelgordillo opened this issue Feb 18, 2025 · 5 comments

Comments

@monelgordillo
Copy link

monelgordillo commented Feb 18, 2025

How can I redirect 80 to 443, and redirect 8000 to 4443 in the same ingress and same load balancer?

Port 80 -> 443 (HTTPS)
Port 8000 -> 4443 (HTTPS)

I tried this, but everything it is not working as expected. Everything is redirecting to 4443.

alb.ingress.kubernetes.io/actions.ssl-redirect-blue: >
      {"Type": "redirect", "RedirectConfig": {"Protocol": "HTTPS", "Port": "443", "StatusCode": "HTTP_302"}, "Port": "80"}
 alb.ingress.kubernetes.io/actions.ssl-redirect-green: >
       {"Type": "redirect", "RedirectConfig": {"Protocol": "HTTPS", "Port": "4443", "StatusCode": "HTTP_302"}, "Port": "8000"}
@shraddhabang
Copy link
Collaborator

@monelgordillo
To achieve different HTTP to HTTPS redirects on the same AWS Load Balancer (ALB) using the AWS Load Balancer Controller for Kubernetes, you'll need to use an Ingress Group and multiple Ingress resources. Direct annotations on a single Ingress won't allow you to differentiate redirects based on the original port. The solution involves creating separate Ingresses for each port/redirect combination and grouping them.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  namespace: test-ns
  name: ingress-blue-443
  annotations:
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/target-type: ip
    alb.ingress.kubernetes.io/group.name: ig-group-1
    alb.ingress.kubernetes.io/certificate-arn: arn:aws:acm:us-east-1:123456789909:certificate/5e6e0e02-be13-481e-b815-147a0514ab10
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTPS": 443}]'
spec:
  ingressClassName: alb
  rules:
    - http:
        paths:
          - path: /echoserver
            pathType: Exact
            backend:
              service:
                name: echoserver # Service Name
                port:
                  number: 80
---
# Ingress
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  namespace: test-ns
  name: ingress-blue-80
  annotations:
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/target-type: ip
    alb.ingress.kubernetes.io/group.name: ig-group-1
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}]'
    alb.ingress.kubernetes.io/actions.redirect-blue: >
      {"Type": "redirect", "RedirectConfig": {"Protocol": "HTTPS", "Port": "443", "StatusCode": "HTTP_302"}, "Port": "80"}
spec:
  ingressClassName: alb
  rules:
    - http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: redirect-blue # Service Name
                port:
                  name: use-annotation
---
# Ingress
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  namespace: test-ns
  name: ingress-green-4443
  annotations:
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/target-type: ip
    alb.ingress.kubernetes.io/group.name: ig-group-1
    alb.ingress.kubernetes.io/certificate-arn: arn:aws:acm:us-east-1:123456789909:certificate/5e6e0e02-be13-481e-b815-147a0514ab10
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTPS": 4443}]'
spec:
  ingressClassName: alb
  rules:
    - http:
        paths:
          - path: /game2048
            pathType: Exact
            backend:
              service:
                name: game-2048 # Service Name
                port:
                  number: 80
---
# Ingress
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  namespace: test-ns
  name: ingress-green-8080
  annotations:
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/target-type: ip
    alb.ingress.kubernetes.io/group.name: ig-group-1
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 8080}]'
    alb.ingress.kubernetes.io/actions.redirect-green: >
      {"Type": "redirect", "RedirectConfig": {"Protocol": "HTTPS", "Port": "4443", "StatusCode": "HTTP_302"}, "Port": "8080"}
spec:
  ingressClassName: alb
  rules:
    - http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: redirect-green # Service Name
                port:
                  name: use-annotation

Here's how it works, based on the provided configuration:

Ingress Group: The alb.ingress.kubernetes.io/group.name: ig-group-1 annotation is crucial. It groups the Ingresses together, ensuring they are handled by the same ALB. This is essential for having all the redirects and routing rules managed by a single load balancer.

Separate Ingresses for HTTP Redirects: You create separate Ingress resources for each HTTP port you want to redirect.

ingress-blue-80: Handles traffic on port 80 and redirects it to HTTPS on port 443. The alb.ingress.kubernetes.io/actions.redirect-blue annotation defines this redirect. Note the dummy backend service redirect-blue that is never actually hit.
ingress-green-8080: Handles traffic on port 8080 and redirects it to HTTPS on port 4443. The alb.ingress.kubernetes.io/actions.redirect-green annotation defines this redirect. Note the dummy backend service redirect-green that is never actually hit.

Separate Ingresses for HTTPS Traffic: You also need separate Ingresses to handle the actual HTTPS traffic on ports 443 and 4443.

ingress-blue-443: Handles HTTPS traffic on port 443 and routes it to your echoserver service. The alb.ingress.kubernetes.io/listen-ports: '[{"HTTPS": 443}]' annotation tells the controller to configure the ALB listener for this port.
ingress-green-4443: Handles HTTPS traffic on port 4443 and routes it to your game-2048 service. The alb.ingress.kubernetes.io/listen-ports: '[{"HTTPS": 4443}]' annotation configures the ALB listener.

Hope this will help.

@monelgordillo
Copy link
Author

Howdy @shraddhabang , thank you for this. I tried this. I noticed that listener 80 and 443 were created on the load balancer, but unfortunately 8000 and 4443 listeners were not created.

kubectl logs -n kube-system deployment.apps/aws-load-balancer-controller | grep -i error

{"level":"error","ts":"2025-02-19T14:32:36Z","msg":"Reconciler error","controller":"ingress","object":{"name":"helloworld-s-ingressgrp-green"},"namespace":"","name":"helloworld-s-ingressgrp-green","reconcileID":"2682d6d8-1eb8-419a-9a6e-34d1469f200c","error":"operation error Elastic Load Balancing v2: CreateLoadBalancer, https response error StatusCode: 400, RequestID: c7b4c883-399f-442b-8823-6f6f8fd1a599, DuplicateLoadBalancerName: A load balancer with the same name 'LB_NAME_HERE' exists, but with different settings"}
$

@monelgordillo
Copy link
Author

monelgordillo commented Feb 19, 2025

I set alb.ingress.kubernetes.io/group.name to be the same for all 4 ingresses. Now, I'm getting this error:

{"level":"error","ts":"2025-02-19T18:15:22Z","msg":"Reconciler error","controller":"ingress","object":{"name":"helloworld-s-ingressgrp"},"namespace":"","name":"helloworld-s-ingressgrp","reconcileID":"2732d414-cacd-4f87-b175-d8a61b9fc593","error":"conflicting sslRedirect port: [443 4443]"}

Image

@monelgordillo
Copy link
Author

monelgordillo commented Feb 19, 2025

I had to get rid of a leftover alb.ingress.kubernetes.io/ssl reference.

This the configuration that worked for me:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: helloworld-s-ingress-blue-443
  namespace: staging
  annotations:
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/target-type: ip
    alb.ingress.kubernetes.io/subnets: subnet-XX,subnet-XX 
    alb.ingress.kubernetes.io/group.name: helloworld-s-ingressgrp
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTPS":443}]'
    alb.ingress.kubernetes.io/certificate-arn: arn:aws:acm:us-east-1:XXXX:certificate/XXXX
    alb.ingress.kubernetes.io/ssl-policy: ELBSecurityPolicy-FS-1-2-Res-2020-10   
    service.beta.kubernetes.io/aws-load-balancer-name: LB_NAME # This naming is not working
    alb.ingress.kubernetes.io/load-balancer-name: LB_NAME # https://kubernetes-sigs.github.io/aws-load-balancer-controller/v2.2/guide/ingress/annotations/
    service.beta.kubernetes.io/aws-load-balancer-target-group-attributes: deregistration_delay.timeout_seconds=30
    alb.ingress.kubernetes.io/security-groups: sg-XX 
    alb.ingress.kubernetes.io/tags: ApplicationName=Hello World,Environment=Staging
    service.beta.kubernetes.io/aws-load-balancer-target-node-labels: ApplicationName=Hello World,Environment=Staging 
    alb.ingress.kubernetes.io/ip-address-type: ipv4
    # Target group must be created ahead of time
    # The action-name in the annotation must match the serviceName in the Ingress rules, and servicePort must be use-annotation.
    alb.ingress.kubernetes.io/actions.helloworld-s-blue-service: >
      {"Type": "forward", "ForwardConfig": {"TargetGroups": [{"TargetGroupArn": "arn:aws:elasticloadbalancing:us-east-1:XXXXXX:targetgroup/XX/XXX"}]}}
spec:
  ingressClassName: alb
  rules:
    - http:
        paths:
        - path: /
          pathType: Prefix
          backend:
            service:
              name: helloworld-s-blue-service
              port:
                name: use-annotation
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: helloworld-s-ingress-blue-80
  namespace: staging
  annotations:
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/target-type: ip
    alb.ingress.kubernetes.io/subnets: subnet-XX,subnet-XX
    alb.ingress.kubernetes.io/group.name: helloworld-s-ingressgrp
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTPS":80}]'
    alb.ingress.kubernetes.io/certificate-arn: arn:aws:acm:us-east-1:XXXXXX:certificate/XXX
    alb.ingress.kubernetes.io/actions.redirect-blue: >
      {"Type": "redirect", "RedirectConfig": {"Protocol": "HTTPS", "Port": "443", "StatusCode": "HTTP_302"}, "Port": "80"}
    alb.ingress.kubernetes.io/ssl-policy: ELBSecurityPolicy-FS-1-2-Res-2020-10   
    service.beta.kubernetes.io/aws-load-balancer-name: LB_NAME # This naming is not working
    alb.ingress.kubernetes.io/load-balancer-name: LB_NAME # https://kubernetes-sigs.github.io/aws-load-balancer-controller/v2.2/guide/ingress/annotations/
    service.beta.kubernetes.io/aws-load-balancer-target-group-attributes: deregistration_delay.timeout_seconds=30
    alb.ingress.kubernetes.io/security-groups: sg-XX 
    alb.ingress.kubernetes.io/tags: ApplicationName=Hello World,Environment=Staging
    service.beta.kubernetes.io/aws-load-balancer-target-node-labels: ApplicationName=Hello World,Environment=Staging 
    alb.ingress.kubernetes.io/ip-address-type: ipv4
spec:
  ingressClassName: alb
  rules:
    - http:
        paths:
        - path: /
          pathType: Prefix
          backend:
            service:
              name: redirect-blue
              port:
                name: use-annotation
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: helloworld-s-ingress-green-4443
  namespace: staging
  annotations:
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/target-type: ip
    alb.ingress.kubernetes.io/subnets: subnet-XX,subnet-XX 
    alb.ingress.kubernetes.io/group.name: helloworld-s-ingressgrp
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTPS":4443}]'
    alb.ingress.kubernetes.io/certificate-arn: arn:aws:acm:us-east-1:XXXXXX:certificate/XXX
    alb.ingress.kubernetes.io/ssl-policy: ELBSecurityPolicy-FS-1-2-Res-2020-10   
    service.beta.kubernetes.io/aws-load-balancer-name: LB_NAME # This naming is not working
    alb.ingress.kubernetes.io/load-balancer-name: LB_NAME # https://kubernetes-sigs.github.io/aws-load-balancer-controller/v2.2/guide/ingress/annotations/
    service.beta.kubernetes.io/aws-load-balancer-target-group-attributes: deregistration_delay.timeout_seconds=30
    alb.ingress.kubernetes.io/security-groups: sg-XX 
    alb.ingress.kubernetes.io/tags: ApplicationName=Hello World,Environment=Staging
    service.beta.kubernetes.io/aws-load-balancer-target-node-labels: ApplicationName=Hello World,Environment=Staging 
    alb.ingress.kubernetes.io/ip-address-type: ipv4
    # Target group must be created ahead of time
    # The action-name in the annotation must match the serviceName in the Ingress rules, and servicePort must be use-annotation.
    alb.ingress.kubernetes.io/actions.helloworld-s-green-service: >
      {"Type": "forward", "ForwardConfig": {"TargetGroups": [{"TargetGroupArn": "arn:aws:elasticloadbalancing:us-east-1:XXXXXX:targetgroup/XX/XXX"}]}}
spec:
  ingressClassName: alb
  rules:
    - http:
        paths:
        - path: /
          pathType: Prefix
          backend:
            service:
              name: helloworld-s-green-service
              port:
                name: use-annotation
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: helloworld-s-ingress-green-8000
  namespace: staging
  annotations:
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/target-type: ip
    alb.ingress.kubernetes.io/subnets: subnet-XX,subnet-XX 
    alb.ingress.kubernetes.io/group.name: helloworld-s-ingressgrp
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTPS":8000}]'
    alb.ingress.kubernetes.io/certificate-arn: arn:aws:acm:us-east-1:XXXXXX:certificate/XXX
    alb.ingress.kubernetes.io/actions.redirect-green: >
      {"Type": "redirect", "RedirectConfig": {"Protocol": "HTTPS", "Port": "4443", "StatusCode": "HTTP_302"}, "Port": "8000"}
    alb.ingress.kubernetes.io/ssl-policy: ELBSecurityPolicy-FS-1-2-Res-2020-10   
    service.beta.kubernetes.io/aws-load-balancer-name: LB_NAME # This naming is not working
    alb.ingress.kubernetes.io/load-balancer-name: LB_NAME # https://kubernetes-sigs.github.io/aws-load-balancer-controller/v2.2/guide/ingress/annotations/
    service.beta.kubernetes.io/aws-load-balancer-target-group-attributes: deregistration_delay.timeout_seconds=30
    alb.ingress.kubernetes.io/security-groups: sg-XX 
    alb.ingress.kubernetes.io/tags: ApplicationName=Hello World,Environment=Staging
    service.beta.kubernetes.io/aws-load-balancer-target-node-labels: ApplicationName=Hello World,Environment=Staging 
    alb.ingress.kubernetes.io/ip-address-type: ipv4
spec:
  ingressClassName: alb
  rules:
    - http:
        paths:
        - path: /
          pathType: Prefix
          backend:
            service:
              name: redirect-green
              port:
                name: use-annotation


















 

@shraddhabang
Copy link
Collaborator

As mentioned here, the alb.ingress.kubernetes.io/ssl-redirect is exclusive across all Ingresses in IngressGroup. It should be consistent across ingresses in ingress group to avoid such conflict. Since we added redirects actions manually for 80 and 8080 ingresses, you don't need to add this annotation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants