Allow custom IP address flag for Impersonation Load Balancer service

[jeuniii]

Is your feature request related to a problem? Please describe.
Currently when pinniped-concierge pod comes up in a managed kubernetes cluster e.g. GKE, it spawns the pinniped-concierge-impersonation-proxy-load-balancer service which is of type LoadBalancer. The LoadBalancer service exposes an EXTERNAL-IP which basically exposes the cluster to the outside world.

This basically creates 2 problems.

1. Security. LoadBalancer type service in managed GKE spawns a LoadBalancer with an external IP address hence exposing the kubernetes cluster to the outside world. Firewall rules then need to be created to deny inbound/outbound traffic that is an overhead. It doesn't make sense to have an external IP address even when using a Private cluster or when deploying clusters in a VPC in cloud environments especially if pinniped and the client (e.g. kubeapps) reside on the same cluster. Even if it does not reside on the same cluster, the other cluster might reside in the same VPC or another VPC that could be connected using VPC peering.

2. Bootstrap. With the current setup, integration with other applications needs to be sequential. The deployment of kubeapps is dependant on deployment of Pinniped first. Once Pinniped is up and has created the service and the generated the relevant certificates, only then can we configure kubeapps to connect to Pinniped using the impersonation proxy. From a bootstrap perspective, this is a nightmare since we cannot bootstrap kubeapps fully until we have the required information from pinniped.

Describe the solution you'd like
It would be nice if pinniped accepted an argument for setting the LoadBalancer IP Address. Basically bring up the pinniped-concierge container with the following arg

--impersonation-proxy-external-ip=X.X.X.X 

The above args will allows users to set their own EXTERNAL-IP for the LB service which can be from their own VPC and is not necessarily a public IP address.

In the event that the specified IP address (public IP if provided) is already in use, the error message from the cloud Load Balancer can be propagated to the user deploying Pinniped.

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Are you considering submitting a PR for this feature?
No

• How will this project improvement be tested?
• How does this change the current architecture?
• How will this change be backwards compatible?
• How will this feature be documented?

Additional context

Kubernetes accepts setting loadBalancerIP in a manifest.

kind: Service
apiVersion: v1
metadata:
  name: load-balancer-service
spec:
  loadBalancerIP: X.X.X.X

So if it can be done using manifest, it can surely be done using the application i.e. pinniped-concierge who is already spawning the service.

[mattmoyer]

@jeuniii: I wanted to acknowledge this issue and thank you for so clearly explaining your use cases. I had been discussing some related problems with the team as well (particularly the internal vs. external load balancer issue).

I wonder if you could help validate the API design proposed in #617. It is a bit different, but I think it has roughly the same goals.

Instead of a command line flag, we would add some fields to the spec of the CredentialIssuer API (it's currently status-only). This would let you configure Service annotations, use a specific load balancer IP, or skip Service creation and bring your own load balancer provisioned externally. It also supports some DNS-friendly configurations to let you avoid hardcoding IP addresses.

I believe your use case would fit into that API design with a configuration like:

apiVersion: config.concierge.pinniped.dev/v1alpha1
kind: CredentialIssuer
metadata:
    name: example
spec:
  impersonationProxy:
    # Enable the impersonation proxy
    mode: enabled

    # Configure how the automatically-provisioned Service is created
    service:
      template:
        # Provision a Service with a particular load balancer IP "X.X.X.X"
        loadBalancerIP: X.X.X.X

        # Request an internal load balancer (I'm not sure if this is important for your use case, but it might be useful)
        annotations:
          networking.gke.io/load-balancer-type: "Internal"

[mattmoyer]

This functionality (in slightly different shape) is now included in v0.9.0. Thanks @jeuniii!