Documentation on external DNS seems out of date

[robinmonjo]

Hello all,

I just setup a Kubernetes cluster with version v0.9.1. I see changes since version 0.8:

• etcd cluster is now deployed on its own instances
• controller is in an autoscalling group
• controller is behind a load balancer

The doc: https://coreos.com/kubernetes/docs/latest/kubernetes-on-aws-launch.html, says:

Otherwise, navigate to the DNS registrar hosting the zone for the provided external DNS name. Ensure a single A record exists, routing the value of externalDNSName defined in cluster.yaml to the externally-accessible IP of the master node instance.

You can invoke kube-aws status to get the cluster API IP address after cluster creation, if necessary. This command can take a while.

I did what I did with version 0.8, I set an A record in my zone file to the public IP address of my controller node. However it is now behind a load balancer and the security group of the controller prevent HTTPS access from "0.0.0.0" and constraints it to only the load balancer and the worker nodes.

To access my cluster, I had to modify the security group and add a HTTPS inbound rule with "0.0.0.0". I guess I'm not supposed to do that. What are the recommandation here ?

Regards,

[camilb]

@robinmonjo Easiest way is to create the DNS record automatically by setting createRecordSet: truein cluster.yaml

Also you can manually create a CNAME record in Route53 for the external LB.
Search for a load balancer that starts with "kubernete-ElbAPISe...."

You don't need to modify the security group.

[robinmonjo]

Thank you for your answer. What if I create a CNAME for my load balancer directly into my zone file ? Will it work with the CA based authentication ?

[camilb]

Yes, just point it to the ELB and that's it. The CA is set in your kubeconfig.

[cmcconnell1]

Just wanted to note the settings that are working for me (using kube-aws version v0.9.1-rc.4) hoping this may help others. As noted above by @camilb kube-aws should take care of your DNS automatically, provided the correct settings are in your cluster.yaml when you provision your cluster.

I.e. with these settings in the cluster/projects cluster.yaml

clusterName: deis-kube1
externalDNSName: deis-kube1.dev.foo.com
createRecordSet: true
recordSetTTL: 300
hostedZoneId: "XXXXXXXXXX" # my internal/private only route53 zone

After cluster deployment, we can validate our DNS record was added, below we just use a simple bash script to search our route53 zones for regex/records, and see that it now exists. This might be useful for dynamic/complex cluster names, etc.

route53-domain-search deis-kube1 
PRIVATE ZONE: 'foo.com' shared for DEV, STAGE, PROD ROUTE53_PRIVATE_ZONEID
deis-kube1.dev	300	IN	CNAME	`deis-kube-ElbAPISe-1NFMQQUZWRAYZ-123456789.us-west-1.elb.amazonaws.com`

So we have validated that our internal private route53 record is an alias for the AWS ELB and we can see that it resolves to our ELB with its three IP's for H/A, multi-zone, scaling, etc on the AWS side.

host deis-kube1.dev
deis-kube1.dev.foo.com is an alias for deis-kube-elbapise-1nfmqquzwrayz-123456789.us-west-1.elb.amazonaws.com.
deis-kube-elbapise-1nfmqquzwrayz-123456789.us-west-1.elb.amazonaws.com has address 54.241.xxx.xxx
deis-kube-elbapise-1nfmqquzwrayz-123456789.us-west-1.elb.amazonaws.com has address 52.9.xxx.xxx
deis-kube-elbapise-1nfmqquzwrayz-174298929.us-west-1.elb.amazonaws.com has address 54.219.xxx.xxx

[robinmonjo]

Thank you for your input. I havent dig much into the all route 53 config. However, adding the ESB domain name as a CNAME of my domain in my hosted zone file produce this error:

Unable to connect to the server: x509: certificate signed by unknown authority

That what I was afraid of @camilb when I was talking about CA based authentication.

[camilb]

Do you have the CA set in your kubeconfig? Please check it by running kubectl config view.

And look for a line similar to this: certificate-authority: ./credentials/ca.pem

Edit:

The type of ELB is TCP, the certificates are still configured on controllers. And if you still get the errors after setting the CA in kubeconfig I think the certificates were generated with a different CA.

[robinmonjo]

Yes I do, my config is fine and credentials paths are properly set. When I open my security group and create an A record with my cluster hostname and the public ip of the controller everything works fine.

I don't know much about CA auth, but it is probably closely related to the DNS name of our cluster. The doc I pointed out in my first post precisely says:

Ensure a single A record exists, routing the value of externalDNSName defined in cluster.yaml to the externally-accessible IP of the master node instance

[camilb]

I don't know much about CA auth, but it is probably closely related to the DNS name of our cluster.

If it works with an A record set directly to your controller, should work with CNAME on ELB too. I suppose they are the same, right?

Like :

kube.yourdomain.tld in A controller_ip_address

kube.yourdomain.tld in CNAME elb_name.zone.elb.amazonaws.com

[robinmonjo]

Indeed it works, I misconfigured my zone file ... Thank you for your help @camilb. However I'd like to keep this issue open since the doc is still not right.

[mumoshu]

Hi @robinmonjo, thanks for reporting this issue and sharing your experience!

We're currently short on resources to improve documentation but would appreciate it you could author a pull request for it 👍

Also, this issue is now tracked in #90