Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Use LoadBalancer IPv6 address #1218

Open
JochemTSR opened this issue Mar 19, 2024 · 11 comments · May be fixed by #1227 or #1334
Open

Use LoadBalancer IPv6 address #1218

JochemTSR opened this issue Mar 19, 2024 · 11 comments · May be fixed by #1227 or #1334

Comments

@JochemTSR
Copy link

/kind feature

Describe the solution you'd like
Currently, the Kubernetes API server endpoint is always set to the load balancer's IPv4 address. I'm setting up a Hetzner cluster with IPv6-only nodes and a private network; all incoming traffic would be handled by Hetzner load balancers. As the API server endpoint is set to an IPv4 address, nodes cannot reach the API server after kubeadm initialization (dial tcp 167.xxx.xxx.xxx: connect: network is unreachable).

As far as I can tell, a pure IPv6 setup is currently not possible without manually configuring a load balancer and DNS so a hostname can be used for spec.controlPlaneEndpoint.host in the HetznerCluster manifest.

Looking at the code that determines the endpoint, I think it should be possible to add a property to the controlPlaneLoadBalancer object (e.g. useV6Endpoint). If this property were set to true, the control plane endpoint would be set to the LB's IPv6 address instead of the IPv4 one in the code section below:

cluster-api-provider-hetzner/controllers/hetznercluster_controller.go

Lines 244 to 265 in 3bc16ac

func processControlPlaneEndpoint(hetznerCluster *infrav1.HetznerCluster) {
if hetznerCluster.Spec.ControlPlaneLoadBalancer.Enabled {
if hetznerCluster.Status.ControlPlaneLoadBalancer.IPv4 != "<nil>" {
defaultHost := hetznerCluster.Status.ControlPlaneLoadBalancer.IPv4
defaultPort := int32(hetznerCluster.Spec.ControlPlaneLoadBalancer.Port)
if hetznerCluster.Spec.ControlPlaneEndpoint == nil {
hetznerCluster.Spec.ControlPlaneEndpoint = &clusterv1.APIEndpoint{
Host: defaultHost,
Port: defaultPort,
}
} else {
if hetznerCluster.Spec.ControlPlaneEndpoint.Host == "" {
hetznerCluster.Spec.ControlPlaneEndpoint.Host = defaultHost
}
if hetznerCluster.Spec.ControlPlaneEndpoint.Port == 0 {
hetznerCluster.Spec.ControlPlaneEndpoint.Port = defaultPort
}
}
conditions.MarkTrue(hetznerCluster, infrav1.ControlPlaneEndpointSetCondition)
hetznerCluster.Status.Ready = true
} else {

Please let me know what you think, I'd be happy to contribute a PR if this seems feasible/desirable.

Anything else you would like to add:
Related issues: #632, #899

Environment:

  • cluster-api-provider-hetzner version: v1.0.0-beta.29
  • Kubernetes version: 1.29.1
  • OS (e.g. from /etc/os-release): ubuntu-22.04
@guettli
Copy link
Contributor

guettli commented Mar 19, 2024 •
edited
Loading

@JochemTSR thank you for your feedback. Up to now we (Syself) and users of our commercial offering are happy with ipv4 only.

Just out of curiosity: Why do you want to use ipv6? What is the benefit, what is your goal?

During the next weeks we are not able to work on that. Maybe later.

But feel free to create PR which works for you.

Ping me if you have difficulties to create a PR and test it in your cluster.

If you want ipv6 for your applications running inside the caph cluster, then this is a different topic.

@JochemTSR
Copy link
Author

@guettli Apart from IPv6 being The Future™, the main benefit would be the 15% discount on the CAX11 nodes for not using a public IPv4 address. Provisioning nodes without IPv4 is already possible with the Hetzner provider by setting enableIPv4 to false in the machine template.

For me personally it's mostly a fun exercise in setting up my hobby cluster.

@JochemTSR
Copy link
Author

I should add, the cluster I'm envisioning does use IPv4 for the internal network, mostly because this is what the Hetzner private networks offer.

@guettli
Copy link
Contributor

guettli commented Mar 19, 2024

@guettli Apart from IPv6 being The Future™, the main benefit would be the 15% discount on the CAX11 nodes for not using a public IPv4 address.

Wait a second. CAX11 are ARM machines. Do you run caph on arm?

Thank you for this hint. This makes sense. If it is a hobby project for you, then why not dive into the big fun create a patch to make it working?

@apricote
Copy link
Contributor

Wait a second. CAX11 are ARM machines. Do you run caph on arm?

Off topic: I do run CAPH on the Arm server types since they launched and it works great 💪

@JochemTSR JochemTSR linked a pull request Mar 25, 2024 that will close this issue
Draft
3 tasks
@JochemTSR
Copy link
Author

@guettli It took some fiddling to get the network to play nice, but I managed to provision a functional cluster with IPv6 only. The development tooling of this project is actually quite nice once you get familiar with it :)

On a related note, I think the development documentation is a bit out of date? It's no biggie but it did take a bit longer to get started. Is that worth opening a seperate issue for?

@janiskemper
Copy link
Contributor

@kranurag7 can you maybe have a look at the development documentation and open an issue? I have also seen that a few days ago but forgot to open an issue. We should really update it!

@kranurag7
Copy link
Contributor

kranurag7 commented Mar 25, 2024 •
edited
Loading

On a related note, I think the development documentation is a bit out of date? It's no biggie but it did take a bit longer to get started. Is that worth opening a seperate issue for?

@JochemTSR If you intend to open an issue then please go ahead and let us know what can be improved here. If not, I'll open up an issue and fix the same.
Please let me know if you've some specific feedback.

@janiskemper
Copy link
Contributor

@kranurag7 it is really outdated. Just open the issue because you know the setup. @JochemTSR can add something on top if he doesn't find his topic in your issue!

@JochemTSR
Copy link
Author

@kranurag7 @janiskemper opened #1233 to track possible improvements to the development documentation.

@janiskemper
Copy link
Contributor

thanks a lot @JochemTSR !

@rbjorklin rbjorklin linked a pull request Jun 8, 2024 that will close this issue
Draft
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
5 participants
@guettli @apricote @JochemTSR @janiskemper @kranurag7