Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"Kernel's RPF check is set to 'loose'" prevents Calico from working #6673

Closed
DevOpsFu opened this issue Feb 18, 2020 · 6 comments · Fixed by #6690
Closed

"Kernel's RPF check is set to 'loose'" prevents Calico from working #6673

DevOpsFu opened this issue Feb 18, 2020 · 6 comments · Fixed by #6690
Labels
area/guest-vm General configuration issues with the minikube guest VM kind/bug Categorizes issue or PR as related to a bug. priority/important-longterm Important over the long term, but may not be staffed and/or may need multiple releases to complete.

Comments

@DevOpsFu
Copy link

The exact command to reproduce the issue:

Provision Minikube with CNI enabled:

minikube start --memory 4096 \
               --network-plugin=cni \
               --extra-config=kubelet.network-plugin=cni \
               --extra-config=kubelet.pod-cidr=192.168.0.0/16 \
               --extra-config=controller-manager.allocate-node-cidrs=true \
               --extra-config=controller-manager.cluster-cidr=192.168.0.0/16 \
               --host-only-cidr=172.17.17.1/24

Install Calico:

kubectl apply -f https://docs.projectcalico.org/v3.9/manifests/calico.yaml  

At this point, the calico-node pod will be failing the readiness check.

The full output of the command that failed:

int_dataplane.go 1037: Kernel's RPF check is set to 'loose'.  
This would allow endpoints to spoof their IP address.  
Calico requires net.ipv4.conf.all.rp_filter to be set to 0 or 1.  
If you require loose RPF and you are not concerned about spoofing,  
this check can be disabled by setting the IgnoreLooseRPF configuration  
parameter to 'true'.

This issue can be worked around by using the following environment variable tweak:

kubectl -n kube-system set env daemonset/calico-node FELIX_IGNORELOOSERPF=true  

The output of the minikube logs command:

N/A

The operating system version:

Windows 10 Professional using the HyperV driver.

@edge0701
Copy link
Contributor

My experience with installing Calico on Minikube was different, although I was using the latest Calico manifest, which points to v3.12.0 instead of v3.9.5.

In my case, I did not get the option to ignore RPF filter/have a loose check. I get the error:
2020-02-18 19:24:56.443 [WARNING][2424] table.go 1159: Failed to execute ip(6)tables-restore command error=exit status 2 errorOutput="iptables-restore v1.8.2 (legacy): Couldn't load match `rpfilter':No such file or directory\n\nError occurred at line: 38\nTry `iptables-restore -h' or 'iptables-restore --help' for more information.\n"

My solution was to add CONFIG_IP_NF_MATCH_RPFILTER=y to the linux_defconfig in this repo and build the minikube ISO myself.
I am using v1.7.0 of the Minikube ISO. Which version are you using?
The Calico documentation here states that ipt_rpfilter and xt_rpfilter are requirements of the host kernel.

I was just about to create an issue for this and then subsequently a PR, if this seemed like a suitable addition?

Mentioning @afbjorklund as they were the one to comment on my PR regarding the Dockerized builds. Apologies if this is not appropriate.

@DevOpsFu
Copy link
Author

My experience with installing Calico on Minikube was different, although I was using the latest Calico manifest, which points to v3.12.0 instead of v3.9.5.

In my case, I did not get the option to ignore RPF filter/have a loose check. I get the error:
2020-02-18 19:24:56.443 [WARNING][2424] table.go 1159: Failed to execute ip(6)tables-restore command error=exit status 2 errorOutput="iptables-restore v1.8.2 (legacy): Couldn't load match `rpfilter':No such file or directory\n\nError occurred at line: 38\nTry `iptables-restore -h' or 'iptables-restore --help' for more information.\n"

@edge0701 I get the same behaviour as you if I use the latest manifest :-(

@tstromberg
Copy link
Contributor

@edge0701 - If you send in a PR for linux_defconfig, we can get it merged in time for the minikube v1.8.0 release.

@tstromberg tstromberg added area/guest-vm General configuration issues with the minikube guest VM kind/bug Categorizes issue or PR as related to a bug. priority/important-longterm Important over the long term, but may not be staffed and/or may need multiple releases to complete. labels Feb 19, 2020
@DevOpsFu
Copy link
Author

v1.7.3 doesn't seem to have completely fixed this for me. With the latest version of Calico, I no longer get any issues with the pods failing their readiness checks, but any pods I create cannot connect out to pull any images.

@edge0701
Copy link
Contributor

It's working as expected on my end.
The only arguments I add to minikube start are:

--network-plugin=cni \
--extra-config=kubeadm.pod-network-cidr=${POD_CIDR} \

And then replace the CIDR in the Calico manifest (I'm setting a different IP range but you should be okay using 192.168.0.0/16 and without modifying the manifest).

Calico is annotating Pods correctly and all seems well. It's definitely pulling images from external repos okay. I haven't yet had time to check if NetworkPolicy is all working, though it's on my list.

@DevOpsFu
Copy link
Author

You're right - this must have been a transient issue on my side - I've tried again and it worked!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/guest-vm General configuration issues with the minikube guest VM kind/bug Categorizes issue or PR as related to a bug. priority/important-longterm Important over the long term, but may not be staffed and/or may need multiple releases to complete.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants