kube-proxy" is forbidden: User "system:node:minikube" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node "minikube" and this object

[veeveksen]

Steps to reproduce the issue:

1.Install minikube and virtualbox.
2. execute command minikube start --force .

:~ vivek$ minikube start --force
😄 minikube v1.9.2 on Darwin 10.14.6
✨ Using the virtualbox driver based on user configuration
👍 Starting control plane node m01 in cluster minikube
🔥 Creating virtualbox VM (CPUs=2, Memory=4000MB, Disk=20000MB) ...
🐳 Preparing Kubernetes v1.18.0 on Docker 19.03.8 ...
🌟 Enabling addons: default-storageclass, storage-provisioner
❗ Enabling 'default-storageclass' returned an error: running callbacks: [Error making standard the default storage class: Error listing StorageClasses: Get "https://192.168.99.131:8443/apis/storage.k8s.io/v1/storageclasses": dial tcp 192.168.99.131:8443: i/o timeout]
❌ Problems detected in kubelet:
Apr 11 06:13:24 minikube kubelet[3983]: E0411 06:13:24.137584 3983 reflector.go:178] object-"kube-system"/"kube-proxy-token-2j4z7": Failed to list *v1.Secret: secrets "kube-proxy-token-2j4z7" is forbidden: User "system:node:minikube" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node "minikube" and this object
Apr 11 06:13:24 minikube kubelet[3983]: E0411 06:13:24.138044 3983 reflector.go:178] object-"kube-system"/"kube-proxy": Failed to list *v1.ConfigMap: configmaps "kube-proxy" is forbidden: User "system:node:minikube" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node "minikube" and this object
❌ Problems detected in kubelet:
Apr 11 06:13:24 minikube kubelet[3983]: E0411 06:13:24.137584 3983 reflector.go:178] object-"kube-system"/"kube-proxy-token-2j4z7": Failed to list *v1.Secret: secrets "kube-proxy-token-2j4z7" is forbidden: User "system:node:minikube" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node "minikube" and this object
Apr 11 06:13:24 minikube kubelet[3983]: E0411 06:13:24.138044 3983 reflector.go:178] object-"kube-system"/"kube-proxy": Failed to list *v1.ConfigMap: configmaps "kube-proxy" is forbidden: User "system:node:minikube" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node "minikube" and this object
❌ Problems detected in kubelet:
Apr 11 06:13:24 minikube kubelet[3983]: E0411 06:13:24.137584 3983 reflector.go:178] object-"kube-system"/"kube-proxy-token-2j4z7": Failed to list *v1.Secret: secrets "kube-proxy-token-2j4z7" is forbidden: User "system:node:minikube" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node "minikube" and this object
Apr 11 06:13:24 minikube kubelet[3983]: E0411 06:13:24.138044 3983 reflector.go:178] object-"kube-system"/"kube-proxy": Failed to list *v1.ConfigMap: configmaps "kube-proxy" is forbidden: User "system:node:minikube" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node "minikube" and this object
❌ Problems detected in kubelet:
Apr 11 06:13:24 minikube kubelet[3983]: E0411 06:13:24.137584 3983 reflector.go:178] object-"kube-system"/"kube-proxy-token-2j4z7": Failed to list *v1.Secret: secrets "kube-proxy-token-2j4z7" is forbidden: User "system:node:minikube" cannot list resource "secrets" in API group "" in the namespace "kube-system": no relationship found between node "minikube" and this object
Apr 11 06:13:24 minikube kubelet[3983]: E0411 06:13:24.138044 3983 reflector.go:178] object-"kube-system"/"kube-proxy": Failed to list *v1.ConfigMap: configmaps "kube-proxy" is forbidden: User "system:node:minikube" cannot list resource "configmaps" in API group "" in the namespace "kube-system": no relationship found between node "minikube" and this object

💣 startup failed: Wait failed: wait for healthy API server: apiserver healthz never reported healthy

[tstromberg]

Can you share the output of:

• minikube start --alsologtostderr
• minikube logs

Is --force necessary, or is it just something you were trying?

It's possible that minikube delete fixes this issue.

[medyagh]

I have a feeling this is because the addons are being enabled before default service account is created. we need to add a verification not to enable addons before default service account.

[medyagh]

could be related to #7704

[medyagh]

@veeveksen do you mind trying starting with --wait=true and see if you still have the problem?

(though that might still error, because we still do enabling addons in parallel)

[medyagh]

anyone knows why we enble storage addon by default ?

1:15
enbling addons by default takes a toll on the verification start time
1:15
currently we need to verify SA is created but because we are doing it in paraelll we get flakes like this: #7613 (edited)
1:15
could we consider disabling all addons that have cluster role bindings by default
1:16
and only enable them if user uses those features?

[tstromberg]

@medyagh - yes, it is required for PV support, and a rather standard/expected feature of Kubernetes. I believe it may also be required for conformance tests, but I'm not absolutely certain.

Alternatively, we could have the retry for addons occur inside of minikube rather than outside, but then we would not be able to provide signal to users when they are broken.

[medyagh]

for this we need to refactor bootstrapper to include Enable addons
so we can apply kic overaly or enable CNI or enable addons right after the Default SA is created
and also do WaitForNode in parallel

[medyagh]

is anyone still interested in this issue ?

[smhmayboudi]

I had the same problem and by adjusting my system proxy and let the kubectl have a direct connection, it solved.

[tstromberg]

I'm closing this issue as it hasn't seen activity in awhile, and it's unclear if this issue still exists. If this issue does continue to exist in the most recent release of minikube, please feel free to re-open it by replying /reopen

If someone sees a similar issue to this one, please re-open it as replies to closed issues are unlikely to be noticed.

Thank you for opening the issue!

[eli-halych]

@tstromberg

I had this issue and resolved it in a simple way.

1. I noticed the keyword forbidden in the error logs and as a rule of thumb ran the Git Bash terminal as an administrator.

2. Removed the minikube cluster by simply running minikube delete

3. Then started it again in the terminal as an administrator.

This resolved the issue.

Some system details:
Microsoft Windows 10 Home 10.0.18363 Build 18363
minikube v1.12.3,
Enabled addons: default-storageclass, storage-provisioner
kubectl.exe is version 1.16.8-eks-e16311

[koushikmgithub]

Hello... I am facing the same issue. I am trying to install minikube on my mac os with M1 chip and following below guideline.

https://minikube.sigs.k8s.io/docs/start/
I am running docker desktop 20.10.8 on my mac and getting the same error when I start minikube. Do I need to do anything for this to work on the new macos chip M1? Already opened an issue : #13265