az aks dashboard is empty "There is nothing to display here"

[ecc256]

What is right way to fix az k8s dashboard?

Kubernetes version 1.16.7
kubectl works properly

Tried this and recreated ClusterRoleBinding

kubectl delete clusterrolebinding kubernetes-dashboard
followed by
kubectl create clusterrolebinding kubernetes-dashboard --clusterrole=cluster-admin --serviceaccount=kube-system:kubernetes-dashboard or
kubectl create clusterrolebinding kubernetes-dashboard -n kube-system --clusterrole=cluster-admin --serviceaccount=kube-system:kubernetes-dashboard
Got dashboard login prompt.
With Token auth got:
Unauthorized (401): Invalid credentials provided

with Kubeconfig it shows dashboard with all items "There is nothing to display here"
and tons of notification similar to this post

cronjobs.batch is forbidden: User "clusterUser" cannot list resource "cronjobs" in API group "batch" in the namespace "default"
secrets is forbidden: User "clusterUser" cannot list resource "secrets" in API group "" in the namespace "default"
events is forbidden: User "clusterUser" cannot list resource "events" in API group "" in the namespace "default"
pods is forbidden: User "clusterUser" cannot list resource "pods" in API group "" in the namespace "default"
replicationcontrollers is forbidden: User "clusterUser" cannot list resource "replicationcontrollers" in API group "" in the namespace "default"
events is forbidden: User "clusterUser" cannot list resource "events" in API group "" in the namespace "default"
pods is forbidden: User "clusterUser" cannot list resource "pods" in API group "" in the namespace "default"
statefulsets.apps is forbidden: User "clusterUser" cannot list resource "statefulsets" in API group "apps" in the namespace "default"
configmaps is forbidden: User "clusterUser" cannot list resource "configmaps" in API group "" in the namespace "default"
persistentvolumeclaims is forbidden: User "clusterUser" cannot list resource "persistentvolumeclaims" in API group "" in the namespace "default"
events is forbidden: User "clusterUser" cannot list resource "events" in API group "" in the namespace "default"
pods is forbidden: User "clusterUser" cannot list resource "pods" in API group "" in the namespace "default"
replicasets.apps is forbidden: User "clusterUser" cannot list resource "replicasets" in API group "apps" in the namespace "default"
ingresses.extensions is forbidden: User "clusterUser" cannot list resource "ingresses" in API group "extensions" in the namespace "default"
services is forbidden: User "clusterUser" cannot list resource "services" in API group "" in the namespace "default"
events is forbidden: User "clusterUser" cannot list resource "events" in API group "" in the namespace "default"
pods is forbidden: User "clusterUser" cannot list resource "pods" in API group "" in the namespace "default"

Tried to google - everybody says recreating ClusterRoleBinding should do it for az aks...
Don't know what to try next...

Clicking on top right icon in all empty dashboard shows:
'logged in with token'

I can perform all admin actions kubectl...

[Zachery2008]

I have the same issue

[ecc256]

Good, I'm not alone! :)

[GustavoAmerico]

I have the same issue.

I think i just need to configure clusterrolebinding. I'm trying to find the right settings

[engineertree5]

Adding to this issue... Running AKS v 1.16.7 and i've been having issues accessing the dashboard. As @ecc256 stated, cluster-role binding is applied and still getting errors. Whatever the errors is Azure should update their docs.

[bq1756]

I deployed a new AKS cluster with k8s v1.15.10 using the Azure Portal. After the deployment was finished, I was able to see relevant data (pods, namespaces, etc) on the k8s Dashboard. I upgraded the cluster to k8s v1.16.7 via the Azure Portal. Viewing the Dashboard after the upgrade, I only see "There is nothing to display here". I deleted clusterrolebinding kubernetes-dashboard, then recreated it as cluster-admin. I am still seeing "There is nothing to display here".

I then upgraded to k8s v1.17.3. I see the same "There is nothing to display here" in the Dashboard.

[ecc256]

I assume it’s due to missing permissions.
Am I wrong about it?
Does ‘clusterUser’ suppose to use (or be assigned to) ‘ClusterRoleBinding’ role for dashboard to work properly?
Anybody has a clue how to debug it?

[kwonhyuck0218]

In My case, I solved this way.

1. kubectl delete clusterrolebinding kubernetes-dashboard
2. kubectl create clusterrolebinding kubernetes-dashboard --clusterrole=cluster-admin --serviceaccount=kube-system:kubernetes-dashboard --user=clusterUser

reference : https://kubernetes.io/docs/reference/generated/kubectl/kubectl-commands#-em-clusterrolebinding-em-

[code-bringer]

@KwonHyuck it works! Thank you!

[ecc256]

In My case, I solved this way.

@KwonHyuck work for me too, except it was not as straightforward.
Here is what happens:

kubectl delete clusterrolebinding kubernetes-dashboard
clusterrolebinding.rbac.authorization.k8s.io "kubernetes-dashboard" deleted

kubectl create clusterrolebinding kubernetes-dashboard --clusterrole=cluster-admin --serviceaccount=kube-system:kubernetes-dashboard --user=clusterUser
Error from server (AlreadyExists): clusterrolebindings.rbac.authorization.k8s.io "kubernetes-dashboard" already exists

I'm like, WTF? :(
Let do one more time:

kubectl delete clusterrolebinding kubernetes-dashboard
clusterrolebinding.rbac.authorization.k8s.io "kubernetes-dashboard" deleted

kubectl delete clusterrolebinding kubernetes-dashboard
Error from server (NotFound): clusterrolebindings.rbac.authorization.k8s.io "kubernetes-dashboard" not found

kubectl create clusterrolebinding kubernetes-dashboard --clusterrole=cluster-admin --serviceaccount=kube-system:kubernetes-dashboard --user=clusterUser
Error from server (AlreadyExists): clusterrolebindings.rbac.authorization.k8s.io "kubernetes-dashboard" already exists

kubectl delete clusterrolebinding kubernetes-dashboard
clusterrolebinding.rbac.authorization.k8s.io "kubernetes-dashboard" deleted

kubectl delete clusterrolebinding kubernetes-dashboard -n kube-system
warning: deleting cluster-scoped resources, not scoped to the provided namespace
clusterrolebinding.rbac.authorization.k8s.io "kubernetes-dashboard" deleted

kubectl create clusterrolebinding kubernetes-dashboard --clusterrole=cluster-admin --serviceaccount=kube-system:kubernetes-dashboard --user=clusterUser
clusterrolebinding.rbac.authorization.k8s.io/kubernetes-dashboard created

And it started working now!
Thanks a lot!

[mchudinov]

The same problem for fresh installed AKS v 1.18.2

[ecc256]

I can see 1.17.5 (preview), but not any 1.18.*

[rathishrms]

same issue, no luck!

[robearlam]

@ecc256 solution above worked for me.

[skillbuilderzone]

As of v1.10.1 of the Kubernetes dashboard or kubernetes v1.16+ the service account "kubernetes-dashboard" can no longer be used to retrieve resources due to a security fix in that release. Thus the 'clusterUser' works.

[alefred]

In My case, I solved this way.

1. kubectl delete clusterrolebinding kubernetes-dashboard
2. kubectl create clusterrolebinding kubernetes-dashboard --clusterrole=cluster-admin --serviceaccount=kube-system:kubernetes-dashboard --user=clusterUser

reference : https://kubernetes.io/docs/reference/generated/kubectl/kubectl-commands#-em-clusterrolebinding-em-

Thats work for me. thanks!

[nitingargprft]

This solves for me
kubectl delete clusterrolebinding kubernetes-dashboard
kubectl create clusterrolebinding kubernetes-dashboard --clusterrole=cluster-admin --serviceaccount=kube-system:kubernetes-dashboard --user=clusterUser

[Leonardo-Ferreira]

I just fixed this issue for a fresh 1.17.7 installation using @ecc256 solution. Not specifying -n kube-system did not resolved.

[mehul9595]

In My case, I solved this way.

1. kubectl delete clusterrolebinding kubernetes-dashboard
2. kubectl create clusterrolebinding kubernetes-dashboard --clusterrole=cluster-admin --serviceaccount=kube-system:kubernetes-dashboard --user=clusterUser

reference : https://kubernetes.io/docs/reference/generated/kubectl/kubectl-commands#-em-clusterrolebinding-em-

finally got this fixed by above lines..

[macikacz]

You don't need to delete the old clusterrolebinding and you can just assign a unique name. --serviceaccount=kube-system:kubernetes-dashboard seems also to be not needed to fix the problem. The command that works for me is just:

kubectl create clusterrolebinding kubernetes-dashboard-cluster-user --clusterrole=cluster-admin --user=clusterUser

[sumonwhoami]

In My case, I solved this way.

1. kubectl delete clusterrolebinding kubernetes-dashboard
2. kubectl create clusterrolebinding kubernetes-dashboard --clusterrole=cluster-admin --serviceaccount=kube-system:kubernetes-dashboard --user=clusterUser

reference : https://kubernetes.io/docs/reference/generated/kubectl/kubectl-commands#-em-clusterrolebinding-em-

it's also worked for me, thanks brother :-)

[ghost]

Action required from @Azure/aks-pm

[ghost]

Action required from @qpetraroia.

[qpetraroia]

Hi everyone,

The reason why you are having issues using the Kubernetes dashboard is because the dashboard no longer uses the service account identity. This was a part of a security fix which you can read here kubernetes/dashboard#3400. If you are just copying the clusterUser role or using the kubeconfig, a workaround you can implement is to grant permissions to clusterUser. An example of this is below.

Kubectl create clusterrolebinding dashboard-users --clusterrole=cluster-admin --user=clusteruser

However, the AKS team recommends that you use an AAD enabled cluster and have each user use its own token and respective permissions instead of giving the clusterUser clusterAdmin privileges. You can also use the Azure Portal as we are in the process of deprecating the dashboard. You can see more here https://docs.microsoft.com/en-us/azure/aks/kubernetes-portal.

Thank you!

[ghost]

Thanks for reaching out. I'm closing this issue as it was marked with "Answer Provided" and it hasn't had activity for 2 days.

[kbayannapelitas]

In My case, I solved this way.

1. kubectl delete clusterrolebinding kubernetes-dashboard
2. kubectl create clusterrolebinding kubernetes-dashboard --clusterrole=cluster-admin --serviceaccount=kube-system:kubernetes-dashboard --user=clusterUser

reference : https://kubernetes.io/docs/reference/generated/kubectl/kubectl-commands#-em-clusterrolebinding-em-

Thats work for me. thanks!

It works for me. Thanks for saving my time.

[viveksi]

Frustating it is, It was working before and suddenly it stopped and getting the above error

Stuck

[palma21]

Please refer to @qpetraroia's comment for explanation and workaround

#1573 (comment)

Please consider using the Azure Portal resource view instead of exposing and granting additional permissions to your clusterUser or continuing to use the k8s dashboard
https://docs.microsoft.com/en-us/azure/aks/kubernetes-portal