Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

dashboard not working #8138

Closed
pkrrsimtian opened this issue May 13, 2020 · 2 comments · Fixed by #8792
Closed

dashboard not working #8138

pkrrsimtian opened this issue May 13, 2020 · 2 comments · Fixed by #8792
Labels
kind/support Categorizes issue or PR as a support question. triage/needs-information Indicates an issue needs more information in order to work on it.

Comments

@pkrrsimtian
Copy link

Steps to reproduce the issue:

  1. Unable to enable dashboard: run callbacks: running callbacks: [sudo KUBECONFIG=/var/lib/minikube/kubeconfig /var/lib/minikube/binaries/v1.18.1/kubectl apply -f /etc/kubernetes/addons/dashboard-ns.yaml -f /etc/kubernetes/addons/dashboard-clusterrole.yaml -f /etc/kubernetes/addons/dashboard-clusterrolebinding.yaml -f /etc/kubernetes/addons/dashboard-configmap.yaml -f /etc/kubernetes/addons/dashboard-dp.yaml -f /etc/kubernetes/addons/dashboard-role.yaml -f /etc/kubernetes/addons/dashboard-rolebinding.yaml -f /etc/kubernetes/addons/dashboard-sa.yaml -f /etc/kubernetes/addons/dashboard-secret.yaml -f /etc/kubernetes/addons/dashboard-svc.yaml: exec: already started
    stdout:

stderr:

Full output of failed command:

Full output of minikube start command used, if not already included:

Optional: Full output of minikube logs command:
@medyagh
Copy link
Member

medyagh commented May 13, 2020

Thank you for sharing your experience! If you don't mind, could you please provide:

  • The exact command-lines used, so that we may replicate the issue
  • The full output of the command that failed
  • The full output of the "minikube logs" command
  • Which operating system version was used

This will help us isolate the problem further. Thank you!

@medyagh medyagh added triage/needs-information Indicates an issue needs more information in order to work on it. kind/support Categorizes issue or PR as a support question. labels May 13, 2020
@priyawadhwa
Copy link

Hey @pkrrsimtian -- Hopefully it's OK if I close this - there wasn't enough information to make it actionable, and some time has already passed. If you are able to provide additional details, you may reopen it at any point by adding /reopen to your comment.

Here is additional information that may be helpful to us:

  • The exact command-lines used, so that we may replicate the issue
  • The full output of the command that failed
  • The full output of the "minikube logs" command
  • Which operating system version was used

Thank you for sharing your experience!

concaf added a commit to concaf/minikube that referenced this issue Jul 21, 2020
@concaf
(Addons) Create a new command in every retry
3181bc0 
Whenever a `kubectl apply` fails while enabling an addon, it is
retried with exponential backoff. The command (type `*exec.Cmd`)
that this retry function runs in created outside the function -
which means that it is reused on every retry. This is a problem
because `exec.Cmd` (https://godoc.org/github.com/pkg/exec#Cmd)
states that "... Cmd cannot be reused after calling its Run or
Start methods."

This retry is a common case due to, say, a CRD and its resource
being present in the same YAML file of the addon which causes
a race condition where the resource is created before its CRD is
created in the cluster - this race is fixed by subsequent retries.
I've noticed this in the dashboard and the ambassador addon.

Due to the above mentioned bug, minikube throws errors like
`exec: already started` in every retry and the retry is never
successful, manifests are never deployed and addon creation errors
out.

Fix kubernetes#8138
Fix kubernetes#8119
Fix a few CI errors in kubernetes#8372
@concaf concaf mentioned this issue Jul 21, 2020
Merged
concaf added a commit to concaf/minikube that referenced this issue Jul 21, 2020
@concaf
(Addons) Create a new command in every retry
f42f79e 
Whenever a `kubectl apply` fails while enabling an addon, it is
retried with exponential backoff. The command (type `*exec.Cmd`)
that this retry function runs in created outside the function -
which means that it is reused on every retry. This is a problem
because `exec.Cmd` (https://godoc.org/github.com/pkg/exec#Cmd)
states that "... Cmd cannot be reused after calling its Run or
Start methods."

This retry is a common case due to, say, a CRD and its resource
being present in the same YAML file of the addon which causes
a race condition where the resource is created before its CRD is
created in the cluster - this race is fixed by subsequent retries.
I've noticed this in the dashboard and the ambassador addon.

Due to the above mentioned bug, minikube throws errors like
`exec: already started` in every retry and the retry is never
successful, manifests are never deployed and addon creation errors
out.

Related to kubernetes#8138 kubernetes#8119 kubernetes#8372
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
kind/support Categorizes issue or PR as a support question. triage/needs-information Indicates an issue needs more information in order to work on it.
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

addons: fix retry command concaf/minikube
3 participants
@medyagh @priyawadhwa @pkrrsimtian