'kubeadm join' failed on cluster node ( @kubernetes/sig-cluster-lifecycle )

[tonyclam]

Is this a request for help? (If yes, you should use our troubleshooting guide and community support channels, see http://kubernetes.io/docs/troubleshooting/.):
No
What keywords did you search in Kubernetes issues before filing this one? (If you have found any duplicates, you should instead reply there.):
'<node/discovery> failed to parse response as JWS object [square/go-jose: compact JWS format must have three parts]'

Is this a BUG REPORT or FEATURE REQUEST? (choose one):
BUG REPORT

Kubernetes version (use kubectl version):
Client Version: version.Info{Major:"1", Minor:"4", GitVersion:"v1.4.4", GitCommit:"3b417cc4ccd1b8f38ff9ec96bb50a81ca0ea9d56", GitTreeState:"clean", BuildDate:"2016-10-21T02:48:38Z", GoVersion:"go1.6.3", Compiler:"gc", Platform:"linux/amd64"}

Environment:

Cloud provider or hardware configuration:
OS (e.g. from /etc/os-release): Centos 7.1
Kernel (e.g. uname -a): 4.1.12-37.5.1.el7uek.x86_64 #2 SMP Thu Jun 9 16:01:20 PDT 2016 x86_64 x86_64 x86_64 GNU/Linux
Install tools:
Others:
What happened:
Failed with error:
<node/discovery> failed to parse response as JWS object [square/go-jose: compact JWS format must have three parts]

What you expected to happen:
The cluster should be able to join the master node

How to reproduce it (as minimally and precisely as possible):

kubeadm init on master
kubeadm join on cluster
Anything else do we need to know:
kubeadm version: version.Info{Major:"1", Minor:"5+", GitVersion:"v1.5.0-alpha.2.380+85fe0f1aadf91e", GitCommit:"85fe0f1aadf91e134102cf3c01a9eed11a7e257f", GitTreeState:"clean", BuildDate:"2016-11-02T14:58:17Z", GoVersion:"go1.7.1", Compiler:"gc", Platform:"linux/amd64"}
For kube-discovery image

docker inspect
[root@ovm21 ~]# docker inspect c5e0c9a457fc
[
{
"Id": "sha256:c5e0c9a457fcb53ac5c564656f3fabba733ab1e8187e98d095c88356b9245de8",
"RepoTags": [
"gcr.io/google_containers/kube-discovery-amd64:1.0"
],
"RepoDigests": [
"gcr.io/google_containers/kube-discovery-amd64@sha256:7ebce8129c41bf64053f56a4f4418e198265b104b17f3f2d5b61667e208528f4"
],
"Parent": "",
"Comment": "",
"Created": "2016-09-24T17:16:49.942311731Z",
"Container": "5581c81bd1defe34d67a2c267841bc164e0decd53b1a9114b745585e628de106",
"ContainerConfig": {
"Hostname": "6250540837a8",
"Domainname": "",
"User": "",
"AttachStdin": false,
"AttachStdout": false,
"AttachStderr": false,
"Tty": false,
"OpenStdin": false,
"StdinOnce": false,
"Env": [
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
],
"Cmd": [
"/bin/sh",
"-c",
"#(nop) ENTRYPOINT \u0026{["/bin/sh" "-c" ""/usr/local/bin/kube-discovery""]}"
],
"Image": "2b3aeb5c0ccc7b85afa3c110b8b6fc3698f1da723f4845cd4250895c599393e1",
"Volumes": null,
"WorkingDir": "",
"Entrypoint": [
"/bin/sh",
"-c",
""/usr/local/bin/kube-discovery""
],
"OnBuild": [],
"Labels": {}
},
"DockerVersion": "1.9.1",
"Author": "",
"Config": {
"Hostname": "6250540837a8",
"Domainname": "",
"User": "",
"AttachStdin": false,
"AttachStdout": false,
"AttachStderr": false,
"Tty": false,
"OpenStdin": false,
"StdinOnce": false,
"Env": [
"PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
],
"Cmd": null,
"Image": "2b3aeb5c0ccc7b85afa3c110b8b6fc3698f1da723f4845cd4250895c599393e1",
"Volumes": null,
"WorkingDir": "",
"Entrypoint": [
"/bin/sh",
"-c",
""/usr/local/bin/kube-discovery""
],
"OnBuild": [],
"Labels": {}
},
"Architecture": "amd64",
"Os": "linux",
"Size": 134164555,
"VirtualSize": 134164555,
"GraphDriver": {
"Name": "btrfs",
"Data": null
},
"RootFS": {
"Type": "layers",
"Layers": [
"sha256:42755cf4ee95900a105b4e33452e787026ecdefffcc1992f961aa286dc3f7f95",
"sha256:7815ca90458efad6a994cbe56b123737a3b92c472493d5581fb960821680a338",
"sha256:5f70bf18a086007016e948b04aed3b82103a36bea41755b6cddfaf10ace3c6ef"
]
}
}
]

[snavien]

I had the same issue, did a kubeadm reset and redid the apply flannel yaml and it worked fine, but let me know if you're able to solve this issue.

[tonyclam]

Thx. I will try it out

…

On Thu, Dec 1, 2016 at 2:27 PM, snavien ***@***.***> wrote: I had the same issue, did a kubeadm reset and redid the apply flannel yaml and it worked fine, but let me know if you're able to solve this issue. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#78 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AUZ7Cwedpz9b59k_XXcYEKvE1B5-iwIDks5rD0nDgaJpZM4LBDpk> .

[pires]

@tonyclam any feedback on this?

[luxas]

We've released a new version since then also.
I'm closing this one as I think it's fixed with the latest version.

If not, just comment here and reopen

[ZachGoldberg]

I'm actually getting the same issue but with apparently a different root cause. I get an identical error with any tokens generate subsequent via "kubeadm token generate." If I use the token originally outputted via kubeadm init it works, but new tokens all produce a failure identical to the above.

[pipejakob]

@ZachGoldberg This probably needs to be cleared up in our documentation, but the command kubeadm token generate just prints out a random token without storing it anywhere or activating it on the master. It's only really appropriate to use before kubeadm init so that you can pass in the value via --token <val> instead of having init generate one for you. This makes it easier to transport or store the token you used duration initialization, instead of having to capture and parse the output of the kubeadm init command when it generates the token for you.

So, the overall flow should look like this:

$ TOKEN=$(kubeadm token generate)
$ kubeadm init --token $TOKEN

Use scp, copy-paste, or any other mechanism to get $TOKEN to your nodes, then on each node:
 
$ kubeadm join --token $TOKEN <master>

I believe the command you're looking for is kubeadm token create, which will actually store and activate the new token so it can be used by nodes.

[eliefly]

In kubeadm 1.6.0 the kubeadm token create is not available, i run this command get the token.
kubectl -n kube-system get secret clusterinfo -o yaml | grep token-map | awk '{print $2}' | base64 -d | awk -F\" '{print $2 "." $4}'

ref: https://stackoverflow.com/questions/40009831/cant-find-kubeadm-token-after-initializing-master