kubeadm should use multi-architecture images when GCR supports Docker distribution API v2 schema 2

[mikedanese]

From @errordeveloper on October 3, 2016 11:19

cc kubernetes/kubernetes#33916

Copied from original issue: kubernetes/kubernetes#33917

[luxas]

I've been told v2 schema 2 will be rolled out to gcr.io in mid-December, so we can probably target this v1.6

[sudhagarc]

I'm new to k8 & stuck/blocked by this issue. I'm using kubeadm v1.6.1. Is the issue addressed post that? Alternatively, if there is a manual way to solve it, I appreciate the instructions.

[codesnk]

Hi all,
Any updates on this? My cluster (Linux hosts & VMs) works fine till I try to join a Pi3 in the mix. The issue is that the Master (Ubuntu) downloads amd64 images for kube-proxy and weave net on the Pi node. I thought this shouldn't happen as the master labels the node as arm architecture.

[jamiehannaford]

@luxas So this is just a case of swapping out the arch type in the URL? If so, I can have a go at this

[jamiehannaford]

Oh it seems that we already do this.

@errordeveloper Can I close this?

[luxas]

This is about k8s pushing manifest lists docker-library/official-images#2289 for all control plane images and kubeadm starting to consume them. We do the ARCH suffixing now, but that's bad. Currently this is stuck on GCR.io not supporting the Docker Registry API v2 schema 2, but hopefully they will in some month

[vielmetti]

GCR.io now supports the new schema and multiarch images and thus this work can be evaluated to proceed.

[aitorhh]

I have being using a multi-platform cluster since last May using the docker registry API v2 and the manifest-tool to push the images to my own private repository.

In kubeadm (v1.6.1), I only needed the following change (extracted from a patch file):

---
 cmd/kubeadm/app/images/images.go | 11 +++++------
 1 file changed, 5 insertions(+), 6 deletions(-)

diff --git a/cmd/kubeadm/app/images/images.go b/cmd/kubeadm/app/images/images.go
index 3f714c9..b0cbfc7 100644
--- a/cmd/kubeadm/app/images/images.go
+++ b/cmd/kubeadm/app/images/images.go
@@ -18,7 +18,6 @@ package images
 
 import (
        "fmt"
-       "runtime"
 
        kubeadmapi "k8s.io/kubernetes/cmd/kubeadm/app/apis/kubeadm"
 )
@@ -39,10 +38,10 @@ func GetCoreImage(image string, cfg *kubeadmapi.MasterConfiguration, overrideIma
        }
        repoPrefix := kubeadmapi.GlobalEnvParams.RepositoryPrefix
        return map[string]string{
-               KubeEtcdImage:              fmt.Sprintf("%s/%s-%s:%s", repoPrefix, "etcd", runtime.GOARCH, etcdVersion),
-               KubeAPIServerImage:         fmt.Sprintf("%s/%s-%s:%s", repoPrefix, "kube-apiserver", runtime.GOARCH, cfg.KubernetesVersion),
-               KubeControllerManagerImage: fmt.Sprintf("%s/%s-%s:%s", repoPrefix, "kube-controller-manager", runtime.GOARCH, cfg.KubernetesVersion),
-               KubeSchedulerImage:         fmt.Sprintf("%s/%s-%s:%s", repoPrefix, "kube-scheduler", runtime.GOARCH, cfg.KubernetesVersion),
-               KubeProxyImage:             fmt.Sprintf("%s/%s-%s:%s", repoPrefix, "kube-proxy", runtime.GOARCH, cfg.KubernetesVersion),
+               KubeEtcdImage:              fmt.Sprintf("%s/%s:%s", repoPrefix, "etcd", etcdVersion),
+               KubeAPIServerImage:         fmt.Sprintf("%s/%s:%s", repoPrefix, "kube-apiserver", cfg.KubernetesVersion),
+               KubeControllerManagerImage: fmt.Sprintf("%s/%s:%s", repoPrefix, "kube-controller-manager", cfg.KubernetesVersion),
+               KubeSchedulerImage:         fmt.Sprintf("%s/%s:%s", repoPrefix, "kube-scheduler", cfg.KubernetesVersion),
+               KubeProxyImage:             fmt.Sprintf("%s/%s:%s", repoPrefix, "kube-proxy", cfg.KubernetesVersion),
        }[image]
 }
-- 

Then, it's just a matter of setting the KUBE_REPO_PREFIX

KUBE_REPO_PREFIX=<docker-registry-v2> /usr/local/bin/kubeadm

If the multi-platform images are already available, and can make a PR.

[luxas]

We still need to do something like this: kubernetes/release#248

If you @aitorhh could open a similar PR that is up-to-date with that functionality and sync with @ixdy to get it merged soon-ish, I'd be more than happy!

We also need to coordinate/sync this up with kubernetes/kubernetes#54174

[timothysc]

/assign @detiber

[neolit123]

/assign
/kind documentation

for docs update:
https://kubernetes.io/docs/reference/setup-tools/kubeadm/kubeadm-init/

[timothysc]

This is done!