Build Kubelet without Docker

[mattjmcnaughton]

A KEP proposing supporting building a Kubelet without any "Docker-specific" code or dependency on the docker/docker Golang library, via the use of build tags.

[mattjmcnaughton]

@dims there were a couple of open questions I had left over from our conversation that I tagged you in - thanks :)

[mattjmcnaughton]

cc @derekwaynecarr @dchen1107

@dims and I had some chats about the benefits of this workstream. Would love to know y'alls thoughts on next steps?

cc @BenTheElder would be curious to get your opinion/expertise given your work on this w/ sig-cloud-provider.

[mattjmcnaughton]

/retest

[BenTheElder]

quick skim 👀

[derekwaynecarr]

An alternative to consider:

• announce deprecation of dockershim
• identify gaps to CRI v1 and set a goal for 1.21
• removal of dockershim with CRI v1

[sftim]

nit: remove unused optional headings; remove [optional] mark from optional headings that are selected.

[mattjmcnaughton]

nit: remove unused optional headings; remove [optional] mark from optional headings that are selected.

Thanks - fixed these :)

[mattjmcnaughton]

Thanks for your responses @dims :) Updated my pr with your answers.

[mattjmcnaughton]

/assign @dchen1107 @derekwaynecarr @Random-Liu

From conversations with @dims, I think this KEP is ready for review when y'all get the chance. I will be working on getting the POC PR in a mergeable state in parallel. Thanks :)

[dims]

/milestone v1.19

[neolit123]

/cc

[mattjmcnaughton]

we need to be cautious about removing Docker support entirely from the kubelet.

while managed solutions and Kubernetes distributions can opt-out of using Docker on their nodes, the wider public is still using Docker. by dropping Docker support the Kubernetes project will be reducing it's ease of consumption.

in a follow up KEP i think we should discuss how the dockershim binary can be separated from the kubelet, managed in separate repository (ideally with community help) and distributed by the Kubernetes project as part of its release.

some numbers from the last kubeadm survey:
https://docs.google.com/forms/d/1A855D054FZfdvNbx6mAa54tcZ204GmPCIhxe1UrXKd8/edit

Thanks for that additional data! Definite +1 from me that we want to be very careful about the path forward to actually remove Docker support entirely from the Kubelet. It requires more discussion and thought. Hopefully this KEP makes testing out different paths, and ultimately pursuing one, easier.

[neolit123]

thank you for helping with this effort @mattjmcnaughton

[mattjmcnaughton]

thank you for helping with this effort @mattjmcnaughton

My pleasure, thanks for adding your insights :)

[derekwaynecarr]

I would make the following updates:

1. drop story 2
2. add under test-criteria "creation of a pre-submit job to ensure we dont regress on dockerless build"
3. add under test-criteria "creation of a node e2e job that passes dockerless"

the node e2e suite itself has docker deps, and so it will also want to get updated to support building a "dockerless" version so that it doesn't inadvertently regress into including them.

[derekwaynecarr]

I agree with everyone enumerated here.

[derekwaynecarr]

i believe this impacts sig-windows, but everything here makes sense.

[mattjmcnaughton]

Worth adding them as a participating sig? Or just something for folks to remain cognizant of moving forward?

[derekwaynecarr]

I would add it as a participating sig unless they have fully transitioned to an alternative.

[derekwaynecarr]

agree this is orthogonal as there is also work to eliminate/reduce cadvisor dep from kubelet itself.

[mattjmcnaughton]

I would make the following updates:

1. drop story 2

2. add under test-criteria "creation of a pre-submit job to ensure we dont regress on dockerless build"

3. add under test-criteria "creation of a node e2e job that passes dockerless"

the node e2e suite itself has docker deps, and so it will also want to get updated to support building a "dockerless" version so that it doesn't inadvertently regress into including them.

Thanks for the feedback @derekwaynecarr - will address within the next day or two, and will ping you when I do.

[mattjmcnaughton]

@derekwaynecarr I updated the testing section with the tests you requested. Please let me know if there's anything I misinterpreted or missed. I also deleted the second story.

W.r.t. kubernetes/kubernetes#87746 (the implementation PR), it currently includes the first test type, but not the second, third, and forth. Do we want to block merging the PR on having all the new tests in place, or merge the PR as is and add them later? I'm open to either path, with a slight preference for keeping PRs as small as possible :)

cc @dims

[dims]

@mattjmcnaughton my preference is to merge this as-is and file subsequent PRs. there is no requirement that one KEP == one PR :)

[derekwaynecarr]

Thank you for enumerating your implementation plan.

I look forward to assisting you in making this succesful.

/approve
/lgtm

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: derekwaynecarr, mattjmcnaughton

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• keps/sig-node/OWNERS [derekwaynecarr]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[dims]

w00t!!! thanks @derekwaynecarr

[mattjmcnaughton]

Wonderful news - big thanks @derekwaynecarr !

[BenTheElder]

this is pretty cool :-)
looking forward to seeing how much binary size we can drop ❤️

[mattjmcnaughton]

cc @dims

what, if anything, do we need to do to get this KEP included in 1.19? Re https://groups.google.com/forum/#!topic/kubernetes-dev/G5iIu3FrRTY

Or is nothing necessary because its not user facing?

[neolit123]

i think this KEP can move to implementable for 1.19. the outlined work is developer facing (i.e. compile options).

[dims]

@mattjmcnaughton like @neolit123 mentioned, please flip to implementable ( example https://github.com/kubernetes/enhancements/pull/1715/files#diff-215637d1a7a561ceea8855c304720e82R10 )