Reduce memory footprint by removing unnecessary cloud provider plugins at runtime

[frolickingferret445]

I am interested in reducing the memory footprint of kubernetes if possible. From what I gather, by default, kubernetes runs with plugins for all cloud providers. I’ve looked into https://k3s.io/ and it mentions that it removes unnecessary in-tree cloud providers and storage drivers. However, it doesn’t offer other important features, so I’d prefer to stick to my existing installation process (with perhaps a couple of additional steps).

Is there a way to do this? (preferably with kubeadm)

I discovered a daemonset called cloud-controller-manager (beta feature) which pulls out some of the cloud-specific tasks from kube-controller-manager.

https://kubernetes.io/docs/tasks/administer-cluster/running-cloud-controller/

However, it still has a cloud-provider command-line flag. Does cloud-controller-manager reduce the memory footprint by only loading plugins for the necessary cloud provider?

I looked into using kubeadm to install cloud-controller-manager, but it seems that it isn’t yet supported, probably because it’s a beta feature. All of the controller-manager flags are for kube-controller-manager:

https://kubernetes.io/docs/setup/independent/control-plane-flags/

It sounds like I could get this working with a few extra steps similar to this :

https://github.com/kubernetes/cloud-provider-openstack/blob/master/docs/using-controller-manager-with-kubeadm.md

Right?

Any help would be appreciated, thanks.

@kubernetes/sig-cloud-provider-maintainers

[k8s-ci-robot]

@frolickingferret445: There are no sig labels on this issue. Please add a sig label by either:

1. mentioning a sig: @kubernetes/sig-<group-name>-<group-suffix>
   e.g., @kubernetes/sig-contributor-experience-<group-suffix> to notify the contributor experience sig, OR

2. specifying the label manually: /sig <group-name>
   e.g., /sig scalability to apply the sig/scalability label

Note: Method 1 will trigger an email to the group. See the group list.
The <group-suffix> in method 1 has to be replaced with one of these: bugs, feature-requests, pr-reviews, test-failures, proposals.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[frolickingferret445]

@sig-cloud-provider

[frolickingferret445]

@kubernetes/sig-cloud-provider

[frolickingferret445]

@kubernetes/sig-cloud-provider-maintainers

[frolickingferret445]

/sig cloud-provider-maintainers

[vllry]

/assign @andrewsykim

[BenTheElder]

also interested in small memory footprints 🙃
cc @cheftako re: cloud-controller-manager
cc @neolit123 re: kubeadm

[neolit123]

I looked into using kubeadm to install cloud-controller-manager, but it seems that it isn’t yet supported, probably because it’s a beta feature. All of the controller-manager flags are for kube-controller-manager:

kubeadm is a node boostrapper. if kubernetes supports cloud-controller-manager as a DaemonSet then kubeadm supports it too.

except that we don't have a guide for that.

All of the controller-manager flags are for kube-controller-manager:
https://kubernetes.io/docs/setup/independent/control-plane-flags/

you still have to use the kubeadm config to pass extraArgs to the control-plane components.

It sounds like I could get this working with a few extra steps similar to this :
https://github.com/kubernetes/cloud-provider-openstack/blob/master/docs/using-controller-manager-with-kubeadm.md

this guide runs the cloud-controller-manager as a static pod.

Does cloud-controller-manager reduce the memory footprint by only loading plugins for the necessary cloud provider?

will leave the memory footprint questions to sig-cloud-provider. :)

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[BenTheElder]

you can exclude all of them at build time now, the CCM is the way forward though as all cloud providers are going out of tree

[fejta-bot]

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten

[cheftako]

/remove-lifecycle rotten

[cheftako]

/lifecycle frozen

[dims]

tracked already kubernetes/enhancements#88 (comment)

/close

[k8s-ci-robot]

@dims: Closing this issue.

In response to this:

tracked already kubernetes/enhancements#88 (comment)

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.