Documentation for dynamic volume limits based on node type #8871

gnufied · 2018-06-01T23:27:03Z

Documentation for dynamic volume limits based on node type.

xref : kubernetes/community#2051

k8sio-netlify-preview-bot · 2018-06-01T23:27:06Z

Deploy preview for kubernetes-io-vnext-staging processing.

Built with commit 6913ea7

https://app.netlify.com/sites/kubernetes-io-vnext-staging/deploys/5b295841c9659223b5dc9b99

tengqm · 2018-06-04T02:11:45Z

content/en/docs/concepts/storage/storage-limits.md

+---
+
+This document describes the concept of number of maximum volumes that can be attached
+to a particular node on various cloud providers.


number of maximum volumes

maximum number of volumes ?

tengqm · 2018-06-04T02:15:09Z

content/en/docs/concepts/storage/storage-limits.md

+### Kubernetes built-in attachable limits for Azure, AWS and GCE
+
+Kubernetes's default scheduler ships with predicates that limit following
+number of volumes attachable to a particular node:


that limit following number of volumes attachable to a particular node:

that limit the number of volumes attached to a particular node as follows:

tengqm · 2018-06-04T02:16:36Z

content/en/docs/concepts/storage/storage-limits.md

+  For Azure Disks Kubernetes permits only 16 volumes to be attached to a node.
+
+Kubernetes admin can change these limits by starting scheduler with environment variable
+`KUBE_MAX_PD_VOLS` and setting a new value.


starting scheduler with environment variable KUBE_MAX_PD_VOLS and setting a new value.

starting the scheduler with the environment variable KUBE_MAX_PD_VOLS set to a new value.

tengqm · 2018-06-04T02:17:12Z

content/en/docs/concepts/storage/storage-limits.md

+Kubernetes admin can change these limits by starting scheduler with environment variable
+`KUBE_MAX_PD_VOLS` and setting a new value.
+
+You must pay extra caution while setting a higher value than defaults and consult corresponding


setting a higher value than defaults

setting a value higher than the defaults

tengqm · 2018-06-04T02:18:47Z

content/en/docs/concepts/storage/storage-limits.md

+
+You must pay extra caution while setting a higher value than defaults and consult corresponding
+cloudprovider documentation to make sure the node can actually support those limits. Also these limits
+are global for entire cluster and hence will affect all nodes.


are global for entire cluster

are applied to the whole cluster

tengqm · 2018-06-04T02:21:57Z

content/en/docs/concepts/storage/storage-limits.md

+### Dynamic volume limits based on node type
+
+Kubernetes 1.11 introduces dynamic volume limits based on node type. It is an Alpha feature and admin
+must enable feature flag `AttachVolumeLimit` to enable this feature.


Please check other docs how alpha/beta features are documented using shortcodes.
Please add the feature gate to the feature gate doc.
You can use #8835 as an example.

gnufied · 2018-06-04T16:40:35Z

/sig storage

cc @saad-ali @childsb @jsafrane

mdlinville · 2018-06-04T18:35:11Z

I rebased to resolve merge conflicts.

mdlinville · 2018-06-12T00:36:25Z

/assign @steveperry-53

Thanks!

steveperry-53 · 2018-06-13T01:18:12Z

@gnufied @MistyHacks @tengqm See this commit for suggested edits.

Preview: https://deploy-preview-8871--kubernetes-io-vnext-staging.netlify.com/docs/concepts/storage/storage-limits/

gnufied · 2018-06-13T02:04:25Z

@steveperry-53 thank you. will take a look.

@msau42 @ravisantoshgudimetla Should we document Balanced volume feature gate somewhere in this document for 1.11 release?

msau42 · 2018-06-13T02:17:48Z

Documenting the balanced volumes feature sounds fine.

In general, do we need a new page for this? Or can we append it to the volumes page?

gnufied · 2018-06-13T02:22:06Z

Yeah I thought about appending this to one of existing pages but in the end I felt we haven't really documented this in k8s and along with balanced volume feature it felt big enough to deserve its own page..

ravisantoshgudimetla · 2018-06-13T03:54:19Z

Should we document Balanced volume feature gate somewhere in this document for 1.11 release?

I think we should and yes it deserves its own page.

tengqm · 2018-06-13T04:35:13Z

content/en/docs/concepts/storage/storage-limits.md


- AWS Elastic Block store
- GCE Persistent Disks
+To enable dynamic volume limits, use the `AttachVolumeLimit` flag.


What kind of a flag is this? A field in kubelet configuration file or a feature gate?

This is a new feature gate.

tengqm · 2018-06-13T04:36:25Z

content/en/docs/concepts/storage/storage-limits.md

+- Amazon EBS
+- Google Persistent Disk
+
+To enable dynamic volume limits, use the `AttachVolumeLimit` flag.


Is AttachVolumeLimit a new feature gate to be documented in the reference?

steveperry-53 · 2018-06-15T19:53:38Z

@gnufied ping

zparnold · 2018-06-18T17:16:12Z

Hello there @gnufied, do you have an update for this? We are past the docs deadline for 1.11 release and need to get this immediately.

gnufied · 2018-06-18T17:27:30Z

@steveperry-53 @zparnold wait, what are we waiting for from me? Are we waiting for documentation of "Balanced volume" feature? I did not implement that feature. that will be a new feature introduced in 1.11 but I just pointed out that - we do not seem to have documentation for it. @ravisantoshgudimetla should ideally be adding docs for that feature.

zparnold · 2018-06-18T17:30:12Z

Ah, I see. Apologies. I picked you out of the list because you opened the PR. I'm scanning my spreadsheet and trying to see what I can get merged. 😄

ravisantoshgudimetla · 2018-06-18T18:20:27Z

@ravisantoshgudimetla should ideally be adding docs for that feature.

That is right. I missed the notification since it was not part of features... Just to be clear, @gnufied, are we fine with creating this document in other page?If yes, where?

steveperry-53 · 2018-06-18T18:40:12Z

I think we should get this merged for the 1.11 release.

But soon we should get @tengqm's question answered:

Is AttachVolumeLimit a new feature gate to be documented in the reference?

And when we say, "use the AttachVolumeLimit flag", we should say on which component. kube-apiserver? Other comonent(s)?

Holding for approval by @MistyHacks.
/hold
It's my understanding that @MistyHacks is out today (Monday), but should be able to approve this Tuesday.

/lgtm

mdlinville · 2018-06-19T19:24:08Z

/lgtm
/approve

Thanks all!

k8s-ci-robot · 2018-06-19T19:24:14Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: mistyhacks

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [mistyhacks]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

steveperry-53 · 2018-06-19T20:58:47Z

/hold cancel

* add cos for storage limits * Update docs specific for aws and gce * fix some minor things * Update storage-limits.md * Add k8s version to feature-state shortcode

* Seperate priority and preemption (#8144) * Doc about PID pressure condition. (#8211) * Doc about PID pressure condition. Signed-off-by: Da K. Ma <[email protected]> * "so" -> "too" * Update version selector for 1.11 * StorageObjectInUseProtection is GA (#8291) * Feature gate: StorageObjectInUseProtection is GA Update feature gate reference for 1.11 * Trivial commit to re-trigger Netlify * CRIContainerLogRotation is Beta in 1.11 (#8665) * Seperate priority and preemption (#8144) * CRIContainerLogRotation is Beta in 1.11 xref: kubernetes/kubernetes#64046 * Bring StorageObjectInUseProtection feature to GA (#8159) * StorageObjectInUseProtection is GA (#8291) * Feature gate: StorageObjectInUseProtection is GA Update feature gate reference for 1.11 * Trivial commit to re-trigger Netlify * Bring StorageObjectInUseProtection feature to GA StorageObjectInUseProtection is Beta in K8s 1.10. It's brought to GA in K8s 1.11. * Fixed typo and added feature state tags. * Remove KUBE_API_VERSIONS doc (#8292) The support to the KUBER_API_VERSIONS environment variable is completely dropped (no deprecation). This PR removes the related doc in release-1.11. xref: kubernetes/kubernetes#63165 * Remove InitialResources from admission controllers (#8293) The feature (was experimental) is dropped in 1.11. xref: kubernetes/kubernetes#58784 * Remove docs related to in-tree support to GPU (#8294) * Remove docs related to in-tree support to GPU The in-tree support to GPU is completely removed in release 1.11. This PR removes the related docs in release-1.11 branch. xref: kubernetes/kubernetes#61498 * Update content updated by PR to Hugo syntax Signed-off-by: Misty Stanley-Jones <[email protected]> * Update the doc about extra volume in kubeadm config (#8453) Signed-off-by: Xianglin Gao <[email protected]> * Update CRD Subresources for 1.11 (#8519) * coredns: update notes in administer-cluster/coredns.md (#8697) CoreDNS is installed by default in 1.11. Add notes on how to install kube-dns instead. Update notes about CoreDNS->CoreDNS upgrades as in 1.11 the Corefile is retained. Add example on upgrading from kube-dns to CoreDNS. * kubeadm-alpha: CoreDNS related changes (#8727) Update note about CoreDNS feature gate. This change also updates a tab as a kubeadm sub-command will change. It looks for a new generated file: generated/kubeadm_alpha_phase_addon_coredns.md instead of: generated/kubeadm_alpha_phase_addon_kube-dns.md * Update cloud controller manager docs to beta 1.11 (#8756) * Update cloud controller manager docs to beta 1.11 * Use Hugo shortcode for feature state * kubeadm-upgrade: include new command `kubeadm upgrade diff` (#8617) Also: - Include note that this was added in 1.11. - Modify the note about upgrade guidance. * independent: update CoreDNS mentions for kubeadm (#8753) Give CoreDNS instead of kube-dns examples in: - docs/setup/independent/create-cluster-kubeadm.md - docs/setup/independent/troubleshooting-kubeadm.md * update 1.11 --server-print info (#8870) * update 1.11 --server-print info * Copyedit * Mark ExpandPersistentVolumes feature to beta (#8778) * Update version selector for 1.11 * Mark ExpandPersistentVolumes Beta xref: kubernetes/kubernetes#64288 * fix shortcode, add placeholder files to fix deploy failures (#8874) * declare ipvs ga (#8850) * kubeadm: update info about CoreDNS in kubeadm-init.md (#8728) Add info to install kube-dns instead of CoreDNS, as CoreDNS is the default DNS server in 1.11. Add notes that kubeadm config images can be used to list and pull the required images in 1.11. * kubeadm: update implementation-details.md about CoreDNS (#8829) - Replace examples from kube-dns to CoreDNS - Add notes about the CoreDNS feature gate status in 1.11 - Add note that the service name for CoreDNS is also called `kube-dns` * Update block device support for 1.11 (#8895) * Update block device support for 1.11 * Copyedits * Fix typo 'fiber channel' (#8957) Signed-off-by: Misty Stanley-Jones <[email protected]> * kubeadm-upgrade: add the 'node [config]' sub-command (#8960) - Add includes for the generated pages - Include placeholder generated pages * kubeadm-init: update the example for the MasterConfiguration (#8958) - include godocs link for MasterConfiguration - include example MasterConfiguration - add note that `kubeadm config print-default` can be used * kubeadm-config: include new commands (#8862) Add notes and includes for these new commands in 1.11: - kubeadm config print-default - kubeadm config migrate - kubeadm config images list - kubeadm config images pull Include placeholder generated files for the above. * administer-cluster/coredns: include more changes (#8985) It was requested that for this page a couple of methods should be outlined: - manual installation for CoreDNS explained at the Kubernetes section of the GitHub project for CoreDNS - installation and upgrade via kubeadm Make the above changes and also add a section "About CoreDNS". This commit also lowercases a section title. * Update CRD subresources doc for 1.11 (#8918) * Add docs for volume expansion and online resizing (#8896) * Add docs for volume expansion going beta * Copyedit * Address feedback * Update exec plugin docs with TLS credentials (#8826) * Update exec plugin docs with TLS credentials kubernetes/kubernetes#61803 implements TLS client credential support for 1.11. * Copyedit * More copyedits for clarification * Additional copyedit * Change token->credential * NodeRestriction admission prevents kubelet taint removal (#8911) * dns-custom-namerserver: break down the page into mutliple sections (#8900) * dns-custom-namerserver: break down the page into mutliple sections This page is currently about kube-dns and is a bit outdated. Introduce the heading `# Customizing kube-dns`. Introduce a separate section about CoreDNS. * Copyedits, fix headings for customizing DNS Hey Lubomir, I coypedited pretty heavily because this workflow is so much easier for docs and because I'm trying to help improve everything touching kubeadm as much as possible. But there's one outstanding issue wrt headings and intro content: you can't add a heading 1 to a topic to do what you wanted to do. The page title in the front matter is rendered as a heading 1 and everything else has to start at heading 2. (We still need to doc this better in the docs contributing content, I know.) Instead, I think we need to rewrite the top-of-page intro content to explain better the relationship between kube-dns and CoreDNS. I'm happy to write something, but I thought I'd push this commit first so you can see what I'm doing. Hope it's all clear -- ping here or on Slack with any questions ~ Jennifer * Interim fix for talking about CoreDNS * Fix CoreDNS details * PSP readOnly hostPath (#8898) * Add documentation for crictl (#8880) * Add documentation for crictl * Copyedit Signed-off-by: Misty Stanley-Jones <[email protected]> * Final copyedit * VolumeSubpathEnvExpansion alpha feature (#8835) * Note that Heapster is deprecated (#8827) * Note that Heapster is deprecated This notes that Heapster is deprecated, and migrates the relevant docs to talk about metrics-server or other solutions by default. * Copyedits and improvements Signed-off-by: Misty Stanley-Jones <[email protected]> * Address feedback * fix shortcode to troubleshoot deploy (#9057) * update dynamic kubelet config docs for v1.11 (#8766) * update dynamic kubelet config docs for v1.11 * Substantial copyedit * Address feedback * Reference doc for kubeadm (release-1.11) (#9044) * Reference doc for kubeadm (release-1.11) * fix shortcode to troubleshoot deploy (#9057) * Reference doc for kube-components (release-1.11) (#9045) * Reference doc for kube-components (release-1.11) * Update cloud-controller-manager.md * fix shortcode to troubleshoot deploy (#9057) * Documentation on lowercasing kubeadm init apiserver SANs (#9059) * Documentation on lowercasing kubeadm init apiserver SANs * fix shortcode to troubleshoot deploy (#9057) * Clarification in dynamic Kubelet config doc (#9061) * Promote sysctls to Beta (#8804) * Promote sysctls to Beta * Copyedits Signed-off-by: Misty Stanley-Jones <[email protected]> * Review comments * Address feedback * More feedback * kubectl reference docs for 1.11 (#9080) * Update Kubernetes API 1.11 ref docs (#8977) * Update v1alpha1 to v1beta1. * Adjust left nav for 1.11 ref docs. * Trim list of old ref docs. * Update Federation API ref docs for 1.11. (#9064) * Update Federation API ref docs for 1.11. * Add titles. * Update definitions.html * CRD versioning Public Documentation (#8834) * CRD versioning Public Documentation * Copyedit Signed-off-by: Misty Stanley-Jones <[email protected]> * Address feedback * More rewrites * Address feedback * Update main CRD page in light of versioning * Reorg CRD docs * Further reorg * Tweak title * CSI documentation update for raw block volume support (#8927) * CSI documetation update for raw block volume support * minor edits for "CSI raw block volume support" Some small grammar and style nits. * minor CSIBlockVolume edits * Update kubectl component ref page for 1.11. (#9094) * Update kubectl component ref page for 1.11. * Add title. Replace stevepe with username. * crd versioning doc: fix nits (#9142) * Update `DynamicKubeletConfig` feature to beta (#9110) xref: kubernetes/kubernetes#64275 * Documentation for dynamic volume limits based on node type (#8871) * add cos for storage limits * Update docs specific for aws and gce * fix some minor things * Update storage-limits.md * Add k8s version to feature-state shortcode * The Doc update for ScheduleDaemonSetPods (#8842) Signed-off-by: Da K. Ma <[email protected]> * Update docs related to PersistentVolumeLabel admission control (#9109) The said admission controller is disabled by default in 1.11 (kubernetes/kubernetes#64326) and scheduled to be removed in future release. * client exec auth: updates for 1.11 (#9154) * Updates HA kubeadm docs (#9066) * Updates HA kubeadm docs Signed-off-by: Chuck Ha <[email protected]> * kubeadm HA - Add stacked control plane steps * ssh instructions and some typos in the bash scripts Signed-off-by: Chuck Ha <[email protected]> * Fix typos and copypasta errors * Fix rebase issues * Integrate more changes Signed-off-by: Chuck Ha <[email protected]> * copyedits, layout and formatting fixes * final copyedits * Adds a sanity check for load balancer connection Signed-off-by: Chuck Ha <[email protected]> * formatting fixes, copyedits * fix typos, formatting * Document the Pod Ready++ feature (#9180) Closes: #9107 Xref: kubernetes/kubernetes#64057 * Mention 'KubeletPluginsWatcher' feature (#9177) * Mention 'KubeletPluginsWatcher' feature This feature is more developers oriented than users oriented, so simply mention it in the feature gate should be fine. In future, when the design doc is migrated from Google doc to the kubernetes/community repo, we can add links to it for users who want to dig deeper. Closes: #9108 Xref: kubernetes/kubernetes#63328, kubernetes/kubernetes#64605 * Copyedit * Amend dynamic volume list docs (#9181) The dynamic volume list feature has been documented but the feature gate related was not there yet. Closes: #9105 * Document for service account projection (#9182) This adds docs for the service account projection feature. Xref: kubernetes/kubernetes#63819, kubernetes/community#1973 Closes: #9102 * Update pod priority and preemption user docs (#9172) * Update pod priority and preemption user docs * Copyedit * Documentation on setting node name with Kubeadm (#8925) * Documentation on setting node name with Kubeadm * copyedit * Add kubeadm upgrade docs for 1.11 (#9089) * Add kubeadm upgrade docs for 1.11 * Initial docs review feedback * Add 1-11 to outline * Fix formatting on tab blocks * Move file to correct location * Add `kubeadm upgrade node config` step * Overzealous ediffing * copyedit, fix lists and headings * clarify --force flag for fixing bad state * Get TOML ready for 1.11 release * Blog post for 1.11 release (#9254) * Blog post for 1.11 release * Update 2018-06-26-kubernetes-1.11-release-announcement.md * Update 2018-06-26-kubernetes-1.11-release-announcement.md * Update 2018-06-26-kubernetes-1.11-release-announcement.md

k8s-ci-robot added the size/M Denotes a PR that changes 30-99 lines, ignoring generated files. label Jun 1, 2018

k8s-ci-robot requested review from tengqm and zhangxiaoyu-zidif June 1, 2018 23:27

k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Jun 1, 2018

gnufied mentioned this pull request Jun 1, 2018

Dynamic Maximum volume count kubernetes/enhancements#554

Closed

steveperry-53 added this to the 1.11 milestone Jun 2, 2018

gnufied mentioned this pull request Jun 3, 2018

Set GCE PD attachable volume limit based on machineType kubernetes/kubernetes#64293

Closed

tengqm reviewed Jun 4, 2018

View reviewed changes

k8s-ci-robot added the sig/storage Categorizes an issue or PR as relevant to SIG Storage. label Jun 4, 2018

mdlinville force-pushed the release-1.11 branch from 4b3430b to 7549d77 Compare June 4, 2018 17:24

gnufied and others added 3 commits June 4, 2018 11:34

add cos for storage limits

295de93

Update docs specific for aws and gce

64abb7e

fix some minor things

406b4b1

mdlinville force-pushed the fix-volume-limit-docs branch from 69164da to 406b4b1 Compare June 4, 2018 18:35

zacharysarah added the Docs Review: Open Issues label Jun 11, 2018

k8s-ci-robot assigned steveperry-53 Jun 12, 2018

Update storage-limits.md

19820f2

tengqm reviewed Jun 13, 2018

View reviewed changes

k8s-ci-robot added do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. lgtm "Looks good to me", indicates that a PR is ready to be merged. labels Jun 18, 2018

steveperry-53 removed the Docs Review: Open Issues label Jun 18, 2018

Add k8s version to feature-state shortcode

6913ea7

k8s-ci-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jun 19, 2018

k8s-ci-robot assigned mdlinville Jun 19, 2018

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jun 19, 2018

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jun 19, 2018

k8s-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jun 19, 2018

k8s-ci-robot merged commit 28262df into kubernetes:release-1.11 Jun 19, 2018

gnufied mentioned this pull request Jun 26, 2018

Make KUBE_MAX_PD_VOLS configurable for all environments kubernetes/kubernetes#65471

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Documentation for dynamic volume limits based on node type #8871

Documentation for dynamic volume limits based on node type #8871

gnufied commented Jun 1, 2018 •

edited

Loading

k8sio-netlify-preview-bot commented Jun 1, 2018 •

edited

Loading

tengqm Jun 4, 2018

tengqm Jun 4, 2018

tengqm Jun 4, 2018

tengqm Jun 4, 2018

tengqm Jun 4, 2018

tengqm Jun 4, 2018

gnufied commented Jun 4, 2018

mdlinville commented Jun 4, 2018

mdlinville commented Jun 12, 2018

steveperry-53 commented Jun 13, 2018

gnufied commented Jun 13, 2018

msau42 commented Jun 13, 2018

gnufied commented Jun 13, 2018

ravisantoshgudimetla commented Jun 13, 2018

tengqm Jun 13, 2018

gnufied Jun 15, 2018

tengqm Jun 13, 2018

steveperry-53 commented Jun 15, 2018

zparnold commented Jun 18, 2018

gnufied commented Jun 18, 2018

zparnold commented Jun 18, 2018

ravisantoshgudimetla commented Jun 18, 2018

steveperry-53 commented Jun 18, 2018 •

edited

Loading

mdlinville commented Jun 19, 2018

k8s-ci-robot commented Jun 19, 2018

steveperry-53 commented Jun 19, 2018

Documentation for dynamic volume limits based on node type #8871

Documentation for dynamic volume limits based on node type #8871

Conversation

gnufied commented Jun 1, 2018 • edited Loading

k8sio-netlify-preview-bot commented Jun 1, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gnufied commented Jun 4, 2018

mdlinville commented Jun 4, 2018

mdlinville commented Jun 12, 2018

steveperry-53 commented Jun 13, 2018

gnufied commented Jun 13, 2018

msau42 commented Jun 13, 2018

gnufied commented Jun 13, 2018

ravisantoshgudimetla commented Jun 13, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

steveperry-53 commented Jun 15, 2018

zparnold commented Jun 18, 2018

gnufied commented Jun 18, 2018

zparnold commented Jun 18, 2018

ravisantoshgudimetla commented Jun 18, 2018

steveperry-53 commented Jun 18, 2018 • edited Loading

mdlinville commented Jun 19, 2018

k8s-ci-robot commented Jun 19, 2018

steveperry-53 commented Jun 19, 2018

gnufied commented Jun 1, 2018 •

edited

Loading

k8sio-netlify-preview-bot commented Jun 1, 2018 •

edited

Loading

steveperry-53 commented Jun 18, 2018 •

edited

Loading