From f609527f1539f860e76136f2a4dc61671f2f3a75 Mon Sep 17 00:00:00 2001 From: Jing Xu Date: Mon, 18 Sep 2017 11:20:55 -0700 Subject: [PATCH 1/7] Add local ephemeral storage alpha feature in managing compute resource Since 1.8, we add the local ephemeral storage alpha feature as one resource type to manage. Add this feature into the doc. --- .../configuration/manage-compute-resources-container.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/docs/concepts/configuration/manage-compute-resources-container.md b/docs/concepts/configuration/manage-compute-resources-container.md index c5a5be4ffc294..f632e2003bbe5 100644 --- a/docs/concepts/configuration/manage-compute-resources-container.md +++ b/docs/concepts/configuration/manage-compute-resources-container.md @@ -303,6 +303,11 @@ LastState: map[terminated:map[exitCode:137 reason:OOM Killed startedAt:2015-07-0 You can see that the Container was terminated because of `reason:OOM Killed`, where `OOM` stands for Out Of Memory. +## Local ephemeral storage (Alpha feature) + +Kubernetes version 1.8 introduces local ephemeral storage resources. The +management of local ephemeral storage is very similar to memory resource. + ## Opaque integer resources (Alpha feature) Kubernetes version 1.5 introduces Opaque integer resources. Opaque From 863a91131bf9ffb0023343d4ae19e020a4de077f Mon Sep 17 00:00:00 2001 From: Jing Xu Date: Mon, 18 Sep 2017 11:33:40 -0700 Subject: [PATCH 2/7] Update manage-compute-resources-container.md --- .../manage-compute-resources-container.md | 43 ++++++++++++++++++- 1 file changed, 42 insertions(+), 1 deletion(-) diff --git a/docs/concepts/configuration/manage-compute-resources-container.md b/docs/concepts/configuration/manage-compute-resources-container.md index f632e2003bbe5..356ea498f95a6 100644 --- a/docs/concepts/configuration/manage-compute-resources-container.md +++ b/docs/concepts/configuration/manage-compute-resources-container.md @@ -305,9 +305,50 @@ where `OOM` stands for Out Of Memory. ## Local ephemeral storage (Alpha feature) -Kubernetes version 1.8 introduces local ephemeral storage resources. The +Kubernetes version 1.8 introduces local ephemeral storage resource. The management of local ephemeral storage is very similar to memory resource. +Each Container of a Pod can specify one or more of the following: + +* `spec.containers[].resources.limits.ephemeral-storage` +* `spec.containers[].resources.requests.ephemeral-storage` + +Limits and requests for `ephemeral-storage` are measured in bytes. You can express storage as +a plain integer or as a fixed-point integer using one of these suffixes: +E, P, T, G, M, K. You can also use the power-of-two equivalents: Ei, Pi, Ti, Gi, +Mi, Ki. For example, the following represent roughly the same value: + +```shell +128974848, 129e6, 129M, 123Mi +``` + +Here's an example. +The following Pod has two Containers. Each one has a request of 2GiB of local ephemeral storage. Each Container has a limit of 4GiB of local ephemeral storage. You can say the Pod has a request of 4GiB of local ephemeral storage, and a limit of 8GiB of storage. + +```yaml +apiVersion: v1 +kind: Pod +metadata: + name: frontend +spec: + containers: + - name: db + image: mysql + resources: + requests: + ephemeral-storage: "2Gi" + limits: + ephemeral-storage: "4Gi" + - name: wp + image: wordpress + resources: + requests: + ephemeral-storage: "2Gi" + limits: + ephemeral-storage: "4Gi" +``` + + ## Opaque integer resources (Alpha feature) Kubernetes version 1.5 introduces Opaque integer resources. Opaque From 44f98157bb4d1d98da9aff6668e2f0a335adc46c Mon Sep 17 00:00:00 2001 From: Zach Corleissen Date: Tue, 19 Sep 2017 13:50:00 -0500 Subject: [PATCH 3/7] Update manage-compute-resources-container.md --- .../manage-compute-resources-container.md | 14 ++++++++------ 1 file changed, 8 insertions(+), 6 deletions(-) diff --git a/docs/concepts/configuration/manage-compute-resources-container.md b/docs/concepts/configuration/manage-compute-resources-container.md index 356ea498f95a6..5b4522d0b3cfe 100644 --- a/docs/concepts/configuration/manage-compute-resources-container.md +++ b/docs/concepts/configuration/manage-compute-resources-container.md @@ -303,10 +303,13 @@ LastState: map[terminated:map[exitCode:137 reason:OOM Killed startedAt:2015-07-0 You can see that the Container was terminated because of `reason:OOM Killed`, where `OOM` stands for Out Of Memory. -## Local ephemeral storage (Alpha feature) +## Local ephemeral storage (alpha feature) -Kubernetes version 1.8 introduces local ephemeral storage resource. The -management of local ephemeral storage is very similar to memory resource. +Kubernetes version 1.8 introduces a resource for local ephemeral storage. The +management of local ephemeral storage is very similar to memory resources. + +**Note:** Local ephemeral storage is an alpha feature in version 1.8. +{: .note} Each Container of a Pod can specify one or more of the following: @@ -322,8 +325,7 @@ Mi, Ki. For example, the following represent roughly the same value: 128974848, 129e6, 129M, 123Mi ``` -Here's an example. -The following Pod has two Containers. Each one has a request of 2GiB of local ephemeral storage. Each Container has a limit of 4GiB of local ephemeral storage. You can say the Pod has a request of 4GiB of local ephemeral storage, and a limit of 8GiB of storage. +For example, the following Pod has two Containers. Each Container has a request of 2GiB of local ephemeral storage. Each Container has a limit of 4GiB of local ephemeral storage. Therefore, the Pod has a request of 4GiB of local ephemeral storage, and a limit of 8GiB of storage. ```yaml apiVersion: v1 @@ -349,7 +351,7 @@ spec: ``` -## Opaque integer resources (Alpha feature) +## Opaque integer resources (alpha feature) Kubernetes version 1.5 introduces Opaque integer resources. Opaque integer resources allow cluster operators to advertise new node-level From b222a3f6bc371d24c98a7163071eb2ad7c8840cb Mon Sep 17 00:00:00 2001 From: Jing Xu Date: Wed, 20 Sep 2017 16:43:23 -0700 Subject: [PATCH 4/7] Update manage-compute-resources-container.md --- .../manage-compute-resources-container.md | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-) diff --git a/docs/concepts/configuration/manage-compute-resources-container.md b/docs/concepts/configuration/manage-compute-resources-container.md index 5b4522d0b3cfe..f8354dddc4d18 100644 --- a/docs/concepts/configuration/manage-compute-resources-container.md +++ b/docs/concepts/configuration/manage-compute-resources-container.md @@ -305,12 +305,9 @@ where `OOM` stands for Out Of Memory. ## Local ephemeral storage (alpha feature) -Kubernetes version 1.8 introduces a resource for local ephemeral storage. The -management of local ephemeral storage is very similar to memory resources. - -**Note:** Local ephemeral storage is an alpha feature in version 1.8. -{: .note} +Kubernetes version 1.8 introduces a new resource "ephemeral-storage" for managing local ephemeral storage. In each kubernetes's node, kubelet's root directory (/var/lib/kubelet by default) and log directory (/var/log) are stored on the root partition of the node. This parition is also shared and consumed by pods via EmptyDir volumes, container logs, image layers and container writable layers. (Notes: If an optional runntime partition is used, root parition will not hold any image layer or writable layers.) This partition is “ephemeral” and applications cannot expect any performance SLAs (Disk IOPS for example) from this partition. Local ephemeral storage management only applies for the root partition, the optional partition for image alyer and writable layer is out of scope. +### Requests and limits setting for local ephemeral storage Each Container of a Pod can specify one or more of the following: * `spec.containers[].resources.limits.ephemeral-storage` @@ -350,6 +347,15 @@ spec: ephemeral-storage: "4Gi" ``` +### How Pods with ephemeral-storage requests are scheduled + +When you create a Pod, the Kubernetes scheduler selects a node for the Pod to +run on. Each node has a maximum amount of local ephemeral storage it can provide for Pods (Please refer to the documentation of [Node Allocata](/docs/tasks/administer-cluster/reserve-compute-resources/). The scheduler ensures that, the sum of the resource requests of the scheduled Containers is less than the capacity of the node. + +### How Pods with ephemeral-storage limits are run + +For container-level isolation, if a Container's writable layer and logs usage exceeds its storage limit, the pod will be evicted. For pod-level isolation, if the sum of the local ephemeral storage usage from all containers and also the pod's EmptyDir volumes exceeds the limit, the pod will be evicted. + ## Opaque integer resources (alpha feature) From ffd553b62fabe4301001aa113c13524a0570ae47 Mon Sep 17 00:00:00 2001 From: Jing Xu Date: Wed, 20 Sep 2017 16:45:18 -0700 Subject: [PATCH 5/7] Update manage-compute-resources-container.md --- .../configuration/manage-compute-resources-container.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/concepts/configuration/manage-compute-resources-container.md b/docs/concepts/configuration/manage-compute-resources-container.md index f8354dddc4d18..2dc457d4fe87d 100644 --- a/docs/concepts/configuration/manage-compute-resources-container.md +++ b/docs/concepts/configuration/manage-compute-resources-container.md @@ -305,7 +305,7 @@ where `OOM` stands for Out Of Memory. ## Local ephemeral storage (alpha feature) -Kubernetes version 1.8 introduces a new resource "ephemeral-storage" for managing local ephemeral storage. In each kubernetes's node, kubelet's root directory (/var/lib/kubelet by default) and log directory (/var/log) are stored on the root partition of the node. This parition is also shared and consumed by pods via EmptyDir volumes, container logs, image layers and container writable layers. (Notes: If an optional runntime partition is used, root parition will not hold any image layer or writable layers.) This partition is “ephemeral” and applications cannot expect any performance SLAs (Disk IOPS for example) from this partition. Local ephemeral storage management only applies for the root partition, the optional partition for image alyer and writable layer is out of scope. +Kubernetes version 1.8 introduces a new resource "ephemeral-storage" for managing local ephemeral storage. In each kubernetes's node, kubelet's root directory (/var/lib/kubelet by default) and log directory (/var/log) are stored on the root partition of the node. This parition is also shared and consumed by pods via EmptyDir volumes, container logs, image layers and container writable layers. (Notes: If an optional runntime partition is used, root parition will not hold any image layer or writable layers.) This partition is “ephemeral” and applications cannot expect any performance SLAs (Disk IOPS for example) from this partition. Local ephemeral storage management only applies for the root partition, the optional partition for image layer and writable layer is out of scope. ### Requests and limits setting for local ephemeral storage Each Container of a Pod can specify one or more of the following: From ad4779ed712f9fd04c4ca3200c88b6418cddfa31 Mon Sep 17 00:00:00 2001 From: Jing Xu Date: Wed, 20 Sep 2017 16:47:47 -0700 Subject: [PATCH 6/7] Update manage-compute-resources-container.md --- .../configuration/manage-compute-resources-container.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/concepts/configuration/manage-compute-resources-container.md b/docs/concepts/configuration/manage-compute-resources-container.md index 2dc457d4fe87d..65676a5f6588e 100644 --- a/docs/concepts/configuration/manage-compute-resources-container.md +++ b/docs/concepts/configuration/manage-compute-resources-container.md @@ -350,7 +350,7 @@ spec: ### How Pods with ephemeral-storage requests are scheduled When you create a Pod, the Kubernetes scheduler selects a node for the Pod to -run on. Each node has a maximum amount of local ephemeral storage it can provide for Pods (Please refer to the documentation of [Node Allocata](/docs/tasks/administer-cluster/reserve-compute-resources/). The scheduler ensures that, the sum of the resource requests of the scheduled Containers is less than the capacity of the node. +run on. Each node has a maximum amount of local ephemeral storage it can provide for Pods (Please refer to the documentation of [Node Allocatable](/docs/tasks/administer-cluster/reserve-compute-resources/#node-allocatable). The scheduler ensures that, the sum of the resource requests of the scheduled Containers is less than the capacity of the node. ### How Pods with ephemeral-storage limits are run From 343aac01024b4ad13a70bc62c7d0769a55ae6f9c Mon Sep 17 00:00:00 2001 From: Zach Corleissen Date: Thu, 21 Sep 2017 17:06:42 -0500 Subject: [PATCH 7/7] Update manage-compute-resources-container.md --- .../manage-compute-resources-container.md | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/docs/concepts/configuration/manage-compute-resources-container.md b/docs/concepts/configuration/manage-compute-resources-container.md index 65676a5f6588e..6f4337157aec9 100644 --- a/docs/concepts/configuration/manage-compute-resources-container.md +++ b/docs/concepts/configuration/manage-compute-resources-container.md @@ -305,7 +305,12 @@ where `OOM` stands for Out Of Memory. ## Local ephemeral storage (alpha feature) -Kubernetes version 1.8 introduces a new resource "ephemeral-storage" for managing local ephemeral storage. In each kubernetes's node, kubelet's root directory (/var/lib/kubelet by default) and log directory (/var/log) are stored on the root partition of the node. This parition is also shared and consumed by pods via EmptyDir volumes, container logs, image layers and container writable layers. (Notes: If an optional runntime partition is used, root parition will not hold any image layer or writable layers.) This partition is “ephemeral” and applications cannot expect any performance SLAs (Disk IOPS for example) from this partition. Local ephemeral storage management only applies for the root partition, the optional partition for image layer and writable layer is out of scope. +Kubernetes version 1.8 introduces a new resource, _ephemeral-storage_ for managing local ephemeral storage. In each Kubernetes node, kubelet's root directory (/var/lib/kubelet by default) and log directory (/var/log) are stored on the root partition of the node. This partition is also shared and consumed by pods via EmptyDir volumes, container logs, image layers and container writable layers. + +This partition is “ephemeral” and applications cannot expect any performance SLAs (Disk IOPS for example) from this partition. Local ephemeral storage management only applies for the root partition; the optional partition for image layer and writable layer is out of scope. + +**Note:** If an optional runntime partition is used, root parition will not hold any image layer or writable layers. +{: .note} ### Requests and limits setting for local ephemeral storage Each Container of a Pod can specify one or more of the following: @@ -350,13 +355,12 @@ spec: ### How Pods with ephemeral-storage requests are scheduled When you create a Pod, the Kubernetes scheduler selects a node for the Pod to -run on. Each node has a maximum amount of local ephemeral storage it can provide for Pods (Please refer to the documentation of [Node Allocatable](/docs/tasks/administer-cluster/reserve-compute-resources/#node-allocatable). The scheduler ensures that, the sum of the resource requests of the scheduled Containers is less than the capacity of the node. +run on. Each node has a maximum amount of local ephemeral storage it can provide for Pods. (For more information, see ["Node Allocatable"](/docs/tasks/administer-cluster/reserve-compute-resources/#node-allocatable) The scheduler ensures that the sum of the resource requests of the scheduled Containers is less than the capacity of the node. -### How Pods with ephemeral-storage limits are run +### How Pods with ephemeral-storage limits run For container-level isolation, if a Container's writable layer and logs usage exceeds its storage limit, the pod will be evicted. For pod-level isolation, if the sum of the local ephemeral storage usage from all containers and also the pod's EmptyDir volumes exceeds the limit, the pod will be evicted. - ## Opaque integer resources (alpha feature) Kubernetes version 1.5 introduces Opaque integer resources. Opaque