From 31794d3cf7d73c5a67999118b498a040989846e0 Mon Sep 17 00:00:00 2001 From: Connor Doyle Date: Mon, 11 Sep 2017 17:01:19 -0700 Subject: [PATCH 1/2] Document extended resources and OIR deprecation. --- .../manage-compute-resources-container.md | 110 ++++++++++++++++++ .../opaque-integer-resource-node.md | 2 +- .../opaque-integer-resource.md | 2 +- 3 files changed, 112 insertions(+), 2 deletions(-) diff --git a/docs/concepts/configuration/manage-compute-resources-container.md b/docs/concepts/configuration/manage-compute-resources-container.md index c5a5be4ffc294..6a35c524770cd 100644 --- a/docs/concepts/configuration/manage-compute-resources-container.md +++ b/docs/concepts/configuration/manage-compute-resources-container.md @@ -305,6 +305,15 @@ where `OOM` stands for Out Of Memory. ## Opaque integer resources (Alpha feature) +{% include feature-state-deprecated.md %} + +**Note:** Opaque Integer Resources will be removed in version 1.9. +[Extended Resources](#extended-resources) are a replacement for Opaque Integer +Resources. Users can use any domain name prefix outside of the `kubernetes.io/` +domain instead of the previous `pod.alpha.kubernetes.io/opaque-int-resource-` +prefix. +{: .note} + Kubernetes version 1.5 introduces Opaque integer resources. Opaque integer resources allow cluster operators to advertise new node-level resources that would be otherwise unknown to the system. @@ -316,6 +325,7 @@ available amount is simultaneously allocated to Pods. **Note:** Opaque integer resources are Alpha in Kubernetes version 1.5. Only resource accounting is implemented; node-level isolation is still under active development. +{: .note} Opaque integer resources are resources that begin with the prefix `pod.alpha.kubernetes.io/opaque-int-resource-`. The API server @@ -395,6 +405,106 @@ spec: pod.alpha.kubernetes.io/opaque-int-resource-foo: 1 ``` +## Extended Resources + +Kubernetes version 1.8 introduces Extended Resources. Extended Resources are +fully-qualified resource names outside the `kubernetes.io` domain. Extended +Resources allow cluster operators to advertise new node-level resources that +would be otherwise unknown to the system. Extended Resource quantities must be +integers and cannot be overcommitted. + +Users can consume Extended Resources in Pod specs just like CPU and memory. +The scheduler takes care of the resource accounting so that no more than the +available amount is simultaneously allocated to Pods. + +The API server restricts quantities of Extended Resources to whole numbers. +Examples of _valid_ quantities are `3`, `3000m` and `3Ki`. Examples of +_invalid_ quantities are `0.5` and `1500m`. + +**Note:** Extended Resources replace [Opaque Integer +Resources](#opaque-integer-resources-alpha-feature). Users can use any domain +name prefix outside of the `kubernetes.io/` domain instead of the previous +`pod.alpha.kubernetes.io/opaque-int-resource-` prefix. +{: .note} + +There are two steps required to use Extended Resources. First, the +cluster operator must advertise a per-node opaque resource on one or more +nodes. Second, users must request the Extended Resource in Pods. + +To advertise a new Extended Resource, the cluster operator should +submit a `PATCH` HTTP request to the API server to specify the available +quantity in the `status.capacity` for a node in the cluster. After this +operation, the node's `status.capacity` will include a new resource. The +`status.allocatable` field is updated automatically with the new resource +asynchronously by the kubelet. Note that because the scheduler uses the +node `status.allocatable` value when evaluating Pod fitness, there may +be a short delay between patching the node capacity with a new resource and the +first pod that requests the resource to be scheduled on that node. + +**Example:** + +Here is an HTTP request that advertises five "example.com/foo" resources on +node `k8s-node-1` whose master is `k8s-master`. + +```http +PATCH /api/v1/nodes/k8s-node-1/status HTTP/1.1 +Accept: application/json +Content-Type: application/json-patch+json +Host: k8s-master:8080 + +[ + { + "op": "add", + "path": "/status/capacity/example.com~1foo", + "value": "5" + } +] +``` + +```shell +curl --header "Content-Type: application/json-patch+json" \ +--request PATCH \ +--data '[{"op": "add", "path": "/status/capacity/example.com~1foo", "value": "5"}]' \ +http://k8s-master:8080/api/v1/nodes/k8s-node-1/status +``` + +**Note**: In the preceding request, `~1` is the encoding for the character `/` +in the patch path. The operation path value in JSON-Patch is interpreted as a +JSON-Pointer. For more details, see +[IETF RFC 6901, section 3](https://tools.ietf.org/html/rfc6901#section-3). +{: .note} + +To consume an Extended Resource in a Pod, include the resource name as a key +in the `spec.containers[].resources.requests` map. + +**Note:** Extended resources cannot be overcommitted, so request and limit +must be equal if both are present in a container spec. +{: .note} + +The Pod is scheduled only if all of the resource requests are +satisfied, including cpu, memory and any Extended Resources. The Pod will +remain in the `PENDING` state as long as the resource request cannot be met by +any node. + +**Example:** + +The Pod below requests 2 cpus and 1 "example.com/foo" (an extended resource.) + +```yaml +apiVersion: v1 +kind: Pod +metadata: + name: my-pod +spec: + containers: + - name: my-container + image: myimage + resources: + requests: + cpu: 2 + example.com/foo: 1 +``` + ## Planned Improvements Kubernetes version 1.5 only allows resource quantities to be specified on a diff --git a/docs/tasks/administer-cluster/opaque-integer-resource-node.md b/docs/tasks/administer-cluster/opaque-integer-resource-node.md index 71925c164247b..26f53f1242f69 100644 --- a/docs/tasks/administer-cluster/opaque-integer-resource-node.md +++ b/docs/tasks/administer-cluster/opaque-integer-resource-node.md @@ -9,7 +9,7 @@ This page shows how to specify opaque integer resources for a Node. Opaque integer resources allow cluster administrators to advertise node-level resources that would otherwise be unknown to Kubernetes. -{% include feature-state-alpha.md %} +{% include feature-state-deprecated.md %} {% endcapture %} diff --git a/docs/tasks/configure-pod-container/opaque-integer-resource.md b/docs/tasks/configure-pod-container/opaque-integer-resource.md index fff8f55703229..d7db46a250132 100644 --- a/docs/tasks/configure-pod-container/opaque-integer-resource.md +++ b/docs/tasks/configure-pod-container/opaque-integer-resource.md @@ -6,7 +6,7 @@ title: Assign Opaque Integer Resources to a Container This page shows how to assign opaque integer resources to a Container. -{% include feature-state-alpha.md %} +{% include feature-state-deprecated.md %} {% endcapture %} From d6ded8acd34394a750db138b72223d5b6b0cc663 Mon Sep 17 00:00:00 2001 From: Connor Doyle Date: Tue, 12 Sep 2017 08:11:12 -0700 Subject: [PATCH 2/2] Updated extended resources doc per reviews. --- .../manage-compute-resources-container.md | 56 ++++--------------- 1 file changed, 12 insertions(+), 44 deletions(-) diff --git a/docs/concepts/configuration/manage-compute-resources-container.md b/docs/concepts/configuration/manage-compute-resources-container.md index 6a35c524770cd..ca19f160b7eb5 100644 --- a/docs/concepts/configuration/manage-compute-resources-container.md +++ b/docs/concepts/configuration/manage-compute-resources-container.md @@ -307,13 +307,6 @@ where `OOM` stands for Out Of Memory. {% include feature-state-deprecated.md %} -**Note:** Opaque Integer Resources will be removed in version 1.9. -[Extended Resources](#extended-resources) are a replacement for Opaque Integer -Resources. Users can use any domain name prefix outside of the `kubernetes.io/` -domain instead of the previous `pod.alpha.kubernetes.io/opaque-int-resource-` -prefix. -{: .note} - Kubernetes version 1.5 introduces Opaque integer resources. Opaque integer resources allow cluster operators to advertise new node-level resources that would be otherwise unknown to the system. @@ -322,9 +315,11 @@ Users can consume these resources in Pod specs just like CPU and memory. The scheduler takes care of the resource accounting so that no more than the available amount is simultaneously allocated to Pods. -**Note:** Opaque integer resources are Alpha in Kubernetes version 1.5. -Only resource accounting is implemented; node-level isolation is still -under active development. +**Note:** Opaque Integer Resources will be removed in version 1.9. +[Extended Resources](#extended-resources) are a replacement for Opaque Integer +Resources. Users can use any domain name prefix outside of the `kubernetes.io/` +domain instead of the previous `pod.alpha.kubernetes.io/opaque-int-resource-` +prefix. {: .note} Opaque integer resources are resources that begin with the prefix @@ -349,22 +344,9 @@ first pod that requests the resource to be scheduled on that node. **Example:** -Here is an HTTP request that advertises five "foo" resources on node `k8s-node-1` whose master is `k8s-master`. - -```http -PATCH /api/v1/nodes/k8s-node-1/status HTTP/1.1 -Accept: application/json -Content-Type: application/json-patch+json -Host: k8s-master:8080 - -[ - { - "op": "add", - "path": "/status/capacity/pod.alpha.kubernetes.io~1opaque-int-resource-foo", - "value": "5" - } -] -``` +Here is an example showing how to use `curl` to form an HTTP request that +advertises five "foo" resources on node `k8s-node-1` whose master is +`k8s-master`. ```shell curl --header "Content-Type: application/json-patch+json" \ @@ -428,7 +410,7 @@ name prefix outside of the `kubernetes.io/` domain instead of the previous {: .note} There are two steps required to use Extended Resources. First, the -cluster operator must advertise a per-node opaque resource on one or more +cluster operator must advertise a per-node Extended Resource on one or more nodes. Second, users must request the Extended Resource in Pods. To advertise a new Extended Resource, the cluster operator should @@ -443,23 +425,9 @@ first pod that requests the resource to be scheduled on that node. **Example:** -Here is an HTTP request that advertises five "example.com/foo" resources on -node `k8s-node-1` whose master is `k8s-master`. - -```http -PATCH /api/v1/nodes/k8s-node-1/status HTTP/1.1 -Accept: application/json -Content-Type: application/json-patch+json -Host: k8s-master:8080 - -[ - { - "op": "add", - "path": "/status/capacity/example.com~1foo", - "value": "5" - } -] -``` +Here is an example showing how to use `curl` to form an HTTP request that +advertises five "example.com/foo" resources on node `k8s-node-1` whose master +is `k8s-master`. ```shell curl --header "Content-Type: application/json-patch+json" \