-
Notifications
You must be signed in to change notification settings - Fork 1.9k
OSDOCS-14540: Content journey #1: Kueue administrator getting started #93005
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| ../_attributes/ |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,47 @@ | ||
| :_mod-docs-content-type: ASSEMBLY | ||
| include::_attributes/common-attributes.adoc[] | ||
| [id="configuring-quotas"] | ||
| = Configuring quotas | ||
| :context: configuring-quotas | ||
|
|
||
| toc::[] | ||
|
|
||
| As an administrator, you can use {product-title} to configure quotas to optimize resource allocation and system throughput for user workloads. | ||
| You can configure quotas for compute resources such as CPU, memory, pods, and GPU. | ||
|
|
||
| You can configure quotas in {product-title} by completing the following steps: | ||
|
|
||
| . Configure a cluster queue. | ||
| . Configure a resource flavor. | ||
| . Configure a local queue. | ||
|
|
||
| Users can then submit their workloads to the local queue. | ||
|
|
||
| include::modules/configuring-clusterqueues.adoc[leveloffset=+1] | ||
|
|
||
| [role="_next-steps"] | ||
| [id="clusterqueues-next-steps_{context}"] | ||
| .Next steps | ||
|
|
||
| The cluster queue is not ready for use until a xref:../configure/configuring-quotas.adoc#configuring-resourceflavors_configuring-quotas[`ResourceFlavor` object] has also been configured. | ||
|
|
||
| include::modules/configuring-resourceflavors.adoc[leveloffset=+1] | ||
|
|
||
| include::modules/configuring-localqueues.adoc[leveloffset=+1] | ||
|
|
||
| [role="_additional-resources"] | ||
| [id="clusterqueues-additional-resources_{context}"] | ||
| == Additional resources | ||
| * xref:../authentication/rbac-permissions.adoc#rbac-permissions[RBAC permissions] | ||
| * link:https://kueue.sigs.k8s.io/docs/concepts/cluster_queue/[Kubernetes documentation about cluster queues] | ||
|
|
||
| //// | ||
| // Save this content for user content journey abstract | ||
| // Users create jobs to be admitted to {product-title} by using the standard link:https://kubernetes.io/docs/concepts/workloads/controllers/job/[Kubernetes `Job` API]. | ||
| // Users can create any supported `Job` object, and then add the `kueue.x-k8s.io/queue-name` label to that object to connect it to a `LocalQueue` resource. | ||
|
|
||
| [role="_additional-resources"] | ||
| [id="additional-resources_{context}"] | ||
| == Additional resources | ||
| * link:https://docs.redhat.com/en/documentation/openshift_container_platform/latest/html/nodes/using-jobs-and-daemon-sets#nodes-nodes-jobs[Running tasks in pods using jobs] | ||
| //// | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| ../images/ |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| ../modules/ |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| ../snippets/ |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,63 @@ | ||
| // Module included in the following assemblies: | ||
| // | ||
| // * /configure/configuring-quotas.adoc | ||
|
|
||
| :_mod-docs-content-type: PROCEDURE | ||
| [id="configuring-clusterqueues_{context}"] | ||
| = Configuring a cluster queue | ||
|
|
||
| A cluster queue is a cluster-scoped resource, represented by a `ClusterQueue` object, that governs a pool of resources such as CPU, memory, and pods. | ||
abrennan89 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| Cluster queues can be used to define usage limits, quotas for resource flavors, order of consumption, and fair sharing rules. | ||
|
|
||
| [NOTE] | ||
| ==== | ||
| The cluster queue is not ready for use until a `ResourceFlavor` object has also been configured. | ||
| ==== | ||
|
|
||
| .Prerequisites | ||
|
|
||
| include::snippets/prereqs-snippet.adoc[] | ||
|
|
||
| .Procedure | ||
|
|
||
| . Create a `ClusterQueue` object as a YAML file: | ||
| + | ||
| .Example of a basic `ClusterQueue` object using a single resource flavor | ||
| [source,yaml] | ||
| ---- | ||
| apiVersion: kueue.x-k8s.io/v1beta1 | ||
| kind: ClusterQueue | ||
| metadata: | ||
| name: example-queue | ||
| spec: | ||
| namespaceSelector: {} <1> | ||
| resourceGroups: | ||
| - coveredResources: ["cpu", "memory", "pods", "foo.com/gpu"] <2> | ||
| flavors: | ||
| - name: "default-flavor" <3> | ||
| resources: <4> | ||
| - name: "cpu" | ||
| nominalQuota: 9 | ||
| - name: "memory" | ||
| nominalQuota: 36Gi | ||
| - name: "pods" | ||
| nominalQuota: 5 | ||
| - name: "foo.com/gpu" | ||
| nominalQuota: 100 | ||
| ---- | ||
| <1> Defines which namespaces can use the resources governed by this cluster queue. An empty `namespaceSelector` as shown in the example means that all namespaces can use these resources. | ||
| <2> Defines the resource types governed by the cluster queue. This example `ClusterQueue` object governs CPU, memory, pod, and GPU resources. | ||
| <3> Defines the resource flavor that is applied to the resource types listed. In this example, the `default-flavor` resource flavor is applied to CPU, memory, pod, and GPU resources. | ||
| <4> Defines the resource requirements for admitting jobs. This example cluster queue only admits jobs if the following conditions are met: | ||
| + | ||
| * The sum of the CPU requests is less than or equal to 9. | ||
| * The sum of the memory requests is less than or equal to 36Gi. | ||
| * The total number of pods is less than or equal to 5. | ||
| * The sum of the GPU requests is less than or equal to 100. | ||
|
|
||
| . Apply the `ClusterQueue` object by running the following command: | ||
| + | ||
| [source,terminal] | ||
| ---- | ||
| $ oc apply -f <filename>.yaml | ||
| ---- | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,40 @@ | ||
| // Module included in the following assemblies: | ||
| // | ||
| // * /configure/configuring-quotas.adoc | ||
|
|
||
| :_mod-docs-content-type: PROCEDURE | ||
| [id="configuring-localqueues_{context}"] | ||
| = Configuring a local queue | ||
|
|
||
| A local queue is a namespaced object, represented by a `LocalQueue` object, that groups closely related workloads that belong to a single namespace. | ||
|
|
||
| As an administrator, you can configure a `LocalQueue` object to point to a cluster queue. This allocates resources from the cluster queue to workloads in the namespace specified in the `LocalQueue` object. | ||
|
|
||
| .Prerequisites | ||
|
|
||
| include::snippets/prereqs-snippet.adoc[] | ||
|
|
||
| * You have created a `ClusterQueue` object. | ||
|
|
||
| .Procedure | ||
|
|
||
| . Create a `LocalQueue` object as a YAML file: | ||
| + | ||
| .Example of a basic `LocalQueue` object | ||
| [source,yaml] | ||
| ---- | ||
| apiVersion: kueue.x-k8s.io/v1beta1 | ||
| kind: LocalQueue | ||
| metadata: | ||
| namespace: team-a | ||
| name: team-a-queue | ||
| spec: | ||
| clusterQueue: cluster-queue | ||
| ---- | ||
|
|
||
| . Apply the `LocalQueue` object by running the following command: | ||
| + | ||
| [source,terminal] | ||
| ---- | ||
| $ oc apply -f <filename>.yaml | ||
| ---- |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,49 @@ | ||
| // Module included in the following assemblies: | ||
| // | ||
| // * /configure/configuring-quotas.adoc | ||
|
|
||
| :_mod-docs-content-type: PROCEDURE | ||
| [id="configuring-resourceflavors_{context}"] | ||
| = Configuring a resource flavor | ||
|
|
||
| After you have configured a `ClusterQueue` object, you can configure a `ResourceFlavor` object. | ||
|
|
||
| Resources in a cluster are typically not homogeneous. If the resources in your cluster are homogeneous, you can use an empty `ResourceFlavor` instead of adding labels to custom resource flavors. | ||
|
|
||
| You can use a custom `ResourceFlavor` object to represent different resource variations that are associated with cluster nodes through labels, taints, and tolerations. You can then associate workloads with specific node types to enable fine-grained resource management. | ||
|
|
||
| .Prerequisites | ||
|
|
||
| include::snippets/prereqs-snippet.adoc[] | ||
|
|
||
| .Procedure | ||
|
|
||
| . Create a `ResourceFlavor` object as a YAML file: | ||
| + | ||
| .Example of an empty `ResourceFlavor` object | ||
| [source,yaml] | ||
| ---- | ||
| apiVersion: kueue.x-k8s.io/v1beta1 | ||
| kind: ResourceFlavor | ||
| metadata: | ||
| name: default-flavor | ||
| ---- | ||
| + | ||
| .Example of a custom `ResourceFlavor` object | ||
| [source,yaml] | ||
| ---- | ||
| apiVersion: kueue.x-k8s.io/v1beta1 | ||
| kind: ResourceFlavor | ||
| metadata: | ||
| name: "x86" | ||
| spec: | ||
| nodeLabels: | ||
| cpu-arch: x86 | ||
| ---- | ||
|
|
||
| . Apply the `ResourceFlavor` object by running the following command: | ||
| + | ||
| [source,terminal] | ||
| ---- | ||
| $ oc apply -f <filename>.yaml | ||
| ---- |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,15 @@ | ||
| // Text snippet included in the following modules: | ||
| // | ||
| // * | ||
| // | ||
| // Text snippet included in the following assemblies: | ||
| // | ||
| // * | ||
|
|
||
| :_mod-docs-content-type: SNIPPET | ||
|
|
||
| Ensure that you have completed the following prerequisites: | ||
|
|
||
| * The {product-title} Operator is installed on your cluster. | ||
| * You have cluster administrator permissions and the `kueue-batch-admin-role` role. | ||
| * You have installed the {oc-first}. |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.