-
Notifications
You must be signed in to change notification settings - Fork 2.1k
MIXEDARCH-37: Arm worker nodes for heterogeneous workflow #31965
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,6 @@ | ||
| approvers: | ||
| - aleskandro | ||
| - deepsm007 | ||
| - jaypoulz | ||
| - jeffdyoung | ||
| - Prashanth684 |
63 changes: 63 additions & 0 deletions
63
ci-operator/step-registry/ipi/install/heterogeneous/ipi-install-heterogeneous-commands.sh
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,63 @@ | ||
| #!/bin/bash | ||
|
|
||
| set -o nounset | ||
| set -o errexit | ||
| set -o pipefail | ||
|
|
||
| export KUBECONFIG=${SHARED_DIR}/kubeconfig | ||
|
|
||
| trap 'CHILDREN=$(jobs -p); if test -n "${CHILDREN}"; then kill ${CHILDREN} && wait; fi' TERM | ||
|
|
||
| # Make sure yq is installed | ||
| if [ ! -f /tmp/yq-v4 ]; then | ||
| # TODO move to image | ||
| curl -L "https://github.com/mikefarah/yq/releases/download/v4.25.3/yq_linux_$(uname -m | sed s/aarch64/arm64/ | sed s/x86_64/amd64/)" -o /tmp/yq-v4 && chmod +x /tmp/yq-v4 | ||
| PATH=${PATH}:/tmp | ||
| fi | ||
|
|
||
| # Make sure jq is installed | ||
| if ! command -v jq; then | ||
| # TODO move to image | ||
| curl -sL https://github.com/stedolan/jq/releases/download/jq-1.6/jq-linux64 > /tmp/jq | ||
| chmod +x /tmp/jq | ||
| PATH=${PATH}:/tmp | ||
| fi | ||
|
|
||
| dir=/tmp/installer | ||
| mkdir "${dir}/" | ||
|
|
||
| REGION=${LEASED_RESOURCE} | ||
|
|
||
| echo "Fetching Worker MachineSet..." | ||
| oc -n openshift-machine-api get -o json machinesets | jq '[.items[] | select(.spec.template.metadata.labels["machine.openshift.io/cluster-api-machine-role"] == "worker")][0]' > ${dir}/99_openshift-cluster-api_worker-machineset-0.yaml | ||
|
|
||
| # AMI for AWS ARM | ||
| echo "Extracting AMI..." | ||
| oc -n openshift-machine-config-operator get configmap/coreos-bootimages -oyaml > ${dir}/coreos-bootimages.yaml | ||
| yq-v4 eval ".data.stream" ${dir}/coreos-bootimages.yaml > ${dir}/machineset.yaml | ||
| amiid_workers_additional=$(yq-v4 ".architectures.${ADDITIONAL_WORKER_ARCHITECTURE}.images.aws.regions.\"${REGION}\".image" ${dir}/machineset.yaml) | ||
| echo "Updating the machineset with ${ADDITIONAL_WORKER_VM_TYPE} and ami ${amiid_workers_additional} ..." | ||
| yq-v4 eval ".metadata.name += \"-additional\" | ||
| | .spec.replicas = ${ADDITIONAL_WORKERS} | ||
| | .spec.selector.matchLabels.\"machine.openshift.io/cluster-api-machineset\" = .metadata.name | ||
| | .spec.template.metadata.labels.\"machine.openshift.io/cluster-api-machineset\" = .metadata.name" \ | ||
| ${dir}/99_openshift-cluster-api_worker-machineset-0.yaml > ${dir}/99_openshift-cluster-api_worker-machineset-additional.yaml | ||
|
|
||
| yq-v4 eval ".spec.template.spec.providerSpec.value.ami.id = \"${amiid_workers_additional}\" | ||
| | .spec.template.spec.providerSpec.value.instanceType = \"${ADDITIONAL_WORKER_VM_TYPE}\" | ||
| " -i ${dir}/99_openshift-cluster-api_worker-machineset-additional.yaml | ||
|
|
||
| echo "Creating ${ADDITIONAL_WORKER_ARCHITECTURE} worker MachineSet" | ||
| exec oc create -f ${dir}/99_openshift-cluster-api_worker-machineset-additional.yaml & | ||
|
|
||
| wait "$!" | ||
| ret="$?" | ||
|
|
||
| if test "${ret}" -eq 0 ; then | ||
| touch "${SHARED_DIR}/success" | ||
| # Save console URL in `console.url` file so that ci-chat-bot could report success | ||
| echo "https://$(env KUBECONFIG=${dir}/auth/kubeconfig oc -n openshift-console get routes console -o=jsonpath='{.spec.host}')" > "${SHARED_DIR}/console.url" | ||
| fi | ||
|
|
||
| echo "Exiting with ret=${ret}" | ||
| exit "${ret}" | ||
12 changes: 12 additions & 0 deletions
12
...rator/step-registry/ipi/install/heterogeneous/ipi-install-heterogeneous-ref.metadata.json
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,12 @@ | ||
| { | ||
| "path": "ipi/install/heterogeneous/ipi-install-heterogeneous-ref.yaml", | ||
| "owners": { | ||
| "approvers": [ | ||
| "aleskandro", | ||
| "deepsm007", | ||
| "jaypoulz", | ||
| "jeffdyoung", | ||
| "Prashanth684" | ||
| ] | ||
| } | ||
| } |
21 changes: 21 additions & 0 deletions
21
ci-operator/step-registry/ipi/install/heterogeneous/ipi-install-heterogeneous-ref.yaml
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,21 @@ | ||
| ref: | ||
| as: ipi-install-heterogeneous | ||
| from: tests | ||
| grace_period: 10m | ||
| commands: ipi-install-heterogeneous-commands.sh | ||
| resources: | ||
| requests: | ||
| cpu: 10m | ||
| memory: 100Mi | ||
| env: | ||
| - name: ADDITIONAL_WORKERS | ||
| default: "2" | ||
| documentation: "Worker count for heterogeneous deploy" | ||
| - name: ADDITIONAL_WORKER_ARCHITECTURE | ||
| default: "aarch64" | ||
| documentation: "Heterogeneous worker architecture, default set to arm64(aarch64)" | ||
| - name: ADDITIONAL_WORKER_VM_TYPE | ||
| default: "m6g.xlarge" | ||
| documentation: "VM Instance type for Heterogeneous worker, default set to arm64 instance type" | ||
| documentation: |- | ||
| The heterogeneous custom step adds additional worker node post configuration for adding custom arch worker node. |
6 changes: 6 additions & 0 deletions
6
ci-operator/step-registry/openshift/e2e/aws/heterogeneous/OWNERS
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,6 @@ | ||
| approvers: | ||
| - aleskandro | ||
| - deepsm007 | ||
| - jaypoulz | ||
| - jeffdyoung | ||
| - Prashanth684 |
12 changes: 12 additions & 0 deletions
12
...ry/openshift/e2e/aws/heterogeneous/openshift-e2e-aws-heterogeneous-workflow.metadata.json
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,12 @@ | ||
| { | ||
| "path": "openshift/e2e/aws/heterogeneous/openshift-e2e-aws-heterogeneous-workflow.yaml", | ||
| "owners": { | ||
| "approvers": [ | ||
| "aleskandro", | ||
| "deepsm007", | ||
| "jaypoulz", | ||
| "jeffdyoung", | ||
| "Prashanth684" | ||
| ] | ||
| } | ||
| } |
39 changes: 39 additions & 0 deletions
39
...ep-registry/openshift/e2e/aws/heterogeneous/openshift-e2e-aws-heterogeneous-workflow.yaml
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,39 @@ | ||
| workflow: | ||
| as: openshift-e2e-aws-heterogeneous | ||
| steps: | ||
| allow_best_effort_post_steps: true | ||
| pre: | ||
| - chain: ipi-aws-pre | ||
| - ref: ipi-install-heterogeneous | ||
| test: | ||
| - ref: openshift-e2e-test | ||
| post: | ||
| - chain: gather-core-dump | ||
| - chain: ipi-aws-post | ||
| env: | ||
| TEST_SKIPS: >- | ||
| The HAProxy router should\| deploymentconfigs\| Users should be able to create and run a job in a user project\| | ||
| Managed cluster should should expose cluster services outside the cluster\| FIPS TestFIPS\| | ||
| TestPodDefaultCapabilities\| Multi-stage image builds should succeed\| Optimized image builds should succeed\| | ||
| build can reference a cluster service with a build being created from new-build should be able to run a build that references a cluster service\| | ||
| custom build with buildah being created from new-build should complete build with custom builder image\| | ||
| oc new-app should succeed with a --name of 58 characters\| oc can run inside of a busybox container\| | ||
| oc debug deployment configs from a build\| oc rsh specific flags should work well when access to a remote shell\| | ||
| prune builds based on settings in the buildconfig should prune completed builds based on the successfulBuildsHistoryLimit setting\| | ||
| s2i build with a quota Building from a template should create an s2i build with a quota and run it\| | ||
| s2i build with a root user image should create a root build and pass with a privileged SCC\| | ||
| verify /run filesystem contents are writeable using a simple Docker Strategy Build\| | ||
| verify /run filesystem contents do not have unexpected content using a simple Docker Strategy Build\| | ||
| Pods cannot access the /config/master API endpoint\| Image append should create images by appending them\| | ||
| Image extract should extract content from an image\| Image info should display information about images\| | ||
| Image layer subresource should return layers from tagged images\| oc tag should change image reference for internal images\| | ||
| Prometheus when installed on the cluster should provide ingress metrics\| | ||
| Prometheus when installed on the cluster should start and expose a secured proxy and unsecured metrics\| | ||
| OpenShift alerting rules should have a runbook_url annotation if the alert is critical\| | ||
| OpenShift alerting rules should have a valid severity label\| OpenShift alerting rules should have description and summary annotations\| | ||
| Prometheus when installed on the cluster should start and expose a secured proxy and verify build metrics\| | ||
| egressFirewall should have no impact outside its namespace\| should ensure ipv4 egressrouter cni resources are created\| | ||
| pod should start with all sysctl on whitelist\| pod sysctls should not affect node\| | ||
| when FIPS is disabled the HAProxy router should serve routes when configured with a 1024-bit RSA key | ||
| documentation: |- | ||
| The Openshift E2E HETEROGENEOUS AWS workflow executes the common end-to-end test suite on AWS with a default cluster configuration. |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.