configs/openshift/installer: Add jobs for Azure testing #3877
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -54,6 +54,15 @@ currently exist: | |
| | `metrics-int.key` | Azure Geneva metrics authentication key | | ||
| | `system-docker-config.json` | Root/node/system level docker config.json file, currently holding access registry.redhat.io | | ||
|
|
||
| #### `cluster-secrets-azure4` | ||
|
|
||
| | Key | Description | | ||
| | ----------------------------------| ----------- | | ||
| | `osServicePrincipal.json` | Credentials for the Azure API. This is a json file that contains fields described in [upstream credentials doc](https://docs.microsoft.com/en-us/azure-stack/operator/azure-stack-create-service-principals#create-a-service-principal-using-a-client-secret). | | ||
|
||
| | `pull-secret` | Credentials for pulling OpenShift images from Quay and for authenticating to telemetry. Retrieved from [try.openshift.com](https://try.openshift.com) under the [email protected] account, and has the service account token from the `ocp` namespace added with `oc registry login --to=/tmp/pull-secret -z default -n ocp`.| | ||
| | `ssh-privatekey` | Private half of the SSH key, for connecting to Azure VMs. | | ||
| | `ssh-publickey` | Public half of the SSH key, for connecting to Azure VMs. | | ||
|
|
||
| #### `cluster-secrets-vsphere` | ||
|
|
||
| | Key | Description | | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -19,8 +19,6 @@ parameters: | |
| - name: RELEASE_IMAGE_LATEST | ||
| required: true | ||
| - name: BASE_DOMAIN | ||
|
|
||
| objects: | ||
|
|
||
|
|
@@ -97,6 +95,8 @@ objects: | |
| env: | ||
| - name: AWS_SHARED_CREDENTIALS_FILE | ||
| value: /tmp/cluster/.awscred | ||
| - name: AZURE_AUTH_LOCATION | ||
| value: /tmp/cluster/osServicePrincipal.json | ||
| - name: ARTIFACT_DIR | ||
| value: /tmp/artifacts | ||
| - name: HOME | ||
|
|
@@ -154,6 +154,12 @@ objects: | |
| # TODO: make openshift-tests auto-discover this from cluster config | ||
| export TEST_PROVIDER='{"type":"aws","region":"us-east-1","zone":"us-east-1a","multizone":true,"multimaster":true}' | ||
| export KUBE_SSH_USER=core | ||
| elif [[ "${CLUSTER_TYPE}" == "azure" ]]; then | ||
| mkdir -p ~/.ssh | ||
| export PROVIDER_ARGS="-provider=azure -gce-zone=centralus" | ||
| # TODO: make openshift-tests auto-discover this from cluster config | ||
| export TEST_PROVIDER='{"type":"azure","region":"centralus","multizone":true,"multimaster":true}' | ||
| export KUBE_SSH_USER=core | ||
| fi | ||
|
|
||
| mkdir -p /tmp/output | ||
|
|
@@ -527,6 +533,10 @@ objects: | |
| value: /etc/openshift-installer/.awscred | ||
| - name: AWS_REGION | ||
| value: us-east-1 | ||
| - name: AZURE_AUTH_LOCATION | ||
| value: /etc/openshift-installer/osServicePrincipal.json | ||
| - name: AZURE_REGION | ||
| value: centralus | ||
| - name: CLUSTER_NAME | ||
| value: ${NAMESPACE}-${JOB_NAME_HASH} | ||
| - name: BASE_DOMAIN | ||
|
|
@@ -567,8 +577,8 @@ objects: | |
|
|
||
| if [[ "${CLUSTER_TYPE}" == "aws" ]]; then | ||
| cat > /tmp/artifacts/installer/install-config.yaml << EOF | ||
| apiVersion: v1 | ||
| baseDomain: ${BASE_DOMAIN:-origin-ci-int-aws.dev.rhcloud.com} | ||
| metadata: | ||
| name: ${CLUSTER_NAME} | ||
| controlPlane: | ||
|
|
@@ -597,6 +607,27 @@ objects: | |
| sshKey: | | ||
| ${SSH_PUB_KEY} | ||
| EOF | ||
| elif [[ "${CLUSTER_TYPE}" == "azure4" ]]; then | ||
| cat > /tmp/artifacts/installer/install-config.yaml << EOF | ||
| apiVersion: v1 | ||
| baseDomain: ${BASE_DOMAIN:-ci.azure.devcluster.openshift.com} | ||
| metadata: | ||
| name: ${CLUSTER_NAME} | ||
| controlPlane: | ||
| name: master | ||
| replicas: 3 | ||
| compute: | ||
| - name: worker | ||
| replicas: 3 | ||
|
||
| platform: | ||
| azure: | ||
| baseDomainResourceGroupName: os4-common | ||
| region: ${AZURE_REGION} | ||
| pullSecret: > | ||
| ${PULL_SECRET} | ||
| sshKey: | | ||
| ${SSH_PUB_KEY} | ||
| EOF | ||
| else | ||
| echo "Unsupported cluster type '${CLUSTER_NAME}'" | ||
| exit 1 | ||
|
|
@@ -621,6 +652,14 @@ objects: | |
| value: ${NAMESPACE}-${JOB_NAME_HASH} | ||
| - name: TYPE | ||
| value: ${CLUSTER_TYPE} | ||
| - name: AWS_SHARED_CREDENTIALS_FILE | ||
| value: /etc/openshift-installer/.awscred | ||
| - name: AWS_REGION | ||
| value: us-east-1 | ||
| - name: AZURE_AUTH_LOCATION | ||
| value: /etc/openshift-installer/osServicePrincipal.json | ||
| - name: AZURE_REGION | ||
| value: centralus | ||
| - name: KUBECONFIG | ||
| value: /tmp/artifacts/installer/auth/kubeconfig | ||
| command: | ||
|
|
@@ -760,7 +799,6 @@ objects: | |
| wait | ||
|
|
||
| echo "Deprovisioning cluster ..." | ||
| openshift-install --dir /tmp/artifacts/installer destroy cluster | ||
| } | ||
|
|
||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -28,8 +28,6 @@ parameters: | |
| - name: RESOURCES_TEST | ||
| value: '{"requests": {"cpu": 1, "memory": "300Mi"}, "limits": {"memory": "3Gi"}}' | ||
| - name: BASE_DOMAIN | ||
|
|
||
| objects: | ||
|
|
||
|
|
@@ -98,6 +96,8 @@ objects: | |
| env: | ||
| - name: AWS_SHARED_CREDENTIALS_FILE | ||
| value: /tmp/cluster/.awscred | ||
| - name: AZURE_AUTH_LOCATION | ||
| value: /tmp/cluster/osServicePrincipal.json | ||
| - name: ARTIFACT_DIR | ||
| value: /tmp/artifacts | ||
| - name: HOME | ||
|
|
@@ -157,6 +157,13 @@ objects: | |
| # TODO: make openshift-tests auto-discover this from cluster config | ||
| export TEST_PROVIDER='{"type":"aws","region":"us-east-1","zone":"us-east-1a","multizone":true,"multimaster":true}' | ||
| export KUBE_SSH_USER=core | ||
| elif [[ "${CLUSTER_TYPE}" == "azure" ]]; then | ||
| mkdir -p ~/.ssh | ||
| export PROVIDER_ARGS="-provider=azure -gce-zone=centralus" | ||
| # TODO: make openshift-tests auto-discover this from cluster config | ||
| export TEST_PROVIDER='{"type":"azure","region":"centralus","multizone":true,"multimaster":true}' | ||
| export KUBE_SSH_USER=core | ||
| fi | ||
|
Member
|
||
| elif [[ "${CLUSTER_TYPE}" == "openstack" ]]; then | ||
| mkdir -p ~/.ssh | ||
| cp /tmp/cluster/ssh-privatekey ~/.ssh/kube_openstack_rsa || true | ||
|
|
@@ -182,6 +189,10 @@ objects: | |
| value: /etc/openshift-installer/.awscred | ||
| - name: AWS_REGION | ||
| value: us-east-1 | ||
| - name: AZURE_AUTH_LOCATION | ||
| value: /etc/openshift-installer/osServicePrincipal.json | ||
| - name: AZURE_REGION | ||
| value: centralus | ||
| - name: CLUSTER_NAME | ||
| value: ${NAMESPACE}-${JOB_NAME_HASH} | ||
| - name: BASE_DOMAIN | ||
|
|
@@ -225,8 +236,8 @@ objects: | |
|
|
||
| if [[ "${CLUSTER_TYPE}" == "aws" ]]; then | ||
| cat > /tmp/artifacts/installer/install-config.yaml << EOF | ||
| apiVersion: v1 | ||
|
Contributor
There was a problem hiding this comment. Is v1 supported back to rc5? Note from now on you can't bump this ever because 4.1 clusters have to be able to install using this template (templates have to work for all 4.y GA versions).
Contributor
Author
There was a problem hiding this comment. We GA'd the v1 API before rc4 openshift/installer#1589 (April 14) |
||
| baseDomain: ${BASE_DOMAIN:-origin-ci-int-aws.dev.rhcloud.com} | ||
| metadata: | ||
| name: ${CLUSTER_NAME} | ||
| controlPlane: | ||
|
|
@@ -255,10 +266,31 @@ objects: | |
| sshKey: | | ||
| ${SSH_PUB_KEY} | ||
| EOF | ||
| elif [[ "${CLUSTER_TYPE}" == "azure4" ]]; then | ||
|
Contributor
There was a problem hiding this comment. I would prefer cluster type to be |
||
| cat > /tmp/artifacts/installer/install-config.yaml << EOF | ||
| apiVersion: v1 | ||
| baseDomain: ${BASE_DOMAIN:-ci.azure.devcluster.openshift.com} | ||
| metadata: | ||
| name: ${CLUSTER_NAME} | ||
| controlPlane: | ||
| name: master | ||
| replicas: 3 | ||
| compute: | ||
| - name: worker | ||
| replicas: 3 | ||
| platform: | ||
| azure: | ||
| baseDomainResourceGroupName: os4-common | ||
|
Contributor
There was a problem hiding this comment. Is this already a set name? Something that is part of the account?
Contributor
Author
There was a problem hiding this comment. installer currently picks the first DNS Zone from the account based on the base domain, while most cloud providers allow multiple DNS zones with the same name. We have https://jira.coreos.com/browse/CORS-1070 already in pipeline to allow users to choose the exact Zone (without the guessing). But Microsoft people were far too un-comfortable with leaving the guessing for Azure addition and insisted we add this field to Azure platform while we were frozen for AWS.
Yes this resource group is present in the CI subscription. |
||
| region: ${AZURE_REGION} | ||
| pullSecret: > | ||
| ${PULL_SECRET} | ||
| sshKey: | | ||
| ${SSH_PUB_KEY} | ||
| EOF | ||
| elif [[ "${CLUSTER_TYPE}" == "openstack" ]]; then | ||
| cat > /tmp/artifacts/installer/install-config.yaml << EOF | ||
| apiVersion: v1 | ||
| baseDomain: ${BASE_DOMAIN:-origin-ci-int-aws.dev.rhcloud.com} | ||
| metadata: | ||
| name: ${CLUSTER_NAME} | ||
| platform: | ||
|
|
@@ -296,6 +328,14 @@ objects: | |
| value: ${NAMESPACE}-${JOB_NAME_HASH} | ||
| - name: TYPE | ||
| value: ${CLUSTER_TYPE} | ||
| - name: AWS_SHARED_CREDENTIALS_FILE | ||
|
Contributor
There was a problem hiding this comment. Was this file inconsistent with the others?
Contributor
Author
There was a problem hiding this comment. All other containers use the ENV to setup the creds. expect teardown, so this change makes it uniform. |
||
| value: /etc/openshift-installer/.awscred | ||
| - name: AWS_REGION | ||
| value: us-east-1 | ||
| - name: AZURE_AUTH_LOCATION | ||
| value: /etc/openshift-installer/osServicePrincipal.json | ||
| - name: AZURE_REGION | ||
| value: centralus | ||
| - name: KUBECONFIG | ||
| value: /tmp/artifacts/installer/auth/kubeconfig | ||
| command: | ||
|
|
@@ -434,7 +474,6 @@ objects: | |
| wait | ||
|
|
||
| echo "Deprovisioning cluster ..." | ||
| openshift-install --dir /tmp/artifacts/installer destroy cluster | ||
| } | ||
|
|
||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| * | ||
| !.type | ||
| !.gitignore | ||
| !OWNERS | ||
| !secret_example |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| azure4 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,6 @@ | ||
| { | ||
| "subscriptionId": "xxxxx", | ||
| "clientId": "xxxxx", | ||
| "clientSecret": "xxxxx", | ||
| "tenantId": "xxxxx" | ||
| } |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please call this
azure-e2e-secretsfor now (azure4 is confusing). @jim-minter since azure is camping on the convention that the rest of the e2e suites run, I would prefer if you guys can over time migrate fromazure-secretstoazure-aro-secretsor similar. The base namespace belongs to the core CI platform.