Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ tests:
cluster_profile: aws-qe
env:
BASE_DOMAIN: qe.devcluster.openshift.com
workflow: cucushift-installer-rehearse-aws-ipi-proxy-workers-rhel8
workflow: cucushift-installer-rehearse-aws-ipi-disconnected-private
- as: installer-rehearse-gcp
interval: 960h
steps:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ tests:
cluster_profile: aws-qe
env:
BASE_DOMAIN: qe.devcluster.openshift.com
workflow: cucushift-installer-rehearse-aws-ipi-sdn
workflow: cucushift-installer-rehearse-aws-ipi-disconnected-private-workers-rhcos-rhel8
- as: installer-rehearse-azure
interval: 960h
steps:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ ref:
as: aws-deprovision-cco-manual-users-static
from_image:
namespace: ocp
name: "4.5"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

don't you need to pin a recent 4.y if you want to rely on openshift/installer#6008? Otherwise 4.8 flows using this step, etc., will fail to find the binary, right?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To be honest, I am a bit confuse with the difference between:

from: upi-installer

v.s

  from_image:
    namespace: ocp
    name: "4.5"
    tag: upi-installer

My understanding is from: upi-installer will rely on the prow ci jobs where defined upi-installer, right? If yes, when some jobs did not defined upi-installer, the job referencing the step will be broken as what you said. If that, I probably need to pin it to 4.12.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For the specific step, presently it is only consumed by QE's jobs, the job we defined upi-installer like https://github.com/openshift/release/blob/master/ci-operator/config/openshift/verification-tests/openshift-verification-tests-master__installer-rehearse-4.12.yaml#L14-L17, that is supposed to works in 4.12 prow jobs. Of course, if we want to make the steps working in broader jobs, maybe it is most safe to pin it to 4.12 tag. WDYT?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I changed back to use a pinned-version (4.12) upi-installer image for the step.

name: "4.12"
tag: upi-installer
grace_period: 10m
commands: aws-deprovision-cco-manual-users-static-commands.sh
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ ref:
as: aws-deprovision-route53-private-hosted-zone
from_image:
namespace: ocp
name: "4.10"
name: "4.12"
tag: upi-installer
grace_period: 10m
commands: aws-deprovision-route53-private-hosted-zone-commands.sh
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ ref:
as: aws-deprovision-s3buckets
from_image:
namespace: ocp
name: "4.5"
name: "4.12"
tag: upi-installer
grace_period: 10m
commands: aws-deprovision-s3buckets-commands.sh
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ ref:
as: aws-deprovision-stacks
from_image:
namespace: ocp
name: "4.5"
name: "4.12"
tag: upi-installer
grace_period: 10m
commands: aws-deprovision-stacks-commands.sh
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,6 @@ trap 'CHILDREN=$(jobs -p); if test -n "${CHILDREN}"; then kill ${CHILDREN} && wa

export AWS_SHARED_CREDENTIALS_FILE="${CLUSTER_PROFILE_DIR}/.awscred"

curl -L https://github.com/mikefarah/yq/releases/download/3.3.0/yq_linux_amd64 -o /tmp/yq && chmod +x /tmp/yq

REGION="${LEASED_RESOURCE}"

# Using source region for C2S and SC2S
Expand All @@ -21,7 +19,7 @@ fi
VpcId=$(cat "${SHARED_DIR}/vpc_id")
echo "VpcId: $VpcId"

PublicSubnet="$(/tmp/yq r "${SHARED_DIR}/public_subnet_ids" '[0]')"
PublicSubnet="$(yq-go r "${SHARED_DIR}/public_subnet_ids" '[0]')"
echo "PublicSubnet: $PublicSubnet"

CLUSTER_NAME="${NAMESPACE}-${JOB_NAME_HASH}"
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
ref:
as: aws-provision-bastionhost
from: upi-installer
from_image:
namespace: ocp
name: "4.12"
tag: upi-installer
cli: latest
grace_period: 10m
commands: aws-provision-bastionhost-commands.sh
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,6 @@ trap 'CHILDREN=$(jobs -p); if test -n "${CHILDREN}"; then kill ${CHILDREN} && wa

export AWS_SHARED_CREDENTIALS_FILE="${CLUSTER_PROFILE_DIR}/.awscred"

curl -L https://github.com/mikefarah/yq/releases/download/3.3.0/yq_linux_amd64 -o /tmp/yq && chmod +x /tmp/yq
curl -L https://github.com/stedolan/jq/releases/download/jq-1.6/jq-linux64 -o /tmp/jq && chmod +x /tmp/jq

REGION="${LEASED_RESOURCE}"

function run_command() {
Expand Down Expand Up @@ -119,10 +116,10 @@ do
#
cr_yaml="${cr_yaml_d}/${item}"
cr_json="${cr_json_d}/${item:0:-5}.json"
/tmp/yq r -j "${cr_yaml}" > "${cr_json}"
yq-go r -j "${cr_yaml}" > "${cr_json}"

name=$(cat "${cr_json}" | /tmp/jq -r '.spec.secretRef.name')
ns=$(cat "${cr_json}" | /tmp/jq -r '.spec.secretRef.namespace')
name=$(cat "${cr_json}" | jq -r '.spec.secretRef.name')
ns=$(cat "${cr_json}" | jq -r '.spec.secretRef.namespace')

# Create policy document
#
Expand All @@ -133,8 +130,8 @@ do
| sed 's/"effect"/"Effect"/g' \
| sed 's/"policyCondition"/"Condition"/g' \
| sed 's/"resource"/"Resource"/g' \
| /tmp/jq '{Version: "2012-10-17", Statement: .spec.providerSpec.statementEntries}' > "${policy_json}"
policy_doc=$(cat "${policy_json}" | /tmp/jq -c .)
| jq '{Version: "2012-10-17", Statement: .spec.providerSpec.statementEntries}' > "${policy_json}"
policy_doc=$(cat "${policy_json}" | jq -c .)
echo "policy_doc: $policy_doc"

# Create policy
Expand All @@ -144,7 +141,7 @@ do
output_policy="${resources_d}/policy_${policy_name}.json"
aws_create_policy $REGION "${policy_name}" "${policy_doc}" "${output_policy}"

policy_arn=$(cat "${output_policy}" | /tmp/jq -r '.Policy.Arn')
policy_arn=$(cat "${output_policy}" | jq -r '.Policy.Arn')

echo "${policy_arn}" >> "${SHARED_DIR}/aws_policy_arns"

Expand All @@ -157,8 +154,8 @@ do
output_access_keys="${resources_d}/accesskey_${user_name}.json"
aws_create_user $REGION "${user_name}" "${policy_arn}" "${output_users}" "${output_access_keys}"

key_id=$(cat "${output_access_keys}" | /tmp/jq -r '.AccessKey.AccessKeyId')
key_sec=$(cat "${output_access_keys}" | /tmp/jq -r '.AccessKey.SecretAccessKey')
key_id=$(cat "${output_access_keys}" | jq -r '.AccessKey.AccessKeyId')
key_sec=$(cat "${output_access_keys}" | jq -r '.AccessKey.SecretAccessKey')
echo "${user_name}" >> "${SHARED_DIR}/aws_user_names"

# Generate users manifests
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
ref:
as: aws-provision-cco-manual-users-static
from: upi-installer
from_image:
namespace: ocp
name: "4.12"
tag: upi-installer
cli: latest
grace_period: 10m
commands: aws-provision-cco-manual-users-static-commands.sh
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ ref:
as: aws-provision-route53-private-hosted-zone
from_image:
namespace: ocp
name: "4.10"
name: "4.12"
tag: upi-installer
grace_period: 10m
commands: aws-provision-route53-private-hosted-zone-commands.sh
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,6 @@ set -o pipefail

export AWS_SHARED_CREDENTIALS_FILE="${CLUSTER_PROFILE_DIR}/.awscred"

# TODO: move to image
curl -L https://github.com/mikefarah/yq/releases/download/3.3.0/yq_linux_amd64 -o /tmp/yq && chmod +x /tmp/yq

EXPIRATION_DATE=$(date -d '4 hours' --iso=minutes --utc)
TAGS="Key=expirationDate,Value=${EXPIRATION_DATE}"

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ ref:
as: aws-provision-vpc-disconnected
from_image:
namespace: ocp
name: "4.5"
name: "4.12"
tag: upi-installer
grace_period: 10m
commands: aws-provision-vpc-disconnected-commands.sh
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,6 @@ set -o pipefail

export AWS_SHARED_CREDENTIALS_FILE="${CLUSTER_PROFILE_DIR}/.awscred"

curl -L https://github.com/mikefarah/yq/releases/download/3.3.0/yq_linux_amd64 -o /tmp/yq && chmod +x /tmp/yq

EXPIRATION_DATE=$(date -d '4 hours' --iso=minutes --utc)
TAGS="Key=expirationDate,Value=${EXPIRATION_DATE}"

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ ref:
as: aws-provision-vpc-shared
from_image:
namespace: ocp
name: "4.5"
name: "4.12"
tag: upi-installer
grace_period: 10m
commands: aws-provision-vpc-shared-commands.sh
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -57,8 +57,7 @@ fi
if [ -z "${VNET_NAME}" ]; then
vnet_file="${SHARED_DIR}/customer_vnet_subnets.yaml"
if [ -f "${vnet_file}" ]; then
curl -L https://github.com/mikefarah/yq/releases/download/3.3.0/yq_linux_amd64 -o /tmp/yq && chmod +x /tmp/yq
bastion_vnet_name=$(/tmp/yq r ${vnet_file} 'platform.azure.virtualNetwork')
bastion_vnet_name=$(yq-go r ${vnet_file} 'platform.azure.virtualNetwork')
else
echo "Did not find ${vnet_file}!"
exit 1
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ ref:
as: azure-provision-bastionhost
from_image:
namespace: ocp
name: "4.5"
name: "4.12"
tag: upi-installer
grace_period: 10m
commands: azure-provision-bastionhost-commands.sh
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,6 @@ set -o nounset
set -o errexit
set -o pipefail

# TODO: move to image
curl -L https://github.com/mikefarah/yq/releases/download/3.3.0/yq_linux_amd64 -o /tmp/yq && chmod +x /tmp/yq

function run_command() {
local CMD="$1"
echo "Running Command: ${CMD}"
Expand Down Expand Up @@ -51,9 +48,9 @@ if [ X"${OUTBOUND_UDR_TYPE}" == X"NAT" ]; then
echo "Use NAT for UserDefinedRouting..."
VNET_FILE="${SHARED_DIR}/customer_vnet_subnets.yaml"
RESOURCE_GROUP=$(cat ${SHARED_DIR}/resourcegroup)
vnet_name=$(/tmp/yq r ${VNET_FILE} 'platform.azure.virtualNetwork')
controlPlaneSubnet=$(/tmp/yq r ${VNET_FILE} 'platform.azure.controlPlaneSubnet')
computeSubnet=$(/tmp/yq r ${VNET_FILE} 'platform.azure.computeSubnet')
vnet_name=$(yq-go r ${VNET_FILE} 'platform.azure.virtualNetwork')
controlPlaneSubnet=$(yq-go r ${VNET_FILE} 'platform.azure.controlPlaneSubnet')
computeSubnet=$(yq-go r ${VNET_FILE} 'platform.azure.computeSubnet')
NAT_for_UDR "$RESOURCE_GROUP" "$vnet_name" "$controlPlaneSubnet" "$computeSubnet" || exit 3
else
echo "UserDefinedRouting is enabled, but does not define steps here, leave them for user"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ ref:
as: azure-provision-nat
from_image:
namespace: ocp
name: "4.5"
name: "4.12"
tag: upi-installer
grace_period: 10m
commands: azure-provision-nat-commands.sh
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ ref:
as: azure-provision-vnet
from_image:
namespace: ocp
name: "4.5"
name: "4.12"
tag: upi-installer
grace_period: 10m
commands: azure-provision-vnet-commands.sh
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,12 @@ ref:
as: gcp-deprovision-bastionhost
from_image:
namespace: ocp
name: "4.10"
name: "4.12"
tag: upi-installer
commands: gcp-deprovision-bastionhost-commands.sh
resources:
requests:
cpu: 10m
memory: 100Mi
documentation: |-
The step destroys all resources configured by "gcp-provision-bastionhost".
The step destroys all resources configured by "gcp-provision-bastionhost".
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,12 @@ ref:
as: gcp-deprovision-vpc
from_image:
namespace: ocp
name: "4.10"
name: "4.12"
tag: upi-installer
commands: gcp-deprovision-vpc-commands.sh
resources:
requests:
cpu: 10m
memory: 100Mi
documentation: |-
The step destroys all resources configured by "gcp-provision-vpc".
The step destroys all resources configured by "gcp-provision-vpc".
Original file line number Diff line number Diff line change
Expand Up @@ -27,10 +27,6 @@ fi
#####################################
##############Initialize#############
#####################################

# TODO: move to image
curl -L https://github.com/mikefarah/yq/releases/download/3.3.0/yq_linux_amd64 -o /tmp/yq && chmod +x /tmp/yq

workdir=`mktemp -d`

curl -L -o ${workdir}/fcos-stable.json https://builds.coreos.fedoraproject.org/streams/stable.json
Expand Down Expand Up @@ -68,8 +64,8 @@ echo "Using region: ${REGION}"

VPC_CONFIG="${SHARED_DIR}/customer_vpc_subnets.yaml"
if [[ -z "${NETWORK}" || -z "${CONTROL_PLANE_SUBNET}" ]]; then
NETWORK=$(/tmp/yq r "${VPC_CONFIG}" 'platform.gcp.network')
CONTROL_PLANE_SUBNET=$(/tmp/yq r "${VPC_CONFIG}" 'platform.gcp.controlPlaneSubnet')
NETWORK=$(yq-go r "${VPC_CONFIG}" 'platform.gcp.network')
CONTROL_PLANE_SUBNET=$(yq-go r "${VPC_CONFIG}" 'platform.gcp.controlPlaneSubnet')
fi
if [[ -z "${NETWORK}" || -z "${CONTROL_PLANE_SUBNET}" ]]; then
echo "Could not find VPC network and control-plane subnet" && exit 1
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ ref:
as: gcp-provision-bastionhost
from_image:
namespace: ocp
name: "4.10"
name: "4.12"
tag: upi-installer
grace_period: 10m
commands: gcp-provision-bastionhost-commands.sh
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ ref:
as: gcp-provision-vpc
from_image:
namespace: ocp
name: "4.10"
name: "4.12"
tag: upi-installer
commands: gcp-provision-vpc-commands.sh
resources:
Expand All @@ -15,4 +15,4 @@ ref:
documentation: |-
This field represents the VPC would be a disconnected network (without NAT gateway), or not.
documentation: |-
The GCP step creates VPC network/subnets/etc. resources, and write the info in "${SHARED_DIR}/customer_vpc_subnets.yaml" for subsequent consumers.
The GCP step creates VPC network/subnets/etc. resources, and write the info in "${SHARED_DIR}/customer_vpc_subnets.yaml" for subsequent consumers.
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,6 @@ set -o pipefail

trap 'CHILDREN=$(jobs -p); if test -n "${CHILDREN}"; then kill ${CHILDREN} && wait; fi' TERM

# TODO: move to image
curl -L https://github.com/mikefarah/yq/releases/download/3.3.0/yq_linux_amd64 -o /tmp/yq && chmod +x /tmp/yq

workdir=`mktemp -d`

CLUSTER_NAME="${NAMESPACE}-${JOB_NAME_HASH}"
Expand Down Expand Up @@ -289,7 +286,7 @@ proxy:
username: "${reg_quay_user}"
password: "${reg_quay_password}"
EOF
/tmp/yq m -x -i "${workdir}/registry_config_file_6001" "${patch_file}"
yq-go m -x -i "${workdir}/registry_config_file_6001" "${patch_file}"

# patch proxy for 6002 brew.registry.redhat.io
reg_brew_url=$(cat "/var/run/vault/mirror-registry/registry_brew.json" | jq -r '.url')
Expand All @@ -301,7 +298,7 @@ proxy:
username: "${reg_brew_user}"
password: "${reg_brew_password}"
EOF
/tmp/yq m -x -i "${workdir}/registry_config_file_6002" "${patch_file}"
yq-go m -x -i "${workdir}/registry_config_file_6002" "${patch_file}"

rm -f "${patch_file}"

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ ref:
as: ignition-bastionhost
from_image:
namespace: ocp
name: "4.10"
name: "4.12"
tag: upi-installer
grace_period: 10m
commands: ignition-bastionhost-commands.sh
Expand Down
2 changes: 2 additions & 0 deletions ci-operator/step-registry/ipi/OWNERS
Original file line number Diff line number Diff line change
Expand Up @@ -5,3 +5,5 @@ approvers:
- deads2k
- staebler
- technical-release-team-approvers
- jianlinliu
- yunjiang29
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,9 @@
"abhinavdahiya",
"deads2k",
"staebler",
"technical-release-team-approvers"
"technical-release-team-approvers",
"jianlinliu",
"yunjiang29"
]
}
}
Loading