Skip to content

Add Proxy CI for OpenStack platform#20800

Merged
openshift-merge-robot merged 1 commit intoopenshift:masterfrom
shiftstack:proxy-with-disconnected
Sep 7, 2021
Merged

Add Proxy CI for OpenStack platform#20800
openshift-merge-robot merged 1 commit intoopenshift:masterfrom
shiftstack:proxy-with-disconnected

Conversation

@MaysaMacedo
Copy link
Contributor

@MaysaMacedo MaysaMacedo commented Aug 2, 2021

This commit includes a new Job for a proxy
installation with a restricted Network. The
bastion VM is configured with two Networks,
one with access to the external world and
another without. The installer will use the
restricted Network and rely on the Proxy
for external access.

The job will run on demand from the installer PRs and also
every 72h in periodic.

@MaysaMacedo MaysaMacedo changed the title Add Proxy CI for OpenStack platform wip: Add Proxy CI for OpenStack platform Aug 2, 2021
@openshift-ci openshift-ci bot requested review from adduarte and mdbooth August 2, 2021 14:10
@openshift-ci openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Aug 2, 2021
@MaysaMacedo MaysaMacedo force-pushed the proxy-with-disconnected branch 2 times, most recently from 438ca5f to 44cbe1c Compare August 2, 2021 14:49
@MaysaMacedo MaysaMacedo marked this pull request as draft August 2, 2021 14:58
@openshift-ci openshift-ci bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Aug 5, 2021
@MaysaMacedo MaysaMacedo force-pushed the proxy-with-disconnected branch from 4e7b673 to d2ffe14 Compare August 10, 2021 13:45
@openshift-ci openshift-ci bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Aug 10, 2021
@MaysaMacedo MaysaMacedo force-pushed the proxy-with-disconnected branch 2 times, most recently from 117c17a to 6653e6d Compare August 10, 2021 20:03
@MaysaMacedo MaysaMacedo marked this pull request as ready for review August 10, 2021 20:07
@MaysaMacedo MaysaMacedo force-pushed the proxy-with-disconnected branch 4 times, most recently from 5fd029b to 0805c03 Compare August 10, 2021 21:36
@MaysaMacedo MaysaMacedo force-pushed the proxy-with-disconnected branch 5 times, most recently from eda90ec to a3d6640 Compare August 11, 2021 12:44
@EmilienM EmilienM force-pushed the proxy-with-disconnected branch 2 times, most recently from 8268f41 to 3505c9c Compare August 25, 2021 00:19
@MaysaMacedo
Copy link
Contributor Author

/retest

@EmilienM
Copy link
Member

/lgtm

@MaysaMacedo
Copy link
Contributor Author

/retest

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: @stbenjam has an approach in PR 26389 to directly exclude the proxy related tests from within the e2e test suites

Copy link
Member

@mandre mandre left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few nits, mostly about logging.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a small change to make it less confusing while debugging:

Suggested change
echo "Skipping step due to CONFIG_TYPE being byon."
echo "Skipping step due to CONFIG_TYPE being \"${CONFIG_TYPE}\"."

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks, I'll add it

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps a comment would help understanding in which cases we need to remove the machines subnet from the router.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
>&2 echo "Created security group for ${CLUSTER_NAME}: ${sg_id}"
>&2 echo "Created bastion security group for ${CLUSTER_NAME}: ${sg_id}"

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You've added DNS to the list, the log message is no longer current:

Suggested change
>&2 echo "Security group rules created in ${sg_id} to allow SSH and squid access"
>&2 echo "Created necessary security group rules in ${sg_id}"

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This assumes there will only ever be one interface for the server on the machines network. If we don't make it more robust, we should at least add a comment explaining our assumption.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see why the bastion would have > 1 port connected on the machines network as of now, but you're right, this can evolve in the future; so we should document it here.

Comment on lines 64 to 65
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we check that ${OPENSTACK_EXTERNAL_NETWORK} != ""?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we need, the previous command would have failed. OPENSTACK_EXTERNAL_NETWORK has a value for each platform, I think this is safe to keep it that way.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
echo "ZONES_COUNT was set to '${ZONES_COUNT}', although CONFIG_TYPE was not set to 'byon'."
echo "ZONES_COUNT was set to '${ZONES_COUNT}', although CONFIG_TYPE was not set to 'byon' or 'proxy'."

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@EmilienM
Copy link
Member

EmilienM commented Sep 1, 2021

/lgtm

1 similar comment
@EmilienM
Copy link
Member

EmilienM commented Sep 1, 2021

/lgtm

@patrickdillon
Copy link
Contributor

/approve

@EmilienM
Copy link
Member

EmilienM commented Sep 1, 2021

/retest

5 similar comments
@MaysaMacedo
Copy link
Contributor Author

/retest

@MaysaMacedo
Copy link
Contributor Author

/retest

@MaysaMacedo
Copy link
Contributor Author

/retest

@MaysaMacedo
Copy link
Contributor Author

/retest

@MaysaMacedo
Copy link
Contributor Author

/retest

@EmilienM
Copy link
Member

EmilienM commented Sep 6, 2021

/lgtm

@MaysaMacedo
Copy link
Contributor Author

/retest

@MaysaMacedo
Copy link
Contributor Author

/assign @petr-muller

@MaysaMacedo
Copy link
Contributor Author

/retest

This commit includes a new Job for a proxy
installation with a restricted Network. The
bastion VM is configured with two networks,
one with access to the external world (Bastion network) and
another without (Machines Network). The installer will use the
restricted Machines network and rely on the Proxy
for external access.

The job will run on demand from the installer PRs and also every 72h in
periodic.

Co-Authored-By: Maysa Macedo <maysa.macedo95@gmail.com>
Co-Authored-By: Emilien Macchi <emilien@redhat.com>
@EmilienM
Copy link
Member

EmilienM commented Sep 6, 2021

/lgtm

@MaysaMacedo
Copy link
Contributor Author

/assign stbenjam

@MaysaMacedo
Copy link
Contributor Author

/retest

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Sep 7, 2021

@MaysaMacedo: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Rerun command
ci/rehearse/periodic-ci-openshift-release-master-ci-4.9-e2e-openstack-parallel 98164da3a3d9dcfa1ff712e4f0146eec56abce9c link /test pj-rehearse
ci/rehearse/periodic-ci-openshift-release-master-nightly-4.10-e2e-openstack-proxy a2e245b link /test pj-rehearse
ci/rehearse/openshift/cluster-cloud-controller-manager-operator/release-4.9/e2e-openstack-ccm a2e245b link /test pj-rehearse
ci/rehearse/openshift/kubernetes/release-4.9/e2e-openstack-csi-manila a2e245b link /test pj-rehearse
ci/rehearse/periodic-ci-openshift-release-master-ci-4.9-e2e-openstack-techpreview-parallel a2e245b link /test pj-rehearse
ci/rehearse/openshift/openstack-cinder-csi-driver-operator/release-4.9/e2e-openstack-csi a2e245b link /test pj-rehearse
ci/rehearse/periodic-ci-openshift-release-master-nightly-4.9-e2e-openstack-az a2e245b link /test pj-rehearse
ci/rehearse/periodic-ci-openshift-release-master-nightly-4.9-e2e-openstack-proxy a2e245b link /test pj-rehearse
ci/rehearse/openshift/installer/master/e2e-openstack-proxy a2e245b link /test pj-rehearse
ci/rehearse/periodic-ci-openshift-release-master-ci-4.9-upgrade-from-stable-4.8-e2e-openstack-upgrade a2e245b link /test pj-rehearse
ci/prow/pj-rehearse a2e245b link /test pj-rehearse
ci/rehearse/periodic-ci-openshift-release-master-nightly-4.10-e2e-openstack-fips a2e245b link /test pj-rehearse

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@stbenjam
Copy link
Member

stbenjam commented Sep 7, 2021

/approve

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Sep 7, 2021

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: EmilienM, MaysaMacedo, patrickdillon, stbenjam

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci
Copy link
Contributor

openshift-ci bot commented Sep 7, 2021

@MaysaMacedo: Updated the following 3 configmaps:

  • ci-operator-master-configs configmap in namespace ci at cluster app.ci using the following files:
    • key openshift-installer-master.yaml using file ci-operator/config/openshift/installer/openshift-installer-master.yaml
    • key openshift-release-master__nightly-4.10.yaml using file ci-operator/config/openshift/release/openshift-release-master__nightly-4.10.yaml
    • key openshift-release-master__nightly-4.9.yaml using file ci-operator/config/openshift/release/openshift-release-master__nightly-4.9.yaml
  • job-config-master configmap in namespace ci at cluster app.ci using the following files:
    • key openshift-installer-master-presubmits.yaml using file ci-operator/jobs/openshift/installer/openshift-installer-master-presubmits.yaml
    • key openshift-release-master-periodics.yaml using file ci-operator/jobs/openshift/release/openshift-release-master-periodics.yaml
  • step-registry configmap in namespace ci at cluster app.ci using the following files:
    • key OWNERS using file ci-operator/step-registry/openshift/e2e/openstack/proxy/OWNERS
    • key openshift-e2e-openstack-proxy-workflow.metadata.json using file ci-operator/step-registry/openshift/e2e/openstack/proxy/openshift-e2e-openstack-proxy-workflow.metadata.json
    • key openshift-e2e-openstack-proxy-workflow.yaml using file ci-operator/step-registry/openshift/e2e/openstack/proxy/openshift-e2e-openstack-proxy-workflow.yaml
    • key openstack-conf-creatednsrecords-ref.yaml using file ci-operator/step-registry/openstack/conf/creatednsrecords/openstack-conf-creatednsrecords-ref.yaml
    • key openstack-conf-createfips-commands.sh using file ci-operator/step-registry/openstack/conf/createfips/openstack-conf-createfips-commands.sh
    • key openstack-conf-createfips-ref.yaml using file ci-operator/step-registry/openstack/conf/createfips/openstack-conf-createfips-ref.yaml
    • key openstack-conf-generateconfig-commands.sh using file ci-operator/step-registry/openstack/conf/generateconfig/openstack-conf-generateconfig-commands.sh
    • key openstack-conf-generateconfig-ref.yaml using file ci-operator/step-registry/openstack/conf/generateconfig/openstack-conf-generateconfig-ref.yaml
    • key openstack-deprovision-bastionproxy-commands.sh using file ci-operator/step-registry/openstack/deprovision/bastionproxy/openstack-deprovision-bastionproxy-commands.sh
    • key openstack-deprovision-bastionproxy-ref.yaml using file ci-operator/step-registry/openstack/deprovision/bastionproxy/openstack-deprovision-bastionproxy-ref.yaml
    • key openstack-deprovision-machinesubnet-commands.sh using file ci-operator/step-registry/openstack/deprovision/machinesubnet/openstack-deprovision-machinesubnet-commands.sh
    • key openstack-deprovision-machinesubnet-ref.yaml using file ci-operator/step-registry/openstack/deprovision/machinesubnet/openstack-deprovision-machinesubnet-ref.yaml
    • key openstack-provision-bastionproxy-commands.sh using file ci-operator/step-registry/openstack/provision/bastionproxy/openstack-provision-bastionproxy-commands.sh
    • key openstack-provision-bastionproxy-ref.yaml using file ci-operator/step-registry/openstack/provision/bastionproxy/openstack-provision-bastionproxy-ref.yaml
    • key openstack-provision-machinesubnet-commands.sh using file ci-operator/step-registry/openstack/provision/machinesubnet/openstack-provision-machinesubnet-commands.sh
    • key openstack-provision-machinesubnet-ref.yaml using file ci-operator/step-registry/openstack/provision/machinesubnet/openstack-provision-machinesubnet-ref.yaml
Details

In response to this:

This commit includes a new Job for a proxy
installation with a restricted Network. The
bastion VM is configured with two Networks,
one with access to the external world and
another without. The installer will use the
restricted Network and rely on the Proxy
for external access.

The job will run on demand from the installer PRs and also
every 72h in periodic.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. lgtm Indicates that a PR is ready to be merged.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants