CNTRLPLANE-947: E2E test adaptations for OIDC

[liouk]

In order to be able to run openshift's conformance e2e test suite on a cluster with external OIDC configured, we must skip any tests that are inherently irrelevant to OIDC (for example, tests against the OAuth APIs must be skipped, as these APIs do not exist in a cluster with external OIDC).

However, there's a number of tests that we want to avoid skipping (e.g. checking apiserver availability); this PR makes adaptations to such tests that currently break when OIDC is configured but we don't want to skip completely.

Summary of changes

• authorization_rbac_proxy: when the users API is not present, the oc user created in this test comes from client.go; the order of the groups is different than the one the test expects, so we must make the test check expect both orders
• apiserver-external-availability monitor test: this test checks all API servers, including the oauth apiserver; we adapt this test to skip the oauth apiserver when OIDC is configured (as it does not exist)
• management_plane_operators: when OIDC is configured, the authentication operator does not have some conditions that are listed as always required in this test; this PR introduces a mechanism to determine some cluster-runtime conditions depending on cluster config/state and moves the respective auth operator ones there

Example failed run of conformance suite with OIDC configured: https://prow.ci.openshift.org/view/gs/test-platform-results/pr-logs/pull/openshift_release/66981/rehearse-66981-periodic-ci-openshift-cluster-authentication-operator-release-4.21-periodics-e2e-aws-external-oidc-conformance-parallel-techpreview/1970076671268622336

[openshift-ci-robot]

@liouk: This pull request references CNTRLPLANE-947 which is a valid jira issue.

In response to this:

This PR makes e2e test adaptations for the case of a cluster with external OIDC authentication configured. These are tests we do not want to skip completely.

Summary of changes

• authorization_rbac_proxy: when the users API is not present, the oc user created in this test comes from client.go; the order of the groups is different than the one the test expects, so we must make the test check more flexible.
• apiserver-external-availability monitor test: this test checks all API servers, including the oauth apiserver; we adapt this test to skip the oauth apiserver when OIDC is configured (as it does not exist)
• management_plane_operators: when OIDC is configured, the authentication operator does not have some conditions that are listed as always required in this test; this PR introduces a mechanism to determine some cluster-runtime conditions depending on cluster config/state and moves the respective auth operator ones there

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[openshift-ci-robot]

@liouk: This pull request references CNTRLPLANE-947 which is a valid jira issue.

In response to this:

This PR makes e2e test adaptations for the case of a cluster with external OIDC authentication configured. These are tests we do not want to skip completely.

Summary of changes

• authorization_rbac_proxy: when the users API is not present, the oc user created in this test comes from client.go; the order of the groups is different than the one the test expects, so we must make the test check expect both orders
• apiserver-external-availability monitor test: this test checks all API servers, including the oauth apiserver; we adapt this test to skip the oauth apiserver when OIDC is configured (as it does not exist)
• management_plane_operators: when OIDC is configured, the authentication operator does not have some conditions that are listed as always required in this test; this PR introduces a mechanism to determine some cluster-runtime conditions depending on cluster config/state and moves the respective auth operator ones there

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[openshift-ci-robot]

@liouk: This pull request references CNTRLPLANE-947 which is a valid jira issue.

In response to this:

This PR makes e2e test adaptations for the case of a cluster with external OIDC authentication configured. These are tests we do not want to skip completely.

Summary of changes

• authorization_rbac_proxy: when the users API is not present, the oc user created in this test comes from client.go; the order of the groups is different than the one the test expects, so we must make the test check expect both orders
• apiserver-external-availability monitor test: this test checks all API servers, including the oauth apiserver; we adapt this test to skip the oauth apiserver when OIDC is configured (as it does not exist)
• management_plane_operators: when OIDC is configured, the authentication operator does not have some conditions that are listed as always required in this test; this PR introduces a mechanism to determine some cluster-runtime conditions depending on cluster config/state and moves the respective auth operator ones there

Example failed run of conformance suite with OIDC configured: https://prow.ci.openshift.org/view/gs/test-platform-results/pr-logs/pull/openshift_release/66981/rehearse-66981-periodic-ci-openshift-cluster-authentication-operator-release-4.21-periodics-e2e-aws-external-oidc-conformance-parallel-techpreview/1970076671268622336

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[openshift-ci-robot]

@liouk: This pull request references CNTRLPLANE-947 which is a valid jira issue.

In response to this:

In order to be able to run openshift's conformance e2e test suite on a cluster with external OIDC configured, we must skip any tests that are inherently irrelevant to OIDC (for example, tests against the OAuth APIs must be skipped, as these APIs do not exist in a cluster with external OIDC).

However, there's a number of tests that we want to avoid skipping (e.g. checking apiserver availability); this PR makes adaptations to such tests that currently break when OIDC is configured but we don't want to skip completely.

Summary of changes

• authorization_rbac_proxy: when the users API is not present, the oc user created in this test comes from client.go; the order of the groups is different than the one the test expects, so we must make the test check expect both orders
• apiserver-external-availability monitor test: this test checks all API servers, including the oauth apiserver; we adapt this test to skip the oauth apiserver when OIDC is configured (as it does not exist)
• management_plane_operators: when OIDC is configured, the authentication operator does not have some conditions that are listed as always required in this test; this PR introduces a mechanism to determine some cluster-runtime conditions depending on cluster config/state and moves the respective auth operator ones there

Example failed run of conformance suite with OIDC configured: https://prow.ci.openshift.org/view/gs/test-platform-results/pr-logs/pull/openshift_release/66981/rehearse-66981-periodic-ci-openshift-cluster-authentication-operator-release-4.21-periodics-e2e-aws-external-oidc-conformance-parallel-techpreview/1970076671268622336

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[liouk]

/retest-required

[sdodson]

@liouk What's the minimal set of required tests you'd like to see pass here before this merges. I don't want to get stuck in a retest quagmire.

[liouk]

@sdodson when it comes to making sure that the changes in this PR do not break existing tests, I've already seen enough successful runs of the tests changed in the jobs that have already run successfully.

From the current failing jobs, the ones that contain the updated tests are the following:

• e2e-aws-ovn
• e2e-aws-ovn-single-node-upgrade
• e2e-aws-proxy
• e2e-openstack-ovn

However these aren't required anyway, so I doubt we should block this PR until these succeed.

Apart from verifying we're not breaking any existing tests, I would like to see the results of the jobs introduced with openshift/release#66981 and I'm planning on running the conformance suites at a local cluster -- however I am not aware of any way to run those jobs on the CI and include the changes of this PR before it merges (let me know if there's a way!).

[sdodson]

/approve

[openshift-ci]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: liouk, sdodson
Once this PR has been reviewed and has the lgtm label, please assign deads2k for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[openshift-ci]

@liouk: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/e2e-aws-ovn-upgrade	c109961	link	false	/test e2e-aws-ovn-upgrade
ci/prow/e2e-aws-ovn-cgroupsv2	c109961	link	false	/test e2e-aws-ovn-cgroupsv2
ci/prow/e2e-metal-ipi-serial-2of2	c109961	link	false	/test e2e-metal-ipi-serial-2of2
ci/prow/e2e-aws-ovn-single-node-upgrade	c109961	link	false	/test e2e-aws-ovn-single-node-upgrade
ci/prow/e2e-aws-ovn-edge-zones	c109961	link	false	/test e2e-aws-ovn-edge-zones
ci/prow/e2e-aws-ovn-single-node-serial	c109961	link	false	/test e2e-aws-ovn-single-node-serial
ci/prow/e2e-openstack-ovn	c109961	link	false	/test e2e-openstack-ovn
ci/prow/e2e-aws-disruptive	c109961	link	false	/test e2e-aws-disruptive
ci/prow/e2e-aws-ovn-kube-apiserver-rollout	c109961	link	false	/test e2e-aws-ovn-kube-apiserver-rollout
ci/prow/e2e-aws-proxy	c109961	link	false	/test e2e-aws-proxy
ci/prow/e2e-metal-ipi-serial-ovn-ipv6-2of2	c109961	link	false	/test e2e-metal-ipi-serial-ovn-ipv6-2of2
ci/prow/e2e-gcp-ovn-techpreview-serial-1of2	c109961	link	false	/test e2e-gcp-ovn-techpreview-serial-1of2
ci/prow/e2e-aws-ovn-single-node	c109961	link	false	/test e2e-aws-ovn-single-node
ci/prow/okd-scos-e2e-aws-ovn	c109961	link	false	/test okd-scos-e2e-aws-ovn
ci/prow/e2e-aws-ovn	c109961	link	false	/test e2e-aws-ovn

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

[liouk]

/retest-required