[CNTRLPLANE-1259]Call OTE in the ci for oauth-apiserver in order to ensure that the tests will be execute always #68202
Conversation
|
/pj-rehearse all |
|
@wangke19: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
|
@wangke19: job(s): , all either don't exist or were not found to be affected, and cannot be rehearsed |
|
/pj-rehearse pull-ci-openshift-oauth-apiserver-master-e2e-tests-extension |
|
@wangke19: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
|
/pj-rehearse pull-ci-openshift-oauth-apiserver-release-4.20-e2e-tests-extension |
|
@wangke19: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
wangke19
changed the title
openshift-ci
bot
added
the
do-not-merge/work-in-progress
|
Failed e2e-tests-extension is fixed in openshift/oauth-apiserver#140 |
wangke19
changed the title
openshift-ci
bot
removed
the
do-not-merge/work-in-progress
|
/lgtm |
|
/pj-rehearse |
|
@wangke19: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
|
Issues in openshift/release go stale after 30d of inactivity. Mark the issue as fresh by commenting If this issue is safe to close now please do so with /lifecycle stale |
openshift-ci
bot
added
the
lifecycle/stale
wangke19
force-pushed
the
oauth-as-ote-ci
branch
from
8fe9d7a to
469cc42
Compare
|
/pj-rehearse |
|
@wangke19: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
wangke19
force-pushed
the
oauth-as-ote-ci
branch
from
469cc42 to
875537d
Compare
|
/remove-lifecycle stale |
openshift-ci
bot
removed
the
lifecycle/stale
|
/pj-rehearse |
|
@wangke19: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
|
/lgtm |
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: gangwgr, wangke19 The full list of commands accepted by this bot can be found here.
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
p0lyn0mial
left a comment
p0lyn0mial
left a comment
There was a problem hiding this comment.
could you split this pr into two commits?
one commit that holds changes made by you and the other commit for holding generated changes.
also could you share a link to the doc that describe ote ?
wangke19
force-pushed
the
oauth-as-ote-ci
branch
from
875537d to
3e0f14e
Compare
|
New changes are detected. LGTM label has been removed. |
|
[REHEARSALNOTIFIER]
Interacting with pj-rehearseComment: Once you are satisfied with the results of the rehearsals, comment: |
|
/pj-rehearse |
|
@wangke19: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
Done. |
|
@wangke19: all tests passed! Full PR test history. Your PR dashboard. Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
| echo "Build binary oauth-apiserver-tests-ext" | ||
| make tests-ext-build | ||
| echo "Running ./oauth-apiserver-tests-ext with sanity test" | ||
| ./oauth-apiserver-tests-ext run-suite openshift/oauth-apiserver/conformance/parallel -j ${ARTIFACT_DIR}/junit_$(shell date +%Y%m%d-%H%M%S).xml |
There was a problem hiding this comment.
shouldn't openshift/oauth-apiserver/conformance/parallel run https://github.com/openshift/oauth-apiserver/blob/34b8999cc261b993cb2a41920f686b88286002d9/test/e2e/tokenreviews.go#L23 ?
how do we add tests to a suite for an existing repo ?
There was a problem hiding this comment.
https://github.com/openshift/oauth-apiserver/blob/34b8999cc261b993cb2a41920f686b88286002d9/test/e2e/tokenreviews.go#L23 is for e2e test, we can see they are in job https://prow.ci.openshift.org/view/gs/test-platform-results/pr-logs/pull/openshift_oauth-apiserver/146/pull-ci-openshift-oauth-apiserver-master-e2e-component/ and https://prow.ci.openshift.org/view/gs/test-platform-results/pr-logs/pull/openshift_oauth-apiserver/146/pull-ci-openshift-oauth-apiserver-master-unit.
how do we add tests to a suite for an existing repo ?
We can add tests into this file https://github.com/openshift/oauth-apiserver/blob/master/test/extended/main.go or some others in directory https://github.com/openshift/oauth-apiserver/blob/master/test/extended, this is the same we did befoe in origin.
You can get detail from workflow of https://github.com/openshift-eng/openshift-tests-extension, with openshift-tests-extension, we can enrich the testing of components and even move some tests closely related to components from origin to within the components.
There was a problem hiding this comment.
I think that the idea behind "openshift-tests-extension" is to be able to execute tests defined for a component/repository/operator. In our case it is about executing tests defined in the oauth-apiserver repo. Right?
Having said that should tests be added to the defined suites ? Is this something we will be doing in the future?
There was a problem hiding this comment.
We will add test cases to the corresponding suite based on the test attributes, such as parallel and serial.
Is this something we will be doing in the future?
Yes, we will add some tests to tests-extension, maybe move some from origin, some from bugs, some from features.
| echo "Build binary oauth-apiserver-tests-ext" | ||
| make tests-ext-build | ||
| echo "Running ./oauth-apiserver-tests-ext with sanity test" | ||
| ./oauth-apiserver-tests-ext run-suite openshift/oauth-apiserver/conformance/parallel -j ${ARTIFACT_DIR}/junit_$(shell date +%Y%m%d-%H%M%S).xml |
There was a problem hiding this comment.
The doc you referenced shows that the name of the binary should be my-component-tests which translates to oauth-apiserver-tests. Should we change the binary name? Is there a convention ?
There was a problem hiding this comment.
no specific convention for the name of the binary, but we use component name of github repo to name, that's easy to manage and identify. From extensionBinaries registry https://github.com/openshift/origin/blob/main/pkg/test/extensions/binary.go#L167, we can see their names are different, but at least components of control plane are unified.
| echo "Build binary oauth-apiserver-tests-ext" | ||
| make tests-ext-build | ||
| echo "Running ./oauth-apiserver-tests-ext with sanity test" | ||
| ./oauth-apiserver-tests-ext run-suite openshift/oauth-apiserver/conformance/parallel -j ${ARTIFACT_DIR}/junit_$(shell date +%Y%m%d-%H%M%S).xml |
There was a problem hiding this comment.
I think that the idea behind "openshift-tests-extension" is to be able to execute tests defined for a component/repository/operator. In our case it is about executing tests defined in the oauth-apiserver repo. Right?
Having said that should tests be added to the defined suites ? Is this something we will be doing in the future?
| echo "Build binary oauth-apiserver-tests-ext" | ||
| make tests-ext-build | ||
| echo "Running ./oauth-apiserver-tests-ext with sanity test" | ||
| ./oauth-apiserver-tests-ext run-suite openshift/oauth-apiserver/conformance/parallel -j ${ARTIFACT_DIR}/junit_$(shell date +%Y%m%d-%H%M%S).xml |
There was a problem hiding this comment.
should we define a makefile target that would run a default test suite ? by doing so we wouldn't have to change the CI job definition if we wanted to change something.
There was a problem hiding this comment.
I think that the idea behind "openshift-tests-extension" is to be able to execute tests defined for a component/repository/operator. In our case it is about executing tests defined in the oauth-apiserver repo. Right?
Our case is executed for oauth-apiserver repo, also register the binary in origin’s extension registry.
If your repo’s extension binary PR hasn’t been merged, you can use multi-PR testing to test them together.
Here we just have merely implemented an example that can run smoothly in the Prow CI.
| test: | ||
| - as: tests-extension | ||
| commands: | | ||
| echo "Build binary oauth-apiserver-tests-ext" |
There was a problem hiding this comment.
which test suites should this job be executing ? is there a convention?
There was a problem hiding this comment.
Basically consistent with the origin we have done before, parallel cases in parallel CI job and serial cases in serial CI job. These will be reflected in the test names, see https://github.com/openshift/oauth-apiserver/blob/master/test/extended/main.go#L9
There was a problem hiding this comment.
In conclusion, the CI job name e2e-tests-extension here may not necessarily be the names we will use in the future, maybe e2e-tests-extension-aws, e2e-tests-extension-aws-serial are CI job names.
|
Issues in openshift/release go stale after 30d of inactivity. Mark the issue as fresh by commenting If this issue is safe to close now please do so with /lifecycle stale |
openshift-ci
bot
added
the
lifecycle/stale
|
Stale issue in openshift/release rot after 15d of inactivity. Mark the issue as fresh by commenting If this issue is safe to close now please do so with /lifecycle rotten |
openshift-ci
bot
added
lifecycle/rotten
|
Rotten issues in openshift/release close after 15d of inactivity. Reopen the issue by commenting /close |
|
@openshift-bot: Closed this PR. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
5 participants
The pr is to verify the of OpenShift Tests Extension scaffold for oauth-apiserver( openshift/oauth-apiserver#138)