hack: fix installing the ginkgo

[cynepco3hahue]

Signed-off-by: Artyom Lukianov alukiano@redhat.com

[jmencak]

I don't think this will work in environments that are cut off from Internet access like the ART pipeline. The question is whether we need this. I specifically wanted to avoid installing ginkgo in the past to keep things a bit more robust with less cruft, but perhaps it is time for a change. Thoughts?

[cynepco3hahue]

I don't think this will work in environments that are cut off from Internet access like the ART pipeline. The question is whether we need this. I specifically wanted to avoid installing ginkgo in the past to keep things a bit more robust with less cruft, but perhaps it is time for a change. Thoughts?

@jmencak Thanks for the comment. Does the ART run e2e tests?

[cynepco3hahue]

I don't think this will work in environments that are cut off from Internet access like the ART pipeline. The question is whether we need this. I specifically wanted to avoid installing ginkgo in the past to keep things a bit more robust with less cruft, but perhaps it is time for a change. Thoughts?

In general, we can do the same as we are doing with other tools, and build it from the vendor directory.

[jmencak]

I don't think this will work in environments that are cut off from Internet access like the ART pipeline. The question is whether we need this. I specifically wanted to avoid installing ginkgo in the past to keep things a bit more robust with less cruft, but perhaps it is time for a change. Thoughts?

@jmencak Thanks for the comment. Does the ART run e2e tests?

It does not to my knowledge and we are running the nightly tests separately on the nightlies instead. So as I say, I'm open to change and perhaps the worry/current config is unfounded.

[jmencak]

I don't think this will work in environments that are cut off from Internet access like the ART pipeline. The question is whether we need this. I specifically wanted to avoid installing ginkgo in the past to keep things a bit more robust with less cruft, but perhaps it is time for a change. Thoughts?

In general, we can do the same as we are doing with other tools, and build it from the vendor directory.

It is an option, but what you're doing is probably simpler and will keep the Makefile cleaner. Let's wait for other thoughts. :) I'm not saying what you do in the PR is a no-go.

[jmencak]

If we do decide to build ginkgo from vendor, then we should also probably change the Makefile to make full use of it when running NTO e2e tests.

[ffromani]

could you please add a comment/note in the commit message about what was the problem and why this fix resolves the problem?

[cynepco3hahue]

Sure will add the comment.

[ffromani]

do we need ginkgo-the-runner or can we just use the ginkgo packages?

[cynepco3hahue]

do we need ginkgo-the-runner or can we just use the ginkgo packages?

We need the runner.

[ffromani]

do we need ginkgo-the-runner or can we just use the ginkgo packages?

We need the runner.

ok, last question: how costly is to avoid using the runner?

[cynepco3hahue]

Ok the CI lane looks better now, it still has failures but at least it succeeded to run
For future investigation:

[Fail] [rfe_id:27368][performance] Validation webhook with API version v1alpha1 profile [It] should reject the creation of the profile with overlapping CPUs 
/go/src/github.com/openshift/cluster-node-tuning-operator/test/e2e/pao/functests/1_performance/performance.go:840
[Fail] [rfe_id:27368][performance] Validation webhook with API version v1alpha1 profile [It] should reject the creation of the profile with overlapping CPUs 
/go/src/github.com/openshift/cluster-node-tuning-operator/test/e2e/pao/functests/1_performance/performance.go:841

[cynepco3hahue]

ok, last question: how costly is to avoid using the runner?

@fromanirh Just to be sure, by this do you mean how costly it is to create a test binary that we can run without ginkgo?

[ffromani]

ok, last question: how costly is to avoid using the runner?

@fromanirh Just to be sure, by this do you mean how costly it is to create a test binary that we can run without ginkgo?

I mean creating a ginkgo entry point like this: https://github.com/openshift-kni/numaresources-operator/blob/main/test/e2e/serial/e2e_serial_test.go which runs all the tests but does not need the ginkgo runner; we can just use go test.

[yanirq]

@jmencak @cynepco3hahue either approach is fine.
I suggest having a fix that will release other PRs if one approach is more costly and will take more time (which we can do in another PR)

[ffromani]

I'm not against the current approach btw

[cynepco3hahue]

I mean creating a ginkgo entry point like this: https://github.com/openshift-kni/numaresources-operator/blob/main/test/e2e/serial/e2e_serial_test.go which runs all the tests but does not need the ginkgo runner; we can just use go test.

I think in this case we probably can not use ginkgo flags, like fail fast. We can consider using it in the future, but first, we should be sure that all our tests are stable for 100%.

[ffromani]

I mean creating a ginkgo entry point like this: https://github.com/openshift-kni/numaresources-operator/blob/main/test/e2e/serial/e2e_serial_test.go which runs all the tests but does not need the ginkgo runner; we can just use go test.

I think in this case we probably can not use ginkgo flags, like fail fast. We can consider using it in the future, but first, we should be sure that all our tests are stable for 100%.

we can, but I agree let's get out of the woods first and if this is the quickest way, let's go with it

[jmencak]

OK, let's wait for ci/prow/e2e-gcp-pao to pass. Then we can squash, merge and cleanup (hopefully...) later.

[yanirq]

OK, let's wait for ci/prow/e2e-gcp-pao to pass. Then we can squash, merge and cleanup (hopefully...) later.

I would still pass this PR even if e2e-gcp-pao fails on actual tests which will need to be addressed (not on setup phase)

[ffromani]

ok, last question: how costly is to avoid using the runner?

@fromanirh Just to be sure, by this do you mean how costly it is to create a test binary that we can run without ginkgo?

I mean creating a ginkgo entry point like this: https://github.com/openshift-kni/numaresources-operator/blob/main/test/e2e/serial/e2e_serial_test.go which runs all the tests but does not need the ginkgo runner; we can just use go test.

NON BLOCKING PoC here: #329 let's see how it goes

[ffromani]

ok, last question: how costly is to avoid using the runner?

@fromanirh Just to be sure, by this do you mean how costly it is to create a test binary that we can run without ginkgo?

I mean creating a ginkgo entry point like this: https://github.com/openshift-kni/numaresources-operator/blob/main/test/e2e/serial/e2e_serial_test.go which runs all the tests but does not need the ginkgo runner; we can just use go test.

NON BLOCKING PoC here: #329 let's see how it goes

the initial experiments demonstrate this is doable BUT having a 1:1 replacement is not so easy. We would possibly need some changes in how we run the suites, so by all means let's unblock quickly the e2e testing with this approach while #329 matures (or a better alternative emerges)

[jmencak]

NON BLOCKING PoC here: #329 let's see how it goes

the initial experiments demonstrate this is doable BUT having a 1:1 replacement is not so easy. We would possibly need some changes in how we run the suites, so by all means let's unblock quickly the e2e testing with this approach while #329 matures (or a better alternative emerges)

I do not have a problem with the approach in this PR to get the tests working fast for the time being.

[yanirq]

/retest

[cynepco3hahue]

/retest

[jmencak]

Great to see the ci/prow/e2e-gcp-pao job succeeded. Can we squash the commits? Then I'm happy to approve.

[cynepco3hahue]

Great to see the ci/prow/e2e-gcp-pao job succeeded. Can we squash the commits? Then I'm happy to approve.

Done.

[jmencak]

Thank you for a clean commit message.
/approve
/lgtm

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: cynepco3hahue, jmencak

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

• OWNERS [jmencak]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[openshift-bot]

/retest-required

Please review the full test history for this PR and help us cut down flakes.

[openshift-ci]

@cynepco3hahue: PR needs rebase.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[openshift-ci]

@cynepco3hahue: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/e2e-upgrade	e51f226	link	true	/test e2e-upgrade
ci/prow/e2e-aws-operator	e51f226	link	true	/test e2e-aws-operator
ci/prow/e2e-aws	e51f226	link	true	/test e2e-aws

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

[cynepco3hahue]

Fixed under different PR.