Tests for Multi Tenancy with Service Mesh by pierDipi · Pull Request #2199 · openshift-knative/serverless-operator

pierDipi · 2023-07-31T07:47:40Z

Based on #2198

Fixes https://issues.redhat.com/browse/SRVCOM-2522

Proposed Changes

Partial PR with changes from #2126:

Rewrite tests from https://github.com/ReToCode/knative-multitenancy in Golang. They're under test/servinge2e/servicemesh/
Remove TestKnativeVersusKubeServicesInOneNamespace as it's redundant. Having Knative and k8s service in one namespace is tested with all Eventing tests which now include KSVC in the test scenarios.
This PR doesn't include any tests for verifying cross-tenant traffic for Eventing. They depend on Send namespace header in MT components knative/eventing#7048 to be merged and backported.
Split the test webhook under serving/metadata-webhook into cluster-resources and namespaced-resources. The second one needs to be applied with -n <namespace>. This is to reduce duplication of code because the webhook is now used in two namespaces: serving-tests, serverless-tests.

TODOs in follow up PRs

Eventing MT tests
Generate authz policies for the tests from the chart container image during make genarated-files
migrate to use mtls: true on the SMCP

This reverts commit 90f5d90.

This reverts commit 9726177.

This reverts commit e00cc73.

Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com>

openshift-ci · 2023-07-31T07:47:44Z

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

pierDipi · 2023-07-31T07:47:53Z

/test 4.10-upstream-e2e-mesh-aws-ocp-410

/test 4.13-upstream-e2e-mesh-aws-ocp-413

Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com>

pierDipi · 2023-07-31T10:58:52Z

Previous failure due to POST method restriction, to be removed in the chart:

=== CONT  TestBrokerConformance/Knative_Broker_Specification_-_Data_Plane/Ingress/Assert/Conformance_Brokers_MUST_reject_all_HTTP_produce_requests_with_a_method_other_than_POST_responding_with_HTTP_status_code_`405_Method_Not_Supported`.
    data_plane.go:350: Expected statuscode 405 for sequence 1 got 403
            --- FAIL: TestBrokerConformance/Knative_Broker_Specification_-_Data_Plane/Ingress/Assert/Conformance_Brokers_MUST_reject_all_HTTP_produce_requests_with_a_method_other_than_POST_responding_with_HTTP_status_code_`405_Method_Not_Supported`. (39.36s)

pierDipi · 2023-07-31T10:59:01Z

/test 4.10-upstream-e2e-mesh-aws-ocp-410

/test 4.13-upstream-e2e-mesh-aws-ocp-413

pierDipi · 2023-07-31T15:12:24Z

Tests are timing out after 4h

pierDipi · 2023-07-31T16:21:46Z

openshift/release#41818

pierDipi · 2023-07-31T16:21:53Z

/test 4.10-upstream-e2e-mesh-aws-ocp-410

/test 4.13-upstream-e2e-mesh-aws-ocp-413

pierDipi · 2023-07-31T20:47:34Z

Passed on 4.10

/test 4.13-upstream-e2e-mesh-aws-ocp-413

pierDipi · 2023-08-01T04:11:11Z

/test 4.13-upstream-e2e-mesh-aws-ocp-413

pierDipi · 2023-08-01T07:59:21Z

APIServer was down:

"https://172.30.0.1/api/v1/namespaces/serverless-tests/pods/t1dlq-mrgeuoyk": dial tcp 172.30.0.1:443: connect: connection refused"

/test 4.13-upstream-e2e-mesh-aws-ocp-413

pierDipi · 2023-08-01T15:32:50Z

From https://gcsweb-ci.apps.ci.l2s4.p1.openshiftapps.com/gcs/origin-ci-test/pr-logs/pull/openshift-knative_serverless-operator/2199/pull-ci-openshift-knative-serverless-operator-main-4.13-upstream-e2e-mesh-aws-ocp-413/1686285486018007040/artifacts/upstream-e2e-mesh-aws-ocp-413/gather-extra/artifacts/pods/serverless-tests_subject-ifdweahz-f6555bd94-dfpmv_user-container.log

2023/08/01 11:15:59 sending cloudevent to http://sink-uzbhnbft.serverless-tests.svc.cluster.local
2023/08/01 11:16:45 failed to send cloudevent: 503:  (10x)

and there is no pod with name sink-uzbhnbft in [1], so it was probably evicted:

[1] https://gcsweb-ci.apps.ci.l2s4.p1.openshiftapps.com/gcs/origin-ci-test/pr-logs/pull/openshift-knative_serverless-operator/2199/pull-ci-openshift-knative-serverless-operator-main-4.13-upstream-e2e-mesh-aws-ocp-413/1686285486018007040/artifacts/upstream-e2e-mesh-aws-ocp-413/gather-extra/artifacts/pods.json

pierDipi · 2023-08-01T15:36:33Z

/test 4.13-upstream-e2e-mesh-aws-ocp-413

pierDipi · 2023-08-02T07:14:54Z

/test 4.13-upstream-e2e-mesh-aws-ocp-413

pierDipi · 2023-08-02T09:18:40Z

/test 4.13-upstream-e2e-mesh-aws-ocp-413

pierDipi · 2023-08-02T09:56:04Z

The failures are not related to the authorization policies, generally, those tests are pretty flaky already but one run on 4.10 succeeded https://prow.ci.openshift.org/view/gs/origin-ci-test/pr-logs/pull/openshift-knative_serverless-operator/2199/pull-ci-openshift-knative-serverless-operator-main-4.10-upstream-e2e-mesh-aws-ocp-410/1686049565687418880, so I think we can merge this PR

/cc @ReToCode @mgencur

pierDipi · 2023-08-02T10:03:01Z

eventing tests flaky reason is that tests' resources are not cleaned up, so the cluster is packed of resources, so sometimes pods just get evicted (we use raw pods in tests).

This will be hopefully fixed by 1.11 and if we really want we can invest some time into backporting the fixes

Refs for commits we need:

Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com>

pierDipi · 2023-08-02T15:40:36Z

/test 4.13-upstream-e2e-mesh-aws-ocp-413

pierDipi · 2023-08-02T15:40:54Z

/test 4.13-upstream-e2e-mesh-aws-ocp-413

pierDipi · 2023-08-02T15:42:03Z

/test 4.10-upstream-e2e-mesh-aws-ocp-410

openshift-ci · 2023-08-02T19:26:18Z

@pierDipi: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/4.13-upstream-e2e-mesh-aws-ocp-413	`2e9c12e`	link	false	`/test 4.13-upstream-e2e-mesh-aws-ocp-413`

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

ReToCode

I'm fine with merging this. Mesh tests are already pretty flaky and we have to continue to improve them anyway. It might be good to do it before 1.11 to see where we stand with our mesh changes for the 1.31 release (which is 1.10).

I'll (try) to do the backports and do an additional PR on top here to use mtls: true, enable the new generation script and see if those back-ports make the tests more stable.

/lgtm
/approve

openshift-ci · 2023-08-03T04:58:45Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: pierDipi, ReToCode

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details

Needs approval from an approver in each of these files:

~~OWNERS~~ [ReToCode,pierDipi]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

ReToCode and others added 10 commits July 31, 2023 08:22

add tests for multitenancy with Service Mesh

59c40a8

add AuthorizationPolicies to mesh_resources

03df549

add DestinationRules and PeerAuthentications to make Domain-Mapping work

87e93f6

Test without forcing mTLS in knative-eventing

4e57576

Skip Long Response message test for now

9329176

Test with http in name

7bfbc5d

Revert "Test with http in name"

cce77a4

This reverts commit 90f5d90.

Revert "Skip Long Response message test for now"

185b6f8

This reverts commit 9726177.

Revert "Test without forcing mTLS in knative-eventing"

c5a4dc9

This reverts commit e00cc73.

Add port name to rekt service

404939a

Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com>

openshift-ci bot added the do-not-merge/work-in-progress label Jul 31, 2023

openshift-ci bot added the approved label Jul 31, 2023

Temp: Remove method restrictions

9cdc4e9

Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com>

ReToCode mentioned this pull request Jul 31, 2023

add tests for multitenancy with Service Mesh #2198

Closed

pierDipi mentioned this pull request Jul 31, 2023

Eventing: Remove POST only requests and add all host variants openshift-knative/knative-istio-authz-chart#7

Merged

pierDipi closed this in openshift-knative/knative-istio-authz-chart#7 Jul 31, 2023

pierDipi reopened this Jul 31, 2023

pierDipi marked this pull request as ready for review August 1, 2023 11:23

openshift-ci bot requested review from lberk and rhuss August 1, 2023 11:23

openshift-ci bot requested review from ReToCode and mgencur August 2, 2023 09:56

pierDipi changed the title ~~[WIP] Tests for Multi Tenancy with Service Mesh~~ Tests for Multi Tenancy with Service Mesh Aug 2, 2023

openshift-ci bot removed the do-not-merge/work-in-progress label Aug 2, 2023

Scale up worker nodes

805914a

Signed-off-by: Pierangelo Di Pilato <pierdipi@redhat.com>

Merge branch 'main' into test-mt-authz-policies

2e9c12e

ReToCode approved these changes Aug 3, 2023

View reviewed changes

openshift-ci bot assigned ReToCode Aug 3, 2023

openshift-ci bot added the lgtm label Aug 3, 2023

openshift-merge-robot merged commit e133050 into openshift-knative:main Aug 3, 2023

Conversation

pierDipi commented Jul 31, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Proposed Changes

Uh oh!

openshift-ci bot commented Jul 31, 2023

Uh oh!

pierDipi commented Jul 31, 2023

Uh oh!

pierDipi commented Jul 31, 2023

Uh oh!

pierDipi commented Jul 31, 2023

Uh oh!

pierDipi commented Jul 31, 2023

Uh oh!

pierDipi commented Jul 31, 2023

Uh oh!

pierDipi commented Jul 31, 2023

Uh oh!

pierDipi commented Jul 31, 2023

Uh oh!

pierDipi commented Aug 1, 2023

Uh oh!

pierDipi commented Aug 1, 2023

Uh oh!

pierDipi commented Aug 1, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pierDipi commented Aug 1, 2023

Uh oh!

pierDipi commented Aug 2, 2023

Uh oh!

pierDipi commented Aug 2, 2023

Uh oh!

pierDipi commented Aug 2, 2023

Uh oh!

pierDipi commented Aug 2, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pierDipi commented Aug 2, 2023

Uh oh!

pierDipi commented Aug 2, 2023

Uh oh!

pierDipi commented Aug 2, 2023

Uh oh!

openshift-ci bot commented Aug 2, 2023

Uh oh!

ReToCode left a comment

Choose a reason for hiding this comment

Uh oh!

openshift-ci bot commented Aug 3, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

pierDipi commented Jul 31, 2023 •

edited

Loading

pierDipi commented Aug 1, 2023 •

edited

Loading

pierDipi commented Aug 2, 2023 •

edited

Loading