Initial Mesh (GAMMA) conformance tests #1878

howardjohn · 2023-03-25T00:16:33Z

What type of PR is this?
/area conformance

What this PR does / why we need it:
This PR introduces the basic scaffolding for Mesh testing. Mesh testing is fundamentally different from ingress testing, as we need to send requests from pod-to-pod instead of "go test" to "reachable IP address".

This PR introduces the basic scaffolding to run this test. With this PR, you can run a test against GAMMA tests.

This will:

Deploy a new namespace and a few applications
Run tests against them. Currently, there is only a trivial test that ensures applications can talk to each other; this PR is just intended to get the base infra, not add useful tests (yet)

Architecturally, we are deploying gcr.io/istio-testing/app:latest to each app. This is a client+server pair. We then on-demand exec into each pod to run client http://some-request, and parse the result and validate against expectations.

Likely desired improvements:

Move image into k8s org, do not use latest
Move to use a long-lived port-forward to each application; exec has a high overhead. The echo app has a grpc server we can control it from (instead of exec, we send it a ForwardMessage request). However, this has a downside that it requires us to pull in gRPC.
Make code a bit more robust, etc

Opening this PR to get early feedback before moving further.

I think we could also merge this as-is, as it doesn't impact non-Mesh cases and will allow further iteration.

Which issue(s) this PR fixes:

Fixes #1340

Does this PR introduce a user-facing change?:

NONE

k8s-ci-robot · 2023-03-25T00:16:35Z

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

conformance/conformance_test.go

shaneutt

I'm OK with this based on the machinery and the decisions from discussions here that led to the follow up PR #1894.

I haven't done significant scrutiny of the tests or the test helpers personally, I've invited several mesh-focused community members to review and hopefully test this out themselves. We should give them a little bit of time to check this out before we merge.

/hold

keithmattix

I personally don't have any reservations about this, but I'll let someone else be the final LGTM

conformance/mesh/manifests.yaml

robscott · 2023-03-31T17:47:54Z

conformance/tests/mesh-split.go

@@ -0,0 +1,70 @@
+/*
+Copyright 2022 The Kubernetes Authors.


A lot of these files have old copyright dates, are they just being copied from Istio codebase?

copied from other files in this repo

michaelbeaumont

Great! LGTM

k8s-ci-robot · 2023-04-01T08:43:57Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: howardjohn, keithmattix, michaelbeaumont, shaneutt

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [shaneutt]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

shaneutt · 2023-04-03T12:21:30Z

Alright, enough people have seen this now so we can

/unhold

howardjohn · 2023-04-03T16:29:44Z

Thanks all. Need 1 lgtm to get this in

kflynn · 2023-04-03T16:47:01Z

Looks good from here, FTR.

robscott · 2023-04-03T16:48:23Z

Thanks @howardjohn!

/lgtm

* With kubernetes-sigs#1878 the existing flag `EnableAllSupportedFeatures` used in implementations to run conformance tests pertaining to all Gateway/Ingress features will now also run Mesh conformance tests which might not be the intent. * rm `EnableAllSupportedFeatures` and ask Ingress implementations to set `SupportedFeatures` to `AllGatewayFeatures` to achieve the same intent as before. Mesh implementations that also support Gateway features can set `SupportedFeatures` to `AllFeatures`. Signed-off-by: Arko Dasgupta <[email protected]>

k8s-ci-robot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. area/conformance cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Mar 25, 2023

k8s-ci-robot requested review from arkodg and youngnick March 25, 2023 00:16

k8s-ci-robot added the size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. label Mar 25, 2023

howardjohn marked this pull request as ready for review March 28, 2023 17:21

k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Mar 28, 2023

k8s-ci-robot requested review from robscott and Xunzhuo March 28, 2023 17:22

howardjohn force-pushed the mesh-conformance branch 4 times, most recently from 595bc40 to 46cd068 Compare March 28, 2023 20:39

shaneutt requested changes Mar 29, 2023

View reviewed changes

conformance/conformance_test.go Outdated Show resolved Hide resolved

shaneutt added this to the GAMMA: HTTPRoute Support Ready For Implementors To Try milestone Mar 29, 2023

shaneutt mentioned this pull request Mar 29, 2023

Initial GRPCRoute Conformance Tests #1884

Closed

k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Mar 29, 2023

gamma: initial mesh conformance tests

ef75c8a

howardjohn force-pushed the mesh-conformance branch from 46cd068 to ef75c8a Compare March 30, 2023 15:04

k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Mar 30, 2023

shaneutt self-requested a review March 30, 2023 15:07

shaneutt self-assigned this Mar 30, 2023

shaneutt requested changes Mar 30, 2023

View reviewed changes

conformance/conformance_test.go Show resolved Hide resolved

shaneutt requested review from michaelbeaumont and kflynn March 30, 2023 15:46

Add TODO

4b00179

howardjohn requested a review from shaneutt March 30, 2023 22:21

shaneutt mentioned this pull request Mar 31, 2023

Conformance Tests: No implicit test runs without features #1894

Merged

shaneutt requested review from mikemorris and keithmattix March 31, 2023 13:16

shaneutt approved these changes Mar 31, 2023

View reviewed changes

k8s-ci-robot added do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. approved Indicates a PR has been approved by an approver from all required OWNERS files. labels Mar 31, 2023

keithmattix approved these changes Mar 31, 2023

View reviewed changes

robscott reviewed Mar 31, 2023

View reviewed changes

conformance/mesh/manifests.yaml Outdated Show resolved Hide resolved

robscott reviewed Mar 31, 2023

View reviewed changes

howardjohn mentioned this pull request Mar 31, 2023

Move mesh test app into k8s org #1895

Closed

pin image

5ca69ae

michaelbeaumont approved these changes Apr 1, 2023

View reviewed changes

k8s-ci-robot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Apr 3, 2023

k8s-ci-robot assigned robscott Apr 3, 2023

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Apr 3, 2023

k8s-ci-robot merged commit 8dde508 into kubernetes-sigs:main Apr 3, 2023

arkodg mentioned this pull request Apr 4, 2023

rm EnableAllSupportedFeatures & add AllGatewayFeatures #1907

Closed

shaneutt mentioned this pull request Apr 5, 2023

Features for every API in conformance tests #1891

Closed

mikemorris mentioned this pull request Apr 22, 2024

GEP: Mesh Conformance Testing Plan #1686

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Initial Mesh (GAMMA) conformance tests #1878

Initial Mesh (GAMMA) conformance tests #1878

howardjohn commented Mar 25, 2023 •

edited

Loading

k8s-ci-robot commented Mar 25, 2023

shaneutt left a comment

keithmattix left a comment

robscott Mar 31, 2023

howardjohn Mar 31, 2023

michaelbeaumont left a comment

k8s-ci-robot commented Apr 1, 2023

shaneutt commented Apr 3, 2023

howardjohn commented Apr 3, 2023

kflynn commented Apr 3, 2023

robscott commented Apr 3, 2023

Initial Mesh (GAMMA) conformance tests #1878

Initial Mesh (GAMMA) conformance tests #1878

Conversation

howardjohn commented Mar 25, 2023 • edited Loading

k8s-ci-robot commented Mar 25, 2023

shaneutt left a comment

Choose a reason for hiding this comment

keithmattix left a comment

Choose a reason for hiding this comment

robscott Mar 31, 2023

Choose a reason for hiding this comment

howardjohn Mar 31, 2023

Choose a reason for hiding this comment

michaelbeaumont left a comment

Choose a reason for hiding this comment

k8s-ci-robot commented Apr 1, 2023

shaneutt commented Apr 3, 2023

howardjohn commented Apr 3, 2023

kflynn commented Apr 3, 2023

robscott commented Apr 3, 2023

howardjohn commented Mar 25, 2023 •

edited

Loading