chore: conformance profiles API version and channel #2062
Conversation
The GEP-1709 has been updated with content on how to detect the APIVersion and the channel and how to check their consistency. Signed-off-by: Mattia Lavacca <[email protected]>
k8s-ci-robot
added
release-note-none
Co-authored-by: Shane Utt <[email protected]>
There was a problem hiding this comment.
/approve
/hold to allow others to take a look
/cc @sunjayBhatia @arkodg @robscott
k8s-ci-robot
added
the
do-not-merge/hold
k8s-ci-robot
added
the
approved
There was a problem hiding this comment.
one issue I could see arising is how to handle an outdated set of CRDs in the cluster in comparison w/ the test suite (e.g. v0.X.Y CRDs vs. running the v1.A.B tests)
I don't expect too much in the way of behavior change between versions going forward since the API is stabilizing, but things that are considered experimental in older versions may no longer be as the tests/versions progress, which could maybe cause a test failure/reporting issue
I wonder if in addition to this we should consider writing in the report what version the test suite is from and discouraging too much version skew (or maybe not allowing any for certified reports)
Test suite version tracking is a good point. I think we could incorporate that here, unless you'd like to do a follow up PR to add this and any other thoughts you have on the proposal so far @sunjayBhatia? |
If @mlavacca wants to add it here that's great, otherwise I'm happy to make a follow-up 👍🏽 |
Thanks for this good point @sunjayBhatia, I've addressed it here |
k8s-ci-robot
added
size/M
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: arkodg, mlavacca, shaneutt, sunjayBhatia The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
shaneutt
removed
the
do-not-merge/hold
|
Thanks for the review @arkodg and @sunjayBhatia! I'll let one of my co-maintainers drop the final LGTM: /cc @robscott @youngnick |
| @@ -280,6 +318,7 @@ implementation: | |||
| - @acme/maintainers | |||
| date: "2023-02-28 20:29:41+00:00" | |||
| gatewayAPIVersion: v0.7.0 | |||
| gatewayAPIChannel: standard | |||
There was a problem hiding this comment.
Does this mean that most implementations would need to submit separate reports for each Gateway API Channel? I'm not sure we gain any value from that since the experimental channel is always a superset of the standard channel, but I may be missing something here.
I agree that the version of tests that are being run is critical though, so I'm glad that's included here.
There was a problem hiding this comment.
Since the experimental channel is a superset of the standard one, we could treat the report in the same way, which means being conformant to the experimental channel implies being conformant to the standard one. So that you just need to submit the experimental report to declare the standard conformance as well. WDYT?
There was a problem hiding this comment.
That's kinda what I'm thinking - we may just want to recommend that implementations that support experimental features run conformance tests with experimental CRDs. I'm not sure that we need to actually state that in the report, but don't feel strongly either way.
There was a problem hiding this comment.
Yeah, I agree we need to add the proper documentation about this as well.
Signed-off-by: Mattia Lavacca <[email protected]>
|
/retest |
|
Thanks @mlavacca! /lgtm |
k8s-ci-robot
added
the
lgtm
What type of PR is this?
/kind gep
/area conformance
What this PR does / why we need it:
In support of #1709, the GEP-1709 has been updated with content on how to detect the
gatewayAPIVersionand thegatewayAPIChanneland how to check their consistency.Does this PR introduce a user-facing change?: